Neural Eulerian Scene Flow Fields
Authors: Kyle Vedder
Neehar Peri
Ishan Khatri
Siyi Li
Eric Eaton
Mehmet Kemal Kocamaz
Yue Wang
Zhiding Yu
Deva Ramanan
Joachim Pehserl
Venue: ICLR 2025 Poster (2025)
Keywords: Scene Flow
Neural Prior
Ordinary Differential Equation
Reconstruction
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We model scene flow as an estimating a ODE over many observations; our unsupervised method is high quality (SotA on important benchmarks) and works out-of-the-box on many diverse domains.

Abstract: We reframe scene flow as the task of estimating a continuous space-time ordinary differential equation (ODE) that describes motion for an entire observation sequence, represented with a neural prior. Our method, EulerFlow, optimizes this neural prior estimate against several multi-observation reconstruction objectives, enabling high quality scene flow estimation via self-supervision on real-world data. EulerFlow works out-of-the-box without tuning across multiple domains, including large-scale autonomous driving scenes and dynamic tabletop settings. Remarkably, EulerFlow produces high quality flow estimates on small, fast moving objects like birds and tennis balls, and exhibits emergent 3D point tracking behavior by solving its estimated ODE over long-time horizons. On the Argoverse 2 2024 Scene Flow Challenge, EulerFlow outperforms all prior art, surpassing the next-best unsupervised method by more than 2.5 times, and even exceeding the next-best supervised method by over 10%. See https://vedder.io/eulerflow for interactive visuals.

Published as a conference paper at ICLR 2025
NEURAL EULERIAN SCENE FLOW FIELDS
Kyle Vedder 12-
Neehar Periz.3
Ishan Khatri
Siyi Li
Eric Eaton'
Mehmet Kocamaz
Yue Wang"
Zhiding Yu²
Deva Ramanan" Joachim Pehser!
University of Pennsylvania
"NVIDIA
Carnegie Mellon University
(a) Small object motion extraction...
(b) diverse, dynamic
w..th emergent 3D point tracking behavior!
Figure 1: EulerFlow is able capture the motion of small, fast moving objects with few lidar points, such a bird flying in front of an autonomous vehicle (Figure Iap. EulerFlow 's flexibility allows it estimate scene flow for fast-moving table top objects without additional hyperparameter tuning (Figure lb). FulerFlow's s ODE estimate exhibits emergent 3D point tracking behavior without explicit long-horizon supervision (Figure Tcl. Note that point clouds are shown in color for visualization purposes only; RGB is not used during optimization.
ABSTRACT
We reframe scene flow as the task of estimating a continuous space-time ordinary differential equation (ODE) that describes motion for an entire observation se- quence, represented with neural prior. Our method, EulerFlow, optimizes this neural prior estimate against several multi-observation reconstruction objectives, enabling high quality scene flow estimation via self-supervision on real-world data. EulerFlow works out-of-the-box without tuning across multiple domains, including large-scale autonomous driving scenes and dynamic tabletop settings. Remarkably, FulerFlow produces high quality flow estimates small, fast moving objects like birds and tennis balls, and exhibits emergent 3D point tracking behavior by solving its estimated ODE over long-time horizons. On the Argoverse 2 2024 Scene Flow Challenge, EulerFlow outperforms all prior art, surpassing the next-best unsuper- vised method by more than 2.5x, and even exceeding the next-best supervised method by over 10%. See vedder. www.::/r//eeeeeeeeeeeo for interactive visuals.
INTRODUCTION
Scene flow estimation is the task of describing motion with per-point 3D motion vectors between
temporally successive point clouds (Dewan et 2016 Liu al. 2019 Erçelik et al., 2022 Jund et al. 2021] Zhang et al. 2024bb Vedder et al. 2024 Khatri et al. 2024). Such per-point motion estimates are critical for autonomy in diverse environments, e.g., maneuvering around open-world objects like debris (Peri et al. 2022a) or folding deformable cloth (Weng et al. 2022). Importantly, scene flow estimation requires not only an understanding of object geometry, but also its motion. However, scene flow methods broadly do not work smaller objects (Khatri al.) (024)). For example, in the autonomous vehicles domain, Khatri et al. highlight that even supervised methods struggle describe the majority of pedestrian motion, with unsupervised methods failing dramatically. Scene flow promises to be powerful primitive for understanding the dynamic world, but such failures
explain why it has limited adoption in downstream applications like tracking (Zhai al. 2020) or open-world object extraction (Najibi et al. 2022).
*Corresponding email: kvedder seas wwwnn

---

Published as a conference paper at ICLR 2025
(a) EulerFlow (Two Frame)
(b) Fast NSF (Two Frame)
(c) Liu ct al. (Two Framc)
(d) Ground Truth (Two Frame)
(e) Elwerllow (Full Sequence)
(1) Fast NSF (Full Sequence)
(g) Liu et al. (Full Sequence) (h) Ground Truth (Full Sequence)
Figure 2: We visualize example of five pedestrians crossing the street in front of stopped car, cherrypicked to have unusually high density lidar returns, making it particularly easy to estimate flow. Figures 2a depict two-frame flow visualization of EulerFlow and several strong baselines. Notably, only visualizing flow over two frames makes it difficult to distinguish flow quality. In contrast, Figures 2e 2h depict flow vectors over the full sequence, making differences in quality clear; for example, FulerFlow is the only one without artifacts on the stopped car.
Scene Flow via ODE. In Figure visual assessment of scene flow quality is much easier in an accumulated global frame; while incomplete due to an implicit time axis, these accumulated flow vectors allow viewers imagine how positions in 3D space evolve over many timesteps, and compare that to predicted flows. This imagination of scene flow as continuous motion over large time intervals motivates us to model scene flow as an ordinary differential equation (ODE) that describes the scene's instantancous motion across continuous position and time. Scene flow estimation then becomes the task of estimating this ODE. We can straightfowardly represent this ODE estimate with a neural prior (Li et al. (2021b) and optimize it against scene flow surrogate objectives, both over single frame pairs and extended across arbitrary time intervals to produce better quality estimates. We formalize this in Section and propose the Scene Flow via ODE framework.
We instantiate Scene Flow via ODE with standard point cloud distance objectives like Chamfer Distance create FalerFlow. Notably, FulerFlow outperforms all prior art, supervised or unsupervised. on the Argoverse 2 2024 Scene Flow Challenge and Waymo Open Scene Flow benchmark. It outperforms prior unsupervised methods by large margin (> 2.5x mean dynamic error reduction), and is able to capture small, fast moving objects, including those outside of labeled taxonomies (e.g. the flying bird in Figure la). Due its simplicity, EulerFlow is able to provide high quality scene flow out-of-the-box tabletop settings (Figure [b} without domain-specifet tuning. Finally, we show that simple ODE solving techniques like Euler integration can be used to extract 3D point tracks (Figure [c]. which serves as both an exciting emergent behavior as well as a mechanism for visualizing and interpreting the quality of the continuous ODE estimate.
EulerFlow.
real-world data for other important domains such as dynamic
We present four primary contributions:
We propose Scene Flow via ODE (SFvODE), a reframing of scene flow estimation as the task of fitting an ODE over all observations to describe the change of continuous positions over continuous time, unlocking new class surrogate objectives that enable better scene flow estimates.
We instantiate SFvODE with EulerFlow, a flexible unsupervised scene flow method that achieves state-of-the-as performance on the Argoverse 2024 Scene Flow Challenge, beating all prior supervised and unsupervised methods.
We study EulerFlow and show its strong performance is derived from its ability to optimize its
arbitrary time horizons. We show that EulerFlow's simple, flexible formulation allows to run unmodified on a variety of domains, with emergent capabilities like 3D point tracking behavior.
ODE estimate against the full sequence of observations
2

---

Published as a conference paper at ICLR 2025
2 BACKGROUND AND RELATED WORK
Evaluation. Dewan et al. formalized scene flow for point clouds as the task of estimating motion between point cloud Pa at time and point cloud Pi+1 at + 1 by estimating the true flow FL,++ i.c. true residual vectors for every point in P. that describe its movement to its associated position at t + 1. Error is computed by measuring the per-point endpoint distance between estimated and ground truth vectors. Historically, these errors are reported with a per-point average (Average EPE); however, as Chodosh et al. show, Average EPE is dominated by background points, preventing meaningful measurement of non-ego object motion descriptions. Khatri et al.) address this shortcoming with Bucket Normalized EPE, which reports per-class performance normalized by speed, allowing for direct comparisons across classes with very different average speeds (e.g. pedestrians and cars). Bucket Normalized EPE is the basis for the Argoverse 2024 Scene Flow Challenge where methods are ranked by the mean error of their motion descriptions (mean Dynamic Normalized EPE).
Input/ Output Formulation. Dewan al. [s choice to formulate scene flow using only two input frames is arbitrary; it's the minimal information needed to extract rigid motion, but there are not real-world problems constrained to only have access to two frames. Indeed, using five or ten frames of past observations is standard practice in the 3D detection literature (Zhu et al. 2019 Vedder & Eaton
2022, Peri et al. 20226 2023 Nalty et al. 22022) , and multi-frame formulations have started to appear in the scene flow literature: Liu et al. (2024) and Flow4D (Kim et al. (024) use three (P.-1, Pr. P1+1)
and five input frames (Pt-3...... Pa+1) respectively to predict Fr,4+1 As we discuss in Section 3 rather than just using more observations to estimate flow for a single frame pair, we formulate scene flow as a joint estimation problem: given the full observation sequence ,Po....,PNy we estimate
all flows FN 1NN between all adjacent observations.
Feedforward Methods. Feedforward networks are popular class of scene flow methods due to their fast inference speed (Liu et al.
2019 Behl et al. al./ 22199; Tishchenko et al., 2020 Kittenplon et al. 2021 Wu et al. 2020, Puy et al. |2000] Li et al. 202 Jund et al. |021 Gu et al. 2019] Battrawy 2022 Wang et al. [022bb Kim et al. 2024; Zhang et al. 2024a). While they are often trained
ct al.
with supervised labels, recent work has developed distillation pipelines that leverage unsupervised
pseudolabelers (Vedder et al. 2024 Zhang et al. 2024b Lin & Caesar 2024).
Neural Scene Flow Prior. Li et al. (2021b) propose Neural Scene Flow Prior (NSFP), a widely
adopted unsupervised scene flow approach. NSFP uses the inductive bias of the smooth, restricted learnable function class of two ReLU MLP coordinate networks (8 hidden layers of 128 neurons); 0 to estimate forward flow and 0' to estimate backwards flow, minimizing
TruncatedChamfer( Pt +0 P.41) + + (P()) +
( (P:+ P?llz
(I)
where TruncatedChamferi is defined as the standard L2 Chamfer distance, but with per-point distances above meters set to zero in order reduce the inffuence of outliers. NSFP is optimized for at most 1000 steps with early stopping.
Motion Beyond Two Frames. Wang et al. (2022a) tackles the adjacent problem of estimating 3D point trajectories over 25 frames with Neural Trajectory Prior (NTP) by jointly optimizing three separate ReLU MLP neural priors: basis estimator (embed(t) + 256 x 25 x 3 tensor, where 256 is the dimension of the trajectory basis), 2) coordinate network bottleneck encoder, and 3) a bottleneck decoder to estimate a per-point linear combination over the learned trajectories. Trajectories are optimized for both a one-frame lookahead L2 Chamfer loss and a cyclic consistency loss over the entire velocity space trajectory.
sinusoidal embedded, time conditioned, 25 frame trajectory
Deformation in Reconstruction. Nerfies (Park et al. 2021> and DynamicFusion (Newcombe et al. 2015) estimate a deformation field to warp a canonical frame to explain the observed frame. While capable of describing small motions, these methods require canonical frame that contains all of the relevant geometry to deform; however, in large, highly dynamic scenes like autonomous driving, there is often no frame that contains all moving objects. By comparison, Scene Flow via ODE does not assume the existence of a canonical frame, instead only describing how the scene changes.
3

---

Published as a conference paper at ICLR 2025
Optimization I oop
Positlon
Oburradie
Direction
Art mo
Plow
Full Observation Sequence
Full Observatio Soguence with Seare Flow
Figure 3: Overview of our Scene Flow via ODE framework, which estimates an ODE across the entire observation sequence by optimizing against multi-frame objectives. This ODE estimate is represented with a neural prior (Li et al. 2021b), providing a flexible, general representation for describing position-time motion.
3
SCENE FLOW VIA ODE
Prior art consumes multiple frames (Pa-N..... P.+1) as input, but these methods are ultimately only tasked with estimating flow vectors between P. and Pe+1- We instead pose the problem of estimating a time-conditioned flow field that describes motion for all adjacent point clouds Pr, Pt+1 in the entire flow as positional change
sequence (Po.-.. PN) To do this, rather than describing fixed interval (F are residual vectors over the interval f to t + 1) as we did in Section can instead express these changes differential equation that describes positional change over continuous time.
2 we
Eulerian View
Lagrangian View
/
Figure Comparison of Fulerian and Lagrangian descriptions of 2D flow. An Fulerian view characterizes flow field via instantaneous velocities many different points, while a Lagrangian view characterizes a flow field via trajectories of many different particles across time. Both approaches are valid ways of describing underlying flow field, and with sufficient characterization one view can be readily converted to another, but the Lagrangian view relies on a definition of the definition of consistent canonical frame
Formally, given a scene, let L( yo, =o, t) be the Lagrangian view of the scene's true flow field, i.e. continuous function that, given canonical particle (Io, Mo. 2u) in canonical frame 0, describes its (x, y. 2) position at frame As we discuss Section 2 this Lagrangian view is common in the the deformable reconstruction literature, and the requirement for canonical frame means such approaches struggle to describe scenes where there is no frame that contains all moving objects.
To break this canonical frame dependence, choose take Eulerian view of motion, i.e. F(a. 2, (bL, aL, 0L=). which describes the velocity at position (x, y. at time t. As we show in our derivation in Appendix C.I. this formulation does not require an external canonical frame to estimate point's trajectory from / I': instead, we can simply set the initial conditions of the ODE at to 24, yt. 2t and utilize off-the-shelf ODE solver (c.g. Euler integration) to extract flow from i to t', expressed as E(xe. V/. 2,, t,)').
8: 0t
We do not know the true flow field F when estimating scene flow; however, with a neural prior 8 (F x 0), and optimize against surrogate objectives. This framing, which formalize into the Scene Flow via ODE framework (SFvODE; Figure 33, allows 0 to benefit from constructive interference between objectives, as well as enables us to formulate objectives over arbitrarily long time horizons, unlocking high quality estimates.
represent F
4

---

Published as a conference paper at ICLR 2025
4 EULERFLOW
Scene Flow via ODE proposes a framework where the neural prior represents an estimate of the Eulerian flow field F (i.e. F 0); however, does not prescribe the optimization objectives for Thus, we instantiate Scene Flow via ODE with EulerFlow, point cloud only scene flow method with reconstruction and cyclic consistency objectives across the entire sequence of observations.
As we show in Equation 17 Appendix C.4 we can use ('s Fulerian flow field estimate to extract an estimated point trajectory from Zt, yt, 24 at t to some future location at time t' via Euler integration over 0 without requiring canonical frame definition, i.e. Eg (x Mts 2t, t, t'). By extracting point trajectories for every point in Pt using Eg. we can not only construct a two-frame scene flow estimate of Ft,t+1, but also estimate flow arbitrary future or prior timesteps (e.g. Fi,t+2 Ft,,-1). This allows us to optimize over multi-frame reconstruction objectives: we can now pose reconstruction surrogate objectives between any two point clouds in our observation sequence, not just adjacent point clouds Pt and P(+1- Similarly, we can straightforwardly pose cyclic consistency objectives by composing Ft,++1 and Fi+1,1. Formally, for Pi's Fi,t+& (for any 2), we define
(Pr - F Vp E Pr. Eo Pa, L, 1 k:) k)
(2)
enabling us to pose 0's optimization objective VP, E (Po. ..., Px) across the window of size W
vk E (-W,.... W1 \ (0) TruncatedChamffe((aueee, (Pt,k), Pata) ||uulero (Eulero (P,, 1). -1) Pall2
arg min Σ
(3)
|
In practice, we set to and 0.01. We provide additional implementation details in Appendix C In order to optimize 6, our estimate of the Eulerian flow field F, we perform Euler integration to extract point cloud flow estimates as part of reconstruction losses. Notably, EulerFlow only requires single optimization loop over single neural prior 0 compared to NSFP's two priors 0 and 0'. Our neural prior is a straightforward extension to NSFP's coordinate network prior. Like with NSFP, TruncatedChamfer is defined as the standard L2 Chamfer distance with per-point distances below meters. As we show in Section [] EulerFlow's 's simple ODE estimation formulation across multiple observations produces high quality flow, and solving this ODE over arbitrary time spans unlocks emergent point tracking behavior.
5 EXPERIMENTS
In order to validate FulerFlow's construction and better understand the impact of its design choices, we perform extensive experiments on the Argoverse 2 (Wilson et al.) 2021) and Waymo Open (Sun)
al. 2020) autonomous vehicle datasets. We compare against open source implementations of FastNSF (Li et al. 2023), Liu et al., NSFP
Li et al. 2021b), FastFlow3D (Jund et al.] 2021]. and variants of ZeroFlow (Vedder et al. 2024 provided by the ZeroFlow model zod third card-party implementation of NTP (Wang et al.) 2022a) from Vidanapathirana et al.) and Argoverse 2 2024
Scene Flow Challenge leaderboard submission results from the authors of Flow4D (Kim al. 2024), TrackFlow (Khatri et al. 2024). DeFloow++DDellow (Zhang et al.] 2024a). ICP Flow (Lin Caesar 2024), and Sellow (Zhang et al. 2024b). As discussed Khatri et al.] and used in the Argoverse 2024 Scene Flow Challenge, methods are ranked by their speed normalized mean Dynamic Normalized EPE.
Implementation Details.
To showcase the flexibility of EulerFlow without hyperparameter tuning, for all quantitative experiments we run with a neural prior of depth 8 (NSFP's default depth), except for submission to the Argoverse 2 2024 Scene Flow Challenge (Section (511) where, based our depth ablation study on the val split (Section 5.2.3). we set the depth of the neural prior to 18. As discussed in NTP's original paper (Wang et al. 2022a) and confirmed by our experiments, NTP struggles to converge beyond 25 frames, so we only compare against it in 20 frame settings. As is
VVisualizations shown in color for hetter viewing. FulerFlow can also use monocular depth estimates
(Appendix A.2
https:/ wtt..ccom/kleeeeeeeeeeeee
from Vedder
2024
5

---

Published as a conference paper at ICLR 2025
typical in the scene flow literature (Chodosh et al. 2023). perform ego compensation and ground point removal on both Argoverse and Waymo Open using the dataset provided map and ego pose.
5.1 How DOES EULERFLOW COMPARE TO PRIOR ART ON REAL DATA?
EulerFlow achieves state-of-the-art performance on the Argoverse 2 2024 Scene Flow Challenge leaderboard. Despite being unsupervised, EulerFlow surpasses all prior art, supervised or un- supervised, including Flow4D (Kim al. 2024 and ICP Flow (Lin & Caesar 2024 Notably, FulerFlow achieves < 2.5x lower error mean Dynamic EPE than ICP Flow and beats Flow4D by over 10%.
EulerFlow (Ouns
0.15M 0.1463
Flow4D
TrackFlow
0128889
DeFlow--
ICP Flow
0,5309
SeFlow
DeFlow
0.5706
Liu et al. 2024
0.4134
0.4219
Zerol low XL. 5x
0.4389
Zecof fow X1.
0.6421
ZemFlow 6x
0:4846
Zero 3x
0.5067
FartFlow3D
0.5323
ZeroFlow lx
0.2841
0.0
0.1
0.2
0.3
0.4
0.5
0.6
mean Dynamic Normallzed EPE
Figure 5: Mean Dynamic Normalized EPE of EulerFlow compared to prior art on the Argoverse 2024 Scene Flow Challenge test set. EulerFlow is state-of-the-arr, beating all supervised (shown in black) and unsupervised (shown in white) methods. Lower is better.
EnierFlow (Ours)
0:2187
FastNFF
NSFP
0.5725
ZaroFoow
0.73%
0.7573
Dynsuic Nocmalised EPE
Figure 6: Mean Dynamic Normalized EPE of EulerFlow compared prior the Waymo Open validation set. EulerFlow is state-of-the-arr, beating all supervised (shown in black) and unsupervised (shown in white) methods. Lower is better.
FulerFlow's dominant performance also holds on Waymo Open (Sun et al.) 2020); we compare against several popular methods (Figure ( and EulerFlow again out-performs the baselines by wide margin, more than halving the error over the next best method.
5.2 WHAT CONTRIBUTES TO EULERFI.OW STATE-OF-THE-ARR PERFORMANCE?
We find that EulerFlow's 's lower mean Dynamic EPE can be attributed better performance on smaller objects. On Argoverse 2, compared to Flow4D, EulerFlow's 's improves on WHEELED VRU (Figure 7 a small, rare, fast moving class. Compared to ICP Flow, EulerFlow's 's improves on all classes (at least halving the error on every class!), with the largest improvements coming from the smaller and harder to detect objects PEDESTRAIN and WHEELED VRU (Figures 7d). Waymo Open, the same story holds; the most dramatic performance improvements come from the small object classes of CYCLIST and PEDESTRIAN (Figure
These results are consistent with our qualitative visualizations. Figure 13 shows EulerFlow is able to cleanly extract the motion of a bird flying past the ego vehicle. Euler integration using EulerFlow's
*Flow4D is the winner of the 2024 Argoverse 2 Scene Flow Challenge supervised track.
'ICP Flow the winner of the 2024 Argoverse Scene Flow Challenge unsupervised track.

---

Published as a conference paper at ICLR 2025
Liu al. 2024
Dysamic Normalad EPE
0.8
Liu et at. 2004
KoroPloe XL. Bx
Zeral XL 30
ZoeodFoo Ix
FadNNF
ZeroFlooe XL. de
Zerolloow XL de
2exo Sx
ZonsPlow St
FondFiow-D
PeroFlow
0.6
Dake' Sow (Ours)
PantNSP
Zerolloow 3b.
0.6
Tracig
DeFlow i-
[CP Flow
Sel
0.4
Trxc
DePPooe 1-4
LCP Flw
Deflow
N
0.4
Flow dD
NSFP
Ploed
Exirr
0.2
(a) CAR
(b) OTHER VEHICLES
Dynanic Normaltel HPE
0.8
0.5
0.6
0.6
0.4
0.4
0.2
(c) PEDESTRIAN
(d) WHEELED VRU
Figure 7: Per class Dynamic Normalized EPE of EulerFlow compared prior art the Argoverse 2024 Scene Flow Challenge test set. Supervised methods shown in black, unsupervised methods shown in white. Methods are ordered left right by increasing mean Dynamic Normalized EPE. Lower is better.
Dynanic Normslited EPE
0.8
Bale (Ous)
ZreeFoow
Fose FlodDD
0.8
0.8
0.6
Fast
NSPP
0.4
0.2
0.2
0.2
(a) VEHICLE
(b) CYCLIST
(C) PEDESTRIAN
Figure 8: Per class Dynamic Normalized EPE of EulerFlow compared prior the Waymo Open validation set. Supervised methods shown black, unsupervised methods shown in white. Methods are ordered left right by increasing mean Dynamic Normalized EPE. Lower is better.
ODE, starting at the bird's takeoff position and ending when it loses lidar returns, produces emergent 3D point tracking behavior on the bird through its trajectory (Figure 9), further demonstrating the quality of EulerFlow's model of the scene's 's motion.
(a) Bird trajectory via Euler integration from takeoff
(b) Bird being tracked
Figure 9: FulerFlow able to track the bird over frames by performing Euler integration starting from takeoff until it loses all point cloud lidar returns.
7

---

Published as a conference paper at ICLR 2025
EulerViow Full
0.1988
Flle:Flow Len 50
0.1948
EuleFfow Ton 20
0.2103
NTP (Len 20)
0.2805
FalerFoww Len
0240189 0.4683
NSFP (Len
0.1
0.5
Dynamic Normalized ETE
Figure 10: Mcan Dynamic Normalized EPE of EulerFlow for various sequence lengths on the Argoverse val split, compared against representative baselines. These results demonstrate that Fu- lerflow improves with sequence length; however, at sequence length of 20, our method significantly outperforms NTP, suggesting that EulerFlow's 's performance cannot solely attributed to longer sequence modeling.
5.2.1 How DOES OBSERVATION SEQUENCE LENGTH IMPACT EULERFLOW?
EulerFlow benefits from constructive interference from ODE estimation over many observations. Does this sufficiently explain EulerFlow's performance? Figure shows the performance of EulerFlow at length 5, 20, 50, and full sequence (roughly 160 frames) compared to NSFP and NTP at length 20. EulerFlow sees clear continual improvements as number of frames increases without signs of saturation. However, sequence length alone does not explain EulerFlow's s performance; even at the same sequence length of 20, EulerFlow demonstrates significantly better performance than NTP.
As we discuss in Section 3
5.2.2
How DO MULTI-FRAME OPTIMIZATION OBJECTIVES IMPACT EULERFLOW?
BulesFlow
0.1588
BaleFFow No Cycle
EalerFlow No R
0,2909
NTP [Len
0,2803
[Len 2)
0.4600
Dynaziic Normalized EPE
Figure 11: Mean Dynamic Normalized EPE of EulerFlow for various losses the Argoverse 2 val split, compared against representative baselines. These results demonstrate that EulerFlow's multi-observation optimization objectives significantly improve overall performance.
Equation 3 outlines two major components of EulerFlow's loss: multi-frame Euler integration for Chamfer Distance reconstruction, and cycle consistency. Figure compares EulerFlow without more than integration step (No k 1) and without cycle consistency rollouts (No Cycle) better understand the impact of these components. Ablating multi-step Euler integrated rollouts results in significant degredation, as they
strong forcing function to have consistent, smooth flow volumes; indeed, despite consuming the entire sequence, EulerFlow (No k >> 1) is only slightly better than NTP with sequence length of 20. These results highlight the power of multi step rollouts and their potential as objective for other test-time optimization methods and feedforward methods.
5.2.3 How DOES THE CAPACITY OF THE NEURAL PRIOR IMPACT BUI.RFILOW?
Li et al. ablate the capacity of NSFP's neural prior to characterize underfitting and overfitting to optimization objective noise, ultimately selecting depth of FulerFlow's neural prior is structured similarly; however, NSFP is fitting single snapshot in time, while EulerFlow is fitting entire ODE over significant time intervals. Intuitively, one would expect that full sequence modeling would benefit from greater network capacity.
To evaluate this, we perform a sweep of EulerFlow's network depth on the Argoverse validation split (Figure 12h. While EulerFlow with NSFP's default of depth performs well Argoverse evaluations (0.1% % worse than the supervised state-of-tte of-the-art Flow4D), we see that performance improves the neural prior's depth increases until depth 18 (indicating underfitting). where start
8

---

Published as a conference paper at ICLR 2025
to see degradation (indicating overfitting noise). Based these results Argoverse 2 2024 Scene Flow Challenge leaderboard submission uses depth 18 neural prior (Figure 5
Buler Flow Depti
0.1472
Flow Depth 20
0.1463
ElleeFlow Depth
Depth
0.1488
Flow Depth
0.1633
ElweeTlow Depth 12
0.1538
How Depth
EulerFlow Depth
0.158
EulerFlo Degth
0:2113
0.026
0.050
0.075
0.100
0.175
0.200
Dynamic Normailed KPB
Figure 12: Mean Dynamic Normalized EPE of EulerFlow on the Argoverse 2 val split for different neural prior capacities. Shallow networks underfit the ODE, while deeper networks overfit to noise in the optimization objectives.
(a) EulerFlow (Ours)
(b) Fast NSF
(c) ZeroFlow 5x
(d) Liu et al.
(c) NSFP
(f) Ground Truth
Figure 13: Visualization of EulerFlow compared to prior art for the same scene as Figure and
Figure Da EulerFlow is able to extract the bird's trajectory; however, all other methods except Liu al. fail to recognize this motion, and Liu et al.'s flow is marred by severe scene artifacts. The bird is outside the labeled object taxonomy, and so its motion is unlabeled ground truth (Figure 131
5.3 BEYOND AUTONOMOUS VEHICLES
Due to dearth of real-world, labeled scene flow data, prior scene flow work real data overwhelm- ingly evaluates on autonomous vehicle datasets (Dewan et al. 2016 et al. 202 Jund et al.
2021 Li al. 2023 Chodosh al.) 2023 Liu et al. 2024 Vedder al. 2024 Khatri al. 2024); consequently, motion understanding in other important domains like tabletop manipulation has been neglected. To showcase FulerFlow's out-of-the-hox flexibility and generalizability.
visualize Fu- lerFlow on several dynamic tabletop scenes we collected using the ORBBEC Astra, low cost depth camera commonly used in roboties (Figure 14). For viewing ease, we paint our point clouds with color; however, RGB information is not provided to EulerFlow during optimization. While EulerFlow only reasons about point clouds, it can leverage video mono depth estimates describe RGB-only scene flow (Appendix A.2 Interactive visuals are available at vedder.. io/eulerflow
CONCLUSION
By reframing scene flow fitting ODE positions for full sequence of observations, able to construct EulerFlow, a simple unsupervised scene flow method that achieves rate-of--he-an performance the Argoverse 2 2024 Scene Flow Challenge and Waymo Scene Flow benchmark,

---

Published as a conference paper at ICLR 2025
Figure 14: Visualizations of FulerFlow's 's emergent 3D point tracking behavior that demonstrate the quality of its ODE estimate, Row I depicts tracking a tomato placed in the sink by a human hand; note the point does not move despite the hand grasping the tomato. Row 2 depicts tracking of painters tape rolling off a table; EulerFlow is able to estimate its trajectory even after it disappears out of frame. Row depicts tracking of the motion of jack commonly used in tabletop manipulation experiments (Venkatesh et al.) 2023). Row 4 depicts tracking of a tennis ball taped to a flexible rod. All tracks are produced by Buler integration through the estimated ODE from the initial conditions shown in the left column. Note that point clouds are shown in color for visualization purposes only.
where it beats all prior art, supervised unsupervised. EulerFlow is able describe motion small, fast moving, out of distribution objects unable to be captured by prior art, suggesting that makes good on the promises of scene flow as powerful primitive for understanding the dynamic world. also exhibits other emergent capabilities, like basic 3D point tracking behavior.
We believe that this ODE formulation has implications for scene flow large, including beyond test- time optimization methods; the power of multi-step Euler integration may translate to feedforward network training. Future work should explore feedforward models that perform autoregressive rollouts directly learn to estimate multiple steps into the future.
6.1 LIMITATIONS AND FUTURE WORK
EulerFlow's 's strong performance opens the book on an exciting new line of work; however, we feel that it's important to be candid about FulerFlow's current limitations in order make future progress.
EulerFlow is point cloud only. Point cloud sparsity bottlenecks performance; for instance, in Figure g and Figure 13 we were only able to track the bird for 20 frames because we lost lidar observations of the bird, while it remained visible in the car's RGB cameras. Future works should explore multi-modal fusion for better long-term motion descriptions.
FulerFlow expensive to optimize. With our implementation, optimizing FulerFlow for a single Argoverse 2 sequence takes 24 hours one NVIDIA V 100 16GB GPU, putting it par with the original NeRF paper's 's computation expense (Mildenhall et al. 202T). However, like with NeRF, we believe algorithmic, optimization, and engineering improvements
significantly reduce runtime.
EulerFlow does not understand ray casting geometry. During ego-motion, a static foreground occluding object casts a moving shadow on the background; this causes Chamfer Distance to estimate this as a leading edge of moving structure, encouraging false motion artifacts (Li et al. 2021b). This can be addressed with optimization losses that model point clouds as originating from a time of flight sensor with limited visibility, as has been successfully demonstrated in the reconstruction (Chodoshl et al. 2024) and forecasting literature (Khurana et al.) 2023 Agro et al.] 2024), rather than an unstructured set of points to be associated via local point distance.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Ben Agro, Quin Sykora, Sergio Casas, Thomas Gilles, and Raquel Urtasun. UnO: Unsupervised Occupancy Fields for Perception and Forecasting. In CVPR, 2024.
Ramy Battrawy, René Schuster, Mohammad-A Ali Nikouei Mahani, and Didier Stricker. RMS-FlowNet: Efficient and Robust Multi-Scale Scene Flow Estimation for Large-Scale Point Clouds. In Int. Conf. Rob, Aut., pp. 883-889. IEFE, 2022.
Aseem Behl, Despoina Paschalidou, Simon Donné, and Andreas Geiger. Pointflownet: Learning representations for rigid motion estimation from point clouds. In Inl. Conf. Compul. Vis., pp. 7962-7971, 2019.
D. J. Butler, J. Wulff, G. B. Stanley, and M. J. Black. A naturalistic open source movie for optical flow evaluation. In A. Fitzgibbon et al. (Eds.) (ed.), European Conf. on Computer Vision (ECCV), Part IV, LNCS 7577, pp. 611-625. Ppringer-Verley October 2012.
Ricky T. Q. Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. Neural ordinary differential equations. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NeurlPS' 18, pp. 6572-6583, Red Hook, NY, USA, 2018.
Shin-Fang Chng. Sameera Ramasinghe, Jamie Sherrah, and Simon Lucey. Gaussian Activated Neural Radiance Fields for High Fidelity Reconstruction and Pose Estimation. Computer Vision ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XXXIII, pp. 264-280, Berlin, Heidelberg, 2022. Springer-Verlay ISBN 978-3-031-19826-7.
Nathaniel Chodosh, Deva Ramanan, and Simon Lucey. Re-Evaluating LiDAR Scene Flow for Autonomous Driving, arXiv preprint, 2023.
Nathaniel Chodosh, Anish Madan, Deva Ramanan, and Simon Lucey. Simultancous Map and Object Reconstruction, 2024.
wwwp:vi.ooo///aaoo aa////0bb/22..... 13896
Ayush Dewan, Tim Caselitz, Gian Diego Tipaldi, and Wolfram Burgard. Rigid scene flow for 3d lidar scans. In Int. Conf. Intel. Rob. Sys., pp. 1765-1770. IEEE, 2016.
Emeç Erçelik, Ekim Yurtsever, Mingyu Liu, Zhijie Yang. Hanzhen Zhang, Pinar Topçam, Maximilian Listl, Yılmaz Kaan Çayh, and Alois Knoll. 3D Object Detection with Self-supervised Lidar Scene Flow Backbone. In Shai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner (eds.), Computer Vision ECCV 2022, pp. 247-2657 Cham, 2022. Springer Nature Switzerland.
Thomas Grandits, Alexander Effland, Thomas Pock, Rolf Krause, Gernot Plank, and Simone Pezzuto. GEASI: Geodesic-based earliest activation sites identification in cardiac models. International Journal for Numerical Methods in Biomedical Engineering, 37(8):33505, 2021. ISSN 2040- 7947. doi: 10. 10.1002/cmm.3500 3505. 10.1002/ cnm. 3505
wiley. com/ doi/abss
Xiuye Gu, Yijie Wang, Chongruo Wu, Yong Jae Lee, and Panqu Wang. Hplftownet: Hierarchical permutohedral lattice flownet for scene flow estimation large-scale point clouds. In IEEE Conf. Comput. Vis. Pattern Recog., pp. 3254-3263, 2019.
Ramin Hasani, Mathias Lechner, Alexander Amini, Daniela Rus, and Radu Grosu. Liquid Time- constant Networks. Proceedings of the AAAI Conference on Artificial Intelligence, 35:7657-7666.
May 2021.
Wenbo Hu, Xiangjun Gao, Xiaoyu Li, Sijie Zhao, Xiaodong Cun, Yong Zhang, Long Quan, and Ying Shan. DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videos. arXiv preprint ar wwxxx:2449.00099 2024.
Philipp Jund, Chris Sweeney, Nichola Abdo, Zhifeng Chen, and Jonathon Shlens. Scalable Scene Flow From Point Clouds the Real World. IEEE Robotics and Automation Letters, 12 2021.
Ishan Khatri, Kyle Vedder, Nechar Peri, Deva Ramanan, and James Hays. Can't Believe It's Not Scene Flow! In European Conference on Computer Vision (ECCV), 2024.

---

Published as a conference paper at ICLR 2025
Tarasha Khurana, Peiyun Hu, David Held, and Deva Ramanan, Point Cloud Forecasting Proxy for 4D Occupancy Forecasting. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
Jaeyeul Kim, Jungwan Woo, Ukcheol Shin, Jean Oh, and Sunghoon Im. Flow4D: : Leveraging 4D Voxel Network for LiDAR Scene Flow Estimation, 2024. URL
14C7.07999
Yair Kittenplon, Yonina Eldar, and Dan Raviv. Flowstep3d: Model unrolling for sulf-suuer scene flow estimation. In IEEE Conf. Comput. Vis. Pattern Recog., pp. 41 41144-122 4123, 2021.
Ruibo Li, Guosheng Lin, Tong He, Fayao Liu, and Chunhua Shen. HCRF-Flow: Scene flow from point clouds with continuous high-order CRFs and position-aware flow embedding. In IEEE Conf. Comput. Vis. Pattern Recog-, pp. 364-373, 2021a.
Xueqian Li, Jhony Kaesemodel Pontes, and Simon Lucey. Neural Scene Flow Prior. Advances in Neural Information Processing Systems, 34, 2021
Xueqian Li, Jianqiao Zheng. Francesco Ferroni. Jhony Kaesemodel Pontes, and Simon Lucey. Fast Neural Scene Flow. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp- 9878-9890, October 2023.
Yancong Lin and Holger Caesar. ICP-Flow: LiDAR Scene Flow Estimation with ICP. 2024.
Dongrui Liu, Daqi Liu, Xueqian Li, Sihao Lin, Hongwei xie, Bing Wang, Xiaojun Chang, and Lei Chu. Self-supervised multi-frame neural scene flow, 2024. URL https::/arx.v..rgg/ abs/
2403.
Xingyu Liu, Charles R Qi, and Leonidas J Guibas. Flow Learning Scene Flow in 3D Point Clouds. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
N. Mayer, Ulg, P. Häusser, Fischer, D. Cremers, Dosovitskiy, and Brox. Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. NeRF: representing scenes as neural radiance fields for view synthesis. Commun. ACM, (5(1):99-106, dec 2021. ISSN 0001-0782.
Mahyar Najibi, Jingwei Ji, Yin Zhou, Charles Qi, Xinchen Yan, Scott Ettinger, and Dragomir Anguelov. Motion Inspired Unsupervised Perception and Prediction in Autonomous Driving, European Conference on Computer Vision (ECCV), 2022.
Chrisopher Nalty, Neehar Peri, Joshua Gleason, Carlos Castillo, Shuowen Hu, Thirimachos Bourlai, and Rama Chellappa. A Brief Survey on Person Recognition at Distance. 12 2022. doi: (0.48550uaxxiv2222..9999.
Richard A. Newcombe, Dieter Fox, and Steven M. Seitz. DynamicFusion: Reconstruction and tracking of non-rigid scenes in real-time. In 2015 IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 343-352, 2015. doi: 10.1109/CVPR.22155.2988
Keunhong Park, Utkarsh Sinha, Jonathan T. Barron, Sofien Bouaziz, Dan B Goldman, Steven M. Seitz, and Ricardo Martin- BruailaBrualla. Nerfies: Deformable Neural Radiance Fields. ICCV, 2021.
Neehar Peri, Achal Dave, Deva Ramanan, and Shu Kong. Towards Long Tailed 3D Detection. CoRL, 2022a.
Nechar Peri, Jonathon Luiten, Mengtian Li, Aljosa Osep. Laura Taixe, and Deva Ramanan. Forecasting from LiDAR via Future Object Detection. 2rXiv:2203.1622 2022b.
Nechar Peri, Mengtian Li, Benjamin Wilson, Yu-Xiong Wang, James Hays, and Deva Ramanan. An empirical analysis of range for 3d object detection. arXiv preprint arXiv:2308. 04054, 2023.
12

---

Published as a conference paper at ICLR 2025
Gilles Puy, Alexandre Boulch, and Renaud Marlet. Flot: Scene flow point clouds guided by optimal transport. In Eur. Conf. Comput. Vis., pp. 527-544. Springer, 2020.
Sameera Ramasinghe, Hemanth Saratchandran, Violetta Shevchenko, Alexander Long, and Simon Lucey. On the Optimality of Activations in Implicit Neural Representations, 2024. URL https
//openreview httpi/ewweeeeeeeeeeeeeeeeeeeee.e fist/fooumms 1il.mmmmttdii0uuumao//oo
Nikhila Ravi, Jeremy Reizenstein, David Novotny, Taylor Gordon, Wan- Yen Lo, Justin Johnson, and Georgia Gkioxari. Accelerating 3D Deep Learning with PyTorch3D. arXiv:2007.08500 2020.
Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, Vijay Vasudevan, Wei Han, Jiquan Ngiam, Hang Zhao, Aleksei Timofeev, Scott Ettinger, Maxim Krivokon, Amy Gao, Aditya Joshi, Yu Zhang, Jonathon Shlens, Zhifeng Chen, and Dragomir Anguelov. Scalability in Perception for Autonomous Driving: Waymo Open Dataset. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
Ivan Tishchenko. Sandro Lombardi. Martin R Oswald, and Marc Pollefeys. Self-supervised learning of non-rigid residual flow and ego-motion. In Int. Conf. 3D Vis., pp. 150-159. IEEE, 2020.
Kyle Vedder and Eric Eaton. Sparse PointPillars:
Maintaining and Exploiting Input Sparsity to Improve Runtime Embedded Systems. In Proceedings the International Conference Intelligent Robots and Systems (IROS), 2022.
Kyle Vedder, Neehar Peri, Nathaniel Chodosh, Ishan Khatri, Eric Eaton, Dinesh Jayaraman, Yang Liu, Deva Ramanan, and James Hays. ZeroFlow: Scalable Scene Flow via Distillation. In Twelfih International Conference on Learning Representations (ICLR), 2024.
Sharanya Venkatesh, Bibit Bianchini, Alp Aydinoglu, and Michael Posa. Sampling-Basee Model Predictive Control for Contact-Rich Manipulation. In IROS 2023 Workshop on Leveraging Models for Contact-Rica Manipulation, 2023.
Kavisha Vidanapathirana, Shin-Fang Chng, Xueqian Li, and Simon Lucey. Multi-body neural scene flow. In 2024 International Conference on 3D Vision (3DV). IEEE, 2024.
Chaoyang Wang, Xueqian Li, Jhony Kaesemodel Pontes, and Simon Lucey. Neural Prior for Trajectory Estimation. In CVPR, pp. 6522-6532, 2022a. doi: 1LL109/CVPR322882222..000
Jun Wang, Xiaolong Li, Alan Sullivan. Lynn Abbott, and Siheng Chen. PointMotionNett Point-Wise Motion Learning for Large-Secr Scale LiDAR Point Clouds Sequences. In 2022 JEEE/CVF Conference Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 4418-4427, 2022b.
Thomas Weng, Sujay Man Bajracharya, Yufei Wang, Khush Agrawal, and David Held. Fabricftownet: Bimanual cloth manipulation with a flow-based policy. In Conference on Robot Learning, pp. 192-202. PMLR, 2022.
Benjamin Wilson, William Qi, Tanmay Agarwal, John Lambert, Jagjeet Singh, Siddhesh Khandelwal, Bowen Pan, Ratnesh Kumar, Andrew Hartnett, Jhony Kaesemodel Pontes, Deva Ramanan, Peter Car,, and James Hays. Argoverse 2: Next Generation Datasets for Self-drivigg Perception and Forecasting. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks (NeurIPS Datasets and Benchmarks 202 1), 2021.
Wenxuan Wu, Zhi Yuan Wang, Zhuwen Li, Wei Liu, and Li Fuxin. Pointpwe-net: Cost volume on point clouds for (self-) supervised scene flow estimation. In Eur. Conf. Comput. Vis., pp. 88-107. Springer, 2020.
Guangyao Zhai, Xin Kong, Jinhao Cui, Yong Liu, and Zhen Yang. FlowMOT: 3D Multi-Object Tracking by Scene Flow Association. ArXiv, abs/2012.07541, 2020.
Qingwen Zhang, Yi Yang, Heng Fang, Ruoyu Geng, and Patric Jensfelt. DeFlow: Decoder of Scene Flow Network in Autonomous Driving. ICRA, 2024a.
Qingwen Zhang, Yi Yang, Peizheng Li, Olov Andersson, and Patric Jensfelt. Seflow: A self- supervised scene flow method in autonomous driving. arXiv preprint arXiv:2407.01700. 2024h.
13

---

Published as a conference paper at ICLR 2025
Benjin Zhu, Zhengkai Jiang, Xiangxin Zhou, Zeming Li, and Gang Yu. Class-balanced Grouping and Sampling for Point Cloud 3D Object Detection, arXiv preprint arXiv:1908.094333 2019.
14

---

Published as a conference paper at ICLR 2025
A ADDITIONAL RESULTS
A.1 HOW DOES THE CHOICE OF LEARNABLE FUNCTION CLASS AND DESIGN OF ENCODINGS IMPACT BULERFLOW?
EulerFlow at its core is optimization loop over simple, feedforward ReLU-based multi-layer perception inherited from Neural Scene Flow Prior (Li et al. 2021b). How does this choice of learnable function class impact the performance of EulerFlow? To better understand these design choices we examine the choice non-linearity and time feature encoding.
FloesFlow Gouselan
2.5192
Buier Floo Flow SuaC
BalerFiw Fourice Time
0.1676
EulerFlow
0.1088
0.0
0.5
1.0
1.5
2.0
25
mean Dynamic Normalzed EDE
Figure 15: Mean Dynamic Normalized EPE of EulerFlow on the Argoverse 2 val split for less-smooth configurations of its learnable function class. These results indicate that the smoothness of the ReLU non-linearity proposed by Li et al. transfers well to EulerFlow.
One of Li et a core theoretical contributions demonstrates that NSFP's ReLU MLP is good prior for scene flow because it represents smooth learnable function class, and scene flow is often locally smooth with respect to input position. However, unlike NSFP, FulerFlow is fitting flow over a full ODE; while it seems reasonable to assume that this ODE is typically also locally smooth, cases like adjacent cars moving rapidly in opposite directions may benefit from the ability to model higher frequency, less locally smooth functions. To test this hypothesis, normalized time with higher frequency sinusoidal time embeddings (mirroring Wang et al. proposed time embedding for NTP), as well as try other popular linearities like SinC (Ramasinghe et al.) 2024) and Gaussian (Chng et al. 2022) from the coordinate network literature. Figure 15 features negative results on these ablations across the board; Gaussians were unable converge due extremely high frequency representation triggering early stopping, while the use of SinC and higher frequency time embeddings both resulted in worse overall performance, indicating that [i et al. ss smooth function prior does indeed seem appropriate for EulerFlow's neural prior.
ablate EulerFlow by replacing its
A.2 EULERFLLOW WITH MONOCULAR DEPTH ESTIMATES
While FulerFlow only consumes point clouds, we can leverage RGB-basee video monocular depth estimators to fit scene flow. In Figure [6 we use DepthCrafter (Hu et al., (024) generate a point cloud from the raw RGB of the tabletop video from Figure [4 Row 4.
Figure Visualizations of EulerFlow's emergent point tracking behavior on monocular depth estimates from DepthCrafter (Hu et al. 2024). Interactive visualizations available at vedder Lo/eulerflow
A.3 How DOES EULERFLOO FAIL?
As we discuss in Section 6.I. EulerFlow does not understand projective geometry - its optimization losses Chamfer Distance which directly associates points, sometimes resulting in moving shadows
15

---

Published as a conference paper at ICLR 2025
Visualizations of one of the failure modes of EulerFlow where flow predicted on the edges of the moving "shadow" in the point cloud. Interactive visualizations available at vedder io/eulerflul
Figure 17:
on background objects. To demonstrate this, select particularly egregious example in Figure 17 featuring a frame from the jack being thrown across the table. Due to the moving shadow cast by the jack onto the table, EulerFlow incorrectly assigns flow the table surface nearby the jack, particularly on the leading edge, even though the table surface is stationary.
B FAQ
B. WHIT DATASETS DID YOU PRETRAIN ON?
FulerFlow is not pretrained on any datasets. a test-time optimization method (akin to NeRFs), and as we show with our tabletop data, this means it runs out-of-the-bon on arbitrary point cloud data.
B.2 WHY DIDN'T T YOU USE NEURAL ODE OR LIQUID NEURAL NETWORK?
Neural ODEs (Chen et al. 2018 take variable size and number steps latent space to do inference;
imagine ResNet that can use ODE solver to dynamically scale the impact of the residual block, as well as decide the number of residual blocks. They are not a function class specially designed to fit derivative estimates well. Similar to Neural ODEs, Liquid Neural Networks (Hasani et al. 2021 focus on the same class of problems and are similarly not applicable.
B.3 WHY DIDN'T T YOU DO EXPERIMENTS ON FLYINGTHINGSSD / <SIMULATED DATASET>?
Most popular synthetic datasets do not contain long observation sequences (Myyer et al. 2016 Butler et al. 2012), but instead include standalone frame pairs. Our method leverages the long sequence of observations to refine neural estimate of the true ODE. Indeed, collapses to NSFP.
frames, FulerFloww
More importantly, these datasets are also not representative of real world environments. To quote Chodosh et al.) "[FlyingThinnsS" has] unrealistic rates of dynamic motion, unrealistic correspon- dences, and unrealistic sampling patterns. As result, progress on these benchmarks is misleading and may cause researchers to focus on the wrong problems." Khatri et al. also make this point by highlighting the importance of meaningfully breaking down the object distribution during evaluation identify performance on rare ccitty- categories.
does not have meaningful semantics; it's ss not obvious what things even matter or how to appropriately break down the scene.
Instead, we want to turn our attention to the sort of workloads that do clearly matter describing motion domains like manipulation or autonomous vehicles, where it seems clear that scene flow, if solved, will serve powerful primitive for downstream systems. This is why we performed qualitative experiments on the tabletop data we collected ourselves; to our knowledge, no real- world dynamic datasets of this nature exist with ground truth annotations, but want to emphasize that EulerFlow works in such domains, and consequently EulerFlow and other Scene Flow via ODE-based methods be used primitive in these real world domains.
16

---

Published as a conference paper at ICLR 2025
C EULERFLOW IMPLEMENTATION DETAILS
Our neural prior 0 is straightforward extension NSFP's coordinate network prio however, instead of taking a 3D space vector (positions X,Y, Z E R) as imput, we encode a 5D space-time- direction vector: positions X, Y, Z; R, sequence normalized time L [-1, 1] (i.e. the point cloud time scaled to this range), and direction d E {BWD = 1, FWD = 1). This simple encoding scheme enables description of arbitrary regions of the ODE, allowing for the ODE to queried at frequencies different from the sensor frame rate. Euler integration enables simple implementation of multi-step forward, backward, and cyclic consistency losses without extra bells and whistles. For efficiency, we use Euler integration with At set as the time between observations for our ODE solver, enabling support for arbitrary sensor frame rates, and set the cycle consistency balancing term a 0.01 and optimization window W 3 for all experiments.
EulerFlow's definition TruncatedChamfer is symmetrid i.e. TruncatedChamfert (A B) TruncatedChamfer( B, A). If this symmetric TruncatedChamter is naively implemented via perfor- mant differentiable CUDA accelerated K=INN computation from A B and from B to A, on an NVIDIA V100, EulerFlow spends roughly 80% of its compute time performing these KNN checks. To accelerate this, we precompute exact GPU accelerated KD-Trees (Grandits et al. 2021 for the input point clouds .P..... Pv] ), and when possible query those trees instead of computing K=INN. In practice, we found these queries are almost instant, and reduce the time spent computing K=INNs about 40% of the total wall-clock time of BulerFlow.
Additionally, while Equation is phrased as independent Euler integration steps for each timestep, we are able to share integration across the losses; we perform two integrations from t W and to t - W, and use the intermediary locations along this trajectory as inputs to intermediary losses.
C.I FORMULATING THE ODE
Given a (possibly moving) particle some canonical frame (i.e. time 0), we define function L(Zo, you 20, that can describe its location at arbitrary future time t, i.. a Lagrangian description of motion (Figure 4).
L L(xo yo: 20, t) = Fe. M/.
(4)
For notational clarity access Fe, Mr, zt individually, can define
La (300 yoo 20, 34 Ty (3o= Mo, 20, 1) =
(5)
(6)
T.g (30: yo, 20, Zt.
(7)
Similarly, we can define F(z,, Vt, 22, t) to describe the instantaneous velocity of a point 24, yt, 21 at some arbitrary time t, i.c. Eulerian description of motion (Figure 4).
dL(ro, yo. 20. t) dL
(dLe dly dL.) dt -(ate dt
= F yc. 22,t)
(8)
dt
dt
F's is defined in terms of the total derivative of L with respect to I, as Fop No: Fy are initial conditions that do not vary with time (i.e. dL We can exactly define recursively terms of the initial conditions and i.e.
AL
ØL daa + aL dgn aL dan aL dea + azo de Pyo dt 8eo di St dt
+
dun dzn
0).
de St
dt
dt
"Hyperparameters (e.g. filter width of 128) of NSFP's prior are kept fixed, except for depth (Section 5.2.3
"Tii in keeping with NSFP, but in opposition other methods like FastNSF [ et al. 2023). We found that a symmetric definition provided non-trivial performance improvements to FulerFlow.
We used Torch3D (Ravi al. 2020), which has custom CUDA operations with CUDA templated support for single neighbor differentiable KNN.
17

---

Published as a conference paper at ICLR 2025
T. Vo: 20, 1) = (Io. No: 20) S/ F(I,z(T00 vo: 20, 20, r), (ay yo: 20, 20, L (T0: yo. 20, r), r)
(9)
or, more compactly,
(2 yo: 20, (No, yo, 20) + ( F( Rr; T)d
(10)
Our function L can thus be defined as a multi-dimennional ODE in terms of F with initial conditions To, yl: 20-
C.2 ARBITRARY START AND END TIMES FROM THE EULERIAN FORMULATION
In the above derivation, I. requires that moving point be defined in terms of canonical frame defined at time 0, as common in the deformation in reconstruction literature. However, the Eulerian formulation has such requirement, allowing us to select arbitrary start and end times across different point queries. To showcase this, we can query to extract the trajectory of a particle at across the range it. f'] starting at Ft. Vts simply by changing the range of the integral in Equation 10 i.e.
[. F(Dr, Zr. T)dr
El yt, t,,t, t,) (2 Vr,
2,) 21,
(11)
While L appear similar on their face, E is strictly more flexible than L. principle you could choose to redefine L to use as the time for your canonical frame, but this is global choice; you cannot do this on a per-query basis. However, with E's Eulerian framing, we can extract a different point's trajectory from the entirely different range et to 1 (i.e. Fl Upt, Zet 1t, ()) without concern for canonical frame definition. It need not even be the case that < t'; indeed, this extraction works even if t', i.. extracting the backwards trajectory through time.
C.3 EULER INTEGRATION TO APPROXIMATELY SOLVE THE ODE
If F is of arbitrary form and we want to compute the concrete values of L,, we cannot exactly compute we must approximate this with finite differences. Thus, we split
the continuous integral from the time range 0 to t into & steps, where each step is of size Thus, can again define L via recursion, but this time explicitly.
L[ yo, 20, 0) = =(xo, yo, =0)
(12)
t x
t
[x yo: 20,
L(xo, you =0, 7), Zr,T);
(13)
or directly without recursion,
k
L(xo, yo. 20. Ro (Fo, you 20) +
Σ Ba 2nt
(14)
k
This finite difference solving approach is Fuler integration.
C.4 ESTIMATING THE FLOW FIELD WITH EULERFLOW'S NEURAL PRIOR
For given scene, do not have access to L or F directly; these uniquely characterize the underlying motion of the scene that we are trying to estimate. For EulerFlow, represent estimate of the scene's 's flow field F with neural prior, 0, i.e.
the true functions that
18

---

Published as a conference paper at ICLR 2025
(ec,y.2, Fo 0(2,8,2,1)
(15)
and thus
(zo, you t0 20 (Io, yo, 20) +
([ Un 2 mTk
(16)
and, using the arbitrary start and end definition from Appendix C.2 t' and d f'-t
with it steps from the range to
Bla,, Ubs (), 2o Eo Ue. a,, t,) (Ie, M/t 2x) + 8. 12 2n84, no t) (17)
This formulation makes EulerFlow highly flexible, enabling optimization of O's estimate of F with objectives that take either an Fulerian view (directly on via Equation 15) Lagrangian view (on
point rollouts for arbitrary start and end ranges via Equation 17.
19


Reviews:
[{"review_id": "mB2wSa1VZh", "reviewer": "Reviewer_V3mD", "summary": "This paper proposes to represent scene flow as a velocity field using a neural prior. Instead of prior art that directly represents per-pair scene flow as neural prior, the authors alternatively propose to use neural prior to model the partial differential equation (PDE) of the position of the point versus the time interval. This novel velocity representation is interesting and could offer flexibility in dealing with long-term sequences of flow estimations as the authors described in the paper. The authors have also done extensive analysis of the proposed method on Argoverse 2 (and Waymo) datasets, comparing the performance with recent scene flow works, and validating the good performance of the proposed method.", "questions": "- The discussion of the different activation functions (appendix) is indeed interesting. And this could be one of the interesting parts of the ablation study. However, it is strange to see that using the Gaussian non-linear function is yielding very bad performance. Perhaps the spectral width needs to be fine-tuned, especially when the distribution of the lidar scene flow is very unique.\n\nPlease also see the above section for detailed comments.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The paper proposes an interesting idea to represent the scene flow as a velocity field using a neural network, making it very easy to combine the temporal information (time) and the spatial information (position of points).\n- The authors have done extensive analysis of the proposed method, and have shown different ablation studies to validate the effectiveness of the method.\n- The proposed method also shows the potential to deal with small objects and emergent flows in robotics scenarios, which could be interesting when applied to highly dynamic environments.\n- Overall, the writing of the paper is clear, and the visualization is easy to understand.", "weaknesses": "- When using the time interval between [-1, 1] for the time encoding, will the proposed method not be able to handle time step outside the range? Given that the representation is a continuous neural network, how does it extrapolate to a longer sequence with the current representation?\n- When comparing with a method like NSFP, I wondered if the authors could show the results of pure Euler integration of the method and highlight the benefits of wrapping a PDE with a neural network.\n- The authors mentioned that they only do sequences of length 20, I wondered if the method failed rapidly with the increase of the sequence length. It would be interesting to show an even longer sequence to highlight the arbitrary time query property of the proposed method.\n- I feel like the authors want to talk about too many things in this paper, so they may overlook the most important part of the method. This method is good at dealing with long-term flow trajectory and has the potential to better capture the small, highly dynamic objects in the scene. The authors could reorganize the motivations and experiments to highlight the advantages of the proposed method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MqmPBX1iiA", "reviewer": "Reviewer_pt2w", "summary": "The paper introduces a novel approach to scene flow estimation by reframing it as the task of estimating a continuous space-time partial differential equation (PDE) that describes motion across an entire observation sequence. The proposed method, called EulerFlow, utilizes a neural prior to represent this PDE and optimizes it against multi-observation reconstruction objectives. This approach allows for high-quality, unsupervised scene flow estimation on real-world data, outperforming existing supervised and unsupervised methods, particularly on small and fast-moving objects like birds and tennis balls. The authors demonstrate the effectiveness of EulerFlow on datasets such as Argoverse 2 and Waymo Open Dataset, and show its applicability across different domains without domain-specific tuning. Additionally, they highlight emergent 3D point tracking behavior as a result of solving the estimated PDE over long time horizons.", "questions": "Overall I believe this paper is in a good shape, the authors discuss the properties and limitations of the proposed method thoroughly in the paper. I have a few more questions:\n\n- How does the method handle scenes with deformable objects?\n- What is the impact of temporal sampling rate on performance?\n- How does the point cloud density affect the performance?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- I'm not up-to-date to the latest scene flow models, but from the results in the paper it surpass the prior art by a large margin, which is very significant\n- Introducing the concept of modeling scene flow as a PDE is innovative and offers a new direction for research in motion estimation.\n- The method is rigorously developed, with comprehensive experiments and ablation studies that validate the approach.\n- The paper is well-written, with clear explanations and effective use of figures to illustrate key points.", "weaknesses": "- As stated in the paper, the speed of the proposed method is a big concern, preventing it from deploying on real world application.\n- Some hyperparameters, such as the depth of the neural prior, seem to require dataset-specific tuning (e.g., increasing depth to 18 for the Argoverse 2 challenge), which may affect the method's out-of-the-box applicability.\n- It would be great if the author could show more failure cases to help readers better understand its limitations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ljQrwUr0Gs", "reviewer": "Reviewer_DftH", "summary": "The paper proposes a neural representation to optimize scene flow as a discrete partial differential equation of scene geometry over time. Compared to previous method, Neural Scene Flow Prior (NSFP), a method is most related to this work in the neural representation, the proposed method introduces a multi-frame formulation and learns bi-directional three-step Euler integration of the geometry consistency using decoded per-frame scene flow. Compared to previous work, the proposed representation can achieve better performance in Autonomous driving datasets and the authors demonstrate qualitative performance on depth camera input as well.", "questions": "Among all the three points I illustrated in the weakness, \n\nFor point 1, I hope the authors can provide concise update on their paper title and contributions in particular for the first bullet time (line 99-100).\n\nFor point 2, the current evaluation is sound and maybe sufficient for this paper. I do believe it is nice to more evaluation on non-AV dataset quantitatively that very likely will benefit this method as a baseline for future work in different domain. \n\nFor point 3, it will be good if the author can provide specific example (as a figure)", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The proposed scene flow representation is simple and technical sound. Compared to prior work NSFP, extending it to multi-frame and learns a bi-directional consistency scene flow is a very intuitive step forward. \n2. The performance of this method (both qualitative and quantitative) is impressive. The method can learn very consistent scene flow in trajectory despite not explicitly considering common issues such as occlusion artifacts. As the paper demonstrated, it can tackle well on small objects (with potentially large motions as well).", "weaknesses": "1. The paper title and introduction is very general and does not provide a precise position of this paper's main contribution. \"Scene flow a partial differential equation\" has been historically formulated long time ago in many prior paper, e.g. [1] as one examplar reference, and it has been proposed as a continuous representation in one early seminal work [2]. Many related work studied this optimization problem using images input and solved it using differential optimization approaches before. In this paper seems only consider related work in the point cloud space, and beneficially solved in using a neural representation. I will suggests to more precise position their scope and contributions  in paper title, introduction and contributions. \n\n2. The evaluation dataset in this paper is mostly on autonomous driving datasets though as the method demonstrated, it should also work on other data domain when depth is available. Though real world depth and flow ground  truth is hard to get, it won't be too hard if evaluated using a more general synthetic dataset that provide different motion patterns, compared to the type of motion and accuracy that autonomous driving dataset can provide. \n\n3. The paper has already discussed the main limitations it section 6.1. Particularly for the last point \"EulerFlow does not understand ray casting geometry\", it was clear how this has been demonstrated in the current results. It will be good if the authors can provide examples and metrics that reflect the challenge in this aspect.\n\n[1] A Variational Method for Scene Flow Estimation from Stereo Sequences, Frederic Huguet and Frederic Devernay, ICCV 2007\n\n[2] Three dimensional scene flow, Vedula et al, CVPR 1999", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "D1P6a18mgV", "reviewer": "Reviewer_XQYd", "summary": "The paper proposes SFvPDE, a framework to cast scene flow estimation as a PDE with a neural prior, and EulerFlow, an example demonstrating how SFvPDE can be trained using the Euler method to locally integrate the PDE during training. A space-time-dependent vector-field is trained to match subsequent point clouds at different timestamps via solving the underlying PDE. The method significantly outperformes both supervised and unsupervised baselines and is especially effective on small objects compared to prior work.", "questions": "Here are some questions and concerns regarding the presentation and the method:\n\n1) In lines 189 and 195 $\\frac{\\partial L^*}{\\partial t}$ is referred to as the partial differential equation, or a PDE. However, $\\frac{\\partial L^*}{\\partial t}$ alone is not a PDE yet, unless it is set equal to something (as in equation 2).\n2) I guess that in equation 2 SFvPDE should also depend on $x$. Could the authors clarify this?\n3) In general, it would be nice to have more formal definitions. E.g. in EulerFlow, an exact formula for solving the PDE, $\\text{Euler}_\\theta(P_t, d, k)$, would improve understanding and reproducibility of the method.\n4) In principle, the PDE can be integrated in both directions by simply reverting the time. The usage of the direction as an extra argument in the model makes the connection between sections 3 and 4 slightly weaker and seems to be a legacy design choice from NSFP. Thus, a question to the authors is whether they have tried training without the direction argument?\n5) Given the high computational complexity of the method, it would be better to see some implementation details on how exactly equation 3 is calculated during training. Are any optimizations already incorporated? E.g. in the current form separate terms in the loss are independent. However, I believe that subsequent Euler steps can use previous estimates instead of recalculating them.\n6) More ablation studies would better highlight the contributions of the paper. E.g. how general and how sensitive is the method to different numerical solvers and sizes of discretization steps? Have the authors tried higher-order PDE solvers or using $\\Delta t$ smaller than the time between observations?\n\nI will adjust my score based on the other reviews and the rebuttal by the authors.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The method is simple and intuitive yet effective. Extensive experiments section shows clearly that the proposed method surpasses prior work.", "weaknesses": "The main weakness of EulerFlow, as also noted by the authors (lines 524-528), is the time it takes to converge on a single scene. But given the performance of the method, this should not be considered critical. However, the presentation of the paper can be improved, the paper lacks some implementation details, as from time to time the reader has to guess what is actually happening (see questions section).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)