Flux4D: Flow-based Unsupervised 4D Reconstruction
Authors: Jingkang Wang
Henry Che
Yun Chen
Ze Yang
Lily Goli
Sivabalan Manivasagam
Raquel Urtasun
Venue: NeurIPS 2025 poster ()
Keywords: unsupervised 4D reconstruction
large reconstruction model
self-driving
Primary area: deep_learning
TL;DR: Flux4D is a simple, scalable framework for unsupervised 4D reconstruction of large-scale driving scenes.

Abstract: Reconstructing large-scale dynamic scenes from visual observations is a fundamental challenge in computer vision, with critical implications for robotics and autonomous systems. While recent differentiable rendering methods such as Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have achieved impressive photorealistic reconstruction, they suffer from scalability limitations and require annotations to decouple actor motion. Existing self-supervised methods attempt to eliminate explicit annotations by leveraging motion cues and geometric priors, yet they remain constrained by per-scene optimization and sensitivity to hyperparameter tuning. In this paper, we introduce Flux4D, a simple and scalable framework for 4D reconstruction of large-scale dynamic scenes. Flux4D directly predicts 3D Gaussians and their motion dynamics to reconstruct sensor observations in a fully unsupervised manner. By adopting only photometric losses and enforcing an "as static as possible" regularization, Flux4D learns to decompose dynamic elements directly from raw data without requiring pre-trained supervised models or foundational priors simply by training across many scenes. Our approach enables efficient reconstruction of dynamic scenes within seconds, scales effectively to large datasets, and generalizes well to unseen environments, including rare and unknown objects. Experiments on outdoor driving datasets show Flux4D significantly outperforms existing methods in scalability, generalization, and reconstruction quality.

Flux4D: Flow-based Unsupervised 4D Reconstruction
Jingkang Wang Henry .he Yun Chen
1.2+ Ze Yang
Lily Goli
Sivabalan Manivasagam
Raquel Urtasun
Waabi
University of Toronto²
UIUC
https://waabi.. http://wabbaaiiffffna ai/flux4d
Abstract
Reconstructing large-scale dynamic scenes from visual observations is a fundamen- tal challenge in computer vision. While recent differentiable rendering methods such as NeRF and 3DGS have achieved impressive photorealistic reconstruction, they suffer from scalability limitations and require annotations to decouple moving actors from the static scene, such as in autonomous driving scenarios. Existing self- supervised methods attempt eliminate explicit annotations by leveraging motion cues and geometric priors, yet they remain constrained by per-scene optimization and sensitivity hyperparameter tuning. In this paper, we introduce Flux4D, a simple and scalable framework for 4D reconstruction of large-scale dynamic driving scenes. Flux4D directly predicts 3D Gaussians and their motion dynamics to reconstruct sensor observations in fully unsupervised manner. By adopting only photometric losses and enforcing an "as static as possible" regularization, Flux4)) learns to decompose dynamic elements directly from raw data without requiring pre-trained supervised models foundational priors simply by training across many scenes. Our approach enables efficient reconstruction of dynamic scenes within seconds, scales effectively to large datasets, and generalizes well to unseen environments, including rare and unknown objects. Experiments on outdoor driving datasets show Flux4D significantly outperforms existing methods in scalability, generalization, and reconstruction quality.
Introduction
Reconstructing the 4D physical world from visual observations captured in the wild is key goal in computer vision, with applications in virtual reality and robotics, including autonomous driving. High-quality reconstructions provide the foundation for scalable simulation environments that enable safer and more efficient autonomy development. Unlike artist-created environments, environments built automatically with data collected by sensor-equipped vehicles are more realistic, are more cost-efficient, and capture the diversity of the real world.
Advances in differentiable rendering approaches such Neural Radiance Field (NeRF) [26] and 3D Gaussian Splatting (3DGS) [17] have enabled high-quality reconstruction of dynamic scenes [53, 50, 62, 39, 18]. These methods decompose scenes into a static background and set of dynamic actors using human annotations such as 3D tracklets or dynamic masks, and then perform rendering on the composed representation, optimizing reconstruct the input observations. While they achieve impressive visual fidelity, their reliance on manual annotations to decompose static and dynamic elements increases costs and time, preventing these methods from scaling to large sets of unlabelled data. Some approaches leverage pre-trained perception models to generate annotations automatically, this can cause artifacts when the model predictions are noisy or incorrect, which can be difficult to recover from during reconstruction. Moreover, these methods typically require hours to reconstruct
"Equal contributions.
Work done while research intern Waabi.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

each scene on consumer GPUs. These two main issues, expensive annotation costs and slow per-seene optimization, limit the scalability of these methods.
Recent works have explored self-super approaches to eliminate the reliance on human an- notations and learn the decomposition of static and dynamic actors directly from data. This is challenging task due to the ambiguity of actor motion over time, coupled with spatial geometry and appearance variations. One strategy attempts improve the decomposition by incorporating additional regularization terms such as geometric constraints [31] or cycle consistency [52], or performing multi-stage training [16]. Another strategy is to leverage foundation models for additional semantic features or priors [31, 7, 52]. However, the resulting complex models can be sensitive to hyperparameters. slow to train, and unable to generalize new scenes. Moreover, they often have poor decomposition results, and struggle to render novel views, limiting their usability.
As an alternative to costly per-scene optimization, generalizable approaches [3, 42, 5, 13, 44, 59] use feed-forward neural networks to predict scene representations directly from observations, enabling efficient reconstruction within seconds. However, these approaches are designed for small-scale environments, can only process a few low-resolution images (lypically 1-4 views with resolutions below 512px), and primarily focus on static scenes [2, 5] or only dynamic objects [33]. When handling large scenes with many dynamic elements, they rely on costly annotations [6, 34j, limiting their scalability. Most recently. DrivingRecon [25] and STORM [51] propose feed-forward, self- supervised approaches for driving scenes. While promising. these methods focus on the sparse reconstruction setting and can only handle small number (5 12) of low-resolution (< 360px) imput views before reaching compute limits, and still depend on pre-trained vision models for semantic guidance, constraining their fidelity, scalability and applicability to downstream simulation.
In this paper, we propose Flux4D, an unsupervised and generalizable reconstruction approach that enables accurate and efficient 4D driving scene reconstruction at scale. Without any annotations, Flux4D predicts 3D Gaussians along with motion parameters directly in 3D space from multi-sensor observations within seconds, enabling efficient scene reconstruction. Our reconstruction paradigm is illustrated in Fig. 1. Flux4D uses a remarkably minimalist design that employs only photometric losses and a simple sutti--reeeeeecc prior, without requiring complex regularization schemes or external supervision to learn the motion that prior works leverage. We find that the key ingredient for Flux4D to accurately recover geometry, appearance, and motion flow comes from learning across diverse range of scenes. Moreover, Flux4D's use of LiDAR data, commonly available in the autonomous driving domain, enable handling of large number (2 60) of high-resolution (1080px) input multi-view images, achieving high-fidelity reconstruction and scalable simulation. Our 3D design yields a compact and geometrically consistent representation across views, improving efficiency, enabling explicit multi-view flow reasoning and reducing appearance-motion ambiguity.
Experiments on outdoor driving datasets PandaSet [48] and WOD [36] demonstrate that Flux4D achieves better scene decomposition and novel view synthesis than previous state-of-the-ars annotation-free reconstruction methods, and is competitive with per-scene optimization methods that use human annotations. We also show that Flux41) can be trained to predict sensor observations in future frames, akin to next-token prediction, but applied to dynamic 3D scenes. Finally, we showcase using Flux4D's ' reconstruction for controllable camera simulation via scene editing and novel view rendering at high resolution (2 1080px). Flux4D highlights the power of unsupervised learning for 4D scene reconstruction, enabling efficient scaling to vast unlabeled datasets.
2 Related Work
Optimization-based 4D reconstruction: Inspired by differentiable rendering [26, 17], recent approaches use deformation fields [32, 30, 56, 46] to model dynamic scenes but still struggle with real-world complexity due to overparameterization and poor static-dynamic decomposition. While some methods address this by using human annotations (3D tracklets, semantic models) to explicitly separate statie and dynamic elements [29, 54, 40, 50, 9, 12], they remain limited by annotation quality and availability. Self-supervised alternatives using motion cues and physics-informer priors [47, 16, 31] reduce dependence on annotations but typically require complex regularization schemes and expensive per-scene optimization. In contrast, our approach reconstructs dynamic 4D scenes without explicit supervision or per-scene optimization, achieving scalable reconstruction through simple photometric losses with minimal regularization.
Generalizable reconstruction: Generalizable methods infer scene representations directly from observations without per-seene optimization [3, 42, 5, 13, 44, 59]. leveraging large training datasets

---

reconstruction with labels
FSNR
w/o kabel
w/ lobel
24-
NeuRAD
Observationa
Per-Scene Opt.
Render
GSR
3D Digitl Twin Lobel
Reconstructin
StreotGs
Ours: unsupervised generalizable 4D scene reconstruction
DesRRoo0S
STORM
Flux4D Netwark
Render
DeothSplot
Rocon Time
4D Digital Twinn
Reconstruction
01x
100s
1000
10000s
Figure 1:
Flux4D is simple and scalable framework for unsupervised 4D reconstruction. Left: Paradigms for 4D reconstruction. Right: realism- speed comparisons with existing works.
to improve reconstruction quality in novel environments. However, existing approaches primarily target static scenes, struggling with dynamic environments due computational constraints and dependence on sparse, low-resolution inputs. Recent advances attempt to overcome these limitations using efficient architectures [63] or iterative refinement [6]. but still rely on 3D annotations. In contrast, Flx4D generalizes to unseen dynamic scenes by predicting 3D Gaussians with their motion directly from observations without external supervision.
Unsupervised world models: Our work relates recent advances in unsupervised world models, which learn predictive representations of environments without explicit supervision. These approaches typically tokenize visual data into discrete or continuous representations [14, 13, 61, 27] processed by autoregressive or diffusion- models to predict future states. While demonstrating impressive visual quality, such methods generally lack interpretable 3D structure, limiting precise control over generated content. Existing solutions often produce lower-resolution outputs with reduced temporal consistency, are typically restricted to single modalities (e.g., camera [14, 11, 22] or LiDAR [60, 55, I),, and require substantial computational resources. While our primary focus is reconstruction, FluxD'ss ability to simultaneously model motion dynamics and predict future frames shares conceptual similarities with world models. Unlike these approaches, Flux4D uses explicit 3D representation, providing 3D interpretability, controllability and spatiotemppral consistency.
Unsupervised generalizable reconstruction: Most recently, DrivingRecon [25] and STORM [51] explore unsupervised generalizable 4D reconstruction for driving scenes, using feed-forward networks to predict the velocities of 3D Gaussians. Despite impressive performance, they can process only sparse (3-4), low-resolution (< 256 512) frames with substantial computational requirements and rely on pre-trained vision models (Deepl Lab33+ [4], SAM [20], ViT-Adapter [8]) for additional supervision, limiting their scalability and applicability. Flux4D achieves better performance with simpler and more scalable approach, and through our novel incorporation of LiDAR to initialize the scene, can handle full HD images with denser views (> 60) while being computationally efficient. Please see supp. for more discussions.
3 Scalable 4D Reconstruction with Flux4D
Given a sequence of camera and LiDAR data captured by robot sensor platform, we aim to reconstruct the underlying 4D scene representation that disentangles static and dynamic entities and supports high-quality rendering at novel viewpoints. Such representation can enable future prediction and counterfactual simulation. To achieve scalable 4D scene reconstruction, our method should unsupervised, meaning uses no annotations, and fast, running in seconds. Towards this goal, propose Flux4D, unsupervised and generalizable approach that learns reconstruct 4D scenes via three simple steps (Fig. 2). We first lift the sensor observations at each timestep to set of initial 3D Gaussians. We then feed the initial representation to network to predict 3D flow and refined attributes for each 3D Gaussian. Finally, we supervise the network solely through reconstruction and static-prefereet losses.
3.1 Scene Representation
Our approach takes a set of posed camera images I (sR) and LiDAR point clouds P (P*)sskSK captured over time by moving platform and outputs scene representation
3

---

Figure Model overview. FluxD reconstructs 4D world by predicting 3D Gaussians with velocities given unlabelled sensor observations, and trained with the photometric reconstruction objective. The resultant model can be used for RGB and flow synthesis from novel views.
with geometry, appearance, and 3D flow. We represent the scene using set of 3D Gaussians Highsiss Each Gaussian point gi parameterized by its center position p; (RR), scale (R orientation (R²"), color (R3) and opacity (IR.) [17]. Additionally, we augment each Gaussian with learnable instantaneous velocity v, E R and fixed capture time I, We denote the sets of velocities and timestamps for all Gaussians as V = {vi]isiMM and 'T = {f4)isisM.
Initialization: We initialize Gaussian positions from LiDAR points P1: from each source frame
in the sequence, set scales based on the average distance to nearby points, and assign colors by projecting these points onto the corresponding camera image [g- Each Gaussian's timestamp 4 assigned the capture time of its source LiDAR frame, and velocities are initialized to zero. We aggregate source frame Gaussians to create Ginit-
3.2 Predicting Flow and Rendering
Inspired by recent advances in 4D reconstruction [47, 52, 31, 58, 25, 51], we propose to learn time-dependent velocity field to model the dynamics of driving scenes. Given the initial velocity- augmented Gaussians Ginit, we leverage neural reconstruction function So that outputs the refined Gaussian parameters G and the predicted velocities
G (9mi. T).
(1)
With the predicted velocities V, each Gaussian can be propagated from its initial timestep any target timestep t' using a linear motion model:
P p +V. (t' t;),
where D is the Gaussian position at time t', V; and t, are its velocity and capture time. This formulation enables continuous, temporally consistent reconstruction under a constant velocity assumption. We find this simple motion model can already achieve reasonable performance when reconstructing outdoor driving scenes with short time horizons (~ 1s), an observation aligned with existing works [31, 25, 21, 51]. Moreover, we investigate higher-order polynomial motion models, as discussed in Sec. 3.4 and Table
3.3 Unsupervised Learning of Dynamies
We now describe how the method learns to disentangle the scene dynamics. The network fo is trained in a fully self-supervised manner, without requiring explicit 3D annotations. Given the predicted Gaussians G, we move the Gaussians to target time t' using Eqn. (2), render the scene using differentiable rasterization [17] to generate color and depth images, and compare them against the real sensor observations I and P. To prevent unnecessary motion and encourage stability, we introduce an "as static as possible" regularization. The total loss defined as:
Lreson Cvel-
where Lecon represents the reconstruction loss, consisting of L, and structural similarity losses w.r. the images, and an L1 depth loss in the image plane compared to the projected LiDAR, and Lvel serves velocity regularization term that minimizes motion magnitudes:
Preen >xgh Logh + ASSIM LSSIM + Adephh
(4)

---

Table 1: Comparison SoTA unsupervised methods novel view synthesis. We evaluate photorealism, geometry, and speed metrics against per-scene optimization methods and generalizable methods. denotes the need for pre-trained vision models. Flux4D surpasses unsupervised and achieves competitive performance with supervised methods (top block), without requiring 3D labels.
Dynamic-only Duams
Full Image
Recon speed
Methods
Unaup.
Gen.
PSNRT
SSIMT
Visas
PSNR
SSIM
Dusaa
Visas
Timel
Recon, with labels (reference)
NeaRAD (39)
X
23.01
0.734
1.98
24.61
0.685
2.30
600mi
StroctSS [50]
20.06
0.605
1,02
23.38
0.680
0.84
222min
21.85
0.670
2.33
Unsupervised recon.
EmeeNeRFF [52]
0.411
DeSiRe-GS" [3]]
19.08
0.477
3,36
0.297
22.25
0.608
24.89
0.322
~ ~-120min
DopthSplat"
L4GM [35)
17.36
0.343
19. 38
0.465
0.32s
STORM [51]
17.65
0.367
5.24
0.203
20.79
0.508
4.80
0.238
0,07
(Ours)
21.99
0.662
1.63
0.157
23.84
Lid
1 M
Σ l/v//22-
(5)
We train fo across diverse set of scenes. Notably, we find that training across many scenes enables the network to automatically decompose static and dynamic components urban scenes without requiring the complex regularizations used in prior per-scene optimization techniques [47, 52, 7. 16, 31j. This highlights the effectiveness of data-driven priors as a powerful form of implicit regularization and the scalability of this simple framework.
3.4 Improving Realism and Flow
The aforementioned components form the core of our approach, termed Flux4D-base. Flux4D-base can already disentangle motion and render novel views with high quality. We further improve Flux4D- base through two enhancements that further recover more fine-grained appearance and refined flow, resulting in our final model, Flux4D)
Iterative refinement: Flux4D-base recovers the overall scene appearance, but often lacks fine- grained details. We hypothesize that this limitation stems from the constrained capacity of single- step feedforward network, and imperfect initialization due to occlusions. To mitigate this, we introduce iterative refinement mechanism inspired by G3R [6], leveraging 3D gradients as feedback to enhance reconstruction quality. Specifically, after each forward pass and generation of rendered color and depth at the supervision views, compute the 3D gradients of the Gaussians according to the loss function Eqn. (3), and provide the generated Gaussians and gradients as input to a network fø to further refine them. This process progressively corrects color inconsistencies and sharpens details within as few as two iterations. By incorporating iterative feedback, our method achieves figher-fideliy reconstruction, particularly in regions with complex appearance variations, while preserving the efficiency and scalability of Flux4D-base.
Motion enhancement: Flux41--base recovers the overall scene flow accurately (Table 7). We fur- ther introduce polynomial motion parameterizations to better model actor behaviors like acceleration, braking turning. Please see supp. for more details and comparisons. Exploring more advanced velocity models [21] or implicit flow representations is an exciting direction for future work. To further improve the flow and appearance quality of dynamic actors, we modify the loss function to focus on dynamic regions. Specifically, we render the flow in the image plane and apply pixel-wise re- eeweeighting to the photometrie loss. This gives higher importance to faster-moving regions during training, which typically occupy fewer pixels and would contribute less to the overall loss.
4 Experiments
We evaluate FluxDD against the current state-of-the-art (SoTA) self-supervised scene reconstruction methods, including both per-scene optimization and generalizable approaches. We also report the performance of supervised methods that do require annotations model dynamics as reference. We perform experiments multiple outdoor dynamic datasets and assess novel view appearance
5

---

CT
DepthSplat
L4GM
EmerNeef
BeBiRe-GS
Ours
Figure 3: Qualitative results for NVS on PandaSet. Rendered RGB images from novel views show that our method achieves better image quality across a variety of urban scenes, with crisper edges and sharper dynamic actors compared to baselines.
Reconstruction w/o label
GT
NeuRAD
GSR
EmerNeAf
DeSiRe GS
Our
Figure 4: NVS on longer-horizon logs. Qualitative comparison shows that our method outperforms SoTA unsupervised baselines, by maintaining better estimation of actor movements in longer horizon. We shrink the gap in quality to supervised methods.
and depth, as well as recovered flow. We also ablate Flux4D's design and show that Flux4D scales with more data. Finally, we demonstrate the controllability of our predicted scene representation for realistic camera simulation.
4.1 Experimental Details
Experiment setup: We conduct experiments on outdoor driving scenes from PandaSet [48] and Waymo Open Dataset (WOD) [36]. From PandaSet's 103 dynamic scenes (1080p cameras, 64-beam LiDARs, 10Hz), we select 10 diverse scenes for validation and use the rest for training. We use the front camera and 360 LiDAR, both collected at 10 Hz. To compare against existing feed-forward generalizable reconstruction methods that can only take small number of frames as input, we report scene reconstruction results on short 1.5s windows within the validation sequences. Each method takes as input frames 0, 2, 4, 6, 8, 10, and is evaluated on frames 1, 3, 5, 7, (interpolation) and 1 11-15 (future prediction). We sample evaluation snippets per log. We also evaluate against per-scene optimization methods over the full duration of the validation sequence (8 seconds) in the interpolation setting (every other frame is held out). For WOD evaluation, we follow the NVS setting in DrivingRecon [25], using the Waymo-NOTR subset with three front cameras, taking {t 2, f 1, + 1} frames input, and generating the interpolated frame at time f, where is every tenth frame each sequence. Finally, we evaluate scene flow estimation perpformance existing scene flow estimation methods cannot directly predict flows at novel timesteps, we evaluate scene flow on the input frames. We restrict evaluation to LiDAR points within the camera field of view (FoV) following [51].
snippet every 20 frames, yielding four non-overlapping
PandaSet and WOD (official validation set with 202 logs). As
Baselines: We compare against SoTA unsupervised scene reconstruction approaches: (1) Self- supervised per-scene optimization: EmerNeRF [52] and DeSiRe-GS [31], which reconstruct dynamic scenes using geometry priors, cycle consistency, and pre-trained vision models (FiT3D [57] and DI-

---

Image
GT
EmerNeRF
DeSiRe-GS
Ours
Figure 5: Estimating motion flows. We compare our estimated motion with prior unsupervised methods through rendered flow, showing accurate static region detection and sharper actor flow edges.
Table Full sequence reconstruction. FluxDD outperforms unsupervised methods for 8-second reconstructions dynamic regions and full image, closing the gap with supervised methods.
Dynamic-only DRMSE
Full image
Methods
PSNR
SSIM
VRMSE:
PSNR1
SSIM
DRMSE:
VRMSE
Recon, labels (reference)
NcuRAD [39]
22.99
0.719
1.71
24.99
0.679
2.29
StreetGS [50]
21.63
0.701
0.94
23.89
0.708
0.87
G3R [6]
20.60
0.573
2.16
23.15
0.636
2.01
Unsupervised recon.
EmerNeRFi [52]
18.65
0.437
4.48
0.478
23.42
0.627
3.09
0.975
DeSiRe-GSt [3]]
19.76
0.544
4.08
0.312
22.91
0.659
4.07
0.395
Flux4D (Ours)
21.94
1.57
0.162
23.72
0.670
1.10
0.186
NOv2 [28]); (2) Generalizable methods: L4GM" [33], 4D reconstruction model adapted driving scenes using depth supervision; DepthSplat".. an extension of [49] that unprojects LiDAR points using estimated depth for 3D Gaussian prediction; DrivingRecon [25], which builds 4D feed forward model utilizing learned priors from pre-trained vision models (SAM [20] and DeepLab-v3 [4]); and STORM [51] which predicts per-pixel Gaussians and their motion in forward manner. For reference, we also include SoTA methods that use ground-truth 3D tracklets: StreetGS [50] and NeuRAD [39] (compositional 3DGS/NeRF), well as G3R [6] (iterative refinement of composi- tional 3DGS). Apart from reconstruction methods, we also compare with representative scene flow estimation methods NSFP [23] and FastNSF [24] as a reference.
Metrics: We report standard metrics to measure the photorealism, geometric and motion accuracy using PSNR, SSIM, and depth RMSE (V RMMs)) and velocity RMSE (VrMse). Results are reported on both full images and dynamically moving regions for a comprehensive assessment. For scene- flow quality, report EPE3D, Ace; and Accin (fraction of points with error < 5/10 cm), angular emor in radians (0.), three-way way EPE [10]: background-stati (BS), foreground-static (FS), and foreground- dynamic (FD), bucketed normalized EPE [19]. and inference speed. On WOD, where semantic labels are coarse, we follow EulerFlow [41] and report bucketed normalized EPE for Background (incl. Signs), Vehicles, Pedestrians, and Cyclists only.
Flux4D implementation details: We adopt 3D U-Net with sparse convolutions [37] for fo. To handle unbounded scenes, we place random points on a spherical plane at far distance to model sky and far-away regions. We also add random points within 3D sphere following [50] increase model robustness. Our model processes full-resolution images (2 1920 x 1080) in all experiments and can be efficiently scaled to higher resolutions without significant overhead. Unless otherwise stated, all models are trained for 30,000 iterations on 4x NVIDIA L40S (48G) GPUs, taking approximately 2 days. The reconstruction loss weights Argh: ASSIM, Adepth are set 0.8, 0.2 and 0.01 respectively. The velocity regularization weight Avel set as 5c-3.

---

Table 4: Future prediction. We surpass
Table 3: NVS on WOD [36]. We achieve significant im- unsupervised and supervised methods.
provements over generalizable baselines.
Methods
PSNRT
SSIM
VRUSE
Methods
Full Imuge
Dynamic PSNR
Static
Recon. with labels
PSNR
SSIM
1.PIPS
SSIM
PSNR
NeuRAD [39]
21.52
0.557
3.03
LGM [38]
17.49
0.47
0.33
17.79
0.49
15.37
0.39
StreetGS
PixelSplat [2]
18.24
G3R (50)
21.13
0.570
2.09
MVSplat [S]
19.00
0.57
0.28
19:29
0.58
17.35
0.47
Unumeerised recon.
L4GM
17.63
0.54
0.31
18.58
0.56
16.78
0.43
EmerNeRF [52]
19.64
0.516
5.00
0.346
DrivingRecon [25]
20.63
20.97
19.70
DeSiRe-G [31]
18.86
0.513
26.07
0.325
Flux4D
26.62
0.82
0.18
26.86
0.83
26.09
0.80
STORM [S]]
0.251
Flux4D) (Ours)
21.81
0.598
1.42
0,193
Flux4D
ROB
Hux4D Flow
Label How
Figure 6: High-fidelity flow and RGB reconstruction. Flux4D not only provides photorealistic reconstruction of the dynamic scene but also estimates actors' motion flow with high precision.
4.2 Scalable 4D Reconstruction
Novel view synthesis on PandaSet: Table 1 and Fig. 3 compare Flux4D against SoTA unsupervised methods on is PandaSet snippets the interpolation setting, with supervised approaches included for reference. Reconstruction speed is measured achieves superior photorealism and geometric accuracy with fast reconstruction speed. We further Longer-horizon reconstruction of 8 second logs (Table 2 and Fig. 4), using
single RTX A5000 GPU (24GB). Flux4D
evaluate our method iterative processing of Is snippets. Our approach outperforms unsupervised per-scene optimization methods by large margin on both Is and 8s reconstruction tasks, without requiring pre- trained models or complex regularization. Our quantitative results as reported in these tables also indicate that Flux4D is competitive even against supervised approaches. Qualitatively, shown in Fig. and 4, Flux4D achieves high-fidelity camera rendering in both static and dynamic regions, while existing unsupervised approaches usually suffer from noticeable artifacts dynamic actors due inaccurate learned dynamics.
We further compare Flux4D with SoTA generalizable methods on WOD in Table 3, where we follow the setup in [25]. The baseline results are from DrivingRecon [25] paper and we confirmed the setup and results with the authors to ensure accurate comparison. Flux4D surpasses ReivingRecon by +5.99 dB in PSNR and +0.21 in SSIM, demonstrating its effectiveness for unsupervised dynamic scene reconstruction. Please see supp. for qualitative comparisons.
Novel view synthesis on WOD:
Flow estimation: We compare the estimated motion flows of Flux4D with existing unsupervised per-scene optimization methods EmerNeRF [52] and DeSiRe-GS [31]. As shown in Table 1, 2 and Fig. 5, Flux4D significantly outperforms prior approaches, learning accurate motion direction and magnitude without any supervision. In contrast, existing methods struggle to learn consistent motion flows and fully decompose dynamic scenes, leading inaccurate and incoherent motion predictions, limiting their applicability in downstream tasks.
Scene flow evaluation: While Flux4D primarily focuses on reconstruction and is not specifically designed for scene flow estimation, we further evaluate its performance on PandaSet compared with representative scene flow estimation methods using standard scene flow metries in Table 5 and Please see supp. for comparisons WOD. Although not designed for scene flow estimation, Flux4D achieves superior performance across most scene flow metries using only reconstruction- supervision (RGB + depth). Notably, it outperforms other methods smaller or less common object categories such as wheeled VRUs, other vehicles, and pedestrians, as shown bucketed evaluations. These results highlight promising path to unifying ofte--of-the-ar scene flow estimation and reconstruction within single framework.
8

---

Table 5: Comparison with scene flow estimation methods.
Method
EPE3D
Aoc; 1
Accua T
02.
EPE-BS
EPE-ES
EPE-FD
EPE.3way
Inference time.
NSFP [23]
0.183
0.558
0.713
0,510
0.106
0.103
0.573
0.227
5555 sftrame
FastNSF [24]
0.194
0.571
0.714
0.471
0.155
0.134
0.428
0.211
0066 s/frame
STORM [5]]
0.120
0.757
0.782
0,489
0,009
0.098
0.536
0.201
-0.01 s/Trame
Flax4D
0.094
0.775
0.807
0.123
0.019
0.117
0.391
0.165
00.31 s/frame
Table 6: Bucketed scene flow error on PandaSet. Normalized EPE3D (4) per class, split into static (S) and dynamic (D) regions. Mean S/D are averages across all buckets. Abbrev.: BG Background, CAR = Car, WVRU = Wheeled VRU, VEH = Other Vehicles, PED = Pedestrian.
Method
BG-S]
CAR-S1
CAR-DI
WVRU-S]
WVRU-DI
VEH-S.
VEH-D.
PED-S]
PED-DI
Mcan SJ
Mcan DJ
NSFP [23]
0.128
0.993
0.668
0.045
0.975
0,060
0.819
0.071
0.945
0.080
0.852
FasNSS [24]
0.196
0.581
0.075
0.894
0.784
STORM [S1]
0.005
0.987
0.713
0.000
1,000
0.195
1,000
0.093
1.012
0.076
0.931
FlavDD
0.019
0.078
0,701
0.011
0.866
0.021
0.661
0.027
0.966
0
0.800
Future prediction: We evaluate Flux4D's ' capability for future frame prediction beyond the ob- served frames. This challenging task requires precise motion estimation, temporal consistency, occlusion reasoning, and a comprehensive 4D scene understanding. As shown in Table 4, Flux4D outperforms existing unsupervised methods in both photometric accuracy and geometric consistency. Moreover, Flux4D even outperforms supervised approaches that rely on imperfect explicit annota- tions for extrapolation, demonstrating the robustness of our predicted scene representation and the effectiveness of unsupervised scene flow prediction. This highlights Flux4D's ss ability to model scene dynamics, which is critical for world modeling, simulation, and scene understanding in autonomous systems. We report full-image metrics in Table 4 and report dynamic-only metrics in supp.
Ablation: Table 7 evaluates Flux4D's key design components. Iterative refinement significantly enhances image quality and geometric accuracy metrics. Polynomial motion modeling improves motion prediction performance. Table 8 demonstrates that our static-preferend prior is essential to learning accurate flow, and that velocity reweighting improves performance on the dynamic elements. Please refer to supp. for qualitative comparisons.
LiDeR-free Flux4D: We show that Flux4D can also operate in a LiDAR-fres mode inference similar to DrivingRecon [25] and STORM [51] by using off- the-shelf monocular depth estimation model [15]. As shown in Table 9, the flow estimation performance remains comparable, and in some cases, the visual realism improves in background regions (e.g.. buildings) due to the broader coverage provided by monocular depth, particularly areas where LiDAR sparsity limits reconstruction quality. Combining both LiDAR and points lifted by monocular depth yields the best overall realism.
Scaling analysis: Flux4D's effectiveness stems from multi-scene training, leveraging diverse driv- ing data as implicit regularization. Unlike per-scene methods that require complex regularizations or pre-trained models, increasing the amount of training data naturally improves scene decompo- sition and motion estimation. Analysis on PandaSet and WOD shows consistent improvements in photometric accuracy and motion estimation as training data scale. This confirms unsupervised 4D reconstruction benefits significantly from diverse real-world scenarios, suggesting Flux4D can continue improving with additional data, making it promising for scalable scene reconstruction.
We showcase applying Flux4D for high-fidelity camera simulation in large- scale driving scenarios. Flux4D produces high-quality motion flows in diverse, large-scale dynamic scenes on PandaSet (Fig. 6). Argoverse [45], and WOD (Fig. 7). This allows accurate scene decomposition across diverse environments which is critical for instance extraction and direct manipulation of dynamic elements (Fig. 9). Compared to existing self-supervised per-scene methods, Flux4D is better suited for interactive and controllable applications, as it reconstructs editable representation that supports instance mask extraction, scene editing and object manipulation for various downstream tasks. In Fig. 9, we demonstrate Flux4D's capability render realistic images of the modified scene representation. Notably, our approach achieves this without requiring labels.
Camera Simulation:
5 Limitations
Although Flux4D achieves SoTA 4D reconstruction without any annotations or pre-trained models, three key limitations remain: (1) flow estimation for highly dynamic actors with complex motion

---

Table 7: Ablation study on Flux4D designs.
Table 8: Ablation study on training strategy.
Dynamic-only
Dynamic-only
Methods
PSNRT
SSIMT
Dussu
Vussu
Methods
PSNR
SSIMT
Dusssu
Vausa:
Flux4D-buse
0.472
1,98
0.165
FladD
21.99
0.662
1.63
0.157
iterative refine
21.32
0.636
1.66
0.167
vel. reweighting
21 45
0.641
1.55
0.167
polyuomial
regularization
Table LiDAR-free Flux4D using off-the- monocular depth estimation model [15].
Methods
SSIM
Dynamic-only
DrassB
Yause
7
SSIM
Full image
Dasss
VAMBE
EPE-Sway
Scene Flow
Flas4D (monnccalar depth only)
21.71
0.668
1.45
0.159
23.87
0.688
1.23
0.186
0.165
(LiDAR, Table 1)
0.157
0.182
FlasDD (LiDAR monocular depth)
21.99
0.682
1.52
0.158
24.55
0.726
1,11
0.184
0.161
hoahhlight
Mightimne
CCyalar
Angovente
Waymo Open DotoSer (WOD)
Figure 7: Flux4D reconstruction Argoverse and WOD.
300
No Mondean P.
idwad
109
3006
Jeber Intriion
orden
Figure 8: Scaling analysis. Increasing num- ber of training scenes for Flux4D consistently improves performance.
Figure Simulation applications. Flux4D can be applied suc- cessfully to different camera simulation tasks, e.g., actor removal, insertion and manipulation.
patterns challenging, which could be mitigated by leveraging larger and more diverse training data; (2) iterative approach for long-horizon reconstruction creates visible inconsistencies at transition points; and (3) the method assumes a simple pinhole camera model with clean LiDAR data, limiting applicability with rolling shutter cameras noisy sensor inputs. Please see supp. for more examples. Future work will focus on scaling to larger datasets, developing a unified temporal representation for seamless long-term reconstruction, and improving robustness to real-world sensor imperfections. Furthermore, Flux4D's explicit 3D representation offers interpretable structure for world models. Overall, believe that our simple and scalable design serves as a foundation for the community to build upon, enabling further advancements in 4D reconstruction.
6 Conclusion
We present Flux4D, scalable flow-based unsupervised framework for reconstructing large-scale dynamic scenes by directly predicting 3D Gaussians and their motion dynamics. By relying solely on photometric losses and enforcing an "as static as possible" regularization, Flux4D effectively decomposes dynamic elements without requiring any supervision, pre-trained models, or foundational priors. Our method enables fast reconstruction, scales efficiently to large datasets, and generalizes well to unseen environments. Extensive experiments on outdoor driving datasets demonstrate state- of-the-art performance in scalability, generalization, and reconstruction quality. We hope this work paves the way for efficient, unsupervised 4D scene reconstruction at scale.
10

---

Acknowledgement
We sincerely thank the anonymous reviewers for their insightful suggestions especially scene flow evaluation, paper framing, and additional experiments using monocular would like thank Andrei Bärsan and Joyce Yang for their feedback the early draft, We also thank the Waabi team for their valuable assistance and support.
depth estimation models. We
References
[1] Ben Agro, Quinlan Sykora, Sergio Casas, Thomas Gilles, and Raquel Urtasun. Uno: Unsuper- vised occupancy fields for perception and forecasting. In CVPR, 2024. 3
[2] David Charatan, Sizhe Lester Li, Andrea Tagliasacchi, and Vincent Sitzmann. pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction. In CVPR, 2024.
2, 8
[3] Anpei Chen, Zexiang Xu, Fuqiang Zhao, Xiaoshuai Zhang, Fanbo Xiang, Jingyi Yu, and Hao
Su. Mvsnerf: Fast generalizable radiance field reconstruction from multi-view stereo. In ICCV,
2021.
[4] Liang-Chiel Chen, George Papandreou, Florian Schroff, and Hartwig Adam. Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXivv1706.05558, 2017. 3,7
[5] Yuedong Chen, Haofei Xu, Chuanxia Zheng, Bohan Zhuang, Mare Pollefeys, Andreas Geiger, Tat-Jen Cham, and Jianfei Cai. Mvsplat: Efficient 3d gaussian splatting from sparse multi-view images. In ECCV, 2024. 2, 8
[6] Yun Chen, Jingkang Wang, Ze Yang, Sivabalan Manivasagam, and Raquel Urtasun. G3R: Gradient guided generalizable reconstruction. ECCV, 2025. 2, 5,
[7] Yurui Chen, Chun Gu, Junzhe Jiang, Xiatian Zhu, and Li Zhang. Periodic vibration gaussian: Dynamic urban scene reconstruction and real-time rendering. arXiv preprint arXiv:2311.18561, 2023.
5,5
[8] Zhe Chen, Yuchen Duan, Wenhai Wang. Junjun He, Tong Lu, Jifeng Dai, and Yu Qiao. Vision transformer adapter for dense predictions. In The Eleventh International Conference on Learning Representations, 2023.
[9] Ziyu Chen, Jiawei Yang, Jiahui Huang. Riccardo de Lutio, Janick Martinez Esturo, Boris Ivanovic, Or Litany, Zan Gojeie, Samja Fidler, Marco Pavone, et al. Omnire: Omni urban scene reconstruction. arXiv preprint arXiv:2408.u 16760, 2024. 2
[10] Nathaniel Chodosh, Deva Ramanan, and Simon Lucey. Re-evaluating lidar scene flow for autonomous driving. WACV, 2024. 7
[11] Shenyuan Gao, Jiazhi Yang, Li Chen, Kashyap Chitta, Yihang Qiu, Andreas Geiger, Jun Zhang, and Hongyang Li. Vista: A generalizable driving world model with high fidelity and versatile controllability. arXiv preprint arXiv:2405. /7398, 2024. 3
[12] Georg Hess, Carl Lindström, Maryam Fatemi, Christoffer Petersson, and Lennart Svensson. Splatad: Real-time lidar and camera rendering with 3d gaussian splatting for autonomous
driving. arXiv preprint arXiv:2411.168/ 2024. 2
[13] Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. LRM: Large reconstruction model for single image to 3d. In ICLR, 2024.7
[14] Anthony Hu. Lloyd Russell, Hudson Yeo, Zak Murez, George Fedoscev, Alex Kendall, Jamie Shotton, and Gianluca Corrado, Gaia-l: A generative world model for autonomous driving. arXiv preprint arXiv:2309. 17080, 2023. 3
[15] Mu Hu, Wei Yin, Chi Zhang, Zhipeng Cai, Xiaoxiao Long, Hao Chen, Kaixuan Wang, Gang Yu, Chunhua Shen, and Shaojie Shen. Metric3d v2: versatile monocular geometric foundation model for zhro metric depth and surface normal estimation. In TPAMI, 2024. 9, 10
11

---

[16] Nan Huang, Xiaobao Wei, Wenzhao Zheng, Pengju An, Ming Lu, Wei Zhan, Masayoshi Tomizuka, Kurt Keutzer, and Shanghang Zhang. S3gaussian: Self-supervised street gaussians for autonomous driving. arXiv preprint arXiv:2405 2024. 2,5
[17] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3D gaussian splatting for real-time radiance field rendering. In TOG, 2023. 1,2,4
[18] Mustafa Khan, Hamidreza Fazlali, Dhruv Sharma, Tongtong Cao, Dongfeng Bai, Yuan Ren, and Bingbing Liu. Autosplat: Constrained gaussian splatting for autonomous driving scene reconstruction. arXiv preprint arXiv:2407.025500 027.0288, 2024.
[19] Ishan Khatri, Kyle Vedder, Neehar Peri, Deva Ramanan, and James Hays. can't believe it's not scene flow! In ECCV, 2024. 7
[20] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yer Lo, et al. Segment anything. In ICCV, 2023. 3,
[21] Jinxi Li, Ziyang Song, and Bo Yang. GVFi: Learning 3d gaussian velocity fields from dynamic videos, 2025. 5,5
[22] Xiaofan Li, Yifu Zhang, and Xiaoqing Ye. Drivingdiffussin: Layout-guided multi-view driving scenarios video generation with latent diffusion model. In ECCV, 2024. 3
[23] Xueqian Li, Jhony Kaesemodel Pontes, and Simon Lucey, Neural scene flow prior. In NeurIPS, 2021. 7,9
[24] Xueqian Li, Jianqiao Zheng. Francesco Ferroni, Jhony Kaesemodel Pontes, and Simon Lucey. Fast neural scene flow. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages
2023. 7,9
[25] Hao Lu, Tianshuo Xu, Wenzhao Zheng, Yunpeng Zhang, Wei Zhan, Dalong Du Masayoshi Tomizuka, Kurt Keutzer, and Yingcong Chen. Drivingrecon: Large 4d gaussian reconstruction
model for autonomous driving. arXiv preprint arXiv:2412.09043 2024. 2, 3, 4, 6, 7, 8, 9
[26] Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. In ECCV, 2020. 1.2
[27] Chen Min, Dawei Zhao, Liang Xiao, Jian Zhao, Xinli Xu, Zheng Zhu, Lei Jin, Jianshu Li, Yulan Guo, Junliang Xing, et al. Driveworld: 4d pre-trained scene understanding via world models for autonomous driving. In CVPR, 2024.
[28] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.0719; 2023. 7
[29] Julian Ost, Fahim Mannan, Nils Thuerey, Julian Knodt, and Felix Heide. Neural scene graphs. In CVPR, 2021. 2
[30] Keunhong Park, Utkarsh Sinha, Jonathan T Barron, Sofien Bouaziz, Dan B Goldman, Steven M Seitz, and Ricardo Brrtilaarr Nerfies: Deformable neural radiance fields. In ICCV, 2021.
2
[31] Chensheng Peng, Chengwei Zhang, Yixiao Wang, Chenfeng Xu, Yichen Xie, Wenzhao Zheng, Kurt Keutzer, Masayoshi Tomizuka, and Wei Zhan. Desire-gs: 4d street gaussians for static- dynamic decomposition and surface reconstruction for urban driving scenes. arXiv preprint arXiv:2411.11921, 2024. 2, 4, 5, 6. 7,
[32] Albert Pumarola, Enric Corona, Gerard Pons-Moll, and Francesc Moreno-Noguer. D-nerf: Neural radiance fields for dynamic scenes. In CVPR, 2021. 2
[33] Jiawei Ren, Cheng Xie, Ashkan Mirzaei, Karsten Kreis, Ziwei Liu, Antonio Torralba, Sanja Fidler, Seung Wook Kim, Huan Ling, et al. 14gm: Large 4d gaussian reconstruction model. In NeurIPS, 2025. 7,8
12

---

[34] Xuanchi Ren, Yifan Lu, Hanxue Liang, Jay Zhangjie Wu, Huan Ling, Mike Chen, Francis Fidler, Sanja annd Williams, and Jiahui Huang. Scube: Instant large-scale scene reconstruction using voxsplats. In NeurIPS, 2024. 2
[35] Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, et al. Scalability in perception for autonomous driving: Waymo open dataset. CVPR, 2020. 17
[36] Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, Vijay Vasudevan, Wei Han, Jiquan Ngiam, Hang Zhao, Aleksei Timofeev, Scott Ettinger, Maxim Krivokon, Amy Gao, Aditya Joshi, Yu Zhang, Jonathon Shlens, Zhifeng Chen, and Dragomir Anguelov. Scalability in perception for autonomous driving: Waymo open dataset. In CVPR, 2020. 2, 6,
[37] Haotian Tang, Shang Yang, Zhijian Liu, Ke Hong, Zhongming Yu, Xiuyu Li, Guohao Dai, Yu Wang, and Song Han. Torchsparse+4: Efficient training and inference framework for sparse convolution on gpus. In MICRO, 2023. 7
[38] Jiaxiang Tang, Zhaoxi Chen, Xiaokang Chen, Tengfei Wang, Gang Zeng, and Ziwei Liu. Lgm: Large multi viili gaussian model for high- 3d content creation. In ECCV, 2024.
[39] Adam Tonderski, Carl Lindström, Georg Hess, William Ljungbergh, Lennart Svensson, and Christoffer Petersson. NeuRAD: Neural rendering for autonomous driving. In CVPR, 2024. 1,
5, 7,8
[40] Haithem Turki, Jason Y Zhang, Francesco Ferroni, and Deva Ramanan. Suds: Scalable urban dynamic scenes. In CVPR, 2023. 2
[41] Kyle Vedder, Neehar Peri, Ishan Khatri, Siyi Li, Eric Eaton, Mehmet Kemal Kocamaz, Yue Wang, Zhiding Yu, Deva Ramanan, and Joachim Pehserl. Neural eulerian scene flow fields. In ICLR, 2025. 7
[42] Qianqian Wang, Zhicheng Wang, Kyle Genova, Pratul Srinivasan, Howard Zhou, Jonathan T Barron, Ricardo multi-view image-based rendering. In CVPR, 2021. 2
Noah Snavely, and Thomas Funkhouser. Ibrnet: Learning
[43] Xiaofeng Wang, Zheng Zhu, Guan Huang, Xinze Chen, Jiagang Zhu, and Jiwen Lu. Drive- dreamer: Towards real-world-drive world models for autonomous driving. In ECCV, 2024.
3
[44] Xinyue Wei, Kai Zhang, Sai Bi, Hao Tan, Fujun Luan, Valentin Deschaintre, Kalyan Sunkavalli, Hao Su, and Zexiang Xu. Meshlrm: Large reconstruction model for high-quality meshes. arXiv preprint arXiv:2404.12385 2024. 2
[45] Benjamin Wilson, William Qi. Tanmay Agarwal, John Lambert, Jagjeet Singh, Siddhesh Khandelwal, Bowen Pan, Ratnesh Kumar, Andrew Hartnett, Jhony Kaesemodel Pontes, et al. Argoverse 2: Next generation datasets for self-driving perception and forecasting, arXiv preprint arXiv:2301.004493 2023. 9, 17
[46] Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4d gaussian splatting for real-time dynamic scene rendering. In CVPR, 2024. 2
[47] Tianhao Wu, Fangcheng Zhong, Andrea Tagliasacchi, Forrester Cole, and Cengiz Oztireli. D 2nerf: Self-supervised decoupling of dynamic and static objects from a monocular video. In NeurlPS, 2022. 2, 4, 5
[48] Pengchuan Xiao, Zhenlei Shao, Steven Hao, Zishuo Zhang, Xiaolin Chai, Judy Jiao, Zesong Li, Jian Wu, Kai Sun, Kun Jiang, et Pandaset: Advanced sensor suite dataset for autonomous driving. In ITSC, 2021. 2, 6, 17
[49] Haofei Xu, Songyou Peng, Fangjinhua Wang. Hermann Blum, Daniel Barath, Andreas Geiger, and Mare Pollefeys. Depthsplat: Connecting gaussian splatting and depth. arXiv preprint axXiv:2410.1386. 2024. 5,
13

---

[50] Yunzhi Yan, Haotong Lin, Chenxu Zhou, Weijie Wang, Haiyang Sun, Kun Zhan, Xianpeng Lang, Xiaowei Zhou, and Sida Peng. Street gaussians for modeling dynamic urban scenes, In ECCV, 2024. 7,
[51] Jiawei Yang, Jiahui Huang, Yuxiao Chen, Yan Wang, Boyi Li, Yurong You, Maximilian Igl, Apoorva Sharma, Peter Karkus, Danfei Xu, Boris Ivanovic. Yue Wang, and Marco Pavone. Storm: Spatio-temporal temporal reconstruction model for large-scale outdoor scenes. arXiv preprint arXiv:2501.00602 2025. 2, 3, 4, 5, 6, 7,
[52] Jiawei Yang, Boris Ivanovic, Or Litany, Xinshuo Weng, Seung Wook Kim, Boyi Li, Tong Che, Danfei Xu, Sanja Fidler, Marco Pavone, and Yue Wang. Emernerf: Emergent spatial-temporal scene decomposition via self-supervision. arXiv preprinl arXiv:231 2023. 2, 4, 5, 6, 8
[53] Ze Yang, Yun Chen, Jingkang Wang. Sivabalan Manivasagam, Wei-Chiu Ma, Anqi Joyce Yang. and Raquel Urtasun. Unisim: A neural closed-loop sensor simulator. In CVPR, 2023.
[54] Ze Yang, Yun Chen, Jingkang Wang, Sivabalan Manivasagam, Wei-Chiu Ma. Anqi Joyce Yang, and Raquel Urtasun. Unisim: neural closed-loop sensor simulator. CVPR, 2023,
[55] Zetong Yang. Li Chen, Yanan Sun, and Hongyang Li. Visual point cloud forecasting enables scalable autonomous driving. In CVPR, 2024.
[56] Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. Deformable 3d gaussians for high-fidelity monocular dynamic scene reconstruction. In CVPR, 2024. 2
[57] Yuanwen Yue, Anurag Das, Francis Engelmann, Siyu Tang, and Jan Eric Lenssen. Improving
2D Feature Representations by 3D-Aware Funing. In ECCV, 2024.
[58] Haiming Zhang, Wending Zhou, Yiyao Zhu, Xu Yan, Jiantao Gao, Dongfeng Bai, Yingjie Cai, Bingbing Liu, Shuguang Cui, and Zhen Li. Visionpad: A vision-centrie pre-training paradigm for autonomous driving. arXiv preprint arXiv:2411.14716 2024.
[59] Kai Zhang, Sai Bi, Hao Tan, Yuanbo Xiangli, Nanxuan Zhao, Kalyan Sunkavalli, and Zexiang Xu. GS-LRM: Large reconstruction model for 3D gaussian splatting. In ECCV, 2025. 2
[60] Lunjun Zhang, Yuwen Xiong, 7e Yang, Sergio Casas, Rui Hu, and Raquel Urtasun. Learning unsupervised world models for autonomous driving via discrete diffusion. arXiv preprint arXiv:2311.01017, 2023.
[61] Wenzhao Zheng, Weiliang Chen, Yuanhui Huang, Borui Zhang, Yueqi Duan, and Jiwen Lu. Occworld: Learning a 3d occupancy world model for autonomous driving. In ECCV, 2024. 3
[62] Xiaoyu Zhou, Zhiwei Lin, Xiaojun Shan, Yongtao Wang, Deqing Sun, and Ming- Hsuan Yang. DrivingGaussian: Composite gaussian splatting for surrounding dynamic autonomous driving scenes. In CVPR, 2024.
[63] Chen Ziwen, Hao Tan, Kai Zhang, Sai Bi, Fujun Luan, Yicong Hong, Li Fuxin, and Zexiang Xu. Long-Irm: Long-sequence large reconstruction model for wide-coverage gaussian splats. arXiv preprini arXiv:2410.1278.1 2024. 3
14

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The claims in the abstract and introduction accurately reflect the contributions and scope of work. The stated claims well aligned with both empirical results presented.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: discuss our limitations in Sec. 5. We further provide qualitative examples of failure cases and more discussions about the limitations in the supplemental material.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
15

---

Answer: [NA]
Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: To the best of our knowledge, we have provided all the information needed to reproduce the main experimental results of the paper. We provide the complete architectural designs, data splits, experiment settings, training hyperparameters, loss weights, and more in both the main paper (Sec. 4) and the supplementary material. Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general, releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
16

---

reproducible, they 5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [No]
Justification: We use public datasets [48, 45, 35] in this work. We are unable to release code at the time of submission. We recognize the importance of reproducibility and are actively exploring the possibility of releasing the code with the camera-ready version. In the meantime, we provide pseudocode and implementation details, as well as training and evaluation procedures in the main paper or supplementary.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.c cc/
icy for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
onPolicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide all implementation, training, testing, and data details in both the main paper and supplementary material. This includes data splits, hyperparameters, loss weights, pseudocode, optimizer type, architectural design, experimental settings, and more.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: The paper does include error bars statistical significance tests. This is consistent with common practice in the 3D vision field, where such results are typically considered self-evident due the large performance gaps.
Guidelines:
17

---

The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report I-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out of range (c.g. negative error rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provide the full breakdown of experiments compute resources for each experiment in the supplementary material. We also provide our rough estimate of the compute resources needed for development.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https://neurs: /ttp///eee...//eeeeee cc http//ulll//ttiiceoeee ctcuideesceeeeleiiccc ines?
Answer: [Yes]
Justification: To the best of our knowledge, our research conforms with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
18

---

Answer: [Yes]
Justification:
We discuss the broader impact in the supplementary material.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymentt of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning conrectly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper does not pose such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We provide a complete summary of the licenses of all assets we used (datasets, codebases) in the supplementary material.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
19

---

The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided. this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
Answer: [NA]
Justification: Our paper experiments with exisitng, open open-source datasets and does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/mmodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
20

---

The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question:
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: Our core method does not involve LLMs as any important, original, or non- standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to LLM policy (https: / http//uurrs.ccc//eee/e http://sseeccooeeeeneeeee/////// /LLM) for what should or should not be described.
21


Reviews:
[{"review_id": "0mroOE3Pf9", "reviewer": "Reviewer_QsPn", "summary": "Flux4D is a method for 4D reconstruction using data driven priors and a (mostly) feed-forward architecture that allows for fast inference without requiring expensive per-scene optimization. The method uses RGB + LiDAR inputs and uses gaussians to render novel-views. The gaussians can be transformed in 3D space using learned velocity offsets (flow) and the newly rendered views are used for supervision with a reconstruction loss. This technique allows for a 3D sparse convolutional U-Net to learn the temporal dynamics of the scene in a fully self-supervised way.\n\nThe authors demonstrate the performance of this 4D scene reconstruction technique on a variety of tasks such as scene editing, static/dynamic decomposition, flow prediction and more. The method achieves SOTA performance on reconstruction metrics compared to other unsupervised methods such as EmerNeRF and DeSiRe-GS.", "questions": "1. Could you evaluate scene flow performance on a common benchmark such as Argoverse 2 or WOD and either submit to the leaderboard or provide a common metric to support the scene flow performance claims? The `bucketed-scene-flow-eval` python package ([bucketed-scene-flow-eval·PyPI](https://pypi.org/project/bucketed-scene-flow-eval/)) provides implementations for both 3-way EPE and bucket normalized EPE with support for both Argoverse 2 and WOD. Some numbers here (even if not SOTA!) would easily raise my score to accept.\n2. Could you please add some 3D visualizations in the appendix (or better an interactive website) so we can view the point clouds/gaussians of the generated novel-views? Similar to those shown in Figure 2.\n3. In the motion enhancement section, it is stated that flows are rendered to the image plane and used to upweight the photometric loss. Is this using the flow predictions? What happens at the start of training when presumably the velocity estimates are not good?\n4. In the paper it is mentioned that you \"support\" LiDAR input which seems to imply that Flux4D could work without it, (though the limitations state that it's necessary). Is it strictly necessary? Have you tried an experiment with monocular depth estimation only?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 4, "originality": 3, "strengths_and_weaknesses": "## Strengths\nThe strengths of this paper are the simple construction of the method and the ability for it to scale with data. The qualitative examples provided, and quantitative metrics support the impressive results, and the baselines are thoughtfully chosen.\n\n## Weaknesses\nThe main weakness in my eyes is the metrics used and the lack of 3D results (either quantitative or qualitative). The authors have not provided any of the commonly used scene flow metrics (at a minimum 3-way EPE, but ideally the newer Bucket Normalized EPE) to support the flow performance. In addition no 3D visualizations are provided, and all metrics are on rendered 2D images or depth maps. These evaluations (while common in the relevant literature) often mask the true performance of reconstruction methods.", "was_revised": true, "final_justification": "The authors addressed my concerns about the limited evaluations for their auxiliary claims around scene flow. The updated metrics show that while the new method provides improvements in flow estimation compared to prior reconstruction methods, it still falls short of dedicated scene flow methods. This contextualization provides important context on real world limitations as well as an exciting direction for future work. As a result, I raise my score to accept.", "rebuttal": "Thank you for the thoughtful review and valuable feedback. We are pleased that the method was recognized for its “simple” design, “ability to scale with data,” and “impressive” results. As suggested, we conducted additional experiments comparing with scene flow methods using standard scene flow metrics, supported interactive 3D visualizations, and evaluated Flux4D without LiDAR (using monocular depth estimation). We will include the results and analysis in the revision.\n\n**Q1: Evaluations on scene flow**\\\n**A1:** Thank you for the suggestion. While Flux4D primarily focuses on reconstruction and is not specifically designed for scene flow estimation, we agree that evaluating its performance compared with scene flow estimation methods using standard scene flow metrics is valuable. To this end, we conducted additional experiments comparing Flux4D with NSFP [1] and FastNSF [2] on PandaSet and Waymo Open Dataset (WOD). We also include comparisons with STORM [3]. \n\nAs existing scene flow estimation methods cannot directly predict flows at novel timesteps, we evaluate scene flow on the input frames, following STORM. We restrict evaluation to LiDAR points within the camera field of view (FoV) and report standard metrics including EPE3D, Acc@5, Acc@10, angular error ($\\theta$ error), Threeway EPE (BS: Background-Static, FS: Foreground-Static, FD: Foreground-Dynamics) [4], bucketed normalized EPE [5], and inference speed. For bucketed normalized EPE on WOD, as the provided semantic labels are not rich, we follow EulerFlow [6] and report the metrics on Background (with Signs), Vehicles, Pedestrian, and Cyclists only. \n\nAlthough not specifically designed for scene flow estimation, Flux4D achieves superior performance across most scene flow metrics using only reconstruction-based supervision (RGB + depth). Notably, it outperforms other methods on smaller or less common object categories such as wheeled VRUs, other vehicles, and pedestrians, as shown in bucketed evaluations. We will include these results, visualizations, and further analysis in the revised version.\n\n***Pandaset Table (1/2)***\n| Method |EPE3D↓|Acc@5↑|Acc@10↑|$\\theta$ error↓|EPE-BS↓|EPE-FS↓|EPE-FD↓|EPE-3way↓|Inference↓|  \n|--------|------|------|-------|---------|-------|-------|------|--------|------------|\n| NSFP   |0.183 |0.558 |0.713  |0.510    |0.106 |0.103 |0.573 |0.227   |~5.57 s/frame |\n| FastNSF|0.194 |0.571 |0.714  |0.471    |0.155 |0.134 |0.428 |0.211   |~0.68 s/frame |\n| STORM  |0.120 |0.757 |0.782  |0.489    |**0.009**|**0.098**|0.536 |0.201   |**~0.01 s/frame**|\n| Flux4D |**0.094**|**0.775**|**0.807**|**0.123**|0.019 |0.117 |**0.391**|**0.165**|~0.31 s/frame |\n\n***Pandaset Table (2/2)*** \n| Method |BG-S↓|CAR-S↓|CAR-D↓|WVRU-S↓|WVRU-D↓|VEH-S↓|VEH-D↓|PED-S↓|PED-D↓|Mean S↓|Mean D↓|  \n|--------|-----|-----|-----|-----|-----|-----|-----|-----|-----|-----|-----|\n| NSFP   |0.128|0.093|0.668|0.046|0.975|0.060|0.819|0.071|0.945|0.080|0.852|\n| FastNSF|0.196|0.153|**0.581**|0.043|0.960|0.075|0.701|0.041|**0.894**|0.102|**0.784**|\n| STORM  |**0.005**|0.087|0.713|**0.000**|1.000|0.195|1.000|0.093|1.012|0.076|0.931|\n| Flux4D |0.019|**0.078**|0.701|0.011|**0.866**|**0.021**|**0.661**|**0.027**|0.966|**0.031**|0.800|\n\n**Legend:**  BG = Background, CAR = Car, WVRU = Wheeled VRU, VEH = Other Vehicles, PED = Pedestrian, S = Static, D = Dynamics\n\n***WOD Table (1/2)***\n| Method |EPE3D↓|Acc@5↑|Acc@10↑|$\\theta$ error↓|EPE-BS↓|EPE-FS↓|EPE-FD↓|EPE-3way↓|Inference↓|  \n|------|------|------|----------|-------|-------|-------|---------|---------|----------|\n| FastNSF|0.162|0.734|0.805|0.908|0.131|0.076|0.650|0.203|~0.28 s/frame|\n| Flux4D |**0.048**|**0.901**|**0.929**|**0.540**|**0.011**|**0.012**|**0.440**|**0.114**|**~0.20 s/frame**|\n\n***WOD Table (2/2)***\n| Method |BG-S↓|VEH-S↓|VEH-D↓|PED-S↓|PED-D↓|CYC-S↓|CYC-D↓|Mean S↓|Mean D↓|  \n|--------|----------|-----|-----|-----|-----|-----|-----|------|------|\n| FastNSF|0.144|0.043|0.653|0.044|1.026|**0.026**|0.744|0.064|0.807|\n| Flux4D |**0.011**|**0.009**|**0.502**|**0.014**|**0.680**|**0.026**|**0.732**|**0.015**|**0.638**|\n\n**Legend:**  BG = Background, VEH = Vehicles, PED = Pedestrian, CYC = Cyclists, S = Static, D = Dynamics\n\n**Q2: 3D visualizations in the appendix (or better an interactive website)?** \\\n**A2:** Thank you for the suggestion. We now support interactive 3D visualizations, including 3D point, camera and flow visualization using the `viser` library, as recommended. Due to NeurIPS rebuttal guidelines prohibiting any links, we are unable to share the interactive visualizations at this stage. However, we will include representative 3D visualizations in the camera-ready version and provide interactive examples on the project website.\n\n**Q3: Velocity reweighting: Is this using the flow predictions? What happens at the start of training when presumably the velocity estimates are not good?** \\\n**A3:** Yes, we use the predicted flow to reweight the photometric loss and enhance the supervision for dynamic regions. As detailed in Supplementary Sec. A.1 (Eq. 1, Line 16-20), we apply a pixel-wise reweighting based on the rendered velocity magnitude:\n\n$$\n\\mathcal{L} = (1 + \\left\\Vert \\text{sg}(\\mathbf{v_r}) \\right\\Vert) \\cdot \\mathcal{L}_{\\text{rgb}},\n$$\n\nwhere $\\text{sg}(\\cdot)$ denotes the stop-gradient operation and $\\mathbf{v_r}$ is the rendered velocity in image space. For stability, we clip $\\left\\Vert\\mathbf{v_r}\\right\\Vert$ to $[0, 10]$.\nAt the start of training, the predicted velocity magnitudes are close to zero, so all pixels are weighted equally. Even when early velocity estimates are inaccurate, the clipping and base weighting ensure stable gradients and effective learning throughout training.\n\n\n**Q4: Flux4D without LiDAR (e.g., monocular depth estimation)** \\\n**A4:**  Thank you for pointing this out. Although LiDAR is an important component of Flux4D, it is not strictly necessary. In our framework, LiDAR primarily serves as an initialization for the 3D scene structure and does not need to be highly accurate. To evaluate performance without LiDAR, we conducted an additional experiment using monocular depth estimated by off-the-shelf Metric3D [7] model, as suggested. Specifically, we removed LiDAR points entirely and instead voxelized the projected monocular depth with a 5 cm resolution, retaining up to 2M points, and then trained the model following the same 3D processing pipeline. The results are shown below. \n\nWe observed that the flow estimation performance remains comparable, and in some cases, the visual realism improves in background regions (e.g., buildings) due to the broader coverage provided by monocular depth, particularly in areas where LiDAR sparsity limits reconstruction quality. Moreover, combining both LiDAR and Metrics3D depth yields the best overall realism. \n\nWe will include the quantitative results, visualizations, and further analysis in the revision.\n\n| Method      |PSNR_D↑|SSIM_D↑|D-RSME_D↓|V-RMSE_D↓|PSNR↑|SSIM↑|D-RSME↓|V-RMSE↓|EPE-3way↓|Mean S↓|Mean D↓|\n|-------------|--------|--------|-----------|------------|------|------|-------|-------|------|-------|-------|\n|STORM        | 17.65  | 0.367  | 5.24      | 0.203      | 20.79| 0.508| 4.80  | 0.238 |0.201 | 0.076|0.931|\n|Flux4D (monocular depth only)  | 21.71  | 0.668  | **1.45**      | 0.159      | 23.87| 0.688| 1.23  | 0.186 |0.165|0.037|0.863\n|Flux4D (lidar, main paper)      | **21.99**  | 0.662  | 1.63      | **0.157**      | 23.84| 0.675| **1.07**  | **0.182** | 0.165|**0.031**|**0.800**|\n|Flux4D (lidar + monocular depth) | **21.99**  | **0.682**  | 1.52      | 0.158      | **24.55**| **0.726**| 1.11  | 0.184 | **0.161**|0.032|0.873\n\n\n**Legend:** _D = Dynamics only\n\n[1] Li et al., Neural Scene Flow Prior, 2021. \\\n[2] Li et al., Fast Neural Scene Flow, 2023. \\\n[3] Yang et al., STORM: Spatio-Temporal Reconstruction Model for Large-Scale Outdoor Scenes, 2025.\\\n[4] Chodosh et al., Re-Evaluating LiDAR Scene Flow for Autonomous Driving, 2024. \\\n[5] Khatri et al., I Can't Believe It's Not Scene Flow!, 2024. \\\n[6] Vedder et al., Neural Eulerian Scene Flow Fields, 2025.  \\\n[7] Hu et al., Metric3d v2: A versatile monocular geometric foundation model for zero-shot metric depth and surface normal estimation, 2024."}, {"review_id": "7sSgtk76My", "reviewer": "Reviewer_tD1B", "summary": "This paper presents a feed-forward method based on 3D U-Net to predict dynamic 3D Gaussian splats from video frames and LiDAR data. The training process is supervised by reconstruction terms of RGB and depth, along with regularization on the velocity of each Gaussian splat. The predicted properties of splats are then iteratively refined by a network that takes in the current properties and backpropagated gradients to achieve higher visual fidelity. Although being surprisingly concise, this method exhibits clear superiority over several state-of-the-art methods, including those relying on labels for motion. As claimed by the authors, the key ingredient for Flex4D's performance is learning from a diverse range of scenes.", "questions": "- What is the consideration of not reporting quantitative comparison with DrivingRecon and STORM is not included?", "limitations": "- The authors made an abundant discussion on the limitations in Section F.2. The artifacts shown in Figure A5 are reasonable and interesting.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "# Strengths\n- This method is simple and effective, achieving faster inference on higher-resolution input while obtaining better results. Being able to predict flow without direct supervision is impressive. In particular, this method outperforms baselines that take motion labels.\n- The authors conduct thorough experiments to support the effectiveness of the method. The ablation study shown in Table 5 indicates a clear advantage of iterative refinement. \n- It is nice to have Section E.2 for a conceptual comparison with DrivingRecon and STORM.\n\n# Weaknesses\n- Besides the comparative discussion in Section E.2, Flux4D uses 3D U-Net with sparse convolution to directly process 3D Gaussian splats in the space, which is fundamentally different from DrivingRecon and STORM, which predict pixel-aligned 3DGS from images. How much performance difference is brought by this architecture change? According to Figure A4, Flex4D produces better visual details. But does this improvement in details mainly come from the iterative refinement?", "was_revised": true, "final_justification": "My questions have been well answered by the authors with sufficient experimental evidence. Therefore, I am glad to still vote for an acceptance.", "rebuttal": "Thank you for the thoughtful review and valuable feedback. We are pleased that Flux4D was recognized as “simple and effective,” delivering “faster inference” with “better results,” and that its ability to predict flow without supervision was found “impressive”. As suggested, we have added quantitative comparisons with STORM, highlighted our ablation to isolate the impact of the iterative refinement, and will include additional results and analysis in the revision.\n\n**Q1: Quantitative comparison with DrivingRecon and STORM**\\\n**A1:** Thank you for pointing this out. We have conducted additional experiments to compare with STORM. The comparisons with DrivingRecon and STORM are summarized as follows.\n\n***Comparison to DrivingRecon:***\nWe report quantitative comparison with DrivingRecon on WOD following their setting (Line 213–215) in Table 3 (main paper) and provide visual comparisons (Supp Fig. A4). We observe significant improvements by Flux4D (**+5.99 dB in PSNR and +0.21 in SSIM**), demonstrating the effectiveness for unsupervised dynamic scene reconstruction.  \n\n***Comparison to STORM:***\nAt the time of submission (May 15), STORM was not open-sourced. Following its release afterwards (May 19), we retrained STORM under the same settings as used in Table 1 of the main paper. Additionally, as suggested by Reviewer QsPn, we also include some representative scene flow metrics, namely Threeway-EPE (EPE-3way) [1] and Bucketed Normalized EPE (Mean S, Mean D) [2]. We observe significant improvements of our method in reconstruction realism and flow estimation. Specifically, Flux4D is able to capture high-speed moving actors (e.g., oncoming actors) and small objects / far-away objects thanks to our architectural design to learn flow and appearance directly in 3D space with an approximate geometry initialization (reduce ambiguity and handle full-resolution $1080 \\times 1920$ inputs). Please see Q2/A2 for more discussions. We will provide more visualizations and analysis in the revision.\n\n| Method |PSNR_D↑|SSIM_D↑|D-RSME_D↓|V-RMSE_D↓|PSNR↑|SSIM↑|D-RSME↓|V-RMSE↓|EPE-3way↓|Mean S↓|Mean D↓|\n|----|----|----|----|----|----|----|----|----|----|----|----|\n| STORM  | 17.65 | 0.367| 5.24| 0.203| 20.79| 0.508| 4.80 | 0.238 | 0.201 | 0.076|0.931|\n| Flux4D |**21.99**|**0.662**|**1.63**|**0.157**|**23.84**|**0.675**|**1.07**|**0.182** |**0.165** |**0.031**|**0.800**|\n\n**Legend:** _D = Dynamics only\n\n**Q2: Directly processing Gaussians in 3D space vs predicting pixel-aligned Gaussians in DrivingRecon and STORM**\\\n**A2:** While common in recent feed-forward reconstruction methods [3,4], predicting pixel-aligned Gaussians introduces substantial memory and computational overheads, especially in large-scale driving scenes. This design forces methods such as STORM and DrivingRecon to significantly downsample the image (e.g., ×8 in STORM and ×5 in DrivingRecon), limiting reconstruction fidelity and scalability. In contrast, Flux4D is designed to support high-fidelity, full-resolution ($1920 \\times 1080$) reconstruction and simulation. To this end, it adopts a fundamentally different architecture that learns flow and appearance directly in 3D space with an approximate geometry initialization. This 3D design yields a more compact and geometrically consistent representation across views, improving scalability and enabling explicit multi-view flow reasoning. It also reduces appearance-motion ambiguity.\n\nAs STORM and DrivingRecon do not rely on LiDAR at test time (they used LiDAR as supervision during training same as Flux4D), we also conduct additional experiments to demonstrate that Flux4D can also operate in a LiDAR-free mode (while still processing Gaussians in 3D space) by using off-the-shelf monocular depth estimation model, namely Metrics3D [5]. Specifically, we removed LiDAR points entirely and instead voxelized the projected monocular depth with a 5 cm resolution, retaining up to 2M points, and then trained the model following the same 3D processing pipeline. \n\nWe observed that the flow estimation performance remains comparable, and in some cases, the visual realism improves in background regions (e.g., buildings) due to the broader coverage provided by monocular depth, particularly in areas where LiDAR sparsity limits reconstruction quality. Moreover, combining both LiDAR and Metrics3D depth yields the best overall realism. \n\n| Method      |PSNR_D↑|SSIM_D↑|D-RSME_D↓|V-RMSE_D↓|PSNR↑|SSIM↑|D-RSME↓|V-RMSE↓|EPE-3way↓|Mean S↓|Mean D↓|\n|-------------|--------|--------|-----------|------------|------|------|-------|-------|------|-------|-------|\n|STORM        | 17.65  | 0.367  | 5.24      | 0.203      | 20.79| 0.508| 4.80  | 0.238 |0.201 | 0.076|0.931|\n|Flux4D (monocular depth only)  | 21.71  | 0.668  | **1.45**      | 0.159      | 23.87| 0.688| 1.23  | 0.186 |0.165|0.037|0.863\n|Flux4D (lidar, main paper)      | **21.99**  | 0.662  | 1.63      | **0.157**      | 23.84| 0.675| **1.07**  | **0.182** | 0.165|**0.031**|**0.800**|\n|Flux4D (lidar + monocular depth) | **21.99**  | **0.682**  | 1.52      | 0.158      | **24.55**| **0.726**| 1.11  | 0.184 | **0.161**|0.032|0.873\n\n\n**Legend:** _D = Dynamics only\n\n**Q3: Improvement from Iterative Refinement**\\\n**A3:** The iterative refinement module in Flux4D is designed primarily to enhance reconstruction and simulation fidelity and aim to bridge the gap with state-of-the-art per-scene optimization approaches like StreetGS and NeuRAD. It helps improve fine-grained visual details and reduce artifacts as shown in Tab. 5 and Fig. A2. However, the core motion reasoning and flow estimation are already well captured by the single forward-pass model, which is named Flux4D-base in the Main paper.\n\nTo illustrate this, we summarize the metrics for STORM, Flux4D-base, and the full Flux4D model below. As shown in the table, Flux4D-base already outperforms STORM significantly across all metrics, especially in visual quality of dynamic regions and flow estimation, while the full model with refinement further improves overall reconstruction fidelity (PSNR, SSIM).\n\n| Method      |PSNR_D↑|SSIM_D↑|D-RSME_D↓|V-RMSE_D↓|PSNR↑|SSIM↑|D-RSME↓|V-RMSE↓|EPE-3way↓|Mean S↓|Mean D↓|\n|----|----|----|----|----|----|----|----|----|----|----|----|\n| STORM  | 17.65 | 0.367| 5.24| 0.203| 20.79| 0.508| 4.80 | 0.238 | 0.201 | 0.076|0.931|\n| Flux4D-base | 18.89  | 0.472  | 1.98  | 0.165 | 20.91| 0.583| 1.34 | 0.193 |**0.164**|**0.030**|0.835\n| Flux4D |**21.99**|**0.662**|**1.63**|**0.157**|**23.84**|**0.675**|**1.07**|**0.182** |0.165|0.031|**0.800**|\n\n**Legend:** D = Dynamics only\n\n[1] Chodosh et al., Re-Evaluating LiDAR Scene Flow for Autonomous Driving, 2024. \\\n[2] Khatri et al., I Can't Believe It's Not Scene Flow!, 2024. \\\n[3] Charatan et al., pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction, 2024. \\\n[4] Xu et al., DepthSplat: Connecting Gaussian Splatting and Depth, 2025. \\\n[5] Hu et al., Metric3d v2: A versatile monocular geometric foundation model for zero-shot metric depth and surface normal estimation, 2024."}, {"review_id": "JpqgYzNSoC", "reviewer": "Reviewer_Y9KP", "summary": "The paper introduces a method for reconstructing vehicle driving scenes. By leveraging multi-sensor observations such as camera images and LiDAR point clouds, the method predicts 3D Gaussians and their motion dynamics. The optimization is guided by photometric losses and regularization terms, enabling the decomposition of dynamic elements from raw data without relying on annotations or pre-trained models.", "questions": "As stated in the Weaknesses section, it would be better to add more evaluations on various 4D scenes, not limited to vehicle driving scenarios. Additionally, even though the method is proposed for vehicle driving, its processing speed constraints limit practical applications.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. The paper is well-written and easy to follow.\n2. The experiments and related analyses in the main paper and supplementary material are sufficient.\n\nWeaknesses:\n1. The experiments are conducted on outdoor driving scenes, which mainly involve sparse rigid motions and represent an easier part of general 4D scenes. It is inappropriate to use \"4D\" to name the method, or more evaluations on diverse 4D scenes—especially non-rigid motions where Eq. 2 is effective only within short time intervals are needed.\n2. The network in Fig. 2 is claimed to be a generalizable module, but its generalization ability does not seem to be evaluated. Can it generalize to unseen 4D motions or just unseen vehicle velocities?\n3. The inference speed of the method is slower than existing approaches, which restricts its application in vehicle driving scenarios.", "was_revised": true, "final_justification": "The authors' detailed rebuttal addressed my concerns. Consequently, I have raised my score to \"Borderline accept\".", "rebuttal": "Thank you for the thoughtful review and valuable feedback. We are pleased that the paper was found to be “well‑written” and that the experiments and analysis are “sufficient.” We now address the concerns.\n\n**Q1: Scope beyond driving scenes: Inappropriate to use \"4D\" to name the method,evaluations on diverse 4D scenes**\\\n**A1:** Thank you for pointing this out. As stated in the paper (Abstract: L8–10; Introduction: L59–60; Method: L118–121), Flux4D is a simple and scalable framework for 4D reconstruction of **large‑scale dynamic driving scenes**. We agree that driving scenes primarily involve sparse rigid motions. To better clarify this scope, we will consider renaming the submission to “FluxDrive: Unsupervised 4D Reconstruction for Driving Scenes.” \n\nWe further note that our method handles short‑horizon non‑rigid motion (e.g., pedestrians and cyclists) on the Waymo Open Dataset and Argoverse 2, as shown in Fig. 7 thanks to the higher‑order polynomial motion models used in the final model (Sec. 3.4, Table 5, Appendix A.1). These motions and scenes are unseen during training, demonstrating the effectiveness and generalizability of our model on driving scenes. Regarding Eq. 2, it represents the motion model used in Flux4D-base, which can already achieve reasonable performance for outdoor driving scenes over short time horizons, as shown in our ablations. This observation aligns with existing works such as GVFi [1] and STORM [2].\n\nFinally, as acknowledged in the method and limitation sections, highly dynamic, complex motions remain challenging (L290–291). Exploring more advanced velocity models or implicit flow representations is an exciting direction for future work (L188–189). \n\n**Q2: Can it generalize to unseen 4D motions or just unseen vehicle velocities?**\\\n**A2:** As discussed in Q1/A1, our method can generalize beyond vehicle velocities. In Fig. 7, we show that Flux4D can generalize to unseen non‑rigid motions (e.g., pedestrians and cyclists) on the Waymo Open Dataset and Argoverse 2. Moreover, as suggested by Reviewer QsPn, we conduct additional experiments to evaluate scene flow against test-time scene flow estimation methods (NSFP [3] and FastNSF[4]) using standard scene flow metrics, namely Threeway-EPE (EPE-3way) [5] and Bucketed Normalized EPE (Mean S, Mean D) [6]. \n\nAs shown in the table, Flux4D achieves competitive performance across most metrics. Notably, although our approach is designed for unsupervised reconstruction and simulation rather than scene flow estimation, it outperforms other methods on challenging cases, including smaller or less common object categories and non‑rigid actors such as wheeled VRUs, other vehicles, and pedestrians, as shown in bucketed evaluations. This highlights the generalizability and effectiveness of large-scale unsupervised training.\n\n***Pandaset Table (1/2)***\n| Method |EPE3D↓|Acc@5↑|Acc@10↑|$\\theta$ error↓|EPE-BS↓|EPE-FS↓|EPE-FD↓|EPE-3way↓|Inference↓|  \n|--------|------|------|-------|---------|-------|-------|------|--------|------------|\n| NSFP   |0.183 |0.558 |0.713  |0.510    |0.106 |0.103 |0.573 |0.227   |~5.57 s/frame |\n| FastNSF|0.194 |0.571 |0.714  |0.471    |0.155 |0.134 |0.428 |0.211   |~0.68 s/frame |\n| STORM  |0.120 |0.757 |0.782  |0.489    |**0.009**|**0.098**|0.536 |0.201   |**~0.01 s/frame**|\n| Flux4D |**0.094**|**0.775**|**0.807**|**0.123**|0.019 |0.117 |**0.391**|**0.165**|~0.31 s/frame |\n\n***Pandaset Table (2/2)*** \n| Method |BG-S↓|CAR-S↓|CAR-D↓|WVRU-S↓|WVRU-D↓|VEH-S↓|VEH-D↓|PED-S↓|PED-D↓|Mean S↓|Mean D↓|  \n|--------|-----|-----|-----|-----|-----|-----|-----|-----|-----|-----|-----|\n| NSFP   |0.128|0.093|0.668|0.046|0.975|0.060|0.819|0.071|0.945|0.080|0.852|\n| FastNSF|0.196|0.153|**0.581**|0.043|0.960|0.075|0.701|0.041|**0.894**|0.102|**0.784**|\n| STORM  |**0.005**|0.087|0.713|**0.000**|1.000|0.195|1.000|0.093|1.012|0.076|0.931|\n| Flux4D |0.019|**0.078**|0.701|0.011|**0.866**|**0.021**|**0.661**|**0.027**|0.966|**0.031**|0.800|\n\n**Legend:**  BG = Background, CAR = Car, WVRU = Wheeled VRU, VEH = Other Vehicles, PED = Pedestrian, S = Static, D = Dynamics\n\n**Q3: The inference speed of the method is slower than existing approaches, which restricts its application in vehicle driving scenarios.**\\\n**A3:** Our method is designed for offline reconstruction and simulation, not real-time onboard perception. In this context, Flux4D offers significant advantages in both speed and reconstruction quality over state of the art, unveiling the possibility for scalable self-driving simulation.\n\nFirst, Flux4D largely bridges the gap compared to state-of-the-art supervised self-driving reconstruction methods (as recognized by Reviewers QsPn, tD1B, Ew2X). It is over **10× faster** than generalizable approaches like G3R while achieving higher-quality results. Compared to per-scene optimization methods such as StreetGS and NeuRAD, Flux4D offers up to **500×** speedup without requiring any labels, making it significantly more scalable for large-scale simulation.\n\nSecond, although system-level efficiency was not the primary focus of our submission, we analyzed the inference pipeline and removed redundant operations, improving our reconstruction time to 1.8 s. If the iterative refinement module is disabled, our model becomes Flux4D-base in the main paper, which reconstructs the scene in **0.13 s** and still achieves significant visual quality improvements over existing unsupervised feed-forward methods (DepthSplat, L4GM, STORM) on dynamic actors as shown below. These results are obtained using a single RTX A5000 GPU. Running on a more advanced GPU such as an H100 would also provide a substantial speedup. Additionally, inference acceleration via TensorRT or quantization is straightforward and expected to yield further gains.\n\n| Method             | PSNR_D↑ | SSIM_D↑ | Recon Speed↓       |  \n|--------------------|---------|---------|--------------------|\n| DepthSplat         | 16.87   | 0.425   | 0.87s     |\n| L4GM               | 17.36   | 0.343   | 0.32s     | \n| STORM              | 17.65   | 0.367   | **0.07s** | \n| Flux4D-base        |**18.89**|**0.472**| 0.13s     | \n|--------------------|---------|---------|-----------|\n| Flux4D (reference) |  21.99  | 0.662   | 1.8s      | \n\n**Legend:** _D = Dynamics only\n\nThird, while some existing feed-forward methods offer faster inference, they typically operate at significantly lower resolutions (e.g., $160 \\times 240$ for STORM) and/or suffer from limited fidelity with noticeable artifacts (see Fig. 3). This compromises downstream usability and leads to a larger domain gap. Moreover, these approaches either do not support key simulation capabilities (e.g., actor removal, insertion) or produce significant artifacts, due to insufficient reconstruction quality and scene decomposition.\n\nIn conclusion, we reiterate that Flux4D is designed for offline driving-scene simulation. Unlike traditional pipelines that require manual annotations and hours of optimization, Flux4D offers an order-of-magnitude speedup with competitive fidelity in a fully unsupervised manner, making it well-suited for scalable, simulation-driven autonomy development. We will include updated timing results and discussion in the revision.\n\n[1] Li et al., GVFi: Learning 3D Gaussian Velocity Fields from Dynamic 360 Videos, 2025.\\\n[2] Yang et al., STORM: Spatio‑Temporal Reconstruction Model for Large‑scale Outdoor Scenes, 2025.\\\n[3] Li et al., Neural Scene Flow Prior, 2021. \\\n[4] Li et al., Fast Neural Scene Flow, 2023. \\\n[5] Chodosh et al., Re-Evaluating LiDAR Scene Flow for Autonomous Driving, 2024. \\\n[6] Khatri et al., I Can't Believe It's Not Scene Flow!, 2024."}, {"review_id": "DirDiovvjj", "reviewer": "Reviewer_Ew2X", "summary": "This paper presents an unsupervised training paradigm for NVS of dynamic scenes. At the core of this work, Lidar is used as a guiding modality for boot-strapping the GS initialization. Using Lidar information, a motion model is fitted as an additional regularization term in addition to a photometric loss composed of (rgb,ssim,depth). Comparisons are made with prior-works in the space showing improvement over unsupervised baselines and \"closing the gap\" to supervised approaches.", "questions": "Main point to address:\nThe applicability and methodology of this work as presented is good. It's more of a framing problem than a method problem. Imo with correct framing and emphasizing the need for lidar information this work can be better appreciated by the community. The presentation makes it hard to disentangle the scientific contribution w.r.t lidar based methods vs. methods that rely on foundation model priors.", "limitations": "See Questions", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strength:\n- Good use of the lidar data for learning the motion model and GS initialization.\n- Research emphasizes real-world applicability showing good runtime performance and good results on NVS for future frame prediction.\n\nWeaknesses:\n- The provided comparisons with previous works makes it hard to evaluate the scientific contributions of this work. While out-performing on the dry metrics, this work uses an additional modality which the others don't. It allows to build a motion model which isn't accesible to the other works.\n- In light of this, I would place this work as an in-between the unsupervised and supervised works. I acknowledge that Lidar data can be collected in an unsupervised manner - in addition it's use is clever in extracting a form of velocity segmentation - other works do not have access to this information.", "was_revised": true, "final_justification": "In the rebuttal the authors show the method is robust to the quality of the geometric prior and can leverage mono-depth estimates as well while still performing well. I raised my score to a borderline accept. \nThe method is technically sound, however, the experiment section mixes methods that use different quality of depth inputs (model based, vs. lidar based). I encourage the authors for the camera ready to improve on this point.", "rebuttal": "Thank you for the thoughtful review and valuable feedback. We are glad that “the applicability and methodology of our work” are recognized as “good”. We now address the reviewer’s concerns regarding the framing of our method, particularly on our use of LiDAR.\n\nTo clarify this, we (1) explain the dependency on LiDAR for baselines and provide additional experiments demonstrating that Flux4D can also operate in a LiDAR-free setting, by using off-the-shelf monocular depth estimation as an alternative, and (2) argue that using LiDAR should not be considered as a form of supervision in the context of self-driving applications, where LiDAR is commonly available and passively collected. \n\n**Q1: Dependence on LiDAR modality – Comparisons with previous works makes it hard to evaluate the scientific contributions**\\\n**A1:** We acknowledge that our method leverages LiDAR to initialize our Gaussian scene representation and uses it for depth supervision. However, we would like to point out that all of our baselines (in Tab. 1, 2, 4) also have access to this data either in reconstruction or during training. Specifically, EmerNeRF, Desire-GS, G3R, StreetGS and NeuRAD all use LiDAR in their proposed reconstruction frameworks, two of which also predict scene flows (EmerNeRF, Desire-GS). For general feed-forward generalizable methods (e.g., DepthSplat, L4GM), we believe we have made considerable effort towards incorporating LiDAR in their frameworks (L219-222 and Supp. 105-123). While STORM and DrivingRecon use LiDAR during training as depth supervision, we acknowledge they do not use it at test time.  \n\nTo demonstrate Flux4D in a LiDAR-free setting at test time, similar to DrivingRecon and STORM, we replaced LiDAR point cloud initialization with metric depth estimated from foundational model Metric3D [1]. Specifically, we removed LiDAR points entirely and instead voxelized the projected monocular depth with a 5 cm resolution, retaining up to 2M points, and then trained the model following the same 3D processing pipeline.  Additionally, as suggested by Reviewer QsPn, we also include some representative scene flow metrics, namely Threeway-EPE (EPE-3way) [2] and Bucketed Normalized EPE (Mean S, Mean D) [3] to better evaluate the motion models. \n\nWe observed that the flow estimation performance remains comparable with the original Flux4D, and in some cases, the visual realism improves in background regions (e.g., buildings) due to the broader coverage provided by monocular depth. Moreover, this LiDAR-free version of Flux4D performs significantly better than STORM. This demonstrates that our unsupervised learning of flow and appearance directly in 3D space with an approximate geometry initialization to be the key scientific contribution (as recognized by Reviewer QsPn), rather than the use of LiDAR.  We report the numbers below and will include the quantitative results, visualizations, and further analysis in the revision.\n\n| Method      |PSNR_D↑|SSIM_D↑|D-RSME_D↓|V-RMSE_D↓|PSNR↑|SSIM↑|D-RSME↓|V-RMSE↓|EPE-3way↓|Mean S↓|Mean D↓|\n|-------------|--------|--------|-----------|------------|------|------|-------|-------|------|-------|-------|\n|STORM        | 17.65  | 0.367  | 5.24      | 0.203      | 20.79| 0.508| 4.80  | 0.238 |0.201 | 0.076|0.931|\n|Flux4D (monocular depth only)  | 21.71  | 0.668  | **1.45**      | 0.159      | 23.87| 0.688| 1.23  | 0.186 |0.165|0.037|0.863\n|Flux4D (lidar, main paper)      | **21.99**  | 0.662  | 1.63      | **0.157**      | 23.84| 0.675| **1.07**  | **0.182** | 0.165|**0.031**|**0.800**|\n|Flux4D (lidar + monocular depth) | **21.99**  | **0.682**  | 1.52      | 0.158      | **24.55**| **0.726**| 1.11  | 0.184 | **0.161**|0.032|0.873\n\n**Q2: Unsupervised nature of Flux4D and framing**\\\n**A2:** Thank you for raising this point. LiDAR is widely available and passively collected in real-world autonomous driving systems. Moreover, LiDAR in our framework is used purely as raw, sparse sensor input, without any human labels, semantic annotations, or ground-truth flow. The learning is driven entirely by photometric and geometric reconstruction losses, and Flux4D does not rely on supervised labels at any stage. This is consistent with definition and common practice in unsupervised learning, where additional modalities (e.g., lidar, stereo or depth) can be used as inputs and help define an energy or reconstruction objective [4, 5, 6]. Importantly, while LiDAR is sparse, Flux4D produces dense, photorealistic 3D reconstructions. We also note that existing unsupervised per-scene reconstruction methods for driving scenes (DesireGS, EmerNerf, UnIRe [7]) also require the usage of LiDAR during their reconstruction process. \n\nWe also provide additional experiments in Q1/A1 demonstrating that Flux4D also performs well without LiDAR, using monocular depth estimated by an off-the-shelf model. We emphasize that this is fundamentally different from methods relying on foundation model priors as supervision, as such supervision can introduce inaccuracies and is not suitable for scalable unsupervised training. The key contribution of Flux4D lies in learning motion and scene dynamics directly from raw RGB and LiDAR inputs, without any annotated supervision.  We will clarify this framing in the revised manuscript and more explicitly discuss/emphasize the role of LiDAR within our unsupervised learning setup. \n\nWe look forward to further discussions if you have any further questions or concerns. \n\n[1] Hu et al., Metric3d v2: A versatile monocular geometric foundation model for zero-shot metric depth and surface normal estimation, 2024. \\\n[2] Chodosh et al., Re-Evaluating LiDAR Scene Flow for Autonomous Driving, 2024. \\\n[3] Khatri et al., I Can't Believe It's Not Scene Flow!, 2024. \\\n[4] Liu et al., Unsupervised Learning of Scene Flow Estimation Fusing with Local Rigidity, 2019. \\\n[5] Li et al., Neural Scene Flow Prior, 2021. \\\n[6] Vedder et al., Neural Eulerian Scene Flow Fields, 2025.  \\\n[7] Mao et al., UnIRe: Unsupervised Instance Decomposition for Dynamic Urban Scene Reconstruction, 2025."}]

Author rebuttal:

Meta-review: All reviewers appreciate the submission and find the solution technically efficient with strong results. The AC recommends accepting the paper.
The AC agrees with reviewer Y9KP that the evaluations are restricted to autonomous driving scenes, while the title and introduction suggest a broader focus on general 4D. The authors should either moderate their claims by explicitly limiting them to autonomous driving scenarios or incorporate more results on general 4D reconstruction during the revision period.
Decision: Accept (poster)