D$^2$GS: Dense Depth Regularization for LiDAR-free Urban Scene Reconstruction
Authors: Kejing Xia
Jidong Jia
Ke Jin
Yucai Bai
Li Sun
Dacheng Tao
Youjian Zhang
Venue: NeurIPS 2025 poster ()
Keywords: Urban scenes reconstruction
Gaussian Splatting
Depth estimation
Primary area: applications

Abstract: Recently, Gaussian Splatting (GS) has shown great potential for urban scene reconstruction in the field of autonomous driving. However, current urban scene reconstruction methods often depend on multimodal sensors as inputs, $\textit{i.e.}$ LiDAR and images. Though the geometry prior provided by LiDAR point clouds can largely mitigate ill-posedness in reconstruction, acquiring such accurate LiDAR data is still challenging in practice: i) precise spatiotemporal calibration between LiDAR and other sensors is required, as they may not capture data simultaneously; ii) reprojection errors arise from spatial misalignment when LiDAR and cameras are mounted at different locations. To avoid the difficulty of acquiring accurate LiDAR depth, we propose ${D}^2GS$, a LiDAR-free urban scene reconstruction framework. In this work, we obtain geometry priors that are as effective as LiDAR while being denser and more accurate. $\textbf{First}$, we initialize a dense point cloud by back-projecting multi-view metric depth predictions. This point cloud is then optimized by a Progressive Pruning strategy to improve the global consistency. $\textbf{Second}$, we jointly refine Gaussian geometry and predicted dense metric depth via a Depth Enhancer. Specifically, we leverage diffusion priors from a depth foundation model to enhance the depth maps rendered by Gaussians. In turn, the enhanced depths provide stronger geometric constraints during Gaussian training. $\textbf{Finally}$, we improve the accuracy of ground geometry by constraining the shape and normal attributes of Gaussians within road regions. Extensive experiments on the Waymo dataset demonstrate that our method consistently outperforms state-of-the-art methods, producing more accurate geometry even when compared with those using ground-truth LiDAR data.

D'GS: Dense Depth Regularization for LiDAR-free Urban Scene Reconstruction
Kejing Xia* Jidong Jia
Ke Jin
Yucai Bai Li Sun't
Dacheng Tao Youjian Zhang't
Wuhan University
Shanghai Jiaotong University 'TongJi University *Bosch NNnyynng Technological University
xiakejing lesia@whu. edu. en, jjd11230sjtu edu. en, 2330379@tongji. edu. dacheng. taoentu. edu. sg, (Yucai.BAl Kevin.SUM, SUN, Youjian 2cn bosch. com
Abstract
Recently, Gaussian Splatting (GS) has shown great potential for urban scene re- construction in the field of autonomous driving. However, current urban scene reconstruction methods often depend on multimodal sensors as inputs, i.e. LiDAR and images. Though the geometry prior provided by LiDAR point clouds can largely mitigate ill-posedeess in reconstruction, acquiring such accurate LiDAR data is still challenging in practice: i) precise spatiotemporal calibration between LiDAR and other sensors is required, as they may not capture data simultaneously; ii) reprojection errors arise from spatial misalignment when LiDAR and cameras are mounted at different locations. To avoid the difficulty of acquiring accurate LiDAR depth, propose D*GS, fireR-frce urban scene reconstruction frame- work. this work, we obtain geometry priors that are as effective as LiDAR while being denser and more accurate. First, initialize dense point cloud by back-projecting multi-view metric depth predictions. This point cloud is then optimized by Progressive Pruning strategy to improve the global consistency. Second, we jointly refine Gaussian geometry and predicted dense metric depth via Depth Enhancer. Specifically, we leverage diffusion priors from depth foundation model to enhance the depth maps rendered by Gaussians. In turn, the enhanced depths provide stronger geometric constraints during Gaussian training. Finally, we improve the accuracy of ground geometry by constraining the shape and normal attributes of Gaussians within road regions. Extensive experiments on the Waymo dataset demonstrate that our method consistently outperforms state- of-the-art art methods, producing more accurate geometry even when compared with those using fround-trute LiDAR data.
Introduction
Modeling urban street scenes is fundamental to autonomous driving (2). Accurate 3D reconstructions of roads, buildings, and street furniture enable the creation of high-fidelity virtual environments, which are crucial for the closed-loop simulation and testing of perception, planning, and control systems (3). The ability reuse and edit these digital urban scenes, rather than repeatedly collecting fresh data for every test scenario, significantly improves the efficiency of data collection.
Recently, methods leveraging Neural Radiance Field (NeRF) 4 and 3D Gaussian Splatting (3DGS) 5S) have become popular in this field for their photorealistic results. However, NeRF-based methods are computationally intensive while 3DGS is favored due to its explicit representation, high
"This work was completed during an internship at Bosch. Equal contribution.
t Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

GT
Omnire
Ours
Reproject jeee
Calibration
Missing
Figure Examples of common LiDAR acquisition issues: a) reprojection error, b) calibration misalignments, and c) LiDAR missing problem. OmniRe [1] yields poor performance in both image reconstruction and depth estimation when using these inaccurate LiDAR measurements.
efficiency, and rapid rendering speed 6 ability for dynamic urban scenes 7
fact, 3DGS has demonstrated its powerful reconstruction
10
Most of the 3DGS urban street scene reconstruction methods rely on LiDAR data to provide an accurate geometry prior. Usually, this prior can be applied in two aspects: i) LiDAR point cloud can be utilized initialize Gaussian points. ii) LiDAR point cloud can provide a sparse depth supervision during the training phase. However, there are several notable drawbacks when acquiring LiDAR data in practice. First, the acquisition of LiDAR data requires specialized vehicles and costly equipment [12], which makes the data collection process challenging and less scalable. Second, the calibration between sensors (LiDARs and cameras) is highly demanding. Since these sensors do not capture data simultaneously. both spatial and temporal calibration are required to accurately align the LiDAR points with images. Furthermore, in practical data collection systems, LiDAR sensors and cameras are inevitably mounted at different positions and heights, causing significant viewpoint disparities and resulting reprojection errors when LiDAR points are projected onto images (Fig. []. However, most existing methods this LiDAR projection result ground-truth depth map. In preliminary experiments, we found that misalignment between LiDAR projections and pixels significantly affects the reconstruction results (as shown in Fig. D.. Finally, LiDAR provides only sparse depth supervision during the GS training phase, limiting the model's ability to learn fine-grained geometric details.
In the meantime, many recent studies incorporate geometric constraints into the optimization of sparse- view scene reconstruction. However, these methods either rely on monocular depth estimation [3]
13
19 20 21 have also explored ways to
14 15
16
17 [8] which only provides
relative depth, or obtain metric depth through multi-view depth estimation 19
20 21), which is
unsuitable for dynamic scenes and still suffers from large absolute errors.
To overcome the challenges of acquiring accurate geometry prior using LiDAR points, while obtaining more robust and dense depth for regularization, we introduce D*GS LiDAR-free dynamic urban street scene reconstruction framework. First, to obtain an initial point cloud as a replacement for LiDAR point clouds, we leverage depth predictions from pre-trained multi-view estimation model 21 Re-projecting these depths yields an initial, dense point cloud. However, this point cloud per-pixel, resulting in an excessive number of points. To manage the computational cost, we introduce a progressive pruning strategy. This strategy starts training the Gaussian field with minimal scene graph components, then gradually prunes the less important Gaussians retain a compact yet representative point set that effectively captures the global scene geometry. Second, for the absence of

---

depth supervision provided by LiDAR data, we propose a novel joint optimization strategy for depth and the Gaussian representation. Inspired by incorporating image-based diffusion priors [22 used in Gaussian training, we introduce Depth Enhancer module that leverages priors from the depth foundation model [33 to refine the depth rendered from the current Gaussian state. Specifically, we iteratively refine and update the densely rendered depth by using the current geometric information as guidance within the diffusion framework, and using refined depth for Gaussian training. The refined depth maps can offer dense and accurate depth supervision that is semantically consistent with the scene, making them a superior alternative to LiDAR point clouds. Finally, since the ground plane possesses strong geometric priors in urban scenes [10), initialize and optimize dedicated road node in the scene graph for a better geometry and depth estimation of the road. Specifically, we constrain each Gaussian in the road node to be planar parallel to precomputed ground plane by regularizing their shape and normal attributes. In summary, our main contributions are as follows:
We introduce DCGS framework for reconstructing dynamic urban street scenes using only camera inputs, eliminating the need for LiDAR points acquisition and the error brought by calibration and reprojection.
We propose a progressive pruning strategy to efficiently manage the dense point cloud derived from initial depth estimates, yielding a compact Gaussian representation that captures global scene geometry.
We develop joint optimization strategy using a diffusion- Depth Enhancer that itera- tively refines estimated depths and the Gaussian representation, providing robust geometric supervision with dense metric depths.
We further enhance the reconstruction accuracy on roads by creating a Road Node in the scene graph, explicitly modeling the ground plane using strong geometric priors.
2
Related Works
Urban scene reconstruction. Neural Radiance Field (NeRF) 4 and 3D Gaussian Splatting (3DGS) (5] have recently become the leading solutions for urban scene reconstruction. Early methods reconstruct entire city blocks and improve scalability to larger environments 24 25 but all assumed static scenes. To handle dynamic elements, methods such as SUDS (26] and EmerN-
cRF (27 employ self-supervised decompositions into static and dynamic components, while neural scene-graph approaches explicitly disentangle motion and structure [28] and then leverage instance- aware frameworks decompose each instance and model each moving object independently [29]. Nowadays, S3GS 77 introduces deformable fields for Gaussians to capture scene dynamics, and PVG g assigns each Gaussian a lifespan attribute for temporal modeling. Then scene graphs are also used in GS-based method, StreetGaussians [9] represents static and dynamic components as individual Gaussians within a Gaussian scene graph and optimizes them jointly, and OmniRe integrates multiple Gaussian types to better reconstruct dynamic elements.
Depths in 3D Gaussian. Depth information provides essential geometric priors for resolving ambiguities in 3D reconstruction [19] and has been widely used in sparse-view reconstruction fields. In scenarios lacking ground-truth -trth depth, many works adopt monocular depth estimates as weak supervision, leveraging scale-invariin loss functions [3
14 [5] and depth ranking constraints [16] 17]. Recent advances such as DNGaussian [8] further improve this paradigm with patch-wise depth normalization. However, these monocular supervision methods inherently suffer from scale ambiguities due to the unknown shift and scale factors predicted depth [2]]. While some feed- forward 3DGS methods like MVSplat (0 and DepthSplat (21] address this by incorporating multi-view depth estimation, but they cannot handle dynamic scenes, and feed-forward depths are imperfect for further training. In contrast, our method leverages metric depth supervision while progressively refining depth predictions with generative priors through an iterative training scheme.
Diffusion priors in 3D Gaussian. Generative priors have recently been incorporated into 3DGS to reduce artifacts arising from geometric inaccuracies in novel-view synthesis (30 22 These approaches leverage diffusion models to enforce semantic coherence, hallucinate photometric details, and extrapolate occluded regions. DriveDreamer4D (3 further adapts this framework to dynamic driving scenarios. However, all of these methods concentrate on image-space priors and overlook the orthogonal role of depth-space generative priors. Diffusion- monocular depth estimators such Marigold [32] establish foundation for learned depth priors, and Marigold-DG (23) shows
3

---

Progressive Pruning
Gaussian Optimization
Rosd mode
Background
Neata
Rackgraand made
Rirobee
Lowe
Other nodes toob) rigid)
Loss
D
07ed
Updalo
Depth Enhancer
degic poit
Eraade
z4
Dicodir
Suuli Skn
Upolas
Salact
1!"
Taneds
Jaker
AV3 AccEss
Figure 2: Pipeline of D2GS. We first employ a Progressive Pruning strategy obtain robust global Gaussian initialization. A Road Node is incorporated into the scene graph structure to regularize road region using strong geometric priors. During training, Gaussian optimization and depth refinement are performed iteratively, allowing depth to be learned jointly from Gaussian supervision and enhanced by diffusion priors from a pretrained depth foundation model.
that classifier-guidet sampling can propagate sparse metric depths into dense predictions. However, our approach diverges by directly integrating Marigold's generative depth prior into the 3DGS optimization loop to refine the dense imperfect depths.
3 Method
Our proposed D2GS (Fig. addresses the challenge of LiDAR-free dynamic urban scene recon- struction through three key components. First, we propose initialization strategy that leverages estimated depth as input and applies progressive pruning to obtain a compact Gaussian representation. (Sec. M.I...IIp. 3.T). Second, during Gaussian optimization, to iteratively refine the rendering depth, providing dense geometric supervision (Sec. (3.2). Third, we incorporate specialized road node into the scene graph representation explicitly model the ground plane using strong geometric priors (Sec. 3.3) We illustrate these components in the following subsections.
propose multi-view Depth Enhancer module
3.1 Initialization via Progressive Pruning
Effective initialization of the 3D Gaussians is crucial for achieving high-quality reconstruction results with Gaussian Splatting. While most current methods 9 I rely on LiDAR point clouds for this initial
purpose, our LiDAR-free setting requires alternative solution. We begin by predicting depth map for each input image using a pre-trained multi-view depth estimation network 211. denoted as D; for iza image. Given the camera pose and D, we unproject each depth map into point cloud. By aggregating these point clouds, we obtain a unified point cloud P.
The point cloud P contains as many points as image pixels, resulting in an excessively large size (about 100 million) that is computationally impractical for Gaussian training. To address this challenge, one common solution is to de de-densify the points by voxel grids. However, this can lead the loss of fine geometric details. To mitigate this, we propose progressive pruning strategy that learns to de-densify the points in a learnable manner. Specifically, we initialize Gaussians with a non-deformable and low-resolution training setting to optimize these points and progressively eliminate less important Gaussians based on their opacity, i.e. the significance of the Gaussian in the scene representation. After each round of progressive pruning, we reset the opacity values and continue the next round. The progressive pruning process be formulated as:

---

G G - = E G(N) oi 2r].
where G represents the set of Gaussians before pruning at round N, N the opacity of one Gaussian point Ii. and is the opacity threshold. This progressive strategy produces a manageable set of Gaussians that capture the essential scene geometry, effectively balancing computational efficiency and the quality of the initial geometric representation.
3.2 Joint Optimization of Depth and Gaussian Splatting
In the absence of LiDAR depth, obtaining reliable metric depth as a geometric constraint becomes challenging. Our experiments found that the estimated metric depths [21] are not sufficiently accurate for training, directly applying these noisy depth maps supervision leads to significant degradation in both reconstruction quality and geometric accuracy. (refer to OmniRe+DS results in Tab.
On the one hand, Gaussian training can infer depth leveraging spatial consistency, and correct inconsistent depth estimates 21 On the other hand, metric depth estimation, which aligns closely with image semantics, can provide smooth and fine-grained depth supervision, especially when training views are insufficient. To marry the advantages of both sides, we propose a joint optimization strategy that iteratively refines both the depth predictions and the Gaussian parameters. We perform iterative depth optimization as part of the Gaussian optimization process. For each depth optimization iteration, first identify the high-confidence region from the previously estimated depth map, then refine the dense depth with the geometric information guided by high-confidence regions, using current Gaussian geometry to condition the Depth Enhancer. We then update the supervision (Multiview) depth pool to guide the Gaussian learning process.
Identify the High-confidence Region. Inspired by methods handling uncertain inputs 33 34 we introduce confidence measure based on the consistency between the previously estimated depth map and the currently rendered depth. The high-confidence region reflects the consistency between Gaussian training and depth estimation, indicating high accuracy. Specifically, for each pixel in view i, we compute the relative difference between the current depth D. and the previous iteration's depth D The confidence map C₁ is formulated as:
C, {{ max( D.ur Ø D.rev, D.rey C D
Ac].
(I)
where Ø denotes element-wiss division, I is an element-wise indicator function that evaluates to 1 if the enclosed condition is true, and 0 otherwise.
Multi-view Depth Enhancer for Depth Refinement. We employ a Multi- Depth Enhancer module to refine the depth map, especially the low-confidence region. Inspired by the Marigold- DC 23 the proposed module leverages generative priors from the pre-trained monocular depth prediction model, iee., Marigold [32], by optimizing latent depth representations and scale-shift parameters through classifier-free guidance. However, it differs from Marigold-DC in the guidance accuracy: instead of relying on sparse ground-truth depth, our Depth Enhancer's 's diffusion steps guided by the potentially noisy depth estimates D, which are rendered by the Gaussian the current state. Therefore, we design three losses to suppress the potential noise and enforce multi-view geometric consistency to obtain more accurate estimates.
i) Reference loss Lyet- Similar to Marigold-DC that use sparse ground- truth depth to inject metric depth information, also the rendered depth reference signal. However, do not use it directly as supervision since the rendered depth D, is not reliable. Instead, we first downsample both the predicted depth D (the predicted depth map for view i at diffusion step k) and the reference depth D, to a patch level (size x p) with average pooling, denoted by J. This reduces the influence of local fluctuations. The loss then computes a combination of L1 and L2 distances between the downsampled maps, modulated by the confidence map C, from Eq. M
Las C O (|LD, - D.),,
+
D. DD
(2)
The oonfidence-wiihhted loss ensures that the refinement closely follows the reference depth in high-confidence regions, while uncertain areas, guides the generation of metric depth based on both the reference depth and the image semantics.
5

---

ii) Multi-view warping loss Ly. is also essential to ensure that the depth remains consistent across multiple viewpoints. Thus, we employ a projection mechanism to further guide the depth generation. Specifically, the depth warping operation W transforms the predicted depth map from another view j to the view of the reference frame using the given camera poses. Then we penalize cross-view inconsistency via an L1 loss between warped and predicted depths:
L = Σ
(D,)(D)) - -
j7i
The gradient of the warping loss enforces the geometry consistency between the predicted depth and the depth of the adjacent frames.
iii) Smooth loss smooth To promote smoothness in the refined depth map, we incorporate an edge-aware smoothness constraint on the predicted depth (()), conditioned on the ground-truth input image 19
1 HW
E(OoD) (z, e- + |o, (z, v) -10
Lamoohh
(4)
where 0₂ and 0g denote the spatial gradients in the horizontal and vertical directions, respectively. The weight e (x.v) reduces the influence of the smooth loss in areas with strong edge.
Finally, the total loss function guiding the diffusion optimization at each step is a weighted sum:
Laiuuse
Are + Aamooth Smmooth + X Lyw
(5)
During the Gaussian training phase, the Depth Enhancer iteratively refines the depth D pest every N iterations in order to balance time costs and effectiveness. More details of the iterative training scheme will be illustrated in the supplementary.
3.3 Optimization of Road Node in Gaussian Scene Graph
Roads are a key component of urban scenes and exhibit strong geometric priors, i.e. specific height that can be precomputed from the point cloud and known planar structure. Similar to [35] 10, 36), we also take advantage of the prior knowledge of roads to obtain a better depth/geometry on road region. Specifically, introduce specialized Road Node within our Gaussian scene graph structure, initializing it with all points identified as belonging to the road through segmentation. Then two geometry regularization are applied:
Position Constraint. We pre-compute the average height of the initial road point cloud and constrain the positions of the road Gaussians stay close to this reference height.
Lplane
Σ (u=))-)=²²
(6)
G
BrEG
where Gread is the set of Gaussians assigned the Road Node, Mlz.i denotes the 7-coordinate of the center Gaussian 9t. and B.g is the mean road height of the initial points on the road.
Planar Constraint. Also, the shape and orientation of individual Gaussians within the Road Node should constrained as a flat planar. Specifically, we encourage the Gaussian to be flat along the ground. Similar to [36], this constraint is formulated as,
Lahape
Σ G.Ga
(Dnormal ||, 1i.-20 22122 + Ana ||smin,=lli)
GG Grand
where Anormal and Atta are hyperparameters. For each Gaussian, Smin, denote the smallest scale and the normal n, derives from the the axis aligns with smallest scale. 2 is the normal of the ground. The overall loss for the Road Node the sum of these two components: Lroad
Lplane + L
3.4 Loss functions for Gaussian Training
In Gaussian training iterations, we follow the training loss of OmniRe [ except for the road loss:
L (1 Ar) L1 + LssIM + Lroad + Adopth Ldopth + Aopacity Lopacity + Liegs
(8)

---

Table 1: Comparison of image reconstruction performances Waymo NOTR Dynamic32 Dataset.
OmniRe+DS refers to OmniRe using DepthSplat's 's estimated depth for depth regularization.
Image reconstruction
Novel view synthesis
Inputs
Methods
PSNR
SSIM
LPIPS
PSNR f
SSIM
LPIPS
EmerNeRF'
28.16
0.806
0.228
25.14
0.747
0.313
3DGS
28.47
0.876
0.136
25.14
0.813
0.165
Img+LiDAR
S3GS
31.35
0.911
0.106
27.44
0.857
0.137
PVG
34.25
0.936
0.111
28.83
0.855
0.159
OmniRe
33.35
0.945
0.067
29.02
0.871
0.115
OmniRe w/o LIDAR
34.07
0.947
0.079
27.69
0.853
0.146
OmniRe+DS
31.26
0.932
0.089
27.37
0.854
0.136
Img
Ours
34.34
0.950
0.074
28.80
0.867
0.129
Table 2: Comparison of depth estimattion performances among multiple LiDAR-free methods and multi-view depth estimation method DepthSplat on Waymo NOTR Dynamic32 Dataset. All metrics are computed with sparse GT depth from LiDAR.
Inputs
Methods
L1
Abs. Rel. |
RMSE |
< 1.25
SppthSplat
8,7088
0.5412
14,6494
0,6020
OmniRe w/o LIDAR
4.2197
0.1977
8.3008
0,7212
Img
OmniRe+DS
5.5345
0,3137
8,5426
0.6422
Ours
2.6043
0.1313
4.6723
0.8404
where L1 and LSSIM are the LI and SSIM losses on rendered images, Ldepth compares the rendered depth of Gaussians with dense reference depth, Louwiit encourages the opacity of the Gaussians to align with the nky-sky mask, and Leeg represents various regularization terms applied to different Gaussian representations.
Experiments
4.1 Experimental Setup
Datasets. We conducted a detailed experimental analysis using the Waymo dataset [37. widely- used dataset for real-world driving scenarios. To evaluate our method's 's performance in dynamic urban environments, we selected the challenging Waymo dataset subset, NOTR Dynamic32, proposed by (27] We utilize data from the three frontal cameras (FRONT, FRONT RIGHT, FRONT LEFT)), resizing the images to resolution of 640 x 960.
Metries. We adopt PSNR, SSIM, and LPIPS metrics evaluate the quality of image reconstruction. Since our method does not use ground-truth point cloud depth for supervision, we employ LI, RMSE, Abs.Rel., and Threshold accuracy(o)
2 to assess the depth estimation performance. Specifically, these metrics are evaluated with the sparse ground-trutt depth obtained from LiDAR.
Baselines. We compare method against several Gaussian Splatting approaches: 3DGS (5). S3GS 77 PVG []. and OmniRe [T] and NeRF-based approach EmerNeRF [27]. However, these methods all require LiDAR point cloud for optimization. For fairer comparison, include two additional baselines, (1) OmniRe w/o LiDAR, which trains OmniRe without any depth regularization, and (2) OmniRe DS, which trains OmniRe using depth estimates from DepthSplat (21) in place of LiDAR depths. For more implementation details, please refer to the supplemental material.
4.2 Evaluation on Urban Driving Dataset
4.2.1 Image Reconstruction Performance
Quantitative results for image reconstruction and novel view synthesis, presented Tab. I demon- strate that our D2GS method consistently outperforms all other methods and is even comparable to methods that use ground-truth (GT) LiDAR. Notably, OmniRe [ without depth supervision shows competitive image quality training views. We infer that the absence of explicit geometric
7

---

GT w/ LiDAR
OmniRe
PVG
S3GS
RGB
Depth
GT w/ LiDAR
Ours
OmniRe w/o LiDAR
OmniRe + DS
RGB
No LIDAR
Depth
Figure 3: Comparison of image reconstruction and depth estimation performances on Waymo NOTR Dynamic32 Dataset with S3GS 77, PVG {8) OmniRe [ and LiDAR-free baselines. Zoom for better visual comparison.
constraints allows its Gaussian representation to overfit to the training views, resulting in poor gener- alization novel views (PSNR drops from 34.07 27 27).9). The results of OmniRe+DS also validates that using the inaccurate depth prediction as supervision will in turn degrade reconstruction quality.
As shown in Fig. D renders more coherent images with superior reconstruction of fine details like vehicle contours and building textures. In contrast, former methods with LiDAR supervision may suffer from quality degradation where the LiDAR is missing: the results of OmniRe [I] PVG [8] and S3GS 7 become blurry in the upper region. We can also observe the inaccurate depth rendering in the corresponding regions, S3GS 7 shows evident dividing line in the depth map. As for LiDAR-free methods, although Gaussian training can overfit to images in the training views, it suffers from distortion and floater artifacts in novel views. Visualizations of the corresponding depth maps further confirm that it is difficult to recover accurate depth either by relying solely on Gaussian training (OmniRe w/o LiDAR) on noisy depth predictions (OmniRe + DS).
4.2.2 Depth Estimation Performance
As shown in Tab. 2 D2GS achieves significant improvement on depth estimation accuracy compared to methods without GT LiDAR supervision. First, DepthSplat, which provides our initial depth estimates, shows inferior performance. This may be due to its inability to handle dynamic scenes. Moreover, compared to DepthSplat (21), the OmniRe DS method shows improvement, suggesting that Gaussian training has some capacity to correct noisy depth predictions. However, it still performs effective
worse than OmniRe w/o LiDAR. We infer that, while depth supervision serve regularizer for Gaussian training, inaccurate depth predictions may mislead the optimization process. In contrast, the depth accuracy of method achieves approximately 53% improvement over OmniRe + DS, validating the effectiveness of our proposed modules, given that the same initialization employed.
similar conclusion can be drawn from the Qualitative results (Fig. 3 as we have provided some analysis in Sec. 4.2. Our proposed method generates accurate dense depth maps, which offer

---

Table 3: Ablation study of our proposed key components and training strategy.
Components
Settings of D.E.
R.N.
P.P.
D.E.
Leef
Liwe
Lsmooth
Update
L1 4
Abs. Rel.
RMSE
1.25 t
Real-time
2.7580
0.1535
4.7748
0.8080
X
v
V
Real-time
2.9121
0.1491
5.1675
0.8041
V
x
X
Real-time
2.8190
0.1595
5.3086
0,8107
V
V
Real-time
2.6477
0.1488
4.6977
0.8163
v
Periodic
2.9168
0.1700
5.0385
0.7701
d
V
v
Real-time
2.3847
0.1303
4.2854
0.8526
w/o Road Node
wio Progroivv Prusing
wio Depth Enhancer
wil L &L
Periodic Update
Ours
GT Image
Dqpih
Figure 4: Visualization comparison of the proposed modules and training strategy. Zoom in for better comparison.
effective regularization and suppress floaters in regions where the depth data is missing unreliable. It is worth noting that, apart from the issue of missing upper LiDAR points, the Waymo dataset exhibits minimal reprojection and calibration emrrr,, it is meticulously calibrated dataset. However, such errors are common in real-world, industrial data, where our method will demonstrate more significant improvements. More visual comparisons provided in Supplementary.
4.3 Ablation Studies
Building upon the previous results, we perform ablation studies on Waymo subset (12 sequences, detailed in supplementary material) to assess the impact D2GS's key components: Progressive Pruning (P.P.). Depth Enhancer (D.E.), and Road Node (R.N.). We evaluate variants by removing each component individually, We further analyze the loss function and update strategy of the Depth Enhancer.
4.3.1 Network Components
Effectiveness of Progressive Pruning. Without progressive pruning (D²GS w/o P.P.), Gaussians are initialized directly from a dense point cloud. The results (Tab. B are worse because initial noise and outliers in the dense points lead to flawed Gaussian representations. Floaters arise when training from this setting (Fig. 4.. Our P.P. strategy successfully filters these initial points, creating a robust global geometry for the following reconstruction.
Removing the Depth Enhancer (D²GS w/o D.E.) degrades depth estimation accuracy and quality (Tab. 3). The D.E. iteratively refines depth rendered from current Gaussians using diffusion priors, establishing a crucial loop for geometric accuracy. Without it, the model struggles with insufficient geometric constraints, failing to capture fine geometric details and exhibiting depth errors and distortions.
Effectiveness of Depth Enhancer.
Effectiveness of Road Node Constraint. Excluding the Road Node constraint (D w/o R.N.) removes the strong ground plane prior, leading to decline depth metrics (Tab. 3 and qualitative artifacts such as uneven ground surfaces (Fig. 4). Inferring geometry for large, textureless surfaces like roads from multi-view images alone is challenging. Road Nodes enforce shape and normal constraints Gaussians in detected road areas, promoting smooth and continuous ground geometry.

---

4.3.2 Settings of Depth Enhancer
Loss Function. We ablate the D.E.'s loss function by removing its projection and smooth loss terms. As shown in Tab. 3 this degrades D.E. performance and overall reconstruction quality. These loss terms provide geometric information from the Gaussian with less error, enabling the D.E. to generate more consistent and accurate depth maps, which in turn provide more effective geometric supervision during iterative optimization, validating our design.
Update Strategy. We compare our adopted Real-time Update strategy for the D.E. with a Periodic Update strategy. In Periodic Update, depth maps for all training views are updated at the same iteration. In our Real-time Update, only a single depth map of one training view is rendered, enhanced, and used for depth regularization during depth enhancement iteration. Experimental results (Tab. 3) show that Real-time Update significantly outperforms Periodic Update. The reason is that Periodic Update suffers from using outdated depth for supervision, which leads to a conflict between Gaussian optimization and depth regularization. Real-time Update ensures the D.E. takes the latest Gaussian geometry as a conditioning signal, providing timely geometric constraints for Gaussian training.
5 Conclusion
In this paper, we presented D'GS, novel LiDAR-free free framework for reconstructing dynamic urban street scenes. By effectively leveraging geometric priors derived from images, D eliminates the dependency on LiDAR data, thus significantly simplifying data acquisition for autonomous driving applications. The core ideas of DCGS include: 1) An initialization strategy using multi-view depth estimation followed by a progressive training and pruning process, which efficiently generates compact and representative initial Gaussian set. 2) A joint optimization approach employing diffusion-basee depth enhancer, which iteratively refines both the depth estimates and the Gaussian representation, providing robust geometric supervision throughout training. 3) The integration of dedicated road node into the scene graph GS representation, explicitly modeling the ground plane using strong geometric priors. Extensive experimental results on the Waymo dataset validate the efficacy of our LiDAR-free reconstruction pipeline. While D2GS successfully eliminates the need for LiDAR, it still relies on accurate camera poses. In future work, will investigate pose- pose-free implementations to further simplify the pipeline and broaden its applicability.
References
[1] Z. Chen, Yang, Huang, de Lutio, J. M. Esturo, Ivanovic, O. Litany, Z. Gojeie, S. Fidler, M. Pavone et al., "Omnire: Omni urban scene reconstruction," arXiv preprint arXiv:2408. 16760, 2024.
[2] H. Zhu, Z. Zhang, Zhao, H. Duan, Y. Ding, Xiao, and Yuan, "Scene reconstruction techniques for autonomous driving: a review of 3d gaussian splatting," Artificial Intelligence Review, vol. 58, no. 1, p. 30, 2024.
[3] Z. Yang. Y. Chen, Wang, Manivasagam, W.-C Ma. A. J. Yang, and Urtasun, "Unisim: A neural llos sensor simulator," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 1389-1399.
[4] B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, Ramamoorthi, and R. Ng. "Nerf: Representing scenes as neural radiance fields for view synthesis," Communications of the ACM, vol. 65, no. 1, pp. 99-106, 2021.
[5] B. Kerbl, Kopanas, T. Leimkühlle,, and G. Drettakis, "3d gaussian splatting for real-time radiance field rendering." ACM Trans. Graph vol. 42, no. 4, pp. 1, 2023.
[6] B. Fei, J. Xu, R. Zhang, Q. Zhou, W. Yang, and Y. He, "3d gaussian splatting as new era: survey," IEEE Transactions on Visualization and Computer Graphics, 2024.
[7] N. Huang, X. Wei, W. Zheng, P. An, M. Lu, W. Zhan, M. Tomizuka, K. Keutzer, and S. Zhang, "S3gaussian: Self-supervised street gaussians for autonomous driving." arXiv preprint arXiv:2405.200 2032.,, 2024.
[8] Y. Chen, C. Gu, J. Jiang, X. Zhu, and L. Zhang, "Periodic vibration gaussian: Dynamic urban scene reconstruction and real- time rendering," arXiv preprint arXiv:2311.18500 2023.
[9] Y. Yan, H. Lin, C. Zhou, W. Wang, H. Sun, K. Zhan, X. Lang. X. Zhou, and S. Peng. "Street gaussians: Modeling dynamic urban scenes with gaussian splatting," in European Conference on Computer Vision. Springer, 2024, pp. 156- 173.
10

---

[10] M. Khan, H. Fazlali, D. Sharma. T. Cao. D. Bai, Y. Ren, and B. Liu, "Autosplat: Constrained gaussian splatting for autonomous driving scene reconstruction," arXiv preprint urXiv:2407.02598, 2024.
[11] X. Zhou, Z. Lin, X. Shan, Y. Wang, D. Sun, and M.-HI Yang, "Drivinggaussin: Composite gaussian splatting for surrounding dynamic autonomous driving scenes," " in Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, 2024, pp. 634-21 643.
[12] Y. Li and Ibanez-Guzman "Lidar for autonomous driving: The principles, challenges, and trends for automotive lidar and perception systems," IEEE Signal Processing Magazine, vol. 37, no. 4, pp. 50-61, 2020.
[13] S. Kim, "Därf: Boosting radiance fields from sparse inputs with monocular depth adaptation," Neural Information Processing Systems (NewrIPS), 2023.
[14] H. Xiong. S. Muttukuru, R. Upadhyay, P. Chari, and A. Kadambi, "Sparsegs: Real-time 360 sparse view synthesis using gaussian splatting," arXiv preprint arXiv:2312.00206 2023.
[15] Z. Zhu, Z. Fan, Y. Jiang, and Z. Wang. "Fsgs: Real-time few-shot view synthesis using gaussian splatting." " in European conference on computer vision. Springer, 2024, pp. 145-163.
[16] G. Wang, Z. Chen, C. C. Loy, and Z. Liu, "Sparsenerf: Distilling depth ranking for few-shot novel view synthesis," in Proceedings of the IEEE/CVF international conference computer vision, 2023, pp. 9065-900 9076.
[17] D. Xu, Y. Jiang, P. Wang, Z. Fan, Y. Wang. and Z. Wang, "Neurallift-360 Lifting an in-the-wild 2d photo to 3d object with 360deg views." " in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 4479-4489.
[18] J. Ii, J. Zhang, X. Bai, J. Zheng, X. Ning, 7hou, and L.. Gu, "Dngaussian: Optimizing sparse-vier 3d gaussian radiance fields with glohal-local depth normalization," in Proceedings of the IFEF/CVF conference on compuder vision and pattern recognition. 2024, pp. 20 775-2077 785.
[19] K. Deng, A. Liu, J. Zhu, and D. Ramanan, Deethh-upprriise nerf: Fewer views and faster training for free," in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 12 882-12 891.
[20] Y. Chen, H. Xu, C. Zheng, B, Zhuang, M. Pollefeys, A. Geiger, T.-J. Cham, and J. Cai, "Mysplat: Efficient 3d gaussian splatting from sparse multi-view images," in European Conference Computer Vision Springer, 2024, pp. 370-386.
[21] H. Xu. S. Peng. F. Wang. H. Blum, D. Barath, A. Geiger, and M. Pollefeys, "Depthsplat: Connecting gaussian splatting and depth," arXiv preprint arXiv:2410. 13862, 2024.
[22] X. Liu, C. Zhou, and S. Huang, "3dgs-enhancer: Enhancing unbounded 3d gaussian splatting with view-consistent 2d diffusion priors." Advances Neural Information Processing Systems, vol. 37, pp. 133 305-133 327, 2024.
[23] Viola, K. Qu. N. Metzger, B. Ke, A. Becker, K. Schindler, and A. Obukhov, "Marigold-de: Zero-shot monocular depth completion with guided diffusion," " arXiv preprint arXiv:2412.13338 2024.
[24] H. Turki, D. Ramanan, and M. Satyanarayanan, "Mega-nerf: Scalable construction of large-scale nerfs for virtual fly-throughs," in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp.
[25] M. Tancik, V. Casser, X. Yan, S. Pradhan, B. Mildenhall, P. P. Srinivasan, J. T. Barron, and H. Kretzschmar, "Block-nerf: Scalable large scene neural view synthesis." in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022. pp. 8248-8258.
[26] H. Turki, J. Y. Zhang, F. Ferroni, and D. Ramanan, "Suds: Scalable urhan dynamic scenes," in Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 12375-12338 385.
[27] J. Yang, B. Ivanovic, O. Litany, X. Weng, S. W. Kim, B. Li, T. Che, D. Xu, S. Fidler, M. Pavone et al., "Ememnerf: Emergent spatial-temporal scene decomposition via self-supervision." arXiv preprint arXiv:2311.022 102077, 2023.
[28] Ost, Mannan, N. Thuerey, J. Knodt, and F. Heide, "Neural scene graphs for dynamic scenes," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 2856- 2865.
[29] Z. Wu, T. Liu, L. Luo, Z. Zhong, J. Chen, H. Xiao, C. Hou, H. Lou, Y. Chen, R. Yang et al., "Mars: An instacce-wart modular and realistic simulator for autonomous driving," in CAAI International Conference on Artificial Intelligence. Springer, 2023, pp. 3-15.
[30] F. Liu, W. Sun, H. Wang, Y. Wang, H. Sun, J. Ye, J. Zhang, and Y. Duan, "Reconx: Reconstruct any scene from sparse views with video diffusion model," arXiv preprint arXiv:2408.1677 2024.
11

---

[31] G. Zhao, Ni, X. Wang. Z. Zhu, X. Zhang, Y. Wang, G. Huang, X. Chen, B. Wang. Y. Zhang et al., "DrivedreamerAd: World models effective data machines for 4d driving representation, arXiv preprint arXiv:2410. 13571, 2024.
[32] B. Ke, A. Obukhov, S. Huang, N. Metzger, R. C. Daudt, and K. Schindler, "Repurposing diffusion based image generators for monocular depth estimation." in Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, 2024, pp. 9492-99 9492-9502.
[33] R. Martin-Brualla, N. Radwan, M. S. Sajjadi, J. T. Barron, A. Dosovitskiy, and D. Duckworth, "Nerf in the wild: Neural radiance fields for unconstrained photo collections," in Proceedings of the IEEE/CVF conference computer vision and pattern recognition, 2021, pp. 7210-7219.
[34] Z. Xie, J. Zhang, W. Li, F. Zhang, and L, Zhang, "S-nerf: Neural radiance fields for street views," arXiv preprint arXiv:2303.00744 2023.
[35] Zhou, I.. Lin, Wang. Y. Lu, D. Bai, B. Liu, Wang, A. Geiger, and Y. Liao, "Hugsim: real-time, photo-realistic and closed-lnop simulator for autonomous driving," arXiv preprint arXiv: 2024.
[36] C. Peng, Zhang, Wang, C. Xu, Y. Xie, W. Zheng, K. Keutzer, M. Tomizuka, and W. Zhan, "Desire-gs: 4d street gaussians static-dynamic decomposition and surface reconstruction for urban driving scenes," arXiv preprint arXiv:2411.11922 2024.
[37] P. Sun, H. Kretzschmar, X. Dotiwalla, A. Chouard, V. Patnaik, P. Tsui, Guo, Y. Zhou, Y. Chai. B. Caine et al., "Scalability perception autonomous driving: Waymo open dataset," in Proceedings of IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 2446-2454.
[38] L. Piccinelli, C. Sakaridis, Y.H.. Yang, M. Segu, S. Li, W. Abbcloos, and L. Van Gool, "Unidepthv2: Universal monocular metric depth estimation made simpler," arXiv preprint arXiv:2502.201 10, 2025.
[39] W. Yin, C. Zhang, H. Chen, Z. Cai, G. Yu, K. Wang, X. Chen, and C. Shen, "Metric3d: Towards zero-shot metric 3d prediction from single image." in Proceedings the IEEE/CVF international conference computer vision, 2023, pp. 9043-9053.
12

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract and introduction can clearly reflect our contribution, includ- ing mentioning our motivation, introduction to our method, and brief summary of the experiment results.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We have mentioned the conclusion and supplementary material.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-specfiiatiinn asymptotic approximations only holding locally). ). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
13

---

Answer:
[Yes]
Justification:
All of the formulas have been checked by the authors.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: All the information needed for reproduction is provided in the Method, Experi- ments, and Supplemental Material.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
14

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
The code will be open-sourced after the company's 's review process.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
//nips //ipss.cc/pill wttpcccpppppppppeeeeeeeeeeeeeeee httaa//gguudee//eeeeabiiissssss /Cc////oddiiisssseeeeeee ssiiiiiisiyyyyyyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification:
We provide all the information in the supplemental material.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: We have tested our method on 32 sequences, which commonly used in the field.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
15

---

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out of range (c.g. negative emror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We have mentioned in the Supplemental Material.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
We have carefully checked the Code of Ethics, and our paper conforms it.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is no socictal impact of the work performed.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
16

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have cited all the papers referenced. And there Licenses problem.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
17

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
18

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA
that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
19

---

Supplemental Material
A Implementation Details
Initialization:
Our pipeline begins by generating initial depth estimates using DepthSplat [1] (tt5:d--2566444--nddeeee22-
which processes input images and camera parameters to produce per-view depth maps. These depth maps are then unprojected into 3D space using camera intrinsics and extrinsics to form dense point cloud. We initialize static Gaussians from this point cloud, and then perform low-resolution training at a resolution of 240 160 for three 3,000-iteration pruning epochs. Through our progressive pruning strategy, we gradually filter out less significant Gaussians based on opacity values (the threshold computed from percentiles of all Gaussians), ultimately retaining a subset of 10 Gaussians. To mitigate the impact of initial depth inaccuracies, we defer the activation of the Depth Enhancer module until the 1.000th iteration after the first epoch. The final rendered depth maps from this initialization phase (denoted as D serve as the foundation for subsequent full training processes. For Road Node initialization, we use the road mask identify the corresponding Gaussians and assign them to the Road Node.
(weight name:
Training Details: The model contains 60,000 iterations of training, divided into 40,000 iterations for RGB-only training and 20,000 iterations for iterative Gaussian and depth enhancement training. The reason why do not perform depth enhancement in the first 40000 iteration is that, the early training stages typically produce meaningless depth and image outputs. There are severe floaters and artifacts resulting from the aggressive densification process. We follow the default learning rate settings for Gaussian properties as established in OmniRe [ Our proposed Road Node's 's learning rate is 0.05. The Depth Enhancer module is activated halfway through training, with the diffusion process consisting of 80 steps and an early stop mechanism that prevents the smoothness loss from increasing for 8 continuous steps. For the warping loss, we utilize the views from 6 closest timestamp of the same camera ID. For quantitative comparisons, results for 3DGS, EmerNeRF, and S3GS are adopted from the S3GS 7 paper. All other methods were evaluated by us using their official codebases on single NVIDIA H20 GPU, Key hyperparameters for our method are detailed in Tab. 4
Table 4: Value of the key hyperparameters in our method.
Hyperparameter
Value
Description
p
Patch size for depth downsampling
Ac
2.0
Confidence threshold Depth Enhancer
Ave
1.0
Weight for reference depth loss Depth Enhancer
Asnooh
$
Weight for dopth smoothness loss in Depth Enhancer
A
1
Weight for warping loss in Depth Enhancer
20
Dopth Enhancer update frequency
Avernal
10.0
Weight for road normal alignment loss in Road Node
Adx
1.0
Weight for road flatness loss in Road Node
Anad
0.1
Weight for Road Node loss
20

---

B Datasets and Baselines:
Our main experiments are conducted on the Waymo NOTR Dynamic32 27 dataset, comparing our method against S3GS 77. PVG [8] OmniRe []] and LiDAR- baselines. The specific segment
IDs are listed in Tab. | Our ablation studies are performed on 12-sequence subset of the Waymo NOTR dataset, with segment IDs provided in Tab. 6
Table 5: Segment IDs of Waymo NOTR Dynamic32 Dataset.
seg (023 19...
sog103913...
seg104444...
seg104980...
sog 10918787...
sog 106250...
seg 106648...
seg109636..
seg 10170...
seg117188...
seg118463...
seg119178...
19g119284...
seg120278...
1212111....
seg122514...
seg 23392...
seg 148106...
seg 168016..
18g18111
seg 191876..
seg 2gg22593....
254789.
seg44
seg508351...
sog522233...
seg583504...
seg624282...
sog767010...
seg882250...
70g990779...
seg990914 14..
Table Segment IDs of 12 sequences subset used in our ablation experiments.
1e239291....
seg 1049800...
seg 105887...
seg109636...
seg11017U...
seg117188...
seg I 19284...
seg 12.1.244...
seg 123392...
seg 16860066...
seg 191876...
seg:411223..
21

---

C Additional Results
In this section, we present more qualitative comparisons of our methods with others. Same as the main submission, we show the results of the LiDAR supervised methods (S3GS (71, PVG [8) OmniRe (T. and LiDAR-free methods (OmniRe w/o LiDAR, OmniRe + DepthSplat, ours).
GT w/ LiDAR
OmniRe
PVG
S3GS
RGB
Depth
GT w/ LiDAR
Ours
OmniRe w/o LiDAR
OmniRe DS
RGB
No LiDAR
Depth
Figure 5: Comparison of our method with S3GS, PVG, OmniRe, and LiDAR free baselines.
GT w/ LiDAR
OmniRe
PVG
S3GS
RGB
Depth
GT wo LiDAR
Ours
OmniRe w/o LiDAR
OmniRe DS
RGB
No LIDAR
Depth
Figure 6: Comparison of our method with S3GS, PVG, OmniRe, and LiDAR-free baselines.
22

---

GT LiDAR
OmniRe
PVG
S3GS
RGB
Depth
GT w/o LiDAR
Ours
OmniRe w/o LiDAR
OmniRe DS
RGB
No LiDAR
Depth
Figure 7: Comparison of our method with S3GS, PVG, OmniRe, and LiDAR-free baselines.
GT w/ LiDAR
OmniRe
PVG
S3GS
RGB
Depth
GT w/ LiDAR
Ours
OmniRe wio LiDAR
OmniRe DS
RGB
No LIDAR
Depth
Figure 8: Comparison of our method with S3GS, PVG, OmniRe, and LiDAR-free baselines.
23

---

GT w/ LÍDAR
OmniRe
PVG
S3GS
RGB
Depth
GT w/ LiDAR
Ours
OmniRe w/o LiDAR
OmniRe DS
RGB
No LIDAR
Depth
Figure 9: Comparison of our method with S3GS, PVG, OmniRe, and LiDAR-free baselines.
GT w/ LiDAR
OmniRe
PVG
S3GS
RGB
Depth
GT w/o LiDAR
Ours
OmniRe w/o LiDAR
OmniRe DS
RGB
No LIDAR
Depth
Figure 10: Comparison of our method with S3GS, PVG, OmniRe, and LiDAR-free baselines.

---

GT w/ LiDAR
OmniRe
PVG
S3GS
RGB
Depth
GT wo LiDAR
Ours
OmniRe w/o LiDAR
OmniRe DS
RGB
No LiDAR
Depth
Figure 11: Comparison of our method with S3GS, PVG, OmniRe, and LiDAR-free baselines.
GT w/ LiDAR
OmniRe
PVG
S3GS
RGB
Depth
GT w/ LiDAR
Ours
OmniRe w/o LiDAR
OmniRe DD
RGB
No LIDAR
Depth
Figure 12: Comparison of our method with S3GS, PVG, OmniRe, and LiDAR-free baselines.
25

---

D Additional Experiments and Analyses
Summary. This section expands the computational analysis, clarifies assumptions about camera poses and our handling of dynamic objects, details sky modeling, strengthens ablations (including image-level metrics and road-node losses), visualizes depth evolution during training, and analyzes the initialization choices (SfM vs. MVS monocular metric depth).
Notation. We use the main main-paper notation throughout. "P.P." denotes Progressive Pruning, "D.E." denotes the Depth Enhancer, and "R.N." denotes the Road Node. NVS denotes novel-view synthesis.
D.1 Computational Analysis of Progressive Pruning
Dense depth initialization yields an extremely large set back-projeccie points. Training Gaussians these points is both redundant and error-prone, because abundant low-accuracy points often produce artifacts. P.P. therefore serves two goals: (i) reduce computation and memory by early removal of inconsistent Gaussians and (ii) deliver higher-quality initial point cloud that improves reconstruction.
Computational Analysis. Tab. profiles memory, runtime, and the evolution from initial to retained training points over the first 30k iterations. The results show that when the initial input point cloud contains as many as 15M points, the Gaussian model retains around 6M points that are not effectively pruned. In contrast, our P.P. method prevents significant waste of computational resources and avoids efficiency bottlenecks. Even with very large initial sets, P.P. can avoid out-of-memoon errors and shorten training by eliminating redundant, inconsistent Gaussians before full optimization, In contrast, the original pruning method prunes primarily during rendering optimization, which defers these savings.
Table 7: Analysis of computational cost, comparing P.P. method with baseline using manage- able number of points to demonstrate the efficiency gains.
Method
Initial Points
Training Points
GPU RAM
Runtime
w/o P.P.
15M
6M 8M
34 GB
10h
P.P.
2M
2M 33
20 GB
8h
Retained points used in optimization within the first 30k iterations.
D.2
The Dependency on Camera Puse Accuracy
Despite the LiDAR calibration issue which aim to solve in method, assume accurate camera poses, consistent with common urban driving scene reconstruction settings and dataset priors.
To further address inaccurate camera poses, common issue in data calibration, we propose the following possible solutions for future work. Specifically, when precise camera poses are unavailable, pose estimation algorithms, such learning-based methods or traditional SiM, can provide initial estimates. Subsequently, a joint optimization module for camera poses can be incorporated during reconstruction, enabling simultancous refinement of both the poses and the reconstructed scene.
D.3 Modeling Dynamic Objects
Our method can handle dynamic objects properly through the following aspects:
Scene graph decomposition. We model and decompose dynamic objects using Scene Graph, an approach that follows our baseline, OmniRe [.. Each dynamic object is represented as an independent node within this graph, equipped with its own set of Gaussian models (for dynamic or rigid representations).
Depth refinement for dynamics. Initial MVS depths inevitably contain errors on moving objects. First, since our method uses given ground-truth camera poses from Waymo, dynamic objects do not affect pose estimation. Depth errors are therefore localized to these dynamic areas, with minimal
26

---

impact on the static background. Furthermore, during the iterative optimization of the Gaussians and the D.E., the depth of dynamic objects is further refined (as shown in Fig. 13]. Specifically, this refinement is driven by both the photometric loss from Gaussian splatting and the accurate geometric supervision provided by our diffusion-based D.E.. This joint optimization loop is effective for both static and dynamic scene components, enabling correction of initial errors and more accurate depth prediction.
D.4 Detailed Ablations with Image-Level Metries
We complement geometric ablations with image reconstruction and NVS metrics, with results shown in Tab. [ The full model performs best across settings, corroborating Tab. 3 in the main paper: degraded depth quality typically yields poorer NVS. Notably, the Periodic Update variant attains high reconstruction quality but markedly worse NVS, suggesting overfitting to wraining-view photometry with stale depth. This also underscores the importance of our real-time iterative refinement.
Table Image-level metries for ablations (image reconstruction and NVS). "Update" denotes D.E. update scheme.
Components
Settings D.E.
Image reconstruction
Novel view synthesis
R.N.
P.P.
D.E.
Lint
Lammah
Update
PSNR
SSIM 1
LPIPS
PSNR
SSIM
LPIPS
Real-time
32.67
0,939
0.087
27.67
0.854
0.139
Real-time
32.25
0,932
0.099
27.17
0.847
0.149
Real-time
32.77
0.939
0.087
27.52
0.851
0,143
v
x
Real-time
32.57
0.940
0.086
27.29
0.852
0,140
Periodic
0.075
26.90
0.823
d
Real-time
33.63
0.947
0.079
28.04
0.863
0.130
D.5 Road Node: Loss Analysis
We ablate the two R.N. loss terms independently, and conduct experiments on a subset of Waymo dataset (sequence id: '031",'333 '084', '089', '102', '111', '222', '382'). As shown in Tab. [ both the loss Implane and the loss Lahupe contribute positively to a more accurate geometry (evaluated with depth results). The best performance is achieved when both are used together.
Table 9: Ablation of road-node losses on Waymo subset.
Method
L1
Abs. Rel. +
RMSE 4
8 < 1.25 1
w/o Lplane
2.7879
0.1395
5,0075
0.9270
w/o Lshape
2.6502
0.1211
5,0192
0.9320
Full loss
2.4245
0.1161
4.5535
0.9459
D.6 Depth Evolution Across Training
Fig. 13 visualizes the online refinement process by showing: (i) the Gaussian-rendered depth (D.E. input), (ii) the refined depth (D.E. output), and (iii) visualizations of this process at three checkpoints (initial, mid, and final training stages). We observe a clear removal of floating fragments, stabilization of large planar regions, and improved cross-view geometric consistency the refinement progresses.
D.7 Initialization: SfM vs. MVS vs. Monocular
In this section, we give more comprehensive analysis of different initialization methods. Existing depth estimation methods are inevitably imperfect and unsuitable to use as strict ground-trute for high-fidelity reconstruction. As discussed in the main paper, monocular methods often yield relative depth with unknown scale, while MVS methods can struggle dynamic scenes. Our core contribution is a framework that siarts from an imperfect metric depth prior and jointly optimizes geometry and depth. Metric initialization crucial for scale-correct point-cloud seeding and for strong, multi tettt-vvee--cossstee supervision.
27

---

Initial
Final
Figure 13: Depth evolution with online updates in three different stages: initial + final.
Recent monocular depth estimation foundation models (e.g., UniDepth V2 33 and Metric3D [39] also report strong metric depth prediction performance. We adopt DepthSplat-style MVS depth estimator in our main paper. Nevertheless, our pipeline is orthogonal to the depth source: any reasonable metric initializer can be used.
Monocular metric depth (UniDepth in place of MVS. We replace the DepthSplat MVS method with UniDepthV2 [38 to initialize our pipeline. We conduct experiments on a subset of Waymo dataset (sequence id: '031','00 ''03',, '084', "089", '102'. '111' '222', '382'), and Tab. I0 shows that our method still improves over baselines, confirming orthogonality to the initializer. Gains smaller than with DS, likely because UniDepth provides stronger initial depths and thus less headroom for refinement.
Why not SfM: a sparse point cloud initialization. We define dense initialization as back baok-projecting pixelwise metric depth maps into redundant but eeverage-complee point cloud. Sparse initialization denotes sampled point clouds (e.g., LiDAR, COLMAP), which provide only sparse supervision and may miss fine-scale or specific parts of the geometry.
In urban driving scenes, SfM still tends to be sparse due to limited overlap, textureless regions, and occlusions. Our contributions align naturally with dense initialization: P.P. compresses dense
28

---

Table 10: Monocular metric depth estimation as an initialization alternative on Waymo subset.
Image reconstruction
Novel view synthesis
Methods
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
OmniRe
33.96
0.955
0.057
28.50
0.874
0.117
OmniRe+DS
31.30
0.936
0.078
28.11
0.876
0.110
OmniRe+UniDepthh V2
32.73
0.944
0.068
29.67
0.885
0.098
Ours+DS
34.35
0.954
0.068
29.75
0.883
0.107
Ours+ Unusepnnitt
34.39
0.954
0.069
29.92
0.891
0.096
initialization priors before full optimization, and D.E. requires dense targets for per-pixel supervision. However, with minor changes (e.g., drop P.P. and seed D.F. with reprojected sparse depths), the pipeline still applies SfM sparse initialization.
29


Reviews:
[{"review_id": "2yc1ukhxij", "reviewer": "Reviewer_eDMK", "summary": "The submission presents a novel framework for urban scene reconstruction that does not require LiDAR data. Gaussian Splatting are a very active area of research and are often considered when thinking of scene reconstruction in autonomous driving scenarios. One of the limitations of these methods is that they require LiDAR data as well. However LiDAR data might be unavailable, not well synchronized with camera data, or the sensors' extrinsics might be decalibrated. The authors address this issue by proposing a camera-only method based on depth estimation.\nAuthors propose a method that uses multi-view depth estimation to generate a dense point cloud, then optimized with a pruning strategy proposed by the authors. The depth predictions and Gaussian representations are the refined using a diffusion-based module. The method also uses a road node in the scene graph to improve the accuracy of ground geometry by adding constraints. The method is evaluated on the Waymo dataset and shows improvements in image reconstruction and depth estimation compared to LiDAR-based methods.", "questions": "How do you see your method performing on other datasets than the one used? Especially with noisier data or more difficult weather conditions ?\n\nThe pruning strategy is presented as a solution to lower the computational cost. The strategy seems effective. Could you provide more details on the required resources and on how the computational complexity scales based on the size and complexity of the scene ?\n\nYou point LiDAR-camera calibration as an issue because of potential pose errors, but your method requires still accurate camera pose knowledge. Do you think this might be an issue? Do you have ideas to mitigate the impact of pose estimation on the reconstruction result?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\nGaussian Splatting are a very active area of research and promising method for scene reconstruction in the context of autonomous driving. However, existing methods require LiDAR data (which causes issues with spatial and temporal calibration to the camera). Proposing a LiDAR-free approach is thus interesting an impactful, as it could ease real-world use of these methods, and improve their robustness (authors show the low quality results of existing methods when LiDAR issues occur). Several innovative components are proposed, which together create a robust solution. Results on the Waymo dataset show improvements over the state of the art in image reconstruction and depth estimation, despite not using LiDAR. Experiments, quantitative and qualitative evaluations support the work, and results are compared with methods from the state of the art. Visualizations like Figure 3 help to better understand the significance of the improvements. The additional implementation details as supplementary material are also appreciated. \n\n## Weaknesses\nThis method still requires camera poses, which are not always available or of good quality (similar issue as the camera-LiDAR calibration mentioned by the authors). The code is not made available. While using multiple modalities (such as LiDAR) can be an issue because of calibration, synchronization, etc., sensor fusion is used very frequently in autonomous driving, and these problems can be mitigated, so I am not entirely convinced of the need of using cameras only, but I am however convinced of the interest of proposing such method.", "was_revised": true, "final_justification": "This work can be a strong contribution to the field. Authors answered attentively to the reviewers' questions and made significant efforts. Since I was already convinced, I don't feel the need to update my rating and will leave it at 5.", "rebuttal": "### **Dear Reviewer eDMK,**\n\nWe sincerely thank Reviewer eDMK for the insightful review and valuable feedback. In this rebuttal, we provide new experimental results on the nuScenes dataset, a detailed computational analysis of our pruning method, and a discussion on addressing the camera pose dependency.\n\n### **1\\. Q1. Performance of your method on other datasets, especially with noisier data or more difficult weather conditions.**\n\nThank you for this question. We agree that evaluating on a new dataset is necessary to verify the generalization of our method, and testing on a dataset with noisier LiDAR data would better highlight our method's effectiveness. Regarding a **\"noisier dataset\",** another reviewer (Cnm4) suggested we test on the nuScenes dataset. We believe this also aligns perfectly with your suggestion. Consequently, we have extended our method to nuScenes and compared it with OmniRe-based approaches. The results are presented in the table below.\n\nTab. A: Quantitative comparison of our method against OmniRe-based approaches on the nuScenes dataset. We select 8 sequences from the nuScenes dataset (**two of which are rainy scenarios**) due to the time limitations of the rebuttal phase. We will provide the full results in the revised manuscript.\n| **Methods** | **Image reconstruction** | | | | **Novel view synthesis** | | |\n| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n| | **PSNR**$\\uparrow$ | **SSIM**$\\uparrow$ | **LPIPS**$\\downarrow$ | | **PSNR**$\\uparrow$ | **SSIM**$\\uparrow$ | **LPIPS**$\\downarrow$ |\n| OmniRe | 30.12 | 0.866 | 0.117 | | 27.50 | 0.787 | 0.166 |\n| OmniRe + DS | 29.78 | 0.900 | 0.127 | | 26.60 | 0.678 | 0.189 |\n| Ours | 30.63 | 0.890 | 0.156 | | 27.22 | 0.724 | 0.175 |\n| |\n\nAs the results show in Tab. A, our method achieves better performance in both image reconstruction and novel view synthesis compared to the LiDAR-free baseline. Moreover, our method remains comparable to the LiDAR-based OmniRe in reconstruction and novel view synthesis. We attribute this to the fact that while the LiDAR data in nuScenes is not as high-quality as that in Waymo's, it remains a mature and widely-used dataset, and the discrepancy between its point clouds and the ture depth is still relatively minor compared to the errors in depth estimation. Due to the NeurIPS rebuttal limitations, we are unfortunately unable to provide visualizations at this time. We will add the experiments on nuScenes, along with corresponding qualitative and quantitative results to the final version of our manuscript to fully demonstrate our method's performance across different datasets and more challenging scenarios.\n\n### **2\\. Q2. Detailed computational complexity regarding our Progressive Pruning method.**\n\nThank you for this question and for recognizing the effectiveness of our Progressive Pruning component. The fundamental reason we proposed this strategy is that initializing from a dense depth map would involve an enormous number of back-projected points (approx. $cams×timestep×H×W$). Using these points directly for Gaussian training would be computationally redundant. Furthermore, an excess of low-accuracy points would lead to artifacts during subsequent training. Therefore, Progressive Pruning is not only a method for optimizing computation but also for providing a higher-quality initial point cloud to improve reconstruction results. Regarding your specific question on computational efficiency, we have profiled the resource usage with different numbers of retained points. \n\n\n**Table B:** Analysis of computational cost, comparing our Progressive Pruning method with a baseline using a manageable number of points to demonstrate the efficiency gains.\n| Method   | Initial Point num.   | Training Point num.* (first 30k iter) | GPU RAM | Runtimes |\n|:--------:|:--------------------:|:-----------------------------------:|:-------:|:--------:|\n| w/o P.P. | $1.5×10^7$                | $6×10^6$ - $8×10^6$                             | 34G     | 10h      |\n| P.P      | $2×10^6$                | $2×10^6$ - $3×10^6$                           | 20G     | 8h       |\n| |\n\nAs shown in Tab.B, when the initial input point cloud contains as many as $1.5×10^7$ points, the gaussian model will maintain around $6×10^6$ points that cannot be effectively pruned. In contrast, our Progressive Pruning method prevents a significant waste of computational resources and avoids efficiency bottlenecks. \n\n### **3\\. Q3. Do you think still requiring accurate camera pose knowledge might be an issue? Do you have ideas to mitigate the impact of pose estimation on the reconstruction result?**\n\nThank you for raising this point; we consider it to be very important. We acknowledge that accurate camera calibration is also a practical challenge in the autonomous driving scenarios. However, our method relies on accurate camera poses, which is a common assumption in the field of street view reconstruction and also a prior knowledge in existing datasets. Therefore,  in this work, we **mainly aim to address problems related to the LiDAR data modality**, such as 1\\) the absence of LiDAR data on vision-only autonomous vehicles, 2\\) projection errors from different mounting views, and 3\\) the inaccuracy of LiDAR calibration. \n\nRegarding how to mitigate the need for accurate camera poses, we have the following thoughts for future work. In the absence of precise poses, we could integrate pose estimation algorithms, such as learning-based methods (e.g., VGGT) or traditional SfM methods (e.g., COLMAP), to obtain initial camera poses. Subsequently, a joint optimization module for camera poses could be used during reconstruction, enabling simultaneous refinement of both camera poses and the reconstructed scene. We will add this discussion to the final version of our manuscript to more comprehensively explain the practical implications of our work and future directions.\n\n### **4\\. W1. The code is not made available.**\n\nFinally, regarding your comment on code availability, we commit to making our code publicly available upon acceptance of the paper.\n\nThank you again for your thorough review and constructive comments. They have been invaluable in helping us improve the quality and completeness of our manuscript."}, {"review_id": "zIyHKEOsqH", "reviewer": "Reviewer_Cnm4", "summary": "The paper proposes a LiDAR-free method for urban scene reconstruction based on Gaussian Splatting. It introduces a dense point cloud initialization method through metric depth estimation and progressive pruning, a joint optimization framework leveraging diffusion priors to refine both predicted and rendered depth maps, and a Gaussian model for road regions to encourage planar geometry. Experimental results show that the proposed approach outperforms existing baselines and performs better than using ground-truth LiDAR data, which can be prone to errors due to multi-sensor calibration and re-projection.", "questions": "1. How does this paper handle dynamic objects? How is dynamic object decomposed? What kind of gaussian model or nodes are used to represent dynamic objects? Does the paper use bounding box or self-supervised method such as PVG? I think this part is missing from the paper.\n2. Following the previous point, I am particularly interested in how the proposed method handles depth estimation for dynamic objects. If the rendered depth from Gaussians contains errors or noise especially in dynamic regions, how robust is the method to such artifacts? This is closely related to the type of dynamic object modeling used in the paper, which is currently not well described. \n3. The paper claims that LiDAR depth is not always reliable due to calibration or re-projection issues. Given this, I am not sure whether purely comparing the estimated depth directly against ground-truth LiDAR, as shown in Table 2 and Table 3, is fully justified. Instead, it might be more meaningful to report image-level reconstruction metrics (e.g., PSNR, SSIM, LPIPS) using both LiDAR-based and proposed depth to support the method.  \n4. Following the last point, can the author show the results of using proposed method w/o depth enhancer but with DepthSplat or adding an ablation table showing image recon/synthesis metrics of removing each proposed components? It would be interesting to see which components contribute more to the image qualities and especially how the proposed depth estimation contributes.", "limitations": "yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The paper is well written and easy to follow.\n2. It addresses an important and timely topic, demonstrating that Gaussian Splatting with estimated metric depths can outperform methods relying on ground-truth LiDAR data. This contribution is especially valuable for scenarios in autonomous driving where only camera data is available.\n\nWeaknesses:\n1. While the proposed method is promising, the main weakness of the paper lies in its limited experimental evaluation. Currently, results are only presented on the NOTR dataset. Given the paper’s claim of outperforming methods that rely on potentially inaccurate LiDAR data, it would be more compelling to include evaluations on the nuScenes dataset. To my knowledge, LiDAR calibration in nuScenes is generally less accurate than in the Waymo dataset. Validating the proposed method on nuScenes would strengthen the paper’s contribution and make the results more convincing.", "was_revised": true, "final_justification": "My questions have been adequately addressed, and the authors have provided additional evaluation results. Based on this, I am inclined to raise my score.", "rebuttal": "### **Dear Reviewer Cnm4,**\n\nWe sincerely thank Reviewer Cnm4 for the thorough review and constructive feedback. In this rebuttal, we provide new experimental results on the nuScenes dataset as you suggested, comprehensive ablation study results, and detailed clarifications on our method for handling dynamic objects.\n\n### **1\\. W1. Validating the proposed method on nuScenes would strengthen the paper’s contribution and make the results more convincing.**\n\nWe completely agree with your assessment. Adding a new dataset is essential for validating the generalization of our method.  Testing our method in scenarios with greater LiDAR noise is indeed an excellent way to highlight its effectiveness. However, while such data is common in industry, it is rarely available in public datasets. We thank you for suggesting the nuScenes dataset and for noting its less accurate LiDAR calibration. Accordingly, we have extended our method to nuScenes and compared it against OmniRe-based approaches. The experimental results are summarized below.\n\nTab. A: Quantitative comparison of our method against OmniRe-based approaches on the nuScenes dataset. We select 8 sequences from the nuScenes dataset (two of which are rainy scenarios) due to the time limitations of the rebuttal phase. We will provide the full results in the revised manuscript.\n| **Methods** | **Image reconstruction** | | | | **Novel view synthesis** | | |\n| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n| | **PSNR**$\\uparrow$ | **SSIM**$\\uparrow$ | **LPIPS**$\\downarrow$ | | **PSNR**$\\uparrow$ | **SSIM**$\\uparrow$ | **LPIPS**$\\downarrow$ |\n| OmniRe | 30.12 | 0.866 | 0.117 | | 27.50 | 0.787 | 0.166 |\n| OmniRe + DS | 29.78 | 0.900 | 0.127 | | 26.60 | 0.678 | 0.189 |\n| Ours | 30.63 | 0.890 | 0.156 | | 27.22 | 0.724 | 0.175 |\n|  |\n\nAs the results show in Tab. A, our method achieves better performance in both image reconstruction and novel view synthesis compared to the LiDAR-free baseline. Moreover, our method remains comparable to the LiDAR-based OmniRe in reconstruction and novel view synthesis. We attribute this to the fact that while the LiDAR data in nuScenes is considered less accurate than Waymo's, it is a mature and widely-used dataset, and its errors are still relatively minor compared to those from image-based depth estimation methods. Due to the NeurIPS rebuttal limitations, we are unfortunately unable to provide visualizations at this time. We will add the experiments on nuScenes, along with corresponding qualitative and quantitative results to the final version of our manuscript to fully demonstrate our method's performance across different datasets and more challenging scenarios.\n\n### **2\\. Q1. How does this paper handle dynamic objects?**\n\nThank you for pointing out the lack of clarity regarding our handling of dynamic objects. We will add a dedicated section in the final manuscript to elaborate on these technical details. In response to your specific questions:\n\n* **Dynamic Object Modeling:** Our approach to modeling and decomposing dynamic objects follows our baseline, OmniRe, by utilizing a Scene Graph. Each dynamic object is represented as an independent node within this graph, possessing its own set of Gaussian models (e.g., dynamic or rigid representations).  \n* **Depth Estimation for Dynamic Objects:** We acknowledge that initial depth estimates derived from MVS will inevitably contain errors for moving objects, a common challenge in depth estimation. First, since our method uses given camera poses from Waymo, dynamic objects do not affect pose estimation. Depth errors are therefore localized to these dynamic areas, with minimal impact on the static background. Furthermore, during the iterative optimization of the Gaussians and the Depth Enhancer, the depth of dynamic objects is further refined. Specifically, this refinement is driven by both the photometric loss from Gaussian splatting and the accurate geometric supervision provided by our diffusion-based Depth Enhancer. This joint optimization loop is **equally effective for both static and dynamic scene components**, allowing it to effectively correct initial errors and achieve accurate depth prediction\n\nWe believe a comprehensive visualization of the depth refinement process would best illustrate the optimization of depth for dynamic objects. As requested by Reviewer e8Wz, we will include such visualizations and a supplementary explanation in the final manuscript to demonstrate the rationale and effectiveness of our method.\n\n### **3\\. Q2. Why purely compare the estimated depth directly against ground-truth LiDAR, as shown in Table 2 and Table 3?**\n\nThis is a very insightful and reasonable point. We agree that using LiDAR as ground truth for depth evaluation can introduce errors, especially in real-world applications where LiDAR data may be faulty. However, for the Waymo dataset, we believe use as a reference is justified for two reasons:\n\n1. The LiDAR data in the Waymo dataset is meticulously calibrated and is widely recognized in the academic community as a high-quality dataset.  \n2. The potential **errors in the** **Waymo LiDAR data are minor compared to the errors produced by image-based depth estimation algorithms**. For instance, in most regions, LiDAR typically produces centimeter-level errors.\n\nTherefore, adopting Waymo's LiDAR as an **\"upper-bound”**  to evaluate the relative performance of different LiDAR-free methods is meaningful. Although the Waymo dataset cannot clearly reveal the impact of LiDAR calibration errors on the accuracy of reconstruction, it can still prove the effectiveness of our method in LiDAR-free setting, i.e. our method achieves performance comparable to LiDAR-based approaches and significantly outperforms other LiDAR-free methods.\n\n### **4\\. Q3. Adding an ablation table showing image recon/synthesis metrics of removing each proposed component?**\n\nThank you for this excellent suggestion. Regarding the results of a DepthSplat baseline without our Depth Enhancer, they are presented in Tab. 1 and Tab. 2 of the original manuscript. Additionally, we fully agree that providing image-level metrics in our ablation study is important for evaluating scene reconstruction and novel view synthesis quality. Our original ablation study emphasized geometric metrics because the ill-posed nature of sparse-view reconstruction can be misleading, and models can easily overfit to training views, yielding high PSNR scores that don't reflect true geometric accuracy. We now have updated our main ablation study to include PSNR, SSIM, and LPIPS, and will add the following table to our manuscript.\n\nTab. B: Image-level performance of all ablation study results in our original manuscript. \n| **R.N.** | **P.P.** | **D.E.** | **$L_{\\text{ref}}$** | **$L_w$** | **$L_{\\text{smooth}}$** | **Update** | **Image reconstruction** | | | **Novel view synthesis** | | |\n| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n| | | | | | | | **PSNR**$\\uparrow$ | **SSIM**$\\uparrow$ | **LPIPS**$\\downarrow$ | **PSNR**$\\uparrow$ | **SSIM**$\\uparrow$ | **LPIPS**$\\downarrow$ |\n| × | ✓ | ✓ | ✓ | ✓ | ✓ | Real-time | 32.67 | 0.939 | 0.087 | 27.67 | 0.854 | 0.139 |\n| ✓ | × | ✓ | ✓ | ✓ | ✓ | Real-time | 32.25 | 0.932 | 0.099 | 27.17 | 0.847 | 0.149 |\n| ✓ | ✓ | × | × | × | × | Real-time | 32.77 | 0.939 | 0.087 | 27.52 | 0.851 | 0.143 |\n| ✓ | ✓ | ✓ | ✓ | × | × | Real-time | 32.57 | 0.940 | 0.086 | 27.29 | 0.852 | 0.140 |\n| ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | Periodic | 33.95 | 0.951 | 0.075 | 26.90 | 0.823 | 0.128 |\n| ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | Real-time | 33.63 | 0.947 | 0.079 | 28.04 | 0.863 | 0.130 |\n|  |\n\nAs shown in Tab. B, the results indicate our full method achieves the best performance compared to variants where other components are removed, underscoring the effectiveness of our proposed modules. When combined with Table 3 in the main manuscript, the results show that, in most cases, poorer depth quality leads to inferior NVS performance. Notably, the 'Periodic Update' variant shows a significant drop in NVS performance compared to our full model, indicating it has overfitted to the training views' photometry with the out-dated depth information. This result underscores the importance of our proposed iterative refinement scheme for achieving both geometric accuracy and high-quality novel view synthesis."}, {"review_id": "BJpcot1pUM", "reviewer": "Reviewer_e8Wz", "summary": "This paper proposes a LiDAR-free Gaussian Splatting reconstruction method for urban scenes, named D^2GS.\nFirst, the authors employ a multi-view depth estimation network to generate dense initial point clouds and reduce their quantity through a progressive pruning strategy.\nSecond, using rendered depth as reference, they online-update a diffusion-based depth estimation model to produce scale-aligned metric depth for geometric supervision.\nFinally, to enhance road surface reconstruction capability, a road node regularization scheme is introduced.\nBoth quantitative and qualitative results demonstrate that the proposed method achieves state-of-the-art performance using only images as input.", "questions": "1. Although LiDAR point clouds are not available, it is still possible to generate an initial point cloud using SfM. In fact, most image-only GS methods use SfM point clouds for initialization. Could you explain why dense depth estimation is used for initialization instead of SfM point clouds?\n2. The optimization of road node includes two different loss functions, but there is no independent ablation for both. Can the impact of both on performance be further analysed quantitatively and qualitatively?\n3. The ablation study only reports depth-related metrics. However, for GS scene reconstruction, rendering performance is also important. Could you report the impact of each component on novel view synthesis (NVS) performance?\n4. One of the core contributions of this paper is the online-updated depth supervision. However, there is no visualization for this part. Could you provide visualizations of the estimated depth and rendered depth at different stages to demonstrate the effect of online updates?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. The proposed reconstruction method eliminates the need for LiDAR sensor data, reducing the data collection costs for urban scene reconstruction while avoiding the challenges associated with precise calibration in multi-sensor systems. This advancement contributes positively to promoting large-scale applications in the field.\n2. The writing and equations are clear and easy to understand.\n\nWeaknesses:\n\n1. The motivation for using estimated depth from multi-view as initialization is not clearly explained.\n2. The ablation experiments of some modules are not sufficient and their effects are not clear enough.", "was_revised": true, "final_justification": "The authors have adequately addressed my concerns regarding initialization and ablation studies. I believe the innovative method proposed in this paper contributes to improving the geometric representation of 3DGS in urban scenes, and I am therefore willing to raise my score.", "rebuttal": "### **Dear Reviewer e8Wz,**\n\nWe sincerely thank Reviewer e8Wz for the insightful review and valuable suggestions. In this rebuttal, we clarify our motivations and provide comprehensive ablation studies to quantitatively analyze each component's impact.\n\n### **1\\. W1 & Q1: Could you explain why dense depth estimation is used for initialization instead of SfM point clouds?**  \nThank you for this important question. We acknowledge that SfM is a common initialization strategy for many image-only GS methods. Our decision to use dense depth maps from a MVS approach instead of SfM was based on the following considerations for our specific problem setting:\n\n1. **Availability of High-Quality Camera Poses:** In our primary dataset (Waymo), high-quality, pre-calibrated **camera poses are already provided**. A key function of SfM is to solve for these poses from images, which is not only redundant in our case but also computationally intensive, especially for large-scale urban scenes with lots of images.  \n2. **Requirement for Metric Depth:** Our method is designed to leverage and refine dense geometric priors with a correct metric scale. Traditional SfM pipelines typically produce point clouds that have an arbitrary scale, which would require an additional, non-trivial step to align to a metric scale. Using an MVS-based method with known camera poses provides a more direct and efficient path to the dense, metric depth maps that serve as a strong geometric starting point for our framework.  \n3. **Orthogonality to Our Method:** our framework is orthogonal to the choice of initial depth source and can be generalized to any method that provides a reasonable metric depth initialization. \n\nIn future work, we plan to extend our method to a camera-pose-free setting. In that scenario, integrating a pose estimation module, such as a learning-based method or a traditional SfM approach, would be a necessary and valuable first step, followed by joint optimization of the poses during reconstruction. We will clarify this motivation in the revised manuscript.\n\n### **2\\. W2 & Q2: The independent ablation for two different loss functions of Road Node.**  \nThank you for this valuable suggestion. We agree that a more detailed ablation is needed to demonstrate the individual contributions of the two loss functions for the road node. We have performed this additional ablation study as follows.\n\nTab. A: Ablation study of the two loss functions for the road node. These results are based on 8 sequences from the Waymo dataset due to the time limitations of the rebuttal phase. We will provide the full results in the revised manuscript.\n| **Methods** | **L1**$\\downarrow$ | **Abs. Rel.**$\\downarrow$ | **RMSE**$\\downarrow$ | **$\\delta <$ 1.25**$\\uparrow$ |\n| :--- | :---: | :---: | :---: | :---: |\n| w/o $L_{plane}$ | 2.7879 | 0.1395 | 5.0075 | 0.9270 |\n| w/o $L_{shape}$ | 2.6502 | 0.1211 | 5.0192 | 0.9320 |\n| Final version | 2.4245 | 0.1161 | 4.5535 | 0.9459 |\n||\n\nAs shown in Tab. A, both the loss $L_{plane}$ and the loss $L_{shape}$ contribute positively to a more accurate geometry (evaluated with depth results). The best performance is achieved when both are used together. We will include these quantitative results and a corresponding qualitative analysis in the final version of the manuscript.\n\n### **3\\. W2 & Q3: The ablation study only reports depth-related metrics. Could you report the impact of each component on novel view synthesis (NVS) performance?**  \nThank you for this excellent suggestion. We completely agree that novel view synthesis (NVS) metrics are crucial for evaluating scene reconstruction quality. Our original ablation study emphasized geometric metrics because the ill-posed nature of sparse-view reconstruction can be misleading, and models can easily overfit to training views, yielding high PSNR scores that don't reflect true geometric accuracy. We now have updated our main ablation study to include PSNR, SSIM, and LPIPS, and will add the following table to the manuscript.\n\nTab. B: Image-level performance of all ablation study results in our original manuscript. \n| **R.N.** | **P.P.** | **D.E.** | **$L_{\\text{ref}}$** | **$L_w$** | **$L_{\\text{smooth}}$** | **Update** | **Image reconstruction** | | | **Novel view synthesis** | | |\n| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n| | | | | | | | **PSNR**$\\uparrow$ | **SSIM**$\\uparrow$ | **LPIPS**$\\downarrow$ | **PSNR**$\\uparrow$ | **SSIM**$\\uparrow$ | **LPIPS**$\\downarrow$ |\n| × | ✓ | ✓ | ✓ | ✓ | ✓ | Real-time | 32.67 | 0.939 | 0.087 | 27.67 | 0.854 | 0.139 |\n| ✓ | × | ✓ | ✓ | ✓ | ✓ | Real-time | 32.25 | 0.932 | 0.099 | 27.17 | 0.847 | 0.149 |\n| ✓ | ✓ | × | × | × | × | Real-time | 32.77 | 0.939 | 0.087 | 27.52 | 0.851 | 0.143 |\n| ✓ | ✓ | ✓ | ✓ | × | × | Real-time | 32.57 | 0.940 | 0.086 | 27.29 | 0.852 | 0.140 |\n| ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | Periodic | 33.95 | 0.951 | 0.075 | 26.90 | 0.823 | 0.128 |\n| ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | Real-time | 33.63 | 0.947 | 0.079 | 28.04 | 0.863 | 0.130 |\n|||\n\nAs shown in the Tab. B, each of our proposed components contributes to the final performance. When combined with Table 3 in the main manuscript, the results show that, in most cases, poorer depth quality leads to inferior NVS performance.  Notably, the 'Periodic Update' variant shows a significant drop in NVS performance compared to our full model, indicating it has overfitted to the training views' photometry with the out-dated depth information. This result underscores the importance of **our proposed iterative refinement scheme** for achieving both geometric accuracy and high-quality novel view synthesis.\n\n### **4\\. Q4: Could you provide visualizations of the estimated depth and rendered depth at different stages to demonstrate the effect of online updates?**  \nThis is an excellent suggestion. Visualizing the depth evolution process provides a clear and intuitive demonstration of our method's core mechanism. Due to the rebuttal format restrictions, we cannot include new figures here. However, the method figure in our main manuscript (Figure 2\\) already provides a snapshot of a single update step, showing the rendered depth from the Gaussians (D.E. input) and the refined depth map (D.E. output). This figure illustrates how our method corrects geometric inconsistencies like floating artifacts in a single step. We will add a new, dedicated figure in the final manuscript to show this process over multiple training stages. This visualization will clearly demonstrate how the depth maps are progressively refined from an initial noisy state to a coherent and accurate representation, powerfully illustrating the online update process."}, {"review_id": "Rm3Fq1QM58", "reviewer": "Reviewer_GUm1", "summary": "This paper present the a LiDAR-free framework for reconstructing dynamic urban street scenes named D^2GS. The key components of this paper involve: 1) using multi-view depth estimation to initialize the 3DGS; 2) employ the Depth-Enhancer to refines both the depth estimates and Gaussian geometric information ;3) apply several regularization items to model road region. The results on Waymo seem to validate the efficacy of the LiDAR-free reconstruction pipeline.", "questions": "*  Moreover, the adoption of multi-view depth estimation inherently struggles to handle dynamic objects, so what is implantation details of this method to estimate the dynamic image sequence.\n\n* As the sky region has the infinite distance which is difficult to initialize with the point cloud. Although I notice the  semantic mask in Figure.2 (main paper), how does the proposed method  model the sky?", "limitations": "yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "## Strengths :\n1) D2GS method consistently outperforms all other methods by a large margin in Table 1  and Figure 3 both in Image reconstruction and NVS quality.\n\n2) The lidar-free method designed for autonomous driving scenario have more practical application\n\n3) The diffusion-based Depth Enhancer Module is interesting and somehow innovative, which simultaneous improve the surprised signal (estimated depth) and 3D Gaussian presentation.\n\n4) The Road Node decomposition make sense and help improve reconstruction quality in qualitative results in Figure 4.\n\n## Weaknesses:\n* Lines 51–55: The motivation of this paper is not clearly presented, particularly regarding why monocular depth estimation is considered unsuitable for dynamic scenes. Existing approaches, such as UniDepth and Metric3Dv2, have already demonstrated strong performance in estimating depth in street-view scenarios. Furthermore, the reason for adopting a DepthSplat-based multi-view stereo (MVS) approach, instead of leveraging monocular depth estimation, remains insufficiently justified.\n\n* It is unclear why a heuristic threshold is used during initialization to reduce the number of Gaussians, given that 3D Gaussian Splatting (3DGS) inherently supports pruning based on radius, opacity, and scale. what is the problem if the proposed method apply the  ADC in vanilla 3DGS?\n\n* As a study focused on street scene reconstruction, an important component is how to handle the dynamic agents. However, I could not find relevant details regarding this issue either in the supplementary materials or in the main paper. \n\n* The paper merely attributes performance degradation to the use of DepthSplat-generated depth maps, without exploring alternative depth estimation methods such as monocular depth which seems to be insufficient to support the proposed method.\n\n\n\n* typos: Line 134  ‘..’   Line135  uing->using  Lin57 TO -> to", "was_revised": true, "final_justification": "I think the rebuttal have addressed my concerns and i would like to improve my rating", "rebuttal": "### **Dear Reviewer GUm1,**\n\nWe sincerely thank Reviewer GUm1 for the insightful review and valuable feedback. In this rebuttal, we provide new experiments using a monocular metric depth estimator to demonstrate our framework's robustness, alongside detailed clarifications on the motivation for our key design choices.\n\n### **1\\. W1 & W4: What is the motivation of the current depth initialization method? And what about alternative monocular metric depth estimation methods?**\n\nWe thank you for the important questions regarding our motivation for choosing depthsplat-based MVS as our initialization method. \n\n**From our perspective, existing depth estimation methods still produce imperfect results, making them unsuitable for direct use as ground truth in high-quality reconstruction tasks.** As we argued in L51-55, monocular depth estimation methods typically generate relative depth with an unknown scale, while stereo-based methods struggle with dynamic scenes. Therefore, we aim to design a framework to take the imperfect, metric depth maps as a starting point and jointly optimize both the scene geometry and the depth, which is also our core contribution. We choose **metric depth** specifically because it is essential for **initializing point clouds** at the correct scale and for enabling **strong, direct, and multi-view consistent geometric supervision** during training.\n\nSecond, with the rapid development of depth foundation models, recent works such as UniDepth and Metric3D as you suggested, have achieved impressive performance in metric depth estimation. We adopt a DepthSplat-like MVS approach mainly because the camera poses are known in our setting. However, we emphasize that our framework is **orthogonal** to the choice of initial depth source and can be generalized to any method that provides a reasonable metric depth initialization, whether it originates from MVS or monocular metric depth estimation.\n\nHere, we provide experiments using the state-of-the-art monocular metric depth estimator,  **UniDepthV2 \\[1\\]**, to initialize our pipeline instead of MVS. The results on the Waymo dataset are as follows:\n\n**Tab. A**: Supplementary comparison to Tab. 1 with quantitative results of different initialization and training methods. These results are based on 8 sequences from the Waymo dataset due to the time limitations of the rebuttal phase. We will provide the full results in the revised manuscript.\n| **Methods** | **Image reconstruction** | | | | **Novel view synthesis** | | |\n| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n| | **PSNR**$\\uparrow$ | **SSIM**$\\uparrow$ | **LPIPS**$\\downarrow$ | | **PSNR**$\\uparrow$ | **SSIM**$\\uparrow$ | **LPIPS**$\\downarrow$ |\n| OmniRe | 33.39 | 0.952 | 0.063 | | 27.89 | 0.864 | 0.133 |\n| OmniRe+DS | 31.99 | 0.943 | 0.070 | | 28.26 | 0.871 | 0.115 |\n| OmniRe+UniDepth | 32.87 | 0.949 | 0.062 | | 29.13 | 0.880 | 0.103 |\n| Ours+DS | 33.93 | 0.947 | 0.078 | | 29.25 | 0.879 | 0.121 |\n| Ours+Unidepth | 33.58 | 0.953 | 0.072 | | 29.42 | 0.888 | 0.111 |\n||\n\nAs shown in Tab. A, the results demonstrate that our framework can effectively leverage and refine the initial depth from a monocular method. This confirms that our proposed optimization pipeline can be generalized to other specific choice of initializer. However, the performance improvement of ours + UniDepthV2 is less significant than that of ours + DS, compared to the OmniRe counterpart. We infer that this is because UniDepthV2 provides more accurate depth estimation, leaving less room for further improvement by our method. We will add these experiments and this discussion to our revised manuscript.\n\n\\[1\\] *Piccinelli, Luigi, et al. \"Unidepthv2: Universal monocular metric depth estimation made simpler.\" arXiv preprint arXiv:2502.20110 (2025).*\n\n### **2\\. W2: What is the difference between our Progressive Pruning method and ADC in vanilla 3DGS?**\n\nThank you for this question, which allows us to clarify the critical role of our Progressive Pruning component and its fundamental differences from the native pruning within 3DGS. \n\n* **Distinct Goals:**   \n  1. The goal of ADC in vanilla 3DGS is to refine rendering quality **during** the optimization by splitting Gaussians in high-gradient areas and removing those with near-zero opacity. In contrast, our Progressive Pruning component is a pre-processing step designed to **(1) ensure computational feasibility** by efficient point pruning and **(2) establish a geometrically accurate and compact starting point** for the main training.  \n* **Why Progressive Pruning is Necessary:**  \n1. **Computational Feasibility:** Initializing from dense depth maps generates an enormous number of points. Attempting to load this point cloud into GPU memory for standard training pipeline **may cause a GPU OOM** error at the very first iteration. The experiment results in Table B show that loading 15 millions of points leads to the model maintaining around 6 million points during training that cannot be effectively pruned, indicating significant **point redundancy** and **a slowdown in training speed**.  \n2. **Geometric Distillation:** More importantly, Progressive Pruning is a scheduled, multi-stage pre-training process that acts as a Geometric Distillation mechanism. First, this process operates on the massive point cloud by **completely disabling densification operations** like splitting and cloning. The learning objective is focused exclusively on refining the geometry. Within this simplified framework, a Gaussian's opacity is optimized to reflect its consistency with the geometric constraints.  Second, the progressive filtering is guided by a final target point count, not a fixed threshold. As mentioned by L344 in our Supplementary Material, in each stage, we rank all Gaussians by their learned opacity and prune those that contribute least to a coherent geometric structure. This **adaptive, ranking-based selection** ensures that a compact and representative set of Gaussians is preserved **at a specific amount**.\n\n\n**Table B:** Analysis of computational cost, comparing our Progressive Pruning method with a baseline using a manageable number of points to demonstrate the efficiency gains.\n| Method   | Initial Point num.   | Training Point num.* (first 30k iter) | GPU RAM | Runtimes |\n|:--------:|:--------------------:|:-----------------------------------:|:-------:|:--------:|\n| w/o P.P. | $1.5×10^7$                | $6×10^6$ - $8×10^6$                             | 34G     | 10h      |\n| P.P      | $2×10^6$                | $2×10^6$ - $3×10^6$                           | 20G     | 8h       |\n| |\n\n\n### **3\\. W3 & Q1: How does this paper handle dynamic objects?**\n\nThank you for pointing out the lack of clarity regarding our handling of dynamic objects. We will add a dedicated section in the final manuscript to elaborate on these technical details. In response to your specific questions:\n\n* **Dynamic Object Modeling:** Our approach to modeling and decomposing dynamic objects follows our baseline, OmniRe, by utilizing a Scene Graph. Each dynamic object is represented as an independent node within this graph, possessing its own set of Gaussian models (e.g., dynamic or rigid representations).  \n* **Depth Estimation for Dynamic Objects:** We acknowledge that initial depth estimates derived from MVS will inevitably contain errors for moving objects, a common challenge in depth estimation. First, since our method uses given camera poses from Waymo, dynamic objects do not affect pose estimation. Depth errors are therefore localized to these dynamic areas, with minimal impact on the static background. Furthermore, during the iterative optimization of the Gaussians and the Depth Enhancer, the depth of dynamic objects is further refined. Specifically, this refinement is driven by both the photometric loss from Gaussian splatting and the accurate geometric supervision provided by our diffusion-based Depth Enhancer. This joint optimization loop is **equally effective for both static and dynamic scene components**, allowing it to effectively correct initial errors and achieve accurate depth prediction\n\nWe believe a comprehensive visualization of the depth refinement process would best illustrate the optimization of depth for dynamic objects. As requested by Reviewer e8Wz, we will include such visualizations and a supplementary explanation in the final manuscript to demonstrate the rationale and effectiveness of our method.\n\n### **4\\. Q2: How does this paper handle the sky?**\n\nThank you for this observation. We follow the standard practice used in OmniRe to model the sky. Specifically, we use a pre-processed sky-mask for calculating **opacity loss**, encouraging Gaussians in the sky region to become fully transparent. The appearance of the sky itself is **handled by a separate model** (e.g., a cubemap representation), which is decoupled from the 3D scene Gaussians. We will clarify this in the implementation details of our final manuscript."}]

Author rebuttal:

Meta-review: The paper received (A, BA, A, BA) with confidence (4, 5, 3, 5).
All four reviewers agree that this paper proposes a timely and impactful contribution to LiDAR-free urban scene reconstruction with Gaussian Splatting. The technical innovations, extensive experiments, and practical significance justify publication. Recommended for acceptance.
Decision: Accept (poster)