Title: D$^2$GS: Dense Depth Regularization for LiDAR-free Urban Scene Reconstruction
Venue: NeurIPS 2025
Accepted: yes
Keywords: Urban scenes reconstruction

Gaussian Splatting

Depth estimation
Abstract: Recently, Gaussian Splatting (GS) has shown great potential for urban scene reconstruction in the field of autonomous driving. However, current urban scene reconstruction methods often depend on multimodal sensors as inputs, $\textit{i.e.}$ LiDAR and images. Though the geometry prior provided by LiDAR point clouds can largely mitigate ill-posedness in reconstruction, acquiring such accurate LiDAR data is still challenging in practice: i) precise spatiotemporal calibration between LiDAR and other sensors is required, as they may not capture data simultaneously; ii) reprojection errors arise from spatial misalignment when LiDAR and cameras are mounted at different locations. To avoid the difficulty of acquiring accurate LiDAR depth, we propose ${D}^2GS$, a LiDAR-free urban scene reconstruction framework. In this work, we obtain geometry priors that are as effective as LiDAR while being denser and more accurate. $\textbf{First}$, we initialize a dense point cloud by back-projecting multi-view metric depth predictions. This point cloud is then optimized by a Progressive Pruning strategy to improve the global consistency. $\textbf{Second}$, we jointly refine Gaussian geometry and predicted dense metric depth via a Depth Enhancer. Specifically, we leverage diffusion priors from a depth foundation model to enhance the depth maps rendered by Gaussians. In turn, the enhanced depths provide stronger geometric constraints during Gaussian training. $\textbf{Finally}$, we improve the accuracy of ground geometry by constraining the shape and normal attributes of Gaussians within road regions. Extensive experiments on the Waymo dataset demonstrate that our method consistently outperforms state-of-the-art methods, producing more accurate geometry even when compared with those using ground-truth LiDAR data.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper present the a LiDAR-free framework for reconstructing dynamic urban street scenes named D^2GS. The key components of this paper involve: 1) using multi-view depth estimation to initialize the 3DGS; 2) employ the Depth-Enhancer to refines both the depth estimates and Gaussian geometric information ;3) apply several regularization items to model road region. The results on Waymo seem to validate the efficacy of the LiDAR-free reconstruction pipeline.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths :\n1) D2GS method consistently outperforms all other methods by a large margin in Table 1  and Figure 3 both in Image reconstruction and NVS quality.\n\n2) The lidar-free method designed for autonomous driving scenario have more practical application\n\n3) The diffusion-based Depth Enhancer Module is interesting and somehow innovative, which simultaneous improve the surprised signal (estimated depth) and 3D Gaussian presentation.\n\n4) The Road Node decomposition make sense and help improve reconstruction quality in qualitative results in Figure 4.\n\n## Weaknesses:\n* Lines 51–55: The motivation of this paper is not clearly presented, particularly regarding why monocular depth estimation is considered unsuitable for dynamic scenes. Existing approaches, such as UniDepth and Metric3Dv2, have already demonstrated strong performance in estimating depth in street-view scenarios. Furthermore, the reason for adopting a DepthSplat-based multi-view stereo (MVS) approach, instead of leveraging monocular depth estimation, remains insufficiently justified.\n\n* It is unclear why a heuristic threshold is used during initialization to reduce the number of Gaussians, given that 3D Gaussian Splatting (3DGS) inherently supports pruning based on radius, opacity, and scale. what is the problem if the proposed method apply the  ADC in vanilla 3DGS?\n\n* As a study focused on street scene reconstruction, an important component is how to handle the dynamic agents. However, I could not find relevant details regarding this issue either in the supplementary materials or in the main paper. \n\n* The paper merely attributes performance degradation to the use of DepthSplat-generated depth maps, without exploring alternative depth estimation methods such as monocular depth which seems to be insufficient to support the proposed method.\n\n\n\n* typos: Line 134  ‘..’   Line135  uing->using  Lin57 TO -> to\n\nQUESTIONS:\n*  Moreover, the adoption of multi-view depth estimation inherently struggles to handle dynamic objects, so what is implantation details of this method to estimate the dynamic image sequence.\n\n* As the sky region has the infinite distance which is difficult to initialize with the point cloud. Although I notice the  semantic mask in Figure.2 (main paper), how does the proposed method  model the sky?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-24"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a LiDAR-free Gaussian Splatting reconstruction method for urban scenes, named D^2GS.\nFirst, the authors employ a multi-view depth estimation network to generate dense initial point clouds and reduce their quantity through a progressive pruning strategy.\nSecond, using rendered depth as reference, they online-update a diffusion-based depth estimation model to produce scale-aligned metric depth for geometric supervision.\nFinally, to enhance road surface reconstruction capability, a road node regularization scheme is introduced.\nBoth quantitative and qualitative results demonstrate that the proposed method achieves state-of-the-art performance using only images as input.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The proposed reconstruction method eliminates the need for LiDAR sensor data, reducing the data collection costs for urban scene reconstruction while avoiding the challenges associated with precise calibration in multi-sensor systems. This advancement contributes positively to promoting large-scale applications in the field.\n2. The writing and equations are clear and easy to understand.\n\nWeaknesses:\n\n1. The motivation for using estimated depth from multi-view as initialization is not clearly explained.\n2. The ablation experiments of some modules are not sufficient and their effects are not clear enough.\n\nQUESTIONS:\n1. Although LiDAR point clouds are not available, it is still possible to generate an initial point cloud using SfM. In fact, most image-only GS methods use SfM point clouds for initialization. Could you explain why dense depth estimation is used for initialization instead of SfM point clouds?\n2. The optimization of road node includes two different loss functions, but there is no independent ablation for both. Can the impact of both on performance be further analysed quantitatively and qualitatively?\n3. The ablation study only reports depth-related metrics. However, for GS scene reconstruction, rendering performance is also important. Could you report the impact of each component on novel view synthesis (NVS) performance?\n4. One of the core contributions of this paper is the online-updated depth supervision. However, there is no visualization for this part. Could you provide visualizations of the estimated depth and rendered depth at different stages to demonstrate the effect of online updates?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a LiDAR-free method for urban scene reconstruction based on Gaussian Splatting. It introduces a dense point cloud initialization method through metric depth estimation and progressive pruning, a joint optimization framework leveraging diffusion priors to refine both predicted and rendered depth maps, and a Gaussian model for road regions to encourage planar geometry. Experimental results show that the proposed approach outperforms existing baselines and performs better than using ground-truth LiDAR data, which can be prone to errors due to multi-sensor calibration and re-projection.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The paper is well written and easy to follow.\n2. It addresses an important and timely topic, demonstrating that Gaussian Splatting with estimated metric depths can outperform methods relying on ground-truth LiDAR data. This contribution is especially valuable for scenarios in autonomous driving where only camera data is available.\n\nWeaknesses:\n1. While the proposed method is promising, the main weakness of the paper lies in its limited experimental evaluation. Currently, results are only presented on the NOTR dataset. Given the paper’s claim of outperforming methods that rely on potentially inaccurate LiDAR data, it would be more compelling to include evaluations on the nuScenes dataset. To my knowledge, LiDAR calibration in nuScenes is generally less accurate than in the Waymo dataset. Validating the proposed method on nuScenes would strengthen the paper’s contribution and make the results more convincing.\n\nQUESTIONS:\n1. How does this paper handle dynamic objects? How is dynamic object decomposed? What kind of gaussian model or nodes are used to represent dynamic objects? Does the paper use bounding box or self-supervised method such as PVG? I think this part is missing from the paper.\n2. Following the previous point, I am particularly interested in how the proposed method handles depth estimation for dynamic objects. If the rendered depth from Gaussians contains errors or noise especially in dynamic regions, how robust is the method to such artifacts? This is closely related to the type of dynamic object modeling used in the paper, which is currently not well described. \n3. The paper claims that LiDAR depth is not always reliable due to calibration or re-projection issues. Given this, I am not sure whether purely comparing the estimated depth directly against ground-truth LiDAR, as shown in Table 2 and Table 3, is fully justified. Instead, it might be more meaningful to report image-level reconstruction metrics (e.g., PSNR, SSIM, LPIPS) using both LiDAR-based and proposed depth to support the method.  \n4. Following the last point, can the author show the results of using proposed method w/o depth enhancer but with DepthSplat or adding an ablation table showing image recon/synthesis metrics of removing each proposed components? It would be interesting to see which components contribute more to the image qualities and especially how the proposed depth estimation contributes.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe submission presents a novel framework for urban scene reconstruction that does not require LiDAR data. Gaussian Splatting are a very active area of research and are often considered when thinking of scene reconstruction in autonomous driving scenarios. One of the limitations of these methods is that they require LiDAR data as well. However LiDAR data might be unavailable, not well synchronized with camera data, or the sensors' extrinsics might be decalibrated. The authors address this issue by proposing a camera-only method based on depth estimation.\nAuthors propose a method that uses multi-view depth estimation to generate a dense point cloud, then optimized with a pruning strategy proposed by the authors. The depth predictions and Gaussian representations are the refined using a diffusion-based module. The method also uses a road node in the scene graph to improve the accuracy of ground geometry by adding constraints. The method is evaluated on the Waymo dataset and shows improvements in image reconstruction and depth estimation compared to LiDAR-based methods.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\nGaussian Splatting are a very active area of research and promising method for scene reconstruction in the context of autonomous driving. However, existing methods require LiDAR data (which causes issues with spatial and temporal calibration to the camera). Proposing a LiDAR-free approach is thus interesting an impactful, as it could ease real-world use of these methods, and improve their robustness (authors show the low quality results of existing methods when LiDAR issues occur). Several innovative components are proposed, which together create a robust solution. Results on the Waymo dataset show improvements over the state of the art in image reconstruction and depth estimation, despite not using LiDAR. Experiments, quantitative and qualitative evaluations support the work, and results are compared with methods from the state of the art. Visualizations like Figure 3 help to better understand the significance of the improvements. The additional implementation details as supplementary material are also appreciated. \n\n## Weaknesses\nThis method still requires camera poses, which are not always available or of good quality (similar issue as the camera-LiDAR calibration mentioned by the authors). The code is not made available. While using multiple modalities (such as LiDAR) can be an issue because of calibration, synchronization, etc., sensor fusion is used very frequently in autonomous driving, and these problems can be mitigated, so I am not entirely convinced of the need of using cameras only, but I am however convinced of the interest of proposing such method.\n\nQUESTIONS:\nHow do you see your method performing on other datasets than the one used? Especially with noisier data or more difficult weather conditions ?\n\nThe pruning strategy is presented as a solution to lower the computational cost. The strategy seems effective. Could you provide more details on the required resources and on how the computational complexity scales based on the size and complexity of the scene ?\n\nYou point LiDAR-camera calibration as an issue because of potential pose errors, but your method requires still accurate camera pose knowledge. Do you think this might be an issue? Do you have ideas to mitigate the impact of pose estimation on the reconstruction result?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}