Unifying Appearance Codes and Bilateral Grids for Driving Scene Gaussian Splatting
Authors: Nan Wang
Lixing Xiao
Yuantao Chen
Weiqing Xiao
Pierre Merriaux
Lei Lei
Ziyang Yan
Saining Zhang
Shaocong Xu
Chongjie Ye
Bohan Li
Zhaoxi Chen
Tianfan Xue
Hao Zhao
Venue: NeurIPS 2025 poster ()
Keywords: Gaussian Splatting
Bilateral Processing
Autonomous driving simulation
Primary area: applications

Abstract: Neural rendering techniques, including NeRF and Gaussian Splatting (GS), rely on photometric consistency to produce high-quality reconstructions. However, in real-world driving scenarios, it is challenging to guarantee perfect photometric consistency in acquired images. Appearance codes have been widely used to address this issue, but their modeling capability is limited, as a single code is applied to the entire image. Recently, the bilateral grid was introduced to perform pixel-wise color mapping, but it is difficult to optimize and constrain effectively. In this paper, we propose a novel multi-scale bilateral grid that unifies appearance codes and bilateral grids. We demonstrate that this approach significantly improves geometric accuracy in dynamic, decoupled autonomous driving scene reconstruction, outperforming both appearance codes and bilateral grids. This is crucial for autonomous driving, where accurate geometry is important for obstacle avoidance and control. Our method shows strong results across four datasets: Waymo, NuScenes, Argoverse, and PandaSet. We further demonstrate that the improvement in geometry is driven by the multi-scale bilateral grid, which effectively reduces floaters caused by photometric inconsistency.

Unifying Appearance Codes and Bilateral Grids for Driving Scene Gaussian Splatting
Nan Wang's | Yuantao Chen', Lixing Xiao', Weiqing Xiao', Bohan Li3,4 Zhaoxi Chen', Chongjie Ye', Shaocong Xu Saining Zhang', Ziyang Yan Pierre Merriaux", Lei Lei", Tianfan Xue Hao Zhao 1,21
BAAI; AIRR, THU; SSTTU 'ETT(Ningbo); CUHK; 6 LeddarTech
High
(n) Appearance code
Camerao
Cameral
Global Affine Transformaiinn
uomeaaa Mar parnduin
(b) Bilateral Grid
Pisel-wit Imunstormaiion
CD 478
(c) Multi-Scale Bilateral Gird
ic
Transfommaionn
Inter-Camem Variation
Optimization friendly
Modeling capability
Figure 1: (a) Appearance codes rely on global affine transformations but have limited modeling capability, (b) Bilateral grids perform pixel-wise transformations, improving photometric consistency but are challenging to optimize. (c) The proposed multi-scale bilateral grid unifies appearance codes and bilateral grids, enabling patch-wise transformations.
Abstract
Neural rendering techniques, including NeRF and Gaussian Splatting (GS), rely on photometric consistency to produce high-quality reconstructions. However, real-world driving scenarios, it is challenging to guarantee perfect photometric consistency in acquired images. Appearance codes have been widely used to address this issue, but their modeling capability is limited, applied to the entire image. Recently, the bilateral grid was introduced to perform pixel-wise color mapping, but it is difficult to optimize and constrain effectively. this paper, we propose a novel multi-scale bilateral grid that unifies appear- ance codes and bilateral grids. We demonstrate that this approach significantly improves geometric accuracy in dynamic, decoupled autonomous driving scene reconstruction, outperforming both appearance codes and bilateral grids. This is crucial for autonomous driving, where accurate geometry is important for obstacle avoidance and control. Our method shows strong results across four datasets: Waymo, NuScenes, Argoverse, and PandaSet. We further demonstrate that the im- provement geometry is driven by the multi-scale bilateral grid, which effectively reduces floaters caused by photometric inconsistency. Our code is open-sourced sourced at https://bigcilengg. httpss//bbgiiiggg.guuu..olllleee githuhbioobillteerl--niii
single code is
bigcileng@gmail com
*Corresponding author: zhachao@air. wttpaaaaaarrtttnuuuaaaaa sesinhhu.eedu.co en
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Introduction
Neural rendering techniques, such as NeRFs
24 37 54
20
55
and Gaussian Splatting (GS)
14 2 6
53 28 have demonstrated significant potential in producing high-quality 3D reconstructions by leveraging photometric consistency. However, ensuring photometric consistency across images in real-world scenarios remains challenge, as lighting conditions, viewpoints, and camera settings
57
can vary considerably [23 56
15 36 8). In autonomous driving applications [46 41 35 3 17
13 34. these challenges are amplified due to the presence of multiple cameras capturing the same we observe that these variations wwhtther temporal (across different time steps) or inter-camera (across different viewpoints) can introduce significant discrepancies in image appearance. These inconsistencies pose difficulties in accurately reconstructing dynamic driving environments.
scene from different angles over time. As illustrated in the left half of Teaser. I
To address this issue, appearance codes B5
23 5 have been introduced to encode per-image information and assist in correcting the photometric discrepancies. These codes, while effective, have limitations in modeling capability since they apply a single transformation across the entire image, often neglecting local variations that may occur in complex scenes. For example, in dynamic scenes, such those encountered in autonomous driving environments, lighting and viewpoints can change across different camera angles and time frames. As shown in the right top of Teaser. [ a global affine transformation based single appearance code may not be sufficient handle such variations (In Teaser. NuScenes (a), the fence is blurred).
Recent advancements 38 10 have introduced bilateral grids to perform pixel-wise transformations, enabling improved photometric consistency by allowing more localized adjustments. However, these grids face significant challenges in optimization, they require complex constraints to avoid instability during training (As shown in Teaser. (b), bilateral grids fail to effectively optimize the reconstruction of large, complex scenes.). In this work, we propose a novel solution-a multi- scale bilateral grid-integrating the strengths of appearance codes and bilateral grids. As shown in Teaser. [ this new approach allows for patch-wise transformations. Interestingly, in extreme cases, the multi-scale bilateral grid naturally converges to either the bilateral grid or appearance code, depending on the scale. At the finest scale, the multi-scale grid behaves like a traditional bilateral grid, performing pixel-wise transformations to adjust for local variations. In contrast, at the coarsest scale, effectively reverts more global transformation, resembling the behavior of appearance codes. This flexible, scale-dependent approach offers the best of both worlds, providing localized fine-tuning where necessary, while maintaining global consistency when appropriate.
We demonstrate that the multi-scale bilateral grid significantly enhances the geometric accuracy of dynamic, decoupled driving scenes, which is essential for autonomous driving applications where precise geometry plays a crucial role in tasks like obstacle avoidance and path planning. This is evidenced by our evaluation
multiple widely used datasets, including Waymo (30), NuScenes (. Argoverse [40], and PandaSet [42). The corresponding Chamfer Distance (CD) values, displayed in the Teaser. [ highlight a marked reduction eгror and improvement in reconstruction quality. Additionally, our method reduces photometric inconsistency, mitigating the appearance of floaters and enhancing the overall realism of the scene.
The contributions of this paper are as follows: First, we introduce a novel mulli--cale bilateral grid that unifies appearance codes and bilateral grids, transitioning either of the two paradigms in extreme cases, thus enhancing both modeling capability and optimization efficiency. Second, we show that by addressing photometric inconsistencies, improves the geometric accuracy of dynamic, decoupled driving scene reconstructions. Third, we provide extensive benchmarking across four widely used datasets waaeeetttWymmm..ooe NuScenes, Argoverse, and PandaSet where our method outperforms previous approaches, showcasing notable improvements in both qualitative and quantitative results.
2
Related Works
2.1 Bilateral Grids and Appearance Codes
Recent advancements in bilateral grids and appearance codes have significantly influenced techniques for addressing photometric inconsistencies, which are particularly critical in dynamic environments such as autonomous driving. The bilateral grids, first introduced for real-time edge-aware image pro- cessing 4), enable efficient manipulation of spatial and intensity variations, forming the foundation

---

Inconsistiot Imper
Renderd Image:
Sliced Conticieets
Processed Image
Reconlrucion Loss
a(")
Fixiio
Slicing
Mult-scale
Canssian Soone Graph
Guidance Map
Bilatera
Coarse Cidd
Intermediate Grid
Refined Gimi
Affine Cransfomaation
Confficionts Fusion
Patch
Illuminaton
Patzis wise
Pixel wise
Consistma Ocuput
Raseliie
Overview of our method. We unify appearance codes with multi-scale bilateral grids. Initially, a coarse rendering is obtained from Gaussian scene graph. This rendered image is then processed by our multi-scale bilateral grids to perform detailed per-pixel color modeling, guided by tetinance-basede map through slice and fusion operations.
Figure 2:
for many modern techniques 12 44 58]. Extensions like deep bilateral learning [10] further en-
hance this framework by predicting local affine transformations bilateral space, achieving real-time image enhancement on cesouucce-cossraile cost volume refinement 44 have proven effective tasks ranging from high-dynamic-ranmy
devices. Similarly, bilateral-guided upsampling (3) and
imaging to stereo matching, demonstrating the versatility of bilateral representations. In the context of neural rendering, methods such as NeRF [31 I2 23 and Gaussian Splatting (GS) [56] 15] have achieved groundbreaking results in novel view synthesis but photometric inconsistency caused by varying illumination or transient occluders [X 56 remains a challenge. Techniques like bilateral guided
radiance field processing (38] address these issues by disentangling camera-specifie enhancements and reapplying them consistently in 3D space. WildGaussians [[3] integrates DINO-based appearance codes into 3D GS to robustly handle occlusions and dynamic lighting uncontrolled scenes. Cross- Ray NeRF [52) employs cross-ray feature covariance and grid-based transient masking to harmonize appearance variations and suppress occlusions in unconstrained image collections. Despite these strides, existing approaches often struggle to balance global consistency with localized adaptability. For instance, while appearance codes provide global adjustments, they lack the granularity model fine-grained variations [7]. Conversely, bilateral grids excel at pixel-wise transformations but are challenging optimize effectively (27). To address this issue, introduce multi-scale bilateral grid which unifies bilateral grids and appearance codes, facilitating high-quality dynamic autonomous driving scene reconstruction with enhanced modeling capability and optimization efficiency.
2.2 Autonomous Driving Simulation
Autonomous driving simulation has emerged critical tool for developing perception, planning, and control systems by generating diverse, realistic driving scenarios [41 26
45
19
33
17 47 9
Recent advancements focus photorealistic rendering. dynamic scene modeling. and multi-modal
sensor simulation
41
16
60
35
45
51]
50 43). For instance, MARS 41 employs a modular
NeRF-based framework to independently control static and dynamic scene elements, while Street Gaussian [45) achieves real-time urban scene rendering using explicit 3D Gaussian representations. DrivingGaussian [60] enhances dynamic scene reconstruction via composite Gaussian Splatting, ensuring occlusion accuracy and multi-camera consistency. NeuRAD 35
integrates sensor-specific
effects (e.g., rolling shutter, LiDAR beam divergence) to improve novel view synthesis. Holistic scene understanding and editing have also seen progress. HUGS 599 combines static and dynamic 3D Gaussians for real-time semantic parsing. ChatSim 339 enables language-driven scene editing with external asset integration. LiDAR data integration has advanced through methods like LiDAR- NeRF 322 which uses structural regularization for low-texture regions. HO-Gaussian [88 merges gaidd volumes with Gaussian Splatting to climinate ceneeture-foo--eere dependencies.

---

Despite progress, gaps persist between simulated and real-world data 33 49 48) AligmMiF 33 addresses LiDAR-camera misalignments via geometry- aliineed implicit fields, while RodUS decomposes urban scenes into static and dynamic components using 4D semantics to reduce arti- facts. Innovations in large-scale generation include InfiniCube [2]], which leverages sparse- voxel representations for unbounded dynamic scenes, and Omnire [5]. which reconstructs diverse dynamic objects (e.g., pedestrians) for human- vehiice interaction simulations. Furthermore, recent works like GaussianPro [6] and SplatAD [ explore to refine 3D GS for real-time rendering of dynamic scenes. GaussianPro [6] introduces progressive propagation for texture-less surfaces, and SplatAD [Σ]] models sensor-specific phenomena (e.g., rolling shutter, LiDAR intensity). In this work, aim to reconstruct high-quality driving scenes by unifying appearance codes and bilateral grids to enhance photometric consistency and modeling capability.
3 Methodology
3.1 Problem Formulation
We aim to reconstruct 3D scene representation G from multi- images and LiDAR depth maps, commonly available in autonomous driving datasets. Given a set images {Te,] and corresponding depths (Dc..) captured by N cameras over T time steps, formulate objective minimizing the discrepancy between rendered and observed images and depths:
Σ e,t
1o + An |ID Do.ell2)
min
where 1h and D are the rendered appearance and geometry of the scene G from camera c at time and An weight balancing the depth term.
However, dynamic driving scenes and varying camera properties cause inconsistent appearances, leading to geometric and texture artifacts. To address inconsistency, decompose each image Ie, into consistent Ce., (e.g., intrinsic scene colors and constant sensor adjustments like normalization exposure), and non-consistent. No.t (e.g., components-varying lighting conditions, different camera settings and diverse image ISP effects):
L Co,t + NN
(2)
We model No, as a non-linear transformation F(C.,)). Substituting into Eq. 2):
Te, = Ce.t + F(C.,))
(3)
Let E(Co,)) Ce, + F(Co.)), we can reformulate the optimization objective in Eq. (1) as:
min E(1E(CE, Ic. 1 Ao |IP -D..
(4)
This reformulation links photometric and geometric consistency optimization within joint objective, guiding C²,, adjustments by geometric constraints (D,,) This mitigates texture-geometry ambiguities (e.g., shadows road surfaces misinterpretable geometry changes). F(C..)) models transient
appearance, while C enforces consistent scene properties. Experiments (Sec. 4 Tab. T show this joint optimization reduces both photometric error and geometric drift comparison baselines.
3.2 Gaussian Splatting for Autonomous Driving Environments
Following prior methodologies 35], we represent autonomous driving environments using a hybrid scene graph, decomposing the scene into sky, background, and dynamic object models. Each dynamic object is represented by 3D model in canonical space and transformed to the scene with an associated sequence of SE(3) transformations. We derive the transformation matrixes from existing object detection pipelines or ground- -ttth annotations.
For the sky, we use an environment map to model sky color based on viewing direction, while static background is represented as set of semi-transparent 3D Gaussians. Each Gaussian characterized by a learnable opacity parameter E (0. I), mean position μ E RR3 and an anisotropic covariance matrix Σ E 3x3 which is parameterized by scale vector S E RR3 and rotation quaternion q E R.. Additionally, Spherical harmonics coefficients e IR are used to model appearance.

---

For the moving object models, we further distinguish between non-deformable (e.g., vehicles) and deformable objects (e.g. pedestrians). don-deformable objects {G E (I...., . nr h are optimized in their local coordinate space and transformed into the global world space via their pose T:
G
(5)
For deformable objects (G (L,...,,1z)) n. we employ a deformation network Fo (parameterized by Φ) to adapt the Gaussian representation based on latent variable e and time t:
G T (6. Fo FolG,
(6)
3.3 Multi-Scale Bilateral Grid for Appearance Enhancement
Solving the photometric correction problem formulated in Eq. (4) fundamentally depends on the ability effectively model the complex and spatially-varying photometric transformations present in real-world driving scenes. The function E.) in our formulation represents this crucial photometric enhancement process. To address this, we propose the Multi-Scale Bilateral Grid architecture, detailed in this section, as novel grounded solution for approximating (-) and achieving high- quality, consistent appearance enhancement.
3.3.1 Multi-Scale Bilateral Grid Architecture
To address limitations of existing methods in handling diverse photometric variations in driving scenes, we propose a multi-scale bilateral grid framework, Our framework achieves this unification by employing hierarchical pyramid of bilateral grids, organized across multiple scales. This multi-scale design is directly inspired by the nature of photometric inconsistencies in real-worke environments, which range from global scene-level changes to highly localized variations (e.g., from overall lighting shifts to fine texture-leww level shadows). By utilizing this grid hierarchy, our framework aims to capture and effectively correct photometric variations their corresponding scales, thus enabling a more comprehensive and spatially adaptable photometric correction.
The multi-scale bilateral grid transformation can be formulated as T = Ā I", where À represents a composite, scale-dependent photometric transformation. Crucially, this composite transformation constructed hierarchically, combining transformations learned by individual bilateral grids at different scales, progressing from coarse to fine. This staged composition is key to achieving scale-dependent adaptation to photometric inconsistencies and allows us move beyond the inherent limitations of single-scale methods. Further details are provided in the subsequent sections.
Our framework achieves this hierarchical representation using three-level bilateral grid pyramid (Fig. 2 (1) Coarse Level
1 X 12 grid) captures global scene structure and approximate global appearance codes; (2) Intermediate Level (4 x 4 x 2 x 12 grid) represents regional features and capture mid-range photometric variations; (3) Fine Level (8 x 12 grid) encodes local details and approximate pixel-wise bilateral grids within the multi-scale framework.
At each level I, the grid tensor is defined A E " (w (xxx(()) where Spatial Dimension D specifies
(H)(), W represents spatial resolution (height, width) at level 1, Guidance Dimension the guidance intensity levels at level and Coefficient Channel C = 12 represents a flattened 3 x affine color transformation matrix.
3.3.2 Guidance Map, Slice Operation, and Multi-Scale Fusion
To achieve adaptive and spatially-varying photometric correction, our multi-scale bilateral grid framework employs guidance map and slicing operation [4] to retrieve localized transformations, followed by hierarchical fusion strategy to combine transformations across scales.
We first derive suminance-based guidance map I (u, v) GrayScale( (u, v)) from the rendered image ("(u, v),, following the approach [4] 38 [0]. This map encodes spatial brightness variations like shadows and highlights, serving spatially-varyinn query
multi-scale grid. For each level 1 and pixel (u, v). the luminance from 1 used to perform a slicing operation, querying the grid's intensity dimension ( to locate an intensity bin. Affine transformation coefficients A (1)
i.kk around this bin are then combined via trilinear interpolation for level-specific transformation
5

---

(u, The slicing operation retrieves per- per-pixel transformations from each grid level, written as: (2, A (u, v)
Σ vd, Jj.k
(7)
To efficiently fuse transformations across scales, we utilize hierarchical function composition. A naive approach of full-resolution slicing at each scale is computationally expensive and redundant due to local photometric coherence. To address this, we employ downsampled guidance maps Io(() for each scale I, performing slicing to obtain low-resolution coefficient fields, which are then upsampled. This approach explicitly links scale-aware guidance to patch operations-the -th eeduced-resolution maps enable efficient spatial aggregation of photometric transformations while enhances efficiency (See ablation study in Tab. 5). The level-specific transformations T(,, decomposed into linear matrices M and translations T() are then sequentially composed from coarse fine produce the final composite transformation A. The refined image 1 (u, v) is obtained by applying A(u, v) to (u, v). with the fusion process expressed as:
..... = =(=((11) .... (T))
8
This hierarchical fusion decomposes the photometric transformation into scale-dependent residual refinements. Coarse scales capture global transformations, intermediate scales refine regional varia- tions. and fine scales address local details (Further elaboration in Sec. 4.3) Further details on the interpolation kernel and computational efficiency considerations are provided in Appendix A1.1.
Table 1: Ablation studies and comparisons four large-scale driving datasets. Our proposed method consistently outperforms OmniRe across all metrics and datasets. w/AC denotes appearance codes; w/BG denotes single bilateral grids (size of 16 16 8).
Reconstruction
Novel View Synthesis
Geometry
Dataset
Method
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
CD
RMSE |
Depth
OmniRe
28.92
0.833
0.295
26.40
0.761
0.311
1.482
2.785
0,540
OmniRe w/AC
28.95
0.835
0.293
26.44
0.759
0.315
1.378
2.760
0.519
Waymo
OmniRe w/B
28.19
0,831
0.292
21.51
0.743
0.333
1.523
2.798
0.492
Ours
29.23
0.836
0.289
26,55
0.762
0.310
0.989
2.744
0.477
OmniRe
26.37
0.837
0.209
23,74
0.733
0,232
1.458
3,420
0,110
N NuScenes
OmniRe w/AC
26.38
0.840
0.204
23.74
0.732
0,229
1.437
3.415
0.106
OmniRe w/BG
25.98
0.837
0.209
23.60
0.705
0.262
1.380
3,390
0.097
Ours
27.69
0,847
0.193
24.64
0.739
0,216
1.161
3,340
0.059
OmniRe
24.59
0,848
0.202
22.53
0.755
0,220
0.954
4.208
0.050
Argoverse
OmniRe w/AC
24.58
0,848
0.201
22.51
0.756
0,219
0.959
4.215
0.051
OmniRe w/BG
23.31
0.842
0.216
21.70
0.725
0.254
0.901
4,218
0.049
Ours
24.68
0,849
0.200
22.58
0.756
0.217
0.807
4.199
0.040
OnmiRe
30.20
0.903
0.219
27.49
0.835
0.240
0.503
2.874
0.018
OmniRe w/AC
30.20
0,903
0.220
0.841
0.242
0.496
2.871
0.020
Pandaset
OmniRe w/B
29.73
0.904
0.220
27.38
0.830
0.246
0.484
2.867
0.013
Ours
30.75
0.906
0.213
27.89
0.847
0.235
0.453
2.852
0.011
4 Experiments
4.1 Experimental Setup
Datasets. We evaluate on four autonomous driving datasets: Waymo 30 NuScenes [0 Argov- erse [40] and PandaSet 42), selected for their diversity in sensor configurations (LiDAR,, camera specifications), environmental conditions (lighting, seasons), and geographic locations. Identical model hyperparameters are used across all datasets to test generalization capability. Technical specifi- cations including camera counts and ego-vehicle view cropping details are provided in Appendix A2.
Baseline. Our focus is on modeling appearance variance across viewpoints, we build upon the open- source works DriveStudio (5), ChatSim [39], StreerGS [45] and test three approaches: appearance codes, standalone bilateral grids, and our multi-scale bilateral grids. This allows for direct comparison of how each method handles photometric inconsistencies across viewpoints.
Implementation details. We evaluate our method on many autonomous driving datasets to demon- strate its robustness. For each dataset, We train all the methods single NVIDIA L40 GPU and

---

Table 2: Comparison with different baseline approaches for scene reconstruction. Performance
is reported as the average over the nuScenes
scenarios.
Reconstruction
Novel View Synthesis
Geometry
Method
PSNR1
SSIM
IPIPS
PSNR
SSIM
LPIPS 4
CD L
RMSE
Depth
ChatSim 19
25.10
0.805
0.252
23.27
0.725
0.270
1.557
3.509
0.106
ChatSim (Ours)
27.04
0.231
24.67
0.735
0.249
1.236
3.412
0.053
StreetCSS 45
25,74
0.822
0.240
23.64
0.736
0.259
1.604
3.544
0.107
StreetCS (Ours)
27.90
0.836
0.219
25.10
0.747
0.238
1.272
3.458
0.055
compare the 3D reconstruction and novel view synthesis (NVS) results between camera and LiDAR data. Additionally, we ablate key components of our method and quantify their impact on both appearance and geometry quality.
Training and Dynamic Rendering.
1) Training. To optimize our multi-scale Gaussian scene representation, we employ a joint training strategy that minimizes composite reconstruction loss:
Lrecon = X, + AgCSSIM + AaCa >oCo
Furthermore, we introduce two regularization terms to enhance image fidelity: Adaptive Total Variation Regularization. This term encourages smoothness and reduces noise while preserving image details, and Circle Regularization Loss. This loss applies inverse transformation to the ground- truth images, preventing discrepancies and image quality degradation. As visually demonstrated in Fig. 5 these terms effectively improve image fidelity. Detailed are provided in Appendix A1.
2) Rendering. For dynamic rendering using our multi-scale bilateral grid framework, we employ a specific interpolation strategy when encountering novel test images, especially those simulating dynamic ISP conditions. We first conducts temporal proximity search to identify the most relevant grids. Following this, we perform scale-specific interpolation using the two nearest grids found.
Table 3: Evaluation on Challenging Scene Subsets. Results on systematically selected extreme
cases. Our method shows substantially improvements when photometric inconsistencies are severe.
Reconstruction
Novel View Synthesis
Geometry
Scene Type
Method
PSNR f
S
LPIPS
PSNR
SSIM 1
LPIPS
CD
RMSE
Depth
Challenging Socnes
OmniRe
26.84
0,847
0.239
24.46
0.758
0.263
0.650
3.176
0.063
scenes)
Ours
28.13
0.856
0.223
25.17
0.763
0.250
0.523
3.120
0.023
Night Scenes (4 scenes)
OmniRe
28.34
0.805
0.338
26.54
0.753
0.360
0.586
2.767
0.041
Ours
29.13
0,812
0,330
27.21
0.762
0.349
0,500
2,731
0,016
Sun Flare Soenes
OmniRe
26.68
0.866
0.195
24.98
0.796
0,212
0.781
3.073
0.079
(5 scenes)
Ours
28.65
0.879
0.173
25.84
0,803
0.192
0.593
2.994
0.022
4.2 Quantitative Evaluation
Reconstruction and Geometric Quality. We present comprehensive quantitative analysis in Tab. comparing our full model against the OmniRe and its variants with appearance codes (w/AC) and single bilateral grid (w/BG). Our method consistently sets state-of-the-ee across all four datasets in both appearance and, most critically, geometric metrics.
In terms of appearance fidelity, our model achieves the highest PSNR and SSIM scores. For example, on NuScenes, our method reaches a PSNR of 27.69, significant jump from 26.37 (OmniRe) and 26.38 (OmniRe w/AC). This demonstrates the superior ability of the multi-scale grid to model complex photometric variations.
However, the most striking result is the substantial improvement in geometric accuracy. Our method drastically reduces the Chamfer Distance, a key indicator of geometric fidelity. On the Waymo dataset, our model hits CD of 0.989, outperforming the best baseline (OmniRe w/AC, 1.378) by a remarkable 28.2% This trend holds across all datasets, with significant reductions in Chamfer Distance (CD), Root Mean Squared Error (RMSE), and Median Squared Depth Error (Depth). This result provides strong evidence for our central hypothesis: by accurately resolving photometric ambiguities, multi-scale approach effectively climinates geometric artifacts like "floaters" and
7

---

(a) Specular highlights
Incomplete Geometry
(d) Duplication Artifacts
(b) Motion Blur
(c) Low Light
Ground Truth
Baseline Ours
Baseline
Ground Truth
Baseline Ours
Baseline
Figure 3: Our Method vs. Baseline Methods on Waymo, NuScenes, Argoverse, and PandaSet.
(at)
CD 0.30
CD 0,502
(b)
0U35
CU
Figure 4: Our proposed framework (c) outperforms appearance codes (a) and single bilateral grids (b) by simultancously addressing optimization challenges and enhancing geometric modeling. This delivers significant geometric accuracy improvements (lower Chamfer Distance (CD) via multi-scale bilateral grid, reducing error and improving reconstruction) and reduced photometric inconsistency (fewer floaters) in dynamic, decoupled driving scenes. Yellow indicates high lidar error, while Purple indicates a low lidar error.
produces much cleaner, accurate 3D reconstruction. The single bilateral grid (w/BG), despite its high expressiveness, often fails to converge and even degrades performance, highlighting the optimization challenges that our multi-scale, coarse-to- fine framework successfully overcomes.
NVS. As shown in Tab. D our model's superior reconstruction quality translates directly to improved NVS. Our method consistently outperforms all baselines in NVS metries across the four datasets. For instance, on PandaSet, we achieve a PSNR of 27.89, surpassing the next best (OmniRe w/AC at 27.51). This indicates that the cleaner geometry and more consistent appearance learned by our model generalize better to unseen viewpoints, producing higher-fidelity renderings.
Generalization to Other Methods. To demonstrate the broad applicability of our approach, integrated our multi-scale bilateral grid into two other statc-of-the thhhh--r-rc-rrt methods, ChatSim [39 and StreetGS [45] The results NuScenes, presented in Tab. 2 are compelling. Our module brings substantial gains to both frameworks. For StreetGS, incorporating our method boosts the recon- struction PSNR from 25.74 to 27.90 and slashes the Chamfer Distance from 1.604 to 1.272. These significant improvements underscore that our multi-scale grid is not just a bespoke enhancement for one architecture but generalizable and powerful module for improving photometric and geometric consistency in Gaussian Splatting-be based scene reconstruction.
Performance on Challenging Scenarios. To validate robustness, we curated a subset of 18 chal- lenging scenarios with extreme photometric inconsistencies. As shown in Table 3 our method's advantages substantially more pronounced here than on the standard datasets. Across this entire

---

Analysis of Learned Affine Transformations. To gain deeper insight into how our model functions, we analyze the distribution of the learned affine transformations in Fig. 6 The histograms for the original single-scale bilateral grid (BG) exhibit distinct bi-modal, peaky subset, our model achieves +1.29 dB PSNR gain. This improvement is even more significant in specific difficult sub-categories; for instance, "Sun Flare Scenes," the gain reaches +1.97 dB. This demonstrates our model's strong effectiveness in resolving severe, real-world edge cases.
4.3 Qualitative Evaluation
Rendering Results. Fig. 3 and Fig. present visualization results of method scenes from
Waymo [30]. NuScenes [I]. Argoverse [40] and PandaSet [42], demonstrating its capability to handle diverse conditions that might otherwise lead inconsistencies.
Chrcle
Carcle Requlaizztion Loss
Adkptive
WoAdsplive Loss
Figure Visualizations of ablations on Circle Regularization loss and Adaptive TV loss.
Malti scale Bilaterral
Distribution of affline trains fransoomaaiio
SELF
Scate 1atte-0...2
Bilatcral Grid
Argoverse
NuScenes
Waymo
Pandusel
Visualization of Affine Transformations and Distributions. The first three rows display the affine transformation and corresponding distributions when applying the first level, the first two levels, and all three levels of our multi-scale scale approach. The final row compares these to results from single bilateral grid, demonstrating that the multi-scale bilateral grid achieves a coarse-to-fine, smooth image enhancement with more diverse processing capabilities.
Figure 6:
Configuration
Grid Params (M)
Training Time (h)
Testing time (FPS)
OmniRe w/AC
0.072
1.93
53
OmniRe w/BG
27.843
2.85
54/ 381
Ours
3.969
2.10
54 429
Table 4: Computational Efficiency Analysis. ? denotes render- ing with bilateral grid processing active.
distribution for each viewpoint. This suggests that the single grid learns limited set of dominant photometric corrections for each view, struggling to capture the full spectrum of variations. In stark contrast, the aggregated histogram from our multi-scale grid (across all levels and viewpoints) significantly flatter and more dispersed. This direct visualization of our framework's 's core strength. The coarse-level grid first establishes view-dependent baseline appearance, capturing the dominant global photometric shift (interestingly, its own histogram also peaky, but varies across views). Then, the medium and fine grids learn residual transformations in a coarse-to-fine manner,

---

correcting the errors from the preceding levels. This hierarchical, residual refinement process allows the model represent much broader and more diverse range of photometric transformations. The flatter histogram is empirical evidence of this enhanced representational power, which is crucial for achieving consistent, high-quality renderings across the diverse and challenging views encountered in autonomous driving.
4.4 Ablation Studies
4 As shown in Tab.
Tab. 5 and Fig. systematically evaluate component contributions through:
Guidance Map Resolution. We test dif-
ferent downsampling factors for the guid- ance map at each grid level. The results in
Settings
Recon.
NVS.1
CD
Tab. 5ta) show that a moderate downsam- pling of (2,2,1) from coarse to fine yields the best balance, achieving the highest NVS PSNR (24.64). Using the full resolu- tion (1,1,1) slightly degrades performance, likely due to overfitting to noise, validating our patch-based aggregation strategy.
Full model
27.69
24.64
1.161
(a) Guidance Map Resolution
(1,1,1)
27.62
24.34
1.271
(2.2.1)
27.68
24.64
1.170
(8.8.))
27.52
24.53
1.187
(16,16,8)
27.50
24.54
1.202
(b) Grid Size Combinations Single Grid (8.8.4) (4.4.2)+(8.8.4) (8.8.4 (2.2.1)+(444,2)
26.56
23.90
1.376
27.49
24.56
1.963
Grid Size Combinations. The combina- tion of grid sizes crucial. As shown in Tab. (])). removing the coarsest grid level (e.g.. using only (4,4.2)+(8,8,4)) leads to a dramatic increase in CD, highlight- ing the importance of the coarse grid for establishing global geometric consistency. Crucially, baseline with only a single fine-resolution grid (8,8,4) performs poorly, confirming that finer grids fail to converge without the stable, coarse-level initializa- tion our method provides.
27.57
24.57
1.940
27.33
24.42
1.213
(c) Loss Functions w/o circle regularization loss w/o adaptive total variance loss
27.47
24.52
1.164
27.64
24.61
1.169
Table 5: Ablations. Numbers in Guidance Map Reso- lution indicate downsample factors from coarse fine level, and numbers in Grid Size Combinations indicate grid size.
Loss Functions. Ablating our proposed regularization losses in Tab. () shows their contribution. Removing the circle regularization loss slightly degrades both reconstruction and NVS performance. The effect of the adaptive total variance loss is more subtle in metrics but visibly improves rendering quality by reducing noise, as shown Fig.
Efficiency. As detailed in Tab. our method strikes an effective balance between performance and computational cost. The single large bilateral grid (w/BG) leads massive parameter count (27.8M) and the longest training time (2.85h). In contrast, our multi-scale approach requires only 3.9M parameters and modest 9% increase training time (2. vs. 1.93h) compared to the appearance code baseline, while delivering substantially better results. During inference, our method maintains high frame rates, demonstrating its practicality.
5 Conclusion
We introduced novel multi-scale bilateral grid that unifies global appearance codes and pixel-wise bilateral grids into single, hierarchical framework. Our extensive experiments demonstrate that this approach not only effectively models and corrects complex photometric inconsistencies in autonomous driving scenes but, more importantly, leads to significant and crucial improvement in the geometric accuracy of the final reconstruction. By establishing this strong link between advanced appearance modeling and geometric fidelity, our work paves the way for more robust and reliable neural rendering systems in safety-critical, real-world applications.
Limitations. While our method shows significant improvements, some limitations remain. (1) The computational overhead, though much lower than a single large bilateral grid, still higher than simple appearance codes, which could be a consideration for resource-constrained scenarios. (2) Modeling extremely fast-moving or highly non-rigid objects where LiDAR and camera data misalign remains an open challenge.
10

---

References
[1] Holger Caesar, Varun Bankiti, Alex H Lang. Sourahh Vora, Venice Frin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom, nuscenes: A multimodal dataset autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11621-11631, 2020.
[2] Danpeng Chen, Hai Li, Weicai Ye, Yifan Wang. Weijian Xie, Shangjin Zhai, Nan Wang, Haomin Liu, Hujun Bao, and Guofeng Zhang. Pgsr: Planar-based gaussian splatiing for efficient and high-fidelity surface reconstruction. IEEE Transactions on Visualization and Computer Graphics, 2024.
[3] Jiawen Chen, Andrew Adams, Neal Wadhwa, and Samuel W Hasinoff. Bilateral guided upsampling. ACM Transactions on Graphics (TOG), 35(6):1-8, 2016.
[4] Jiawen Chen, Sylvain Paris, and Frédo Durand. Real-time edge-aware image processing with the bilateral grid. ACM Transactions on Graphics (TOG), 26(3):103-es, 2007.
[5] Ziyu Chen, Jiawei Yang. Jiahui Huang, Riccardo de Lutio, Janick Martinez Esturo, Boris Ivanovic, I. Litany, Zan Gojeie, Sanja Fidler, Marco Pavone, et al. Omnire: Omni urban scene reconstruction. arXiv preprint arX::22408. 16760, 2024.
[6] Kai Cheng, Xiaoxiao Long, Kaizhi Yang, Yao Yao, Wei Yin, Yuexin Ma, Wenping Wang, and Xuejin Chen. Gaussianpro: 3d gaussian splatting with progressive propagation. In Forty-first International Conference on Machine Learning, 2024.
[7] Frédo Durand and Julie Dorsey. Fast bilateral filtering for display of nigh-dynamic-range images. In Proceedings of the 29th annual conference on Computer graphics and interactive techniques, pages 257-266. 2002.
[8] Jian Gao, Chun Gu, Youtian Lin, Zhihao Li, Hao Zhu, Xun Cao, Li Zhang, and Yao Yao. Relightable 3d gaussians: Realistic point cloud relighting with brdf decomposition and ray tracing. In European Conference on Computer Vision, pages 73-89. Springer, 2024.
[9] Ruiyuan Gao, Kai Chen, Enze Xie, Lanqing Hong, Zhenguo Li, Dan Yeung, and Qiang Xu. Magicdrive: Street view generation with diverse 3d geometry control. arXiv preprint arXiv:3310.0661 2023.
[10] Michaël Gharbi, Jiawen Chen, Jonathan T Barron, Samuel W Hasinoff, and Frédo Durand. Deep bilateral learning for real-time image enhancement. ACM Transactions on Graphics (TOG), 36(4):1-12, 2017.
[11] Georg Hess, Carl Lindström, Maryam Fatemi, Christoffer Petersson, and Lennart Svensson. Splatad: Real-time lidar and camera rendering with 3d gaussian splatting for autonomous driving. arXiv preprint arXiv:2411. 16816,88666 2024.
[12] Wonbong Jang and Lourdes Agapito. Codenerf: Disentangled neural radiance fields for object categories. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 19949-129588 2021.
[13] Bu Jin, Yupeng Zheng. Pengfei Li, Weize Li, Yuhang Zheng, Sujie Hu, Xinyu Liu, Jinwei Zhu, Zhijie Yan, Haiyang Sun, et al. Tod3cap: Towards 3d dense captioning in outdoor scenes. In European Conference on Computer Vision, pages
Springer, 2024.
[14] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 139-), 2023.
[15] Jonas Kulhanek, Songyou Peng, Zuzana Kukelova, Mare Pollefeys, and Torsten Sattler. Wildgaussians: 3d gaussian splatting the wild. arXiv preprint 2iix::240770844777 2024.
[16] Bohan Li, Jiajun Deng, Wenyao Zhang, Zhujin Liang, Dalong Du, Xin Jin, and Wenjun Zeng. Hierarchical temporal context learning for camera-based semantic scene completion. In Furopean Conference on Computer Vision, pages 131-148. Springer, 2024.
[17] Bohan Li, Jiazhe Guo, Hongsi Liu, Yingshuang Zou, Yikang Ding, Xiwu Chen, Hu Zhu, Feiyang Tan, Chi Zhang, Tiancai Wang, et al. Uniscene: Unified occupancy-contric driving scene generation. arXiv preprint arXiv:2412.05433 2024.
[18] Zhuopeng Li, Yilin Zhang, Chenming Wu, Jianke Zhu, and Liangjun Zhang. Ho- gaussian: Hybrid optimization of 3d gaussian splatting for urban scenes. In European Conference on Computer Vision, pages 39-3..65 Springer, 2024.
11

---

[19] Carl Lindström, Georg Hess, Adam Lilja. Maryam Fatemi, Lars Hammarstrand, Christoffer Petersson, and Lennart Svensson. Are nerls ready for autonomous driving? towards closing the real-to-simulation gap. In Pruceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4461-4471, 2024.
[20] Junchen Liu, Wenbo Hu, Zhuo Yang. Jianteng Chen. Guoliang Wang. Xiaoxue Chen, Yantong Cai, Huan- ang Gao, and Hao Zhao. Rip-nerf: Anti-aliasing radiance fields with ripmap-encoded platonic solids. In ACM SIGGRAPH 2024 Conference Papers, pages 1-11, 2024.
[21] Yifan Lu, Xuanchi Ren, Jiawei Yang, Tianchang Shen, Zhangjic Wu, Jun Gao, Yue Wang. Siheng Chen, Mike Chen, Sanja Fidler, et Infimicube: Unbounded and controllable dynamic 3d driving scene generation with world-guided video models. arXiv preprint arXiv:2412.03954. 2024.
[22] Nicolas Märki, Federico Perazzi, Oliver Wang, and Alexander Dorkine-Hornung Bilateral space video segmentation. In Proceedings of the IEFE conference on computer vision and pattern recognition, pages 743-751, 2016.
[23] Ricardo Martin-Brudie Brualla, Noha Radwan, Mehdi SM Sajjadi, Jonathan T Barron, Alexey Dosovitskiy, and Daniel Duckworth. Nerf in the wild: Neural radiance fields for unconstrained photo collections. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7210-7219, 2021.
[24] Ben Mildenhall, Pratul Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes neural radiance fields for view synthesis. Communicutions of the ACM, 65(1):99-106, 2021.
[25] Thang-Am Anh-Quan Nguyen, Luis Roldão, Nathan Piasco, Moussab Bennehar, and Dzmitry Tsishkou. Rodus: Robust decomposition of static and dynamic elements urban scenes. In European Conference on Computer Vision, pages 122-130. Springer, 2024.
[26] James Okae, Bohan Li, Juan Du, and Yueming Hu. Robust scale-aware stereo matching network. IEEE Transactions on Artificial Intelligence, 3(2):244-255 2021.
[27] Sylvain Paris and Frédo Durand. A fast approximation of the bilateral filter using signal processing approach. In Computer Vision-ECCV 2006: 9th European Conference on Computer Vision, Graz, Austria, May 7-13, 2006, Proceedings, Part pages 568-580. Springer, 2006.
[28] Yinghao Shuai, Ran Yu, Yuantao Chen, Zijian Jiang, Xiaowei Song, Nan Wang. Jv Zheng, Jianzhu Ma, Meng Yang, Zhicheng Wang, et al. Pugs: Zero-shot physical understanding with gaussian splatting. arXiv preprint arXiv:2502. 12231 2025.
[29] Xiaowei Song, Jv Zheng, Shiran Yuan, Huan-ang Gao, Jingwei Zhao, Xiang He, Weihao Gu, and Hao Zhao. Sa- Scale-adaptic gaussian splatting for training free anti anii-aliasing. arXiv preprint arXiv:2403. 19615, 2024.
[30] Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, et al. Scalability in perception for autonomous driving: Waymo open dataset. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2446 2454, 2020.
[31] Matthew Tancik, Vincent Casser, Xinchen Yan, Sabeek Pradhan, Ben Mildenhall, Pratul P Srinivasan, Jonathan T Barron, and Henrik Kretzschmar. Block-nerf: Scalable large scene neural view synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8248-8258, 2022.
[32] Tang Tao, Longfei Gao, Guangrun Wang, Yixing Lao, Peng Chen, Hengshuang Zhao, Dayang Hao, Xiandan Liang, Mathieu Salzmann, and Kaicheng Yu. Lidar-nerf: Novel lidar view synthesis via neural radiance fields. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 390-398, 2024.
[33] Tang Tao, Guangrun Wang, Yixing Lao, Peng Chen, Jie Liu, Liang Lin, Kaicheng Yu, and Xiaodan Liang. Alignmif:: Geometry-aligned multimodal implicit field for lidar-camera joint synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1230-21240, 2024.
[34] Beiwen Tian, Mingdao Liu, Huan-ang Gao, Pengfei Li. Hao Zhao, and Guyue Zhou. Unsupervised road anomaly detection with language anchors. In 2023 IEEE international conference on robotics and automation (ICRA). pages 1778-7785. IEEE, 2023.
12

---

[35] Adam Tonderski. Carl Lindström, Georg Hess, William Ljungbergh, Lennart Svensson, and Christoffer Petersson. Neurad: Neural rendering for autonomous driving. In Proceedings the IEEE/CVT Conference on Computer Vision and Puttern Recognition. pages 14895-14904, 2024.
[36] Nan Wang, Xiaohan Yan, Xiaowei Song, and Zhicheng Wang. Semantic-guided gaussian splatting with deferred rendering. In ICASSP 2025-2025 IEEF International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1-5. IEEE, 2025.
[37] Peng Wang, Lingjie Liu, Yuan Liu, Christian Theobalt, Taku Komura, and Wenping Wang. Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. arXiv preprint arXiv:2 106. 10689, 2021.
[38] Yuehao Wang, Chaoyi Wang, Bingchen Gong, and Tianfan Xue. Bilateral guided radiance field processing. ACM Transactions Graphics (TOG), 43(4):1-13, 2024.
[39] Yuxi Wei, Zi Wang, Yifan Lu, Chenxin Xu, Changxing Liu, Hao Zhao, Siheng Chen, and Yanfeng Wang. Editable scene simulation for autonomous driving via collaborative Ilm-agents. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 15077-15087, 2024.
[40] Benjamin Wilson, William Qi, Tammay Agarwal, John Lambert, Jagject Singh, Siddhesh Khandelwal, Bowen Pan, Ratnesh Kumar, Andrew Hartnett, Jhony Kacsemodel Pontes, et al. Argoverse 2: Next generation datasets for self-driving perception and forecasting, arXiv preprint arXiv:2301..0444 2023.
[41] Zirui Wu, Tianyu Liu, Liyi Luo, Zhide Zhong, Jianteng Chen, Hongmin Xiao, Chao Hou, Haozhe Lou, Yuantao Chen, Runyi Yang, ct al. Mars: An instance-aware, modular and realistic simulator for autonomous driving. In CAAI International Conference Artificial Intelligence, pages 15. Springer, 2023.
[42] Pengchuan Xiao, Zhenlei Shao, Steven Hao, Zishuo Zhang, Xiaolin Chai, Judy Jiao, Zesong Li, Jian Wu, Kai Sun, Kun Jiang, et al. Pandaset: Advanced sensor suite dataset for autonomous driving. In 2021 IEEE international intelligent transportation systems conference (ITSC), pages 3095-3101 3101. IEEE, 2021.
[43] Weiqing Xiao, Hao Huang, Chonghao Zhong, Yujic Lin, Nan Wang, Xiaoxue Chen, Zhaoxi Chen, Saining Zhang, Shuocheng Yang. Pierre Merriaux, et al. Simulate any radar: contolute radar simulation via waveform parameter embedding. arXiv preprint arXiv:2506.03134 2025.
[44] Bin Xu, Yuhua Xu, Xiaoli Yang, Wei Jia, and Yulan Guo. Bilateral grid learning for steren matching networks. In Proceedings of the IREF/CVF Conference on Computer Vision and Pattern Recognition, pages 1 12497-12506, 2021.
[45] Yunzhi Yan, Haotong Lin, Chenxu Zhou, Weijie Wang, Haiyang Sun, Kun Zhan, Xianpeng Lang, Xiaowei Zhou, and Sida Peng. Street gaussians: Modeling dynamic urban scenes with gaussian splatting. In European Conference on Computer Vision, pages 156 173. Springer, 2024.
[46] Ziyang Yan, Wenzhen Dong, Yihua Shao, Yuhang Lu, Liu Haiyang. Jingwen Liu, Haozhe Wang, Zhe Wang. Yan Wang, Fabio Remondino, al. Renderworld: World model with superviee 3d label. arXiv preprint arXiv:2409. 1135,, 2024.
[47] Jiawei Yang, Jiahui Huang. Yuxiao Chen, Yan Wang, Boyi Li, Yurong You, Apoorva Sharma, Maximilian Igl, Peter Karkus, Danfei Xu, et al. Storm: Spatio-temporal reconstruction model for large-scale outdoor scenes. arXiv preprint arXiv:2501.0060) 00602, 2024.
[48] Xiuyu Yang, Bohan Li, Shaocong Xu, Nan Wang, Chongjie Ye, Zhaoxi Chen, Minghan Qin, Yikang Ding, Xin Jin, Hang Zhao, al. Orv: 4d occupancy-cone robot video generation. arXiv preprint 0oXiv:556603077 2025.
[49] Xiuyu Yang. Yunze Man, Jun-Kun Chen, and Yu-Xiong Wang. Scenecraft: Layout-guided 3d scene generation. In Advances in Neural Information Processing Systems, 2024.
[50] Xiuyu Yang, Shuhan Tan, and Philipp Krähenbühl. Long-term traffic simulation with interleaved autore- gressive motion and scenario generation. arXiv preprint arXiv:2506. 17213, 2025.
[51] Xiuyu Yang, Zhuangyan Zhang, Haikuo Du, Sui Yang, Fengping Sun, Yanbo Liu, Ling Pci, Wenchao Xu, Weiqi Sun, and Zhengyu Li. Rmmdet: Road-side multitype and multigroup sensor detection system for autonomous driving. arXiv preprint arXiv:2303.05203, 2023.
[52] Yifan Yang. Shuhai Zhang, Zixiong Huang. Yubing Zhang, and Mingkui Tan. Cross-ray neural radiance fields for novel-view synthesis from unconstrained image collections. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages
2023.
13

---

[53] Baijun Ye, Minghui Qin. Saining Zhang, Moonjun Gong, Shaoting Zhu. Zebang Shen, Luan Zhang. Lu Zhang, Hao Zhao, and Hang Zhao. Gs-oce3d: Scaling vision-only occupancy reconstruction for autonomous driving with gaussian splatting. arXiv preprint arXiv:2507.194 19451. 2025.
[54] Zehao Yu, Songyou Peng. Michael Niemeyer, Torsten monocular geometric cues for neural implicit surface reconstruction. Advances in neural information processing systems, 35:25018-250333 2022.
Sattler, and Andreas Geiger. Monosdf: Exploring
[55] Shiran Yuan and Hao Zhao. Slimmerf: Slimmable radiance fields. 2024 International Conference on 3D Vision (3DV), pages 64 74. IEEE, 2024.
[56] Dongbin Zhang, Chuming Wang, Weitao Wang, Peihao Minghan Qin, Haoqian Wang. Gaussian the wild: 3d gaussian splatting for unconstrained image collections. In European Conference on Compuler Vision, pages 341-359. Springer, 2024.
[57] Saining Zhang, Baijun Ye, Xiaoxue Chen, Yuantao Chen, Zongzheng Zhang, Cheng Peng, Yongliang Shi. and Hao Zhao. Drone-assisted road gaussian splatting with cross-view uncertainty. arXiv preprint mnXiv:2408. 15242, 2024.
[58] Zhuoran Zheng, Wenqi Ren, Xiaochun Cao, Xiaobin Hu, Tao Wang, Fenglong Song, and Xiuyi Jia. Ultra Ultra-high- high-definitton image dehazing via multi-guided bilateral learning. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16180 16189. IEEE, 2021.
[59] Hongyu Zhou, Jiahao Shao, Lu Xu, Dongfeng Bai, Weichao Qiu, Bingbing Liu, Yue Wang, Andreas Geiger, and Yiyi Liao. Hugs: Holistic urban 3d scene understanding via gaussian splatting. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages
2024.
[60] Xiaoyu Zhou. Zhiwei Lin, Xiaojun Shan, Yongtao Wang, Deqing Sun, and Ming-Hsuan Yang. Drivinggaus- sian: Composite gaussian splatting for surrounding dynamic autonomous driving scenes. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 21634-21643, 2024.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract clearly state the claims.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
They are discussed in the conclusion.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
15

---

Justification:
the proof' are in the methodology part.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: All the information are included in the experimental setup part in section 4.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
16

---

Answer:
[Yes]
The code is avaliable in the anonymous GitHub repository in last line of abstract.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm ec/
for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
//nips. //nips.cc/ppl cc/public/guu httgggidddee//eebbbiiinnne
for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
All the information are included in the experimental setup part in section 4.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification:
All results are accompanied by statistical significance tests.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
17

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
They are described in the implementation details part in Section
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics http:///wwrppccccaaaaa/a//aaaaaa
Answer: [Yes]
Justification: [NA]
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We have discussed both positive and negative impacts.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
18

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
We identify our paper as having no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have given the creators or original owners of assets used in the paper proper credits.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the Datasets.papersstt paperswi thcode. cww/ datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
19

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: We have not released new assets at the submission time. We will carefully document our data and model when we release the code and data.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
20

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: The paper does not involve LLM.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to L.LM policy hateap//meerrpppc.cc///ofeeerere for what should or should not be described.
21

---

Appendix:
Unifying Appearance Codes and Bilateral Grids for Driving Scene Gaussian Splatting
A Additional Implementation Details
A.1 Loss Functions and Optimization
We jointly optimize our multi-scale Gaussian scene representation by minimizing a composite loss function Liotal:
Listal Lrecon + λTV + AcireteCeircle
(A1)
where Crecon is the primary reconstruction loss, and LTv and Ceircle are regularization terms.
A.1.1 Reconstruction Loss
The core reconstruction loss, Lrecon: drives the accurate reproduction of both RGB and depth informa- tion:
Lreom Xr + (1 Xr) LSSIM + daha + XaC.
(A2)
Here, L₁ and LSIM measure the image-space difference. La represents the loss between the rendered depth and the ground truth LiDAR depth. Co is an opacity regularization term that encourages alignment with a non-sky mask.
A.1.2 Adaptive Total Variation (TV) Regularization
To encourage smoothness and reduce noise in the optimized grid representations, we incorporate TV regularization term, LTv, at each level I the multi-scale representation:
CTV Σ (0)
Σ
(1..k))))))
(A3)
L.ihh (E(.y..)
The adaptive weight k:()) for each level I is proportional to the grid size, applying stronger smoothing to finer, higher-resolution grids and lighter regularization to coarser grids.
h av H w()) - +
(A4)
A.1.3 Circle Regularization for Photometric Consistency
To constrain the noise introduced by photometric corrections and prevent overly aggressive alterations, we introduce circle regularization loss, Leintte:
Leirete
Σ (u, - (u, v)))
(A5)
(u,))s
where denotes the of pixel coordinates. This loss encourages the rendered image I" to be reconstructible from the ground truth image 19t via an inverse appearance transformation A-1
A.1.4 Coarse-to-Fine Optimization Strategy
We employ coarse-to-fine optimization strategy by utilizing level-dependent learning rates (e.g., 1 x 10-5.3 3,3 x 10-5 10-1 from coarse to fine). Coarser grids, assigned higher learning rates, rapidly learn the global scene illumination, while finer grids, with lower learning rates, hierarchically refine high-frequency photometric details. This staged optimization enhances stability.
A.2 Dynamic Rendering for Real-World ISP Adaptation
To bridge the domain gap between the fixed ISP used during training and dynamic real-world ISP pipelines encountered during inference, we employ an interpolation strategy. For a novel test image with timestamp fnoel, proceed as follows:
Temporal Proximity Search: We identify the two training timestamps from the same camera, t and fg, that are temporally closest to toovll (such that t S tnoeel S tz).

---

2. Scale-Specific Grid Interpolation: We perform linear interpolation specifically on the coarse and medium-scale bilateral grids (A4 Aa)) derived from these two timestamps.
The interpolated grid A for E {0, 1) is formulated as:
A A +
w A
(A6)
The temporal interpolation weight, w, is determined by proximity:
l/ nooell
(A7)
12 4
The fine-scale grid (1 = 2) is not used during interpolation, it is primarily intended to capture scene-intrinsic details rather than global ISP variations.
B
Dataset Details
This section provides datasct-specific details regarding our evaluation protocol, including sequence IDs for reproducibility.
B.1 Waymo Open Dataset
2
We use five cameras and all LiDAR sensors. We conduct our experiments on the following sequences, selected according to 4
8ggment-1108322271144222233
$ggmant-1500087777/99993333
segment-58350049442 segment--633044444000001311
B.2 NuScenes [1
We utilize all six available cameras and all LiDAR sensors. We seleet the following 8 sequences: 152, 164, 171, 200, 209, 359, 529, 916, which is extension of the dataset used by [3]. To address ego-vehicle visibility, we crop the bottom 80 pixels from the back camera images.
B.3 PandaSet
We utilize six cameras and one LiDAR unit. We specifically evaluate on the following 10 challenging nighttime sequences: 063, 066, 070, 073, 074, 077, 078, 079, 088, 149. To mitigate ego-vehicle artifacts, we apply bottom crop of 260 pixels to the back camera images.
B.4 Argoverse2
We leverage the seven ring cameras and both LiDAR sensors available in the dataset. We conduct our experiments on the following 9 sequences, in line with B
To minimize ego-vehicle interference, we apply bottom crop of 250 pixels to the front center, rear
left, and rear right camera views.

---

C Additional Quantitative Results
This section presents a comprehensive breakdown of our model's performance, including per-scene metrics, integration results with SOTA methods, and robustness checks challenging subsets.
C.1 Per-Scene Geometry and Appearance Evaluation
We provide detailed metrics for geometry (Tab. A1
A4
and appearance (Tab.
A5
A8
C
S
Ground Thuth
S
C
Baseline+AP
S
S
Baseline
S
Baseline+tOus
Figure A1: Qualitative Comparison of Photometric Correction with Baseline Methods. These figures visualize the output of our method (" ('Baseline+Ours") against the ground truth, a baseline with appearance codes ('Baselin+AAP', and baseline with single bilateral grid ("Baselin++BG'. The accompanying error maps (blue indicating lower error, red higher) and highlighted red boxes demonstrate our method's superior ability handle complex illumination and reduce artifacts compared to traditional approaches.
C.2 Integration Evaluation with SOTA Methods
Tab. A10 provide comparisons with SOTA methods like ChatSim and StreetGS on the NuScenes dataset.

---

Ground Truth
Baseline
Baseline+BB
Baseline-tOurs
Figure A2: Qualitative Comparison of Photometric Correction with Baseline Methods.
Geometry Evaluation
Scene
CDL
RMSE
Depth
152
1.227
3.463
0.042
164
0.889
2.469
0.103
1.042
2.813
0.075
200
1.300
3.617
0.020
209
1.294
3.524
0.065
359
1.238
3.542
0.036
529
0.678
2.675
0.021
916
1.623
4.617
0.109
Average
1.161
3.340
0.059
Table A1: Detailed Scene-by-Scene Geometry Evaluation on the NuScenes [1] This table presents key geometry metrics-CCamfee Distance (CD), Root Mean Square Error (RMSE), and Depth error-for individual scenes and averaged across the evaluated sequences from the NuScenes dataset.

---

OmniRe
OmniRe+Ours
Figure A3: Qualitative Comparison with the OmniRe. This figure showcases side-by-side visual comparison of renderings and depth maps produced by the original OmniRe framework versus OmniRe integrated with our proposed method ('OmniRe+Ours') ). The results highlight the enhancements image fidelity and depth map coherence achieved by our approach.
Geometry Evaluation
Scene
CD1
RMSEL
Depth
L6688
3,217
1.733
1.427
2.905
0.350
31
0.278
1,636
0.026
233
1.149
3.465
0.594
551
1.339
3.362
0.095
0.103
0.007
Average
0.989
2.744
0.467
Table A2: Detailed Scene-by-Scene Geometry Evaluation on the Waymo Open Dataset 2 This table showcases the Chamfer Distance (CD), Root Mean Square Error (RMSE), and Depth error for specific scenes and their average, evaluating the geometric reconstruction accuracy on the Waymo Open Dataset.
5

---

ChalSim
ChatSim+Ours
Figure A4: Qualitative Comparison with the ChatSim. Visual results comparing the ChatSim framework with ChatSim augmented by our method ( "CChatSimOurs'' The rendered images and corresponding depth maps illustrate the improvements in visual quality and geometric detail provided by multi-scale bilateral grid framework.
StredGS
StrvAGS+OOrs
Figure A5: Qualitative Comparison with the StreetGS. This figure presents visual comparison between the StreetGS framework and StreetGS enhanced with our method ("StreetGS+Ours ) The displayed images and depth maps demonstrate the capability of our approach improve rendering realism and depth accuracy in challenging driving scenarios.

---

Geometry Evaluation
Scene
CD}
RMSEL
Depth
63
0.760
3,812
0.015
66
0.517
2.657
0.008
70
0.310
2.244
0.027
73
0.280
2.012
0.007
0.680
3,240
0.009
0.190
2.926
0.004
0.403
2.819
0.008
79
0.198
2.081
0.004
88
0.968
4.674
0.039
149
0.228
2.052
0.007
Average
0.453
2.852
0.013
Table A3: Detailed Scene-by-Seee Geometry Evaluation on the PandaSet [6] This table provides
a per-scene breakdown and average of Chamfer Distance (CD), Root Mean Square Error (RMSE), and Depth error, assessing geometric reconstruction performance on challenging nighttime scenarios from PandaSet.
Geometry Evaluation
Scene
CD1
RMSEL
Depth.
0.522
3.001
0.023
2.071
5.845
0.085
0.461
3.226
0.011
0.348
3.677
0.014
1.198
4.485
0.059
0.694
5.069
0.040
0.967
6.193
0.075
0.743
4,436
0.045
0.257
1.952
0.005
Average
0.807
4.209
0.040
Table A4: Detailed Scene-by- Scene-by-Scene Geometry Evaluation on the Argoverse2 Dataset (5). This table displays Chamfer Distance (CD), Root Mean Square Error (RMSE), and Depth error for individual sequences and their average, evaluating geometric accuracy on the Argoverse2 dataset.
Reconstruction
View Synthesis
Scume
Full Image
human
vehicle
Full Image
human
vehicle
PSNR1
SSIM?
LPIPS.
PSNR1
SSIM
PSNR1
SSIM"
PSNR
SSIM
LPIPS.
PSNK
SSIM
PSNR
SSIM
0.203
0.873
0.862
0.254
01655
0.57%
24.75
0.793
0.454
171
26.97
0.833
0.249
24.05
0.652
26.22
0.812
24.39
0.748
0.273
22.28
0.568
22.51
0.639
200
27.80
0.835
0.193
N/A
N/A
28.08
0.832
25
0.745
0.211
N/A
NIA
24.11
0.643
209
29.07
0,866
0.185
25.20
0.802
28.84
0.836
26.18
0.785
0.204
25.42
0.691
25.07
0.683
0.839
0.647
529
29.90
0.893
0.133
24.25
0.736
27.80
0.856
27.40
0.826
0.146
24.18
0.712
24.84
0.745
916
25.88
0.819
0,181
28.16
0.792
26.95
0.839
22.73
0.678
0.208
25.14
0.637
22.75
0.634
Average
27.69
0.847
0.193
26.53
0.777
27.31
0.834
24.64
0.739
0.216
23.75
0.623
23.23
0.646
Table A5: Detailed Appearance Evaluation for Scene Reconstruction and Novel View Synthesis on the NuScenes Dataset [1]. This table presents PSNR, SSIM, and LPIPS metrics for the full image, as well as for 'human' and 'vehicle' classes, for both scene reconstruction and novel view synthesis tasks on various NuScenes sequences.
Scone Reconstruction
Novel View Synthesss
Scene
Full Image
buran
vehiele
Full Image
human
Vehicle
PSNR1
SSIM?
LPPISS]
PSNR1
SSIM?
PENRT
SSIM
PSNK
SSIM
1.PIPS.
PSNR"
SSIM
PSNR
SSIM
29.25
0.8410
0.311
N/A
N/A
23.33
0.510
23.54
0.770
0.311
NIA
N/A
23.39
0.331
27.94
0.841
0.241
21.82
0.632
NAA
N/A
24.30
0.737
0.269
20.45
0.505
N/A
N/A
0.224
0.784
0.683
23.9%
0.250
0.657
33.71
0.482
N/A
0.686
0.48%
NIA
22.37
551
24.89
0.748
0.409
20.77
0.470
21.66
0.683
22.54
0.670
0.437
19.99
0.407
18.76
0.471
621
31.91
0.939
0.067
26.68
0.846
26.40
0.845
29.73
0.895
0.078
24.94
0.789
24.15
0.765
Average
29.23
0.836
0.289
25.55
0.683
23.50
0.707
26.55
0.762
0.310
23.97
0.590
21.36
0.567
Table A6: Detailed Appearance Evaluation for Scene Reconstruction and Novel View Synthesis on the Waymo Open Dataset (2). This table details PSNR, SSIM, and LPIPS metrics for full images and specific object classes (" ("human', 'vehicle") during scene reconstruction and novel view synthesis on selected Waymo sequences.

---

Scene Reconstruction
Novel View Synthesin
Scenc
Full Image
bursan
vchicle
Full Image
human
vehicle
PSNR1
LPIPS
SSIMT
PSNR-
LPIPS.
SSIMT
29.95
0,503
0.226
23.44
0.783
22:32
0.726
27.32
0.854
0.258
26.93
0.613
18.45
0.447
66
31.73
0.931
0.198
29.94
0.845
20.78
0.681
28.92
0.890
0.215
26,84
0.738
19:55
0.59]
31,09
0.509
0.260
N/A
N/A
21.35
0.740
28.35
0.862
0.287
N/A
N/A
21.20
0.674
0.806
29.16
0.889
0.214
N/A
29.85
0.914
0.207
30.99
0.874
21.44
0.819
27.13
0.862
0.228
27.82
0.740
22.23
0,740
77
32.60
0.937
0.165
32.28
0.852
25.42
0.855
29.77
0.895
0.178
29.02
0.730
22.12
0.721
$1.30
0.522
0.181
35.54
0.896
25.27
0.812
78.39
0.869
0.197
30.74
0.809
22.86
0,683
0.843
28.
0.251
0.726
0.702
0.787
0.670
0.738
20.19
0.362
149
32.31
0.926
0.202
N/A
NAA
24.04
0.791
29.73
0.891
0.215
N/A
NIA
23.37
0.728
Average
30.75
0.906
0.213
30.18
0.830
23.19
0.784
27.89
0.847
0.235
27.22
0.674
21.32
0.614
Table A7: Detailed Appearance Evaluation for Scene Reconstruction and Novel View Synthesis on
PandaSet [6]. This table outlines PSNR, SSIM, and LPIPS metrics for full image and object-specific
("human', 'vehicle') evaluations in both scene reconstruction and novel view synthesis tasks using
PandaSet nighttime sequenees.
Scene Reconstruction
Novel View Synthesis
Full Image
Full Image
PSNR1
SSIM1
LPIPS.
PSNR1
SSIM
PSNR1
SSIM.
PSNR
SSIM
LPIPS.
PSNR
SSIM
PSNR
SSIMI
25.72
0.871
0.181
25.34
0.822
36.58
0.825
22.83
0.765
0.199
23.73
0.734
23.31
0.682
22.18 18
0.769
0.302
21.76
0.681
22.87
0.798
20.20
0.655
0.323
18.22
0.472
18.48
0.478
0.884
0.183
0.620
26.55
0.893
0.123
23.24
0.811
25.58
0.875
21.69
0.826
0.141
21.99
0.756
23.10
0.769
22.65
0.804
0.272
22.75
0.720
23.76
0.812
20.55
0.685
0.290
19.95
0.518
19.58
0.493
24.19
0,848
0.207
23.69
0.760
23.75
0.790
22.07
0.738
0.226
21.12
0.598
20.32
0.580
23,47
0.850
0.183
25.63
0.787
23,66
0.851
21 15
0.756
0.199
20.57
0.531
19.58
0.628
26.48
B.925
0.191
22. 44
0.700
24.42
0.853
24.91
0.875
0.100
21.04
0.614
21
0.732
Average
24.68
0.849
0.200
23.35
0.739
24.51
0.827
22 22.58
0.756
0.217
20.79
0.588
20.86
0.615
Table A8: Detailed Appearance Evaluation for Scene Reconstruction and Novel View Synthesis on the Argoverse2 Dataset [5]. This table shows PSNR, SSIM, and LPIPS for full image and object- focused ("human', 'vehicle") assessments across scene reconstruction and novel view synthesis on the Argoverse2 dataset.
Reconstruction
Novel View Synthesis
Geometry
Scene
Method
PSNR f
SSIM
LPIPS
PSNR
SSIM f
LPIPS +
CD 4
RMSE +
Depth
ChatSim
25.37
0,811
0.251
1.592
3.637
0.085
22.72
0.694
0.276
152
Ours
27.09
0.825
0.231
1.309
3,540
0.040
23.80
0.703
0.256
ChatSim
24.08
0.788
0.299
1.198
2.615
0.186
22.07
0.708
0.321
164
Ours
26.16
0,802
0,273
0.972
2.523
0.101
23.79
0.718
0.293
ChatSim
25.00
0.801
0.310
1.420
2.981
0.105
23.31
0.736
0.326
171
Ours
26.95
0.813
0.289
1.114
2,895
0.057
24.77
0.744
0.307
ChatSim
24.85
0.799
0.230
1.690
3,740
0.035
23.47
0.732
0.246
200
Ours
27.54
0.816
0.220
1,289
3.619
0.017
25.43
0.745
0.235
ChatSim
25.77
0.818
0.268
1.841
3.805
0.118
24.30
0.763
0.281
209
Ours
27.93
0.833
0.243
1.445
3.691
0.061
25.93
0.776
0.255
ChatSim
25.76
0.823
0.220
L.498
3.701
0.073
23.46
0.714
0.238
359
Ours
27.17
0.834
0.208
1,317
3.647
0.036
24.31
0.719
0.227
ChatSim
27.19
0.864
0,191
1,071
2.853
0.077
25.70
0.812
0.201
529
Ours 29.60
0.878
0.159
0.692
2.749
0.018
27.35
0.822
0.169
ChatSim
22.81
0.742
0.249
2.145
4.745
0.173
21.19
0.641
0.271
916
Ours
23.95
0.759
0.232
1,750
4.635
0.102
21.96
0.655
0.253
Table A9: Comparative Performance Analysis against ChatSim the NuScenes Dataset. This table provides a scene-by-scent comparison of our method ("Ours') with the ChatSim baseline across scene reconstruction (PSNR, SSIM, LPIPS), novel view synthesis (PSNR, SSIM, LPIPS), and geometry (CD, RMSE, Depth) metrics.
8

---

Reconstruction
Novel View Synthesis
Geometry
Scene
Method
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
CD 4
RMSE
Depth
StreetGS
25.49
0.813
0.254
1.611
3.648
0.089
22.73
0.695
0.280
152
Ours
27.34
0.828
0.231
1.343
3.560
0.038
23.90
0.706
0.256
StreetGS
25.04
0.813
0.273
1.187
2.608
0.179
22.55
0.724
0.296
Ours
27.41
0.827
0.248
0.956
2.523
0.096
24.29
0.734
0.269
StreetGS
25.62
0.811
0.305
1.355
2.958
0.115
23.63
0.743
0.323
Ours
27.79
0.822
0.281
1.184
2,910
0.067
25.21
0.751
0.300
StreetGS
25.32
0.811
0.230
1.878
3.925
0.041
23.75
0.739
0.247
200
Ours
28. 12
0.829
0.219
1.418
3.817
0.020
25.67
0.752
0.235
StreetGS
26.53
0.840
0.243
1.932
3.813
0.125
24.77
0.779
0.258
209
Ours
0.854
0.217
1.481
3.697
0.061
26.53
0.791
0,231
StreetGS
26.08
0.831
0.211
1.539
3.710
0.075
23.53
0.716
0.230
359
Ours
27.63
0.843
0.199
1.299
3.659
0.039
24.45
0.723
0.218
StreetGS
27.50
0.871
0.186
1.065
2.883
0.063
25.96
0.819
0.197
529
Ours
30.09
0.884
0.158
0.718
2,795
0.020
27.69
0.827
0.169
StreetGS
24.34
0.787
0.223
2.268
4.813
0.175
22.21
0.680
0.245
Ours
25.80
0.804
0.204
1.781
4.709
0.106
23.11
0.692
0.226
Table A10: Comparative Performance Analysis against StreetGS the NuScenes Dataset. This table presents a detailed per-scene comparison our approach (' ('Ours') with the StreetGS baseline, evaluating scene reconstruction (PSNR, SSIM, LPIPS), novel view synthesis (PSNR, SSIM, LPIPS), and geometry (CD, RMSE, Depth) metrics.
Table All: Detailed comparison on Challenging Argoverse Scenarios (6 scenes).
Reconstruction
Novel View Synthesis
Method
PSNR
SSIM f
CD 4
RMSE +
Depth +
PSNR 1
SSIM
OmniRe (Baseline)
23.95
0.850
0.964
3.647
0.098
22.10
0.761
OmniRe w/ AC
23.97
0.848
0.923
3.642
0.097
22.09
0.759
OmniRe w/ BG
24.29
0.855
0.807
3.586
0.047
22.32
0.768
Ours
25.29
0.863
0.726
3.564
0.027
22.96
0.773
Table A12: Detailed comparison on Challenging Waymo Scenarios (6 scenes).
Reconstruction
Novel View Synthesis
Method
PSNR
SSIM f
CD |
RMSE 1
Depth 4
PSNR f
SSIM
OmniRe (Baseline)
29.28
0.833
0.352
1.917
0.061
27.37
0.791
OmniRe AC
29.31
0.833
0.343
1.915
0.053
27.30
0.789
OmniRe w/ BG
29.61
0.836
0.333
1.893
0.042
27.56
0.792
Ours
31.23
0.841
0.272
1.846
0.021
28.34
0.793
Table A13: Detailed comparison on Challenging Pandaset Scenarios (6 scenes).
Reconstruction
Novel View Synthesis
Method
PSNR
SSIM f
CD |
RMSE |
Depth J
PSNR 1
SSIM
OmniRe (Baseline)
27.28
0.859 0.863
0.634 0.572
3.963
0.029
23.91 24.22
0.721
Ours
27.88
3.951
0.022
0.724
Table A14: Comparison results of vehicle-specific metrics, averaged across scenes from all four datasets.
Method
PSNR
SSIM
CD 1
OmniRe (Baseline)
8.675 24.24
0.784
Ours
24.80
0.791
7.400

---

C.3 Performance on Challenging Scenarios
To validate robustness, we curated subsets of challenging scenarios with extreme photometric inconsistencies (e.g., night scenes, sun glare, rain, and complex reflections). As shown in Tables A11 A12 and A13] our method's performance gap widens significantly compared to all baselines (OmniRe, w/ AC, w/ BG) in these difficult conditions, demonstrating the superior robustness of our multi-scale design.
Challenging Argoverse (6 scenes): 49e970c4 (Extreme Lighting, Overexposure, Lens Flare), 8184872e (Complex Shadows, Direct Sunlight), 91923e20 (Extreme Lighting, Camera Artifacts), a8a2fbc2 (Low-Angle Sun), b403f8a3 (Overpass Structure, Extreme Lighting Transitions), 5adf (Multiple Artificial Lights, Wet Surfaces, Rainy Condi- tions).
Challenging Waymo (6 scenes): segment-10 (Night Scene, Artificial Lighting, Lens Flare), segment- (Night Scene, Rainy Conditions, Specular Reflections), segment-539 (High Dynamic Range, Low-Angle Sun, Sun Glare), segment-550 (High Dynamic Range, Storefront Reflections, Hard Shadows), segment- -611 (Low-Angle Sun, Lens Flare, Over- exposure), segment-570 (Rainy Conditions, Low Light, Motion Blur).
Challenging Pandaset (6 scenes): 19. 21, 29, 48, 52 (all Low-Angle Sun with Hard/Deep Shadows or High Dynamic Range), 63 (Night Scene, Multiple Light Sources, Glare).
C.4 Vehicle-Specific Metrics
To specifically evaluate performance challenging, highly reflective surfaces such windows, we used semantic masks to isolate vehicles. Table [114 shows that our method improves not only appearance (PSNR) but also, crucially, geometry (CD) for these specific objects. This supports claim that our method mitigates the negative geometric impact of such severe view-dependent effects.
D Quantitative Analysis of Photometric Inconsistency
To rigorously validate the learned affine transformations discussed in the main paper, we compute the Kullback-Leibler (KI.) Divergence between our model's learned distribution and a ground-truth (GT) distribution real-world photometric inconsistency.
Experimental Setup: We construct the GT distribution P(AL) by sampling corresponding pixels representing the same 3D world points across different camera views in the nuScenes dataset, using ground-truth LiDAR points and camera parameters. The luminance difference AL for all corresponding pairs forms the GT histogram. We then measure the KL Divergence between this P(AL) and the distributions learned by our model and the baseline.
Results: As shown in Table A15 the distribution learned by our multi-scale method achieves a significantly lower KI. Divergence (0.79) compared to the single-scale baseline (1,25). This quantitatively confirms that our model learns distribution that more accurately reflects real-world photometric inconsistencies.
Table A15: Comparison KL Divergence against the ground-truth (GT) photometric inconsistency distribution from the nuScenes dataset. Lower is better, indicating more realistic learned distribution.
Method
KL Divergence (vs. Dataset GT)
Single Bilateral Grids' Affine Transformation Multi-scale Bilateral Grids' Affine Transformation (Ours)
1.25
0.79
References
[1] Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh Vora, Venice Erin Liong, Qiang Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multimodal dataset for autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11621-11633 2020.
10

---

[2] Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo. Yin Zhou, Yuning Chai. Benjamin Caine, al. Scalability in perception for autonomous driving: Waymo open dataset. Proceedings of the IEEE/CVF conference on computer vision and puttern recognition. pages 2446-2455 2020.
[3] Adam Tonderski, Carl Lindström. Georg Hess, Williamm Ljungbergh. Lennart Svensson, Christoffer Petersson. Neurad: Neural rendering for autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14895-14904, 2024.
[4] Yuxi Wei, Zi Wang. Yifan Lu, Chenxin Xu, Changxing Liu, Hao Zhao, Siheng Chen, and Yanfeng Wang. Editable scene simulation for autonomous driving via collaborative Ilm agents. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 15077-15087, 2024.
[5] Benjamin Wilson, William Qi, Tanmay Agarwal, John Lambert, Jagject Singh, Siddhesh Khandelwal, Bowen Pan, Ramesh Kumar, Andrew Hartnett, Jhony Kaesemodel Pontes, et al. Argoverse 2: Next generation datasets for self-driving perception and forecasting. arXiv preprint arXiv:2301.000 004003 2023.
[6] Pengchuan Xiao, Zhenlei Shao, Steven Hao, Zishuo Zhang, Xiaolin Chai, Judy Jiao, Zesong Li, Jian Wu, Kai Sun, Kun Jiang, al. Pandaset: Advanced sensor suite datasct for autonomous driving. In 202/ IEEE international intelligent transportation systems conference (ITSC), pages 3095-3101, IEEE, 2021.
[7] Yunzhi Yan, Haotong Lin, Chenxu Zhou, Weijie Wang. Haiyang Sun, Kun Zhan, Xianpeng Lang, Xiaowei Zhou, and Sida Peng. Street gaussians: Modeling dynamic urhan scenes with gaussian splatting. In European Conference on Computer Vision, pages 156-173. Springer, 2024.


Reviews:
[{"review_id": "jyxN7F2T05", "reviewer": "Reviewer_5GsZ", "summary": "This paper proposes a method to learn to represent the appearance variation in autonomous driving scenes reconstruction with 3D gaussian splatting scene graph.\nInstead of relying on a single appearance embedding that provides a global transform on the image level but fails to represent local variation, or a bilateral grid that will learn pixel-wise variation but is hard to optimize, this paper proposes to use a composition of multi-scale bilateral grids.\nThe coarse bilateral grid will ressemble the appearance code by  modeling an image level transformation, then progressively  reduce patch size to learn more local transformations.\nBy doing so, the model is able to learn more local variation than appearance embedding, while also guiding the bilateral grid training with a composition of progressively more detailed transformation.\nBy doing so, this paper is able to provide better geometric and photometric results than using either appearance code or bilateral grid, as it is shown through extensive benchmarking on 4 different driving datasets.", "questions": "How would this method perform in very difficult scenarios (night scenes, sun flare)?\nAre there cases where the multi-scale bilateral grids are able to overfit to some dynamic element instead of representing it correctly in 3D?", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- SOTA reconstruction quality compared to appearance code and bilateral grids.\n- The idea is simple and intuitive yet provides good results.\n- Paper is overall clear, with benchmarking on 4 different driving datasets. \n- Significantly less parameters to process compared to purely pixel-wise bilateral grids.\n\nWeaknesses:\n- Computation is much higher compared to simply using appearance code.", "was_revised": true, "final_justification": "My questions were answered. I think it is overall a good paper with interesting contributions.", "rebuttal": "We thank **5GsZ** for the positive evaluation and for recognizing our state-of-the-art reconstruction quality and the efficiency of our approach compared to pixel-wise bilateral grids.\n\n---\n\n#### **W1: Computation is much higher compared to simply using appearance code.**\n\n**We focus on favorable trade-off.** We designed the framework to ensure this overhead is moderate and justified by the significant improvements in reconstruction quality.\n\n- **Modest Computational Cost.** Our approach increases training time by only **9%** over the AC baseline (from 1.93 to 2.10 hours). \n- **Substantial Performance Gain.** In return for this modest overhead, our method achieves a **19% reduction in geometric error** on the NuScenes dataset (**Table 1**), with the Chamfer Distance dropping from 1.437 to **1.161**.\n- We believe this demonstrates a valuable trade-off, where a small increase in computation yields a substantial and critical improvement in geometric accuracy, which is crucial for autonomous driving applications.\n\n---\n\n#### **Q1: How would this method perform in very difficult scenarios (night scenes, sun flare)?**\n\nOur results show that the framework can work well and demonstrate even **greater advantages in these challenging environments.**\n\n- **Qualitative Results. Figure 3(e)** and the **left panel of** **Figure 4** provide qualitative visualizations of our method's ability to produce high-fidelity, coherent reconstructions in low-light environments. Sun flare is also included in our test scenarios. We will add more examples in our final version.\n- **Quantitative Improvement.** We select a **subset** consisting of scenarios featuring **night scenes and sun flare** to make further evaluation. Our method shows a marked quantitative improvement over baseline approaches.\n\n> Table R1: Additional comparison results on Night Scenes: waymo-10, waymo-30, argoverse-eaaf5ad3, pandaset-63.\n|  | Recon.  PSNR$\\uparrow$ | Recon.  SSIM$\\uparrow$ | Recon.  CD$\\downarrow$ | Recon.  RMSE$\\downarrow$ | Recon.  Depth$\\downarrow$ | NVS PSNR$\\uparrow$ | NVS SSIM$\\uparrow$ |\n| --- | --- | --- | --- | --- | --- | --- | --- |\n| OmniRe | 28.34 | 0.805 | 0.586 | 2.767 | 0.041 | 26.54 | 0.753 |\n| Ours | **29.13** | **0.812** | **0.500** | **2.731** | **0.016** | **27.21** | **0.762** |\n\n> Table R2: Additional comparison results on Sun flare scenes: waymo-550, waymo-561, argoverse-49e970c4, argoverse-91923e20, argoverse-a8a2fbc2.\n|  | Recon. PSNR$\\uparrow$ | Recon.  SSIM$\\uparrow$ | Recon.  CD$\\downarrow$ | Recon.  RMSE$\\downarrow$ | Recon.  Depth$\\downarrow$ | NVS PSNR$\\uparrow$ | NVS SSIM$\\uparrow$ |\n| --- | --- | --- | --- | --- | --- | --- | --- |\n| OmniRe | 26.68 | 0.866 | 0.781 | 3.073 | 0.079 | 24.98 | 0.796 |\n| Ours | **28.65** | **0.879** | **0.593** | **2.994** | **0.022** | **25.84** | **0.803** |\n\n---\n\n#### **Q2: Are there cases where the multi-scale bilateral grids are able to overfit to some dynamic element instead of representing it correctly in 3D?**\n\nThe reviewer's concerns are insightful. Actually it may happen without enough constrains, as we mentioned in our limitations.\n\n- Our framework prevent this type of overfitting by explicitly linking appearance modeling with **geometric constraints**.\n\n- As formulated in **Equation 4** . Our framework does not solely minimize photometric error; it simultaneously minimizes a **geometric loss** term against ground-truth LiDAR depth maps:\n\n> $\\left\\| D_{c,t}^r - D_{c,t} \\right\\|_2^2$.\n> \n- This design creates a powerful synergy between appearance and geometry:\n    - Any appearance transformation learned by the bilateral grids must also support a **geometrically accurate 3D reconstruction**.\n    - If the grid were to \"overfit\" to a transient visual effect (e.g., a shadow or reflection) by altering its appearance, it would likely create a **discrepancy with the ground-truth LiDAR data**.\n    - This discrepancy would incur a higher geometry loss, penalizing the model and guiding it to learn**corrections** **for true photometric inconsistencies** (like lighting changes) rather than incorrectly baking dynamic effects into the 3D geometry.\n- This process is a key factor in our method's ability to reduce geometric artifacts like \"floaters\" and achieve superior geometric fidelity across our experiments.\n\n---\n\nWe appreciate all your constructive comments and we will improve our work according to them in the near future. If the reviewer has any follow-up questions, we are happy to discuss them."}, {"review_id": "0viL4yTD7a", "reviewer": "Reviewer_yZ4i", "summary": "This paper proposes multi-scale bilateral grids to address the photometric inconsistency in a temporal multi-camera setting. The proposed method combines the advantages of appearance code and vanilla bilateral grid, thus being optimization-friendly and having better modeling capacity. The paper conducts extensive experiments on four driving datasets, demonstrating good performance.", "questions": "1. Why are the performance improvements on the Argoverse and Pandaset datasets less prominent compared with those on the Waymo and nuScenes datasets, including both the 3D reconstruction and novel view synthesis tasks?\n2. How is the performance of the baseline model with a smaller sized single-scale bilateral grid, e.g., with only the finest bilateral grid in the proposed multi-scale setting?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. Good motivation. The photometric inconsistency is an important problem in 3D reconstruction. The paper proposes to address it by combining the advantages of appearance code and bilateral grids.\n2. Good writing. The paper is well written and easy to follow.\n\nWeaknesses:\n1. Lack of novelty. The extension of single-scale bilateral grid to the multi-scale counterpart is quite straightforward and does not reflect much insight.\n2. Inconsistency in performance improvement. The performance improvements on the Argoverse and Pandaset datasets are less prominent and even minimal compared with those on the Waymo and nuScenes datasets. The paper does not include any discussion on it.", "was_revised": true, "final_justification": "Thanks for the thoughtful response.\nW1: The explanation about novelty focuses on the unification of two paradigms and indeed addresses my concern.\nW2 & Q1: The results of additional experiments are impressive and demonstrate the advantages of the proposed method in extreme cases.\nQ2: The comparison with the single-scale baseline validates the effectiveness of the multi-scale design.\nGenerally, the rebuttal has addressed all of my concerns and I will raise my score to 4. It is recommended to integrate rebuttal experiments and analysis into the revised version.", "rebuttal": "We thank **yZ4i** for the feedback and for recognizing our good motivation and clear writing. We address the concerns about novelty and performance consistency below.\n\n---\n\n#### **W1: Lack of novelty & does not reflect much insight.**\n\n- We respectfully disagree with the assessment that our work lacks insight.\n    - Our primary contribution is not the simple extension to a multi-scale structure, but rather the insightful **unification of two distinct paradigms**: global appearance codes and pixel-wise bilateral grids.\n- Our framework has a principled design that provides significant advantages by addressing key limitations in prior work:\n    - **Optimization Challenges**: Our method overcomes the critical optimization challenges and instability inherent to single high-resolution bilateral grids by decomposing the complex transformation across multiple, more manageable scales.\n    - **Hierarchical Refinement**: We introduces a **hierarchical, residual refinement** process where the coarse grid learns a global base appearance (akin to an appearance code), while finer grids learn residual corrections for regional and local details. This structured, coarse-to-fine learning process is a key insight that enables stable optimization.\n    - **Bridges Global and Local Modeling**: Our approach provides the localized, pixel-wise adjustment that global appearance codes lack, while maintaining the global consistency and optimization stability that single bilateral grids struggle with.\n- Therefore, the novelty also lies not in how we leverage this structure to create a unified and robust framework that solves core limitations of existing methods.\n\n---\n\n#### **W2 & Q1: Improvements on the Argoverse and Pandaset datasets less prominent**\n\n- The performance variation across datasets stems from **a. the characteristics of the test scenes** and **b. the intrinsic capabilities of Gaussian Splatting**.\n    - We would like to clarify that our method provides a consistent and crucial improvement in **geometric accuracy** across all tested conditions.\n    - **Dataset Characteristics.** In our evaluation, the selected test sequences from the Argoverse and PandaSet datasets feature relatively more consistent lighting conditions.\n        1. Since our method's primary function is to resolve appearance inconsistencies, the margin for improvement on appearance-based metrics is naturally smaller in these scenarios.\n    - **Gaussian Splatting has capabilities of fitting appearance variance using \"floater\".** \n        1. It can model some inconsistencies by adjusting its Spherical Harmonics (SH) coefficients and by using semi-transparent Gaussians.\n        2. This allows the baseline to achieve a reasonable appearance match in less challenging scenes, which is why the PSNR/SSIM improvement of our method is more modest on those datasets.\n        3. However, this baseline behavior of **\"absorbing\" appearance errors into the Gaussian representation** often comes at the **expense of geometric accuracy**.\n- Our method resolves these appearance inconsistencies first, which enables the optimizer to achieve a more accurate geometric reconstruction, reducing the Chamfer Distance on Argoverse by **15%** (from 0.954 to 0.807) and on PandaSet by **10%** (from 0.503 to 0.453) over the baseline.\n- **Additional experiments to validate.** We conduct additional experiments on a curated subset of scenes featuring more extreme lighting and weather conditions. The scenes are listed here:\n\n> Table R3: Additional challenging Argoverse scenarios.\n|scene ID|description |\n| --- | --- |\n| 49e970c4 | Extreme Lighting, Overexposure, Lens Flare |\n| 8184872e | Complex Shadows, Direct Sunlight, Information Loss |\n| 91923e20 | Extreme Lighting, Camera Artifacts. |\n| a8a2fbc2 | Low-Angle Sun, Texture Disentanglement. |\n| b403f8a3 | Overpass Structure, Extreme Lighting Transitions|\n| eaaf5ad3 | Multiple Artificial Lights, Wet Surfaces, Rainy Conditions |\n\n> Table R4: Evaluation results of our multi-scale bilateral grids on additional challenging Argoverse scenarios:\n|  | Recon. PSNR$\\uparrow$ | Recon. SSIM$\\uparrow$ | Recon. CD$\\downarrow$ | Recon. RMSE$\\downarrow$ | Recon. Depth$\\downarrow$ | NVS PSNR$\\uparrow$ | NVS SSIM$\\uparrow$ |\n| --- | --- | --- | --- | --- | --- | --- | --- |\n| 49e970c4 | 23.35 | 0.789 | 1.229 | 5.175 | 0.060 | 21.33 | 0.673 |\n| 8184872e | 24.23 | 0.864 | 0.666 | 3.303 | 0.022 | 21.76 | 0.766 |\n| 91923e20 | 25.86 | 0.876 | 0.820 | 3.342 | 0.019 | 23.48 | 0.789 |\n| a8a2fbc2 | 25.13 | 0.823 | 0.774 | 3.513 | 0.033 | 22.65 | 0.716 |\n| b403f8a3 | 27.55 | 0.926 | 0.298 | 2.822 | 0.004 | 25.25 | 0.870 |\n| eaaf5ad3 | 25.60 | 0.899 | 0.566 | 3.228 | 0.022 | 23.28 | 0.824 |\n| Average | **25.29** | **0.863** | **0.726** | **3.564** | **0.027** | **22.96** | **0.773** |\n\n> Table R5: Evaluation results of OmniRe on additional challenging Argoverse scenarios:\n|  | Recon. PSNR$\\uparrow$ | Recon. SSIM$\\uparrow$ | Recon. CD$\\downarrow$ | Recon. RMSE$\\downarrow$ | Recon. Depth$\\downarrow$ | NVS PSNR$\\uparrow$ | NVS SSIM$\\uparrow$ |\n| --- | --- | --- | --- | --- | --- | --- | --- |\n| 49e970c4 | 21.99 | 0.772 | 1.474 | 5.270 | 0.129 | 20.37 | 0.659 |\n| 8184872e | 22.79 | 0.850 | 0.862 | 3.389 | 0.101 | 20.79 | 0.751 |\n| 91923e20 | 24.47 | 0.859 | 1.171 | 3.463 | 0.124 | 22.67 | 0.776 |\n| a8a2fbc2 | 23.13 | 0.805 | 1.107 | 3.602 | 0.140 | 21.20 | 0.701 |\n| b403f8a3 | 26.65 | 0.922 | 0.396 | 2.862 | 0.014 | 24.77 | 0.869 |\n| eaaf5ad3 | 24.69 | 0.889 | 0.776 | 3.295 | 0.083 | 22.79 | 0.810 |\n| Average | 23.95 | 0.850 | 0.964 | 3.647 | 0.098 | 22.10 | 0.761 |\n\nIn these challenging Argoverse scenarios, our method outperforms OmniRe by an average of **1.338 dB** in PSNR.\n\n> Table R6: Additional challenging Pandaset scenarios.\n|scene ID|description |\n| --- | --- |\n| 19 | Low-Angle Sun, Hard Shadows |\n| 21 | Low-Angle Sun, High Dynamic Range |\n| 29 | Low-Angle Sun, Deep Shadows |\n| 48 | Low-Angle Sun, High Dynamic Range |\n| 52 | Low-Angle Sun, Sun Glare |\n| 63 | Night Scene, Multiple Light Sources, Glare |\n\n> Table R7: Evaluation results of our multi-scale bilateral grids on additional challenging Pandaset scenarios:\n|  | Recon.  PSNR$\\uparrow$ | Recon.  SSIM$\\uparrow$ | Recon.  CD$\\downarrow$ | Recon.  RMSE$\\downarrow$ | Recon.  Depth$\\downarrow$ | NVS PSNR$\\uparrow$ | NVS SSIM$\\uparrow$ |\n| --- | --- | --- | --- | --- | --- | --- | --- |\n| 19 | 24.97 | 0.812 | 0.639 | 5.697 | 0.040 | 21.94 | 0.645 |\n| 29 | 27.95 | 0.864 | 0.274 | 2.559 | 0.012 | 23.94 | 0.725 |\n| 48 | 28.30 | 0.878 | 0.260 | 3.026 | 0.020 | 24.21 | 0.709 |\n| 21 | 26.98 | 0.849 | 0.947 | 4.886 | 0.024 | 22.91 | 0.686 |\n| 52 | 28.87 | 0.870 | 0.557 | 3.737 | 0.023 | 24.83 | 0.727 |\n| 63 | 30.20 | 0.902 | 0.755 | 3.801 | 0.014 | 27.49 | 0.853 |\n| Average | **27.88** | **0.863** | **0.572** | **3.951** | **0.022** | **24.22** | **0.724** |\n\n> Table R8: Evaluation results of OmniRe on additional challenging Pandaset scenarios:\n|  | Recon.  PSNR$\\uparrow$ | Recon.  SSIM$\\uparrow$ | Recon.  CD$\\downarrow$ | Recon.  RMSE$\\downarrow$ | Recon.  Depth$\\downarrow$ | NVS PSNR$\\uparrow$ | NVS SSIM$\\uparrow$ |\n| --- | --- | --- | --- | --- | --- | --- | --- |\n| 19 | 24.60 | 0.811 | 0.771 | 5.702 | 0.045 | 21.78 | 0.643 |\n| 29 | 27.61 | 0.863 | 0.313 | 2.566 | 0.016 | 23.85 | 0.726 |\n| 48 | 27.70 | 0.873 | 0.288 | 3.031 | 0.027 | 23.96 | 0.705 |\n| 21 | 26.35 | 0.842 | 1.035 | 4.900 | 0.035 | 22.69 | 0.679 |\n| 52 | 28.06 | 0.868 | 0.583 | 3.757 | 0.030 | 24.44 | 0.726 |\n| 63 | 29.35 | 0.898 | 0.812 | 3.820 | 0.021 | 26.74 | 0.845 |\n| Average | 27.28 | 0.859 | 0.634 | 3.963 | 0.029 | 23.91 | 0.721 |\n\nIn these challenging Pandaset scenarios, our method outperforms OmniRe by an average of **0.6 dB** in PSNR, showing significant improvements when facing scenarios with obvious appearance inconsistencies.\n\n---\n\n#### **Q2: How is the performance of the baseline model with a smaller sized single-scale bilateral grid?**\n\n- **Smaller sized bilateral grids do not lead to better results.** Motivated by reviewer **yZ4i's** suggestion, we ran a **new experiment** using only the finest grid from our hierarchy $(8\\times 8\\times 4\\times 12)$ as a single-scale baseline.\n\n> Table R9: Additional ablation study on grid size.\n|  | Recon. PSNR$\\uparrow$ | NVS PSNR$\\uparrow$ | CD$\\downarrow$ |\n| --- | --- | --- | --- |\n| Ours | **27.69** | **24.64** | **1.161** |\n| OmniRe w/single grid (16,16,8) | 25.98 | 23.60 | 1.380 |\n| OmniRe w/single grid (8,8,4) | 26.56 | 23.90 | 1.376 |\n| OmniRe w/grid (8,8,4)+(2,2,1) | 27.57 | 24.57 | 1.340 |\n- The single fine-scale grid performs poorly compared to our full model.\n\nThe central premise of our work is that single-scale bilateral grids, especially fine ones, are difficult to optimize for complex scenes because they lack a stable, global initialization.\n- Our ablations in **Table 5(b)** (\"Grid Size Combinations\") also provide evidence for this.\n    - Removing the essential coarse grid (the `(4,4,2)+(8,8,4)` configuration) causes a severe degradation in geometric performance, with the Chamfer Distance (CD) increasing from 1.161 to **1.963**.\n    - This shows that without the coarse grid providing a stable base transformation, the finer grids fail to converge properly, a challenge we also illustrate qualitatively in **Figure 1(b)** and **Figure 4(b)**.\n---\n\nWe appreciate all your constructive comments and we will improve our work according to them in the near future. We respectfully hope that this clarification will prompt you to reconsider your assessment and potentially raise the score of our submission. If the reviewer has any follow-up questions, we are happy to discuss them."}, {"review_id": "wlkSC8RBDT", "reviewer": "Reviewer_2hvk", "summary": "This paper tackles inconsistent appearance in 3DGS driving scenes with a multi-scale bilateral grid. It works hierarchically to provide both global (coarse levels) and local (fine levels) appearance correction. Experiments on four driving datasets show improved geometry and reduced visual artifacts, outperforming standard baselines like appearance codes.", "questions": "1. Please clarify the implementation details for the \"Baseline\" method used in comparisons (e.g., Fig. 3).\n2. The paper evaluates dynamic scenes using static frames, which can be prone to cherry-picking favorable results. Could the authors provide video results for the driving sequences?\n3. Autonomous driving scenarios are inherently dynamic. The paper mentions that modeling \"extremely complex, fast-moving, or highly non-rigid objects\" remains a challenge.  Could you provide some qualitative examples of these failure cases?\n4. The ablation in Table 5 is concerning: adding finer grids can harm geometric performance. This challenges the robustness of the fixed grid architecture. How sensitive is performance to this specific three-level design? Does the optimal grid configuration change per-scene (e.g., night vs. day), requiring manual tuning?\n5. The method performs appearance correction in 2D as a post-processing step. How does this compare conceptually to methods that disentangle appearance in 3D (e.g., Relightable 3DGS)? What are the fundamental limitations of a 2D approach for handling 3D view-dependent effects like specular highlights?", "limitations": "Yes. The authors discuss limitations in the conclusion, primarily: increased computational overhead and challenges with modeling highly dynamic or non-rigid objects.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**:\n- The paper tackles a critical and well-known challenge in real-world 3D reconstruction. Photometric inconsistency in driving data is a known source of artifacts and geometric errors for 3DGS. Solving this is valuable for downstream tasks like simulation and planning.\n- The method shows strong quantitative and qualitative results. Evaluation is thorough, covering four large driving datasets (Waymo, etc.) and multiple baselines. The handling of dynamic objects further demonstrates practical relevance.\n\n\n**Weaknesses**:\n- The novelty is limited. The proposed multi-scale grid is a heuristic that combines two existing ideas (global and per-pixel correction), rather than a fundamental new technique. The presentation overstates the novelty of what is effectively a smart engineering combination of known components.\n- The analysis in Section 4.3 is unconvincing. The central claim—that a \"flatter\" histogram signifies a better model—is an unproven assertion. A flat distribution could equally imply that the model is learning unstructured noise or is poorly constrained, whereas a peaky distribution might indicate an efficient model that has correctly identified the primary sources of error. The analysis shows a correlation between flatter histograms and better results, but fails to establish causation; Also Lacking a ground-truth comparison, it arbitrarily claims one learned distribution is superior to another, making the entire analysis feels hand-wavy and lacks rigor.", "was_revised": true, "final_justification": "After considering the rebuttal and the discussion, I find that some of my initial concerns have been clarified, but the key issues are not fully resolved. I weighed the strengths of the work against the remaining weaknesses and concluded that my original score best reflects the overall quality and contribution of the paper.", "rebuttal": "Thank reviewer **2hvk** for detailed review and constructive feedback. We appreciate the opportunity to clarify our work.\n\n---\n\n#### **W1: The novelty is limited.**\n\n- Our multi-scale bilateral grids architecture is a deliberate design choice, aimed at creating a **pragmatic and robust solution**, and we believe the specific way these components are unified represents a novel and valuable contribution.\n    - Our goal was to address the critical problem of geometric inaccuracy caused by photometric inconsistencies in autonomous driving scenes.\n    - After exploring various approaches, including more complex 3D disentanglement techniques, we focused on developing a method that strikes a crucial balance between reconstruction performance, computational efficiency, and ease of integration.\n- While we build upon known components, our contribution is a **principled unification** that resolves their opposing weaknesses.\n    - Our framework overcomes the optimization instability of fine-grained bilateral grids while adding the local modeling capability that global appearance codes lack.\n    - The core technical insight is the **coarse-to-fine residual refinement** process, where a stable global transformation is learned first, followed by residual corrections from finer grids.\n- The success of this approach is demonstrated by its strong and consistent performance.\n    - The **identical architecture and hyperparameters** were used across four diverse datasets (including day and night scenes) without any per-scene tuning, proving its robustness and generalizability.\n    - We believe this balance of performance, efficiency, and stability offers a valuable and practical solution that addresses a key gap for real-world driving scene reconstruction.\n\n---\n\n#### **W2: The analysis in Section 4.3 is unconvincing.**\n\n- We appreciate reviewer **2hvk** for this sharp critique and acknowledge that our original explanation required further clarification.\n- We totally agree that a more rigorous analysis is necessary to elucidate why the multi-scale bilateral grids achieve superior performance and provide valuable practical guidance for future research\n\nTo quantify photometric inconsistency between camera views, we design an experiment to make further analysis from the real-world driving dataset.\n\n- We regret the front camera view as reference, left-front and right-front camera as target views\n- Using ground truth LiDAR points $L_w$, camera poses $P_{ref},P_{trg}^i$, and intrinsic matrix $K_{ref},K_{trg}^i$, we get corresponding pixels that represent the same 3D world position among reference views and target views.\n    - For each corresponding pair, the color values are sampled from their respective images. These colors are converted to luminance, and the difference $\\Delta L$ between them can be calculated.\n    - This process is repeated for all initial LiDAR points to create a large set of luminance differences. A histogram of this set provides the final statistical distribution, $P(\\Delta L)$ which characterizes the inherent photometric inconsistency between the camera views.\n    - Compare $P(\\Delta L) $ distribution of affine transformation using KL Divergence. We can further prove which distribution better models the real-world appearance inconsistency of the dataset.\n- The experiment on our NuScenes test set show that our method has more similar distribution with GT.\n\n> Table R10: Comparison of KL Divergence.\n|             | KL Divergence with dataset distribution |\n|---------|----------|\n| multi-scale bilateral grids' affine transformation | 0.79  |\n| single bilateral grids' affine transformation      | 1.25 |\n\n- Our insight is color inconsistency in real scenes is affected by random factors such as lighting, which requires more diverse modeling capabilities.\n\n---\n\n#### **Q1: Please clarify the implementation details for the \"Baseline\" method used in comparisons (e.g., Fig. 3).**\n\n- The \"**Baseline**\" method used in our main comparison tables (**Table 1, 3, 4**) and figures (**Figure 2, 3, 4**) refers to **OmniRe [A]**. As stated in **Section 4.1**.\n- The variants presented in our main ablation study are:\n    - **Baseline**: OmniRe.\n    - **Baseline w/AC**: OmniRe with appearance codes added.\n    - **Baseline w/BG**: OmniRe with single bilateral grids added.\n    - **Ours**: OmniRe [5] with our proposed multi-scale bilateral grids.\n- We will revise to make these definitions explicit in the text and figure captions to avoid confusion. Thank reviewer **2hvk** for suggestions.\n\n---\n\n#### **Q2: Could the authors provide video results for the driving sequences?**\n\n- We have already included a comprehensive set of video comparisons in the **supplementary material.**\n- To view them, please unzip the supplementary folder and open the `index.html` file in a web browser.\n    - These videos showcase the temporal consistency and high geometric fidelity of our reconstructions across various driving sequences.\n    - We believe these results strongly support our paper's claims.\n\n---\n\n#### **Q3: Could you provide some qualitative examples of these failure cases (extremely complex, fast-moving, or highly non-rigid objects)?**\n\nCause we couldn't upload PDFs. We claim the mechanism for ****Failure cases **as follows:**\n\n- Our method relies on LiDAR data to provide strong geometric supervision.\n    - In these extreme cases, the sparse LiDAR points may not align perfectly with the object's appearance in the camera image due to the motion blur.\n- When LiDAR points for fast-moving dynamic objects poor, the optimization process may prioritize minimizing the 2D photometric error.\n    - It may cause it to \"bake\" the motion blur artifact into the rendered texture instead of reconstructing a crisp 3D shape.\n    - This can be considered a form of overfitting to a transient, motion-induced visual artifact.\n- We will add a dedicated section with qualitative examples of these failure cases to the supplementary material in the next version.\n\n---\n\n#### **Q4: Adding finer grids can harm geometric performance.**\n\n- Our ablation results in **Table 5** validate our hierarchical design by demonstrating that performance depends on a balanced coarse-to-fine structure, not merely on grid resolution.\n- The experiments show that:\n    - Both naively adding excessively fine grids (which overfits photometric noise) and removing the essential coarse grid (which destabilizes optimization) significantly harm geometric accuracy.\n    - Respectively. Our full three-level model achieves the best balance and performance with a CD of **1.161**.\n\n---\n\n#### **Q4: The robustness of fixed grid architecture & Does the optimal grid configuration change per-scene (e.g., night vs. day), requiring manual tuning?**\n\n- Crucially, our architecture is robust and does not require per-scene tuning.\n- We used the **identical three-level architecture and hyperparameters** across all four diverse datasets—which included varied conditions like day, night, and different sensors—and achieved consistent results.\n- This proves the design generalizes well without manual adjustments.\n\n---\n\n#### **Q5: The method performs appearance correction in 2D as a post-processing step.**\n\n- Our method offers a pragmatic and efficient alternative to more complex 3D disentanglement techniques.\n- It is not a simple post-processing step but is **deeply integrated into the reconstruction pipeline**.\n- Through a joint optimization objective, the 2D appearance transformations are guided by the rendered 3D scene and constrained by 3D LiDAR data, ensuring they remain geometrically consistent.\n\n---\n\n#### **Q5: How does this compare conceptually to methods that disentangle appearance in 3D (e.g., Relightable 3DGS)?**\n\n- Our methods:\n    - The 2D approach we proposed is not only more efficient and easier to plug into other methods, but also boasts strong practicality and feasibility in real-world applications.\n    - Our approach can be easily integrated into various reconstruction and neural rendering pipelines to improve their reconstruction quality, without additional modifications to the representation.\n        - as demonstrated in our comparisons with OmniRe, ChatSim, and StreetGS.\n- The 3D approaches:\n    - The 3D approach is capable of creating a physical representation of the object/environment\n    - 3D approach e.g., Relightable 3DGS can better model the underlying physics of 3D view-dependent effects by scene properties like roughness, and albedo.\n        - But they usually need additional modifications to the representation.\n    - For large-scale, in-the-wild driving scenarios, accurately modeling these properties for all scene elements is a challenging, under-constrained problem.\n    - Often significantly more complex and computationally intensive.\n- We believe our method offers a valuable and practical solution for a critical problem in autonomous driving, while we appreciate the reviewer's insight and full 3D disentanglement remains a worthwhile but distinct future research direction.\n\n---\n\n#### **Q5: What are the fundamental limitations of a 2D approach for handling 3D view-dependent effects like specular highlights?**\n\n- **Limitations**: It cannot model the underlying physics of 3D view-dependent effects.\n- For a specular highlight, it does not create a physically accurate 3D representation of the highlight that would behave correctly from all novel viewpoints.\n- However, our method has the ability to **effectively mitigate their negative impact** on geometric reconstruction.\n\n---\n\nWe appreciate all your constructive comments and we will improve our work according to them in the near future. We respectfully hope that this clarification will prompt you to reconsider your assessment and potentially raise the score of our submission. If the reviewer has any follow-up questions, we are happy to discuss them.\n\n> Chen, Ziyu, et al. \"Omnire: Omni urban scene reconstruction.\" arXiv preprint arXiv:2408.16760 (2024)."}, {"review_id": "kjCESJ7tkC", "reviewer": "Reviewer_dvs1", "summary": "This paper proposes a novel multi-scale bilateral grid that unifies appearance codes and bilateral grids to handle the photometric inconsistency problem in reconstructing autonomous driving scenes. The multi-scale bilateral grid provides affine color transformation matrices for pixels in a coarse-to-fine manner, thereby achieving global and local photometric corrections. The experiments demonstrate the effectiveness of proposed methods.", "questions": "+ Can the multi-scale bilateral grid handle the reflection problem of several car window glass for better geometric accuracy?\n+ The improvements shown in Table 1 on the Waymo and Argoverse datasets are not obvious compared to the \"Baseline\" which has no photometric corrections.\n+ The efficiency analysis in Table 4 probably requires a \"Baseline\"(without any photometric corrections) model to further show the computational overhead of the proposed methods.", "limitations": "Yes, the limitation has been mentioned.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n+ The paper is well written and clearly expressed.\n+ The proposed multi-scale bilateral grid seems novel and effective.\n+ The experiments are thorough. The authors evaluate their methods on different datasets and baselines, and the improvements shown in several experiments are obvious.\n\n**Weakness**\n+ The improvements shown in Table 1 on the Waymo and Argoverse datasets are not obvious compared to the \"Baseline\" which has no photometric corrections.\n+ The efficiency analysis in Table 4 probably requires a \"Baseline\"(without any photometric corrections) model to further show the computational overhead of the proposed methods.", "was_revised": true, "final_justification": "My main concerns about the performance of the proposed methods have been addressed. In more challenging scenes provided by the authors, the proposed methods obtain an obvious improvement in comparison to the traditional appearance codings. However, just as Reviewer 2hvk said, the proposed appearance coding might be a smart engineering combination of known components. In general, I think it is a simple but effective work. Therefore, I tend to keep my rating as boardline accept.", "rebuttal": "We thank reviewer **dvs1** for the constructive review and for recognizing the novelty and thoroughness of our experiments.\n\n---\n\n#### **W1 / Q2: The improvements shown in Table 1 on the Waymo and Argoverse datasets are not obvious compared to the \"Baseline\" which has no photometric corrections.**\n\n- The core reasons for this are **a.** **dataset characteristics** and **b. intrinsic capabilities of gaussian splatting**. \n- We would like to clarify that our method still provides a significant improvement in **geometric accuracy** under these reason. Meanwhile, we provide additional evaluation.\n\n    - **Dataset Characteristics.** In our evaluation, the selected test sequences from the Argoverse and PandaSet datasets feature relatively more consistent lighting conditions. And our method mainly focuses on appearance inconsistencies, making improvements not obvious.\n    - **Gaussian Splatting has capabilities of fitting appearance variance using \"floater\" [A].** Gaussian Splatting model has a native capacity to handle minor appearance variations.\n        1. It can model some inconsistencies by adjusting its **Spherical Harmonics (SH) coefficients** and by using **semi-transparent Gaussians**.\n        2. When these inconsistencies are less severe, a baseline model might \"bake\" these small errors into 3D geometry, which is why the PSNR/SSIM improvement of our method is more modest on those datasets.\n        3. However, it would harm geometry fidelity, as proved in **Table 1,** geometry metrics of baseline obviously worse.\n    - Our method prevents this, leading to consistently better geometric outcomes across all tested datasets, which is the core contribution of our paper.\n- **Additional experiments to validate.** To further validate our method's performance, we conduct additional experiments on a curated subset of scenes featuring more extreme lighting and weather conditions. The scenes are listed here:\n\n> Table R10: Additional challenging Argoverse scenarios.\n|scene ID|description |\n| --- | --- |\n| 49e970c4 | Extreme Lighting, Overexposure, Lens Flare |\n| 8184872e | Complex Shadows, Direct Sunlight, Information Loss |\n| 91923e20 | Extreme Lighting, Deep Shadows, Camera Artifacts. |\n| a8a2fbc2 | Low-Angle Sun, Complex Shadows, Texture Disentanglement. |\n| b403f8a3 | Overpass Structure, Extreme Lighting Transitions, Complex Indirect Illumination |\n| eaaf5ad3 | Multiple Artificial Lights, Wet Surfaces, Rainy Conditions |\n\n> Table R11: Evaluation results of our multi-scale bilateral grids on additional challenging Argoverse scenarios:\n|  | Recon. PSNR$\\uparrow$ | Recon. SSIM$\\uparrow$ | Recon. CD$\\downarrow$ | Recon. RMSE$\\downarrow$ | Recon. Depth$\\downarrow$ | NVS PSNR$\\uparrow$ | NVS SSIM$\\uparrow$ |\n| --- | --- | --- | --- | --- | --- | --- | --- |\n| 49e970c4 | 23.35 | 0.789 | 1.229 | 5.175 | 0.060 | 21.33 | 0.673 |\n| 8184872e | 24.23 | 0.864 | 0.666 | 3.303 | 0.022 | 21.76 | 0.766 |\n| 91923e20 | 25.86 | 0.876 | 0.820 | 3.342 | 0.019 | 23.48 | 0.789 |\n| a8a2fbc2 | 25.13 | 0.823 | 0.774 | 3.513 | 0.033 | 22.65 | 0.716 |\n| b403f8a3 | 27.55 | 0.926 | 0.298 | 2.822 | 0.004 | 25.25 | 0.870 |\n| eaaf5ad3 | 25.60 | 0.899 | 0.566 | 3.228 | 0.022 | 23.28 | 0.824 |\n| Average | **25.29** | **0.863** | **0.726** | **3.564** | **0.027** | **22.96** | **0.773** |\n\n> Table R12: Evaluation results of OmniRe on additional challenging Argoverse scenarios:\n|  | Recon. PSNR$\\uparrow$ | Recon. SSIM$\\uparrow$ | Recon. CD$\\downarrow$ | Recon. RMSE$\\downarrow$ | Recon. Depth$\\downarrow$ | NVS PSNR$\\uparrow$ | NVS SSIM$\\uparrow$ |\n| --- | --- | --- | --- | --- | --- | --- | --- |\n| 49e970c4 | 21.99 | 0.772 | 1.474 | 5.270 | 0.129 | 20.37 | 0.659 |\n| 8184872e | 22.79 | 0.850 | 0.862 | 3.389 | 0.101 | 20.79 | 0.751 |\n| 91923e20 | 24.47 | 0.859 | 1.171 | 3.463 | 0.124 | 22.67 | 0.776 |\n| a8a2fbc2 | 23.13 | 0.805 | 1.107 | 3.602 | 0.140 | 21.20 | 0.701 |\n| b403f8a3 | 26.65 | 0.922 | 0.396 | 2.862 | 0.014 | 24.77 | 0.869 |\n| eaaf5ad3 | 24.69 | 0.889 | 0.776 | 3.295 | 0.083 | 22.79 | 0.810 |\n| Average | 23.95 | 0.850 | 0.964 | 3.647 | 0.098 | 22.10 | 0.761 |\n- In these challenging Argoverse scenarios, our method outperforms OmniRe by an average of **1.338 dB** in PSNR.\n\n> Table R13: Additional challenging Waymo scenarios.\n|scene ID|description |\n| --- | --- |\n| 10 | Night Scene, Artificial Lighting, Lens Flare |\n| 30 | Night Scene, Rainy Conditions, Specular Reflections |\n| 539 | High Dynamic Range, Low-Angle Sun, Sun Glare |\n| 550 | High Dynamic Range, Storefront Reflections, Hard Shadows |\n| 561 | Low-Angle Sun, Lens Flare, Overexposure |\n| 570 | Rainy Conditions, Low Light, Motion Blur |\n\n> Table R14: Evaluation results of our multi-scale bilateral grids on additional challenging Waymo scenarios:\n|  | Recon. PSNR$\\uparrow$ | Recon. SSIM$\\uparrow$ | Recon. CD$\\downarrow$ | Recon. RMSE$\\downarrow$ | Recon. Depth$\\downarrow$ | NVS PSNR$\\uparrow$ | NVS SSIM$\\uparrow$ |\n| --- | --- | --- | --- | --- | --- | --- | --- |\n| 10 | 31.46 | 0.716 | 0.155 | 1.167 | 0.007 | 30.10 | 0.674 |\n| 30 | 29.29 | 0.734 | 0.528 | 2.731 | 0.024 | 27.98 | 0.698 |\n| 539 | 28.17 | 0.826 | 0.385 | 2.530 | 0.052 | 24.60 | 0.746 |\n| 550 | 36.26 | 0.962 | 0.050 | 1.049 | 0.001 | 32.24 | 0.943 |\n| 561 | 32.68 | 0.944 | 0.094 | 1.894 | 0.002 | 29.51 | 0.898 |\n| 570 | 29.52 | 0.861 | 0.421 | 1.707 | 0.043 | 25.60 | 0.802 |\n| Average | **31.23** | **0.841** | **0.272** | **1.846** | **0.021** | **28.34** | **0.793** |\n\n> Table R15: Evaluation results of OmniRe on additional challenging Waymo scenarios.\n|  | Recon. PSNR$\\uparrow$ | Recon. SSIM$\\uparrow$ | Recon. CD$\\downarrow$ | Recon. RMSE$\\downarrow$ | Recon.  Depth$\\downarrow$ | NVS PSNR$\\uparrow$ | NVS SSIM$\\uparrow$ |\n| --- | --- | --- | --- | --- | --- | --- | --- |\n| 10 | 30.80 | 0.711 | 0.197 | 1.189 | 0.020 | 29.79 | 0.672 |\n| 30 | 28.52 | 0.724 | 0.559 | 2.765 | 0.043 | 26.88 | 0.689 |\n| 539 | 25.01 | 0.822 | 0.617 | 2.704 | 0.146 | 21.86 | 0.739 |\n| 550 | 32.74 | 0.953 | 0.052 | 1.133 | 0.001 | 32.13 | 0.946 |\n| 561 | 31.12 | 0.942 | 0.102 | 1.898 | 0.003 | 28.57 | 0.898 |\n| 570 | 27.50 | 0.847 | 0.585 | 1.815 | 0.155 | 24.97 | 0.800 |\n| Average | 29.28 | 0.833 | 0.352 | 1.917 | 0.061 | 27.37 | 0.791 |\n- In these challenging Waymo scenarios, our method outperforms OmniRe by an average of **1.950 dB** in PSNR.\n\n---\n\n#### **W2 / Q3: The efficiency analysis in Table 4 probably requires a \"Baseline\"(without any photometric corrections) model to further show the computational overhead of the proposed methods.**\n\n- We thank the reviewer for this constructive suggestion.\n    - We agree that including a \"Baseline\" model without photometric corrections will provide a clearer benchmark for the efficiency analysis. We will update Table 4 accordingly in the final version.\n- We wish to clarify that the computational overhead of the baseline is negligible and **nearly identical to our \"Baseline w/AC\" model**, that is the reason why we didn't add another line to show.\n\n---\n\n#### **Q1: Can the multi-scale bilateral grid handle the reflection problem of several car window glass for better geometric accuracy?**\n\n- Our method can **effectively mitigate their negative impact** on geometric reconstruction.\n- It achieves this by treating reflections as a source of severe, view-dependent photometric inconsistency.\n    - When reflections cause a car window to appear bright from one angle and dark from another, our model learns a transformation to reduce this mismatch.\n    - By improving photometric consistency on these surfaces, our method prevents the optimization process from generating erroneous geometric artifacts, such as floaters, which leads to improved overall geometric accuracy.\n- We provide both qualitative and quantitative evidence to support this:\n    - **Qualitative Results:**\n        - Figure 3(a) in the main paper demonstrates our method’s ability to handle specular highlights, a phenomenon physically related to reflections.\n        - Figure A2 in the supplementary material provides a direct visual comparison, showing that our approach produces a better visual appearance and fewer artifacts in the reflective regions of a vehicle compared to baselines.\n    - **Quantitative Results:** To quantitatively evaluate our method on challenging surfaces, we focus on vehicle-specific metrics.\n        - We use semantic masks to isolate vehicles—which often feature reflective windows—and compute metrics for both appearance and geometric.\n        - The following Table presents the results, averaged across 20 scenes (5 scenes each from the Waymo, Pandaset, Argoverse, and NuScenes).\n        - The data demonstrate its effectiveness in mitigating the negative impact of reflective surfaces.\n\n> Table R16: Comparison results of vehicle-specific metrics.\n|  | PSNR$\\uparrow$ | SSIM$\\uparrow$ | CD$\\downarrow$ |\n| --- | --- | --- | --- |\n| OmniRe | 24.24 | 0.784 | 8.675 |\n| Ours | **24.80** | **0.791** | **7.400** |\n\n---\n\nWe appreciate all your constructive comments and we will improve our work according to them in the near future. If the reviewer has any follow-up questions, we are happy to discuss them.\n\n> [A] Zhang, Dongbin, et al. \"Gaussian in the wild: 3d gaussian splatting for unconstrained image collections.\" European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024."}]

Author rebuttal:

Meta-review: The paper tackles inconsistent appearance in 3DGS driving scenes through a multi-scale bilateral grid that operates hierarchically to provide both global (coarse-level) and local (fine-level) corrections. Initial feedback of the reviewers was generally positive, highlighting the paper’s strengths while raising questions regarding performance in challenging scenarios, the degree of novelty, limited improvements on the Argoverse and Pandaset datasets, and the impact of using a smaller single-scale bilateral grid. The rebuttal successfully addressed these concerns, and overall the paper is considered strong, offering interesting and valuable contributions.
Decision: Accept (poster)