Title: Unifying Appearance Codes and Bilateral Grids for Driving Scene Gaussian Splatting
Venue: NeurIPS 2025
Accepted: yes
Keywords: Gaussian Splatting

Bilateral Processing

Autonomous driving simulation
Abstract: Neural rendering techniques, including NeRF and Gaussian Splatting (GS), rely on photometric consistency to produce high-quality reconstructions. However, in real-world driving scenarios, it is challenging to guarantee perfect photometric consistency in acquired images. Appearance codes have been widely used to address this issue, but their modeling capability is limited, as a single code is applied to the entire image. Recently, the bilateral grid was introduced to perform pixel-wise color mapping, but it is difficult to optimize and constrain effectively. In this paper, we propose a novel multi-scale bilateral grid that unifies appearance codes and bilateral grids. We demonstrate that this approach significantly improves geometric accuracy in dynamic, decoupled autonomous driving scene reconstruction, outperforming both appearance codes and bilateral grids. This is crucial for autonomous driving, where accurate geometry is important for obstacle avoidance and control. Our method shows strong results across four datasets: Waymo, NuScenes, Argoverse, and PandaSet. We further demonstrate that the improvement in geometry is driven by the multi-scale bilateral grid, which effectively reduces floaters caused by photometric inconsistency.


Peer reviews:
comments: SUMMARY:
This paper proposes a novel multi-scale bilateral grid that unifies appearance codes and bilateral grids to handle the photometric inconsistency problem in reconstructing autonomous driving scenes. The multi-scale bilateral grid provides affine color transformation matrices for pixels in a coarse-to-fine manner, thereby achieving global and local photometric corrections. The experiments demonstrate the effectiveness of proposed methods.

STRENGTHS AND WEAKNESSES:
**Strengths**
+ The paper is well written and clearly expressed.
+ The proposed multi-scale bilateral grid seems novel and effective.
+ The experiments are thorough. The authors evaluate their methods on different datasets and baselines, and the improvements shown in several experiments are obvious.

**Weakness**
+ The improvements shown in Table 1 on the Waymo and Argoverse datasets are not obvious compared to the "Baseline" which has no photometric corrections.
+ The efficiency analysis in Table 4 probably requires a "Baseline"(without any photometric corrections) model to further show the computational overhead of the proposed methods.

QUESTIONS:
+ Can the multi-scale bilateral grid handle the reflection problem of several car window glass for better geometric accuracy?
+ The improvements shown in Table 1 on the Waymo and Argoverse datasets are not obvious compared to the "Baseline" which has no photometric corrections.
+ The efficiency analysis in Table 4 probably requires a "Baseline"(without any photometric corrections) model to further show the computational overhead of the proposed methods.

LIMITATIONS:
Yes, the limitation has been mentioned.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-04

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper tackles inconsistent appearance in 3DGS driving scenes with a multi-scale bilateral grid. It works hierarchically to provide both global (coarse levels) and local (fine levels) appearance correction. Experiments on four driving datasets show improved geometry and reduced visual artifacts, outperforming standard baselines like appearance codes.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**:\n- The paper tackles a critical and well-known challenge in real-world 3D reconstruction. Photometric inconsistency in driving data is a known source of artifacts and geometric errors for 3DGS. Solving this is valuable for downstream tasks like simulation and planning.\n- The method shows strong quantitative and qualitative results. Evaluation is thorough, covering four large driving datasets (Waymo, etc.) and multiple baselines. The handling of dynamic objects further demonstrates practical relevance.\n\n\n**Weaknesses**:\n- The novelty is limited. The proposed multi-scale grid is a heuristic that combines two existing ideas (global and per-pixel correction), rather than a fundamental new technique. The presentation overstates the novelty of what is effectively a smart engineering combination of known components.\n- The analysis in Section 4.3 is unconvincing. The central claim—that a \"flatter\" histogram signifies a better model—is an unproven assertion. A flat distribution could equally imply that the model is learning unstructured noise or is poorly constrained, whereas a peaky distribution might indicate an efficient model that has correctly identified the primary sources of error. The analysis shows a correlation between flatter histograms and better results, but fails to establish causation; Also Lacking a ground-truth comparison, it arbitrarily claims one learned distribution is superior to another, making the entire analysis feels hand-wavy and lacks rigor.\n\nQUESTIONS:\n1. Please clarify the implementation details for the \"Baseline\" method used in comparisons (e.g., Fig. 3).\n2. The paper evaluates dynamic scenes using static frames, which can be prone to cherry-picking favorable results. Could the authors provide video results for the driving sequences?\n3. Autonomous driving scenarios are inherently dynamic. The paper mentions that modeling \"extremely complex, fast-moving, or highly non-rigid objects\" remains a challenge.  Could you provide some qualitative examples of these failure cases?\n4. The ablation in Table 5 is concerning: adding finer grids can harm geometric performance. This challenges the robustness of the fixed grid architecture. How sensitive is performance to this specific three-level design? Does the optimal grid configuration change per-scene (e.g., night vs. day), requiring manual tuning?\n5. The method performs appearance correction in 2D as a post-processing step. How does this compare conceptually to methods that disentangle appearance in 3D (e.g., Relightable 3DGS)? What are the fundamental limitations of a 2D approach for handling 3D view-dependent effects like specular highlights?\n\nLIMITATIONS:\nYes. The authors discuss limitations in the conclusion, primarily: increased computational overhead and challenges with modeling highly dynamic or non-rigid objects.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-09"}

comments: SUMMARY:
This paper proposes multi-scale bilateral grids to address the photometric inconsistency in a temporal multi-camera setting. The proposed method combines the advantages of appearance code and vanilla bilateral grid, thus being optimization-friendly and having better modeling capacity. The paper conducts extensive experiments on four driving datasets, demonstrating good performance.

STRENGTHS AND WEAKNESSES:
Strengths:
1. Good motivation. The photometric inconsistency is an important problem in 3D reconstruction. The paper proposes to address it by combining the advantages of appearance code and bilateral grids.
2. Good writing. The paper is well written and easy to follow.

Weaknesses:
1. Lack of novelty. The extension of single-scale bilateral grid to the multi-scale counterpart is quite straightforward and does not reflect much insight.
2. Inconsistency in performance improvement. The performance improvements on the Argoverse and Pandaset datasets are less prominent and even minimal compared with those on the Waymo and nuScenes datasets. The paper does not include any discussion on it.

QUESTIONS:
1. Why are the performance improvements on the Argoverse and Pandaset datasets less prominent compared with those on the Waymo and nuScenes datasets, including both the 3D reconstruction and novel view synthesis tasks?
2. How is the performance of the baseline model with a smaller sized single-scale bilateral grid, e.g., with only the finest bilateral grid in the proposed multi-scale setting?

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 2
DATE: 2025-07-02

comments: SUMMARY:
This paper proposes a method to learn to represent the appearance variation in autonomous driving scenes reconstruction with 3D gaussian splatting scene graph.
Instead of relying on a single appearance embedding that provides a global transform on the image level but fails to represent local variation, or a bilateral grid that will learn pixel-wise variation but is hard to optimize, this paper proposes to use a composition of multi-scale bilateral grids.
The coarse bilateral grid will ressemble the appearance code by  modeling an image level transformation, then progressively  reduce patch size to learn more local transformations.
By doing so, the model is able to learn more local variation than appearance embedding, while also guiding the bilateral grid training with a composition of progressively more detailed transformation.
By doing so, this paper is able to provide better geometric and photometric results than using either appearance code or bilateral grid, as it is shown through extensive benchmarking on 4 different driving datasets.

STRENGTHS AND WEAKNESSES:
Strengths:
- SOTA reconstruction quality compared to appearance code and bilateral grids.
- The idea is simple and intuitive yet provides good results.
- Paper is overall clear, with benchmarking on 4 different driving datasets. 
- Significantly less parameters to process compared to purely pixel-wise bilateral grids.

Weaknesses:
- Computation is much higher compared to simply using appearance code.

QUESTIONS:
How would this method perform in very difficult scenarios (night scenes, sun flare)?
Are there cases where the multi-scale bilateral grids are able to overfit to some dynamic element instead of representing it correctly in 3D?

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03