Title: Robust Neural Rendering in the Wild with Asymmetric Dual 3D Gaussian Splatting
Venue: NeurIPS 2025
Accepted: yes
Keywords: Novel View Synthesis; 3D Reconstruction; In-the-wild Data
Abstract: 3D reconstruction from in-the-wild images remains a challenging task due to inconsistent lighting conditions and transient distractors. Existing methods typically rely on heuristic strategies to handle the low-quality training data, which often struggle to produce stable and consistent reconstructions, frequently resulting in visual artifacts. In this work, we propose Asymmetric Dual 3DGS, a novel framework that leverages the stochastic nature of these artifacts: they tend to vary across different training runs due to minor randomness. Specifically, our method trains two 3D Gaussian Splatting (3DGS) models in parallel, enforcing a consistency constraint that encourages convergence on reliable scene geometry while suppressing inconsistent artifacts. To prevent the two models from collapsing into similar failure modes due to confirmation bias, we introduce a divergent masking strategy that applies two complementary masks: a multi-cue adaptive mask and a self-supervised soft mask, which leads to an asymmetric training process of the two models, reducing shared error modes. In addition, to improve the efficiency of model training, we introduce a lightweight variant called Dynamic EMA Proxy, which replaces one of the two models with a dynamically updated Exponential Moving Average (EMA) proxy, and employs an alternating masking strategy to preserve divergence. Extensive experiments on challenging real-world datasets demonstrate that our method consistently outperforms existing approaches while achieving high efficiency. Codes and trained models will be released.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper propose a novel and interesting framework to tackle distractors for in-the-wild scene reconstruction. The authors leverage the stochastic nature of artifacts caused by distractors to form a mutual consistency loss for different runs. To avoid both runs converage toward the same reconstruction errors, the authors apply two types of dynamic masking strategy for two models. For computational efficiency, the authors apply EMA strategy during optimization to avoid double gradient descent. In a nutshell, this paper is interesting and somewhat intuitive.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:** \n\n1. This paper propose to leverage the stochastic nature of artifacts to tackle distractors, which is insteresting. \n2. To obtain mutual consistency across different runs, this paper trains two models in parallel, and further accelerate the training speed with EMA strategy.\n3. The ablation study is comprehensive and convincing.\n\n**Weakness:**\n\n1. My major concern is that some statements in this paper requires further visual analysis to support, especially the stochastic nature of artifacts, which is the core idea in this paper. \n    - For example, I wanna see more examples across various scenes and different runs to demonstrate the appearance of artifcats is stochastic. \n    - The visual comparison between applying the same mask or different mask on two parallel models is also necessary to show whether this design can avoid converging toward the same reconstruction errors\n    - Does the above conclusion also work for EMA stategy? A visual comparison is also needed here.\n\nQUESTIONS:\nFor Eq.(7), will it still work in the case that the reconstructed 3D scene overfits to the distractors? If not, how does this method avoid overfitting to distractors with 3D Gaussians floating in the sky?  \nMore can be seen in \"Weakness\"\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a new method to **reconstruct scenes despite transient objects** being present in only some of the frames (e.g. a passerby in an outdoor scene). Specifically, Asymmetric Dual 3DGS optimizes two 3DGS in parallel and leverages the consistency between the two reconstructions to improve them. This is achieved in two steps: \n* **Enforcing consistency between the models**: their renderings are supervised to be identical.\n* **Preventing the two reconstructions from being identical**: the two 3DGS identify transient objects in different ways, the first 3DGS uses a combination of SAM+COLMAP+DINOv2 and residual error, while the second uses the DINOv2 residual error only (i.e. the difference in DINO features between render and ground-truth).\nThen, the authors apply both transient strategies to the first model, and replace the second model by a moving average of the first, reducing the computational overhead of their technique.\nBoth approaches perform well on a number of datasets with transient objects and varying lighting, comparing against numerous methods such as HybridGS and WildGaussians.\n\nSTRENGTHS AND WEAKNESSES:\n+: The overall observation and philosophy behind the approach are sound and interesting: training two similar 3DGS models to be similar, while making sure they remain different.\n\n-: However in practice, the way these reconstructions are kept different is by introducing more losses to perform the same task of transient mask estimation.\n\n-: The paper does not attempt to meaningfully explain how it differs from previous methods, and instead describes the entire literature of transient NeRF/3DGS as \"heuristic strategies\". This description is very vague and could be used to describe the proposed method too, which is instead called a \"principled framework\". For example, the estimation of multi-cue adaptive mask is very \"heuristic\".\n\n-: In fact, some of these works have a fairly similar idea, such as SimpleNeRF which also uses a simpler model to regularize the main one, or HybridGS which uses an additional 2DGS to separate moving elements. To be clear, these methods are not exactly identical, but the paper would benefit from discussing these differences in detail and why they matter.\n\n+: The experiments compare with several methods for radiance field reconstruction that are robust to transient objects. An extensive ablation study is also performed.\n\nQUESTIONS:\n- It is a bit counter-intuitive that the dual-models would perform better than a single model combining both masking strategies (Single w/ Mh/s). What is the interpretation of this result?\n- There are many other ways to keep the models different, and using two different losses for the same goal of masking transient objects is not the most intuitive. Have you experimented with other ideas, such as using different initializations or training on different subsets of images?\nMinor points and typos:\n- The term \"frame-independent\" is used with and without dash. It also does not exactly convey the meaning intended.\n- In the EMA section, it is unclear at the beginning that the main model uses both masking strategies. It should be mentioned early.\n\nLIMITATIONS:\n- yes, limitations are discussed but only in the supplementary material.\nDespite their contributions, these methods largely rely on heuristic strategies to suppress\ncorrupted supervision signals from low-quality training data. [...] In this work, we propose a\nprincipled framework, Asymmetric Dual 3DGS.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a novel method for outlier suppression in 3DGS training. Their method is motivated by the observation that training the same model twice, but with different sample ordering, results in output stochasticity in uncertain areas. They therefore train two model simultaneously enforcing a cross-consistency loss that encourages model to agree photometrically when rendered from the same view. Furthermore, they give each model separate masking strategies to mitigate confirmation bias. These masking strategies are similar to some previous works, but allow them to combine strengths of each. The method also involves some extra machinery to account for appearance modeling and offers an EMA variant offers efficiency gains. The method achieves state of the art results over most datasets and the design choices are thoroughly ablated.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper is generally well written in all aspects, and the technique is sufficiently described. \n\nThe method, though it is somewhat complicated in it's complete implementation, it achieves state of the art results over a sufficient set of the most relevant datasets, making it a significant contribution. This perhaps isn't too surprising, as their method allows them to combine the strengths of previous methods through the use of different masking strategies in the models. All design choices are thoroughly ablated, justifying the architecture.  \n\nDetails are scant in the description of the Multi-Cue Adaptive Mask. I would prefer to see some of this in mathematical notation. The method also sounds somewhat similar to the newly released RoMo [1], in that it combines SAM features with epipolar cues.\n\nIf I further wanted to nitpick at weaknesses, one could say that because the method could be seen as one that combines strengths of previous works, it could be classed as less than original. The authors observation that retrained models tend to disagree in areas of uncertainty is simply empirical and could have been better motivated. Yet this is difficult to do in a limited amount of space.\n\nIt could also be faulted for being somewhat complex - yet there is no other simpler technique that can achieve the same metrics.\n\n[1] https://romosfm.github.io/\n\nQUESTIONS:\n1. Can you elaborate on the technique of the multi-cue adaptive mask? I'm a bit confused - are \"static\" regions always included? note that semantically static objects could still be considered as outliers due to difficult to model specular reflections. Are then only the \"dynamic\" regions considered for the union of the other cues?\n\n2. How is the self-supervised mask regularized against collapsing to zero?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the challenging problem of 3D scene reconstruction from \"in-the-wild\" image collections, which are often plagued by inconsistent lighting and transient objects (e.g., pedestrians, vehicles) that corrupt the training signal and lead to visual artifacts in the final rendering. Existing methods often rely on heuristic strategies such as per-image appearance embeddings or hand-crafted outlier filtering rules, which can lack stability and generalizability.\nAuthors propose Asymmetric Dual 3DGS -- a novel framework that enforces consistency between two concurrently trained 3DGS models.\nThe authors claim state-of-the-art performance in reconstruction quality (PSNR, SSIM, LPIPS) and training efficiency, validated through extensive experiments on the challenging NeRF On-the-go, RobustNeRF, and PhotoTourism datasets. The authors claim state-of-the-art performance in reconstruction quality (PSNR, SSIM, LPIPS) and training efficiency, validated through extensive experiments on the challenging NeRF On-the-go, RobustNeRF, and PhotoTourism datasets.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n* Principled novel core Idea. By leveraging consistency between two independent training processes as a supervisory signal, the method learns to distinguish reliable scene structure from spurious, inconsistent information, which is typically present in the in-the-wild data.\n* The asymmetric dual 3DGS framework, with its divergent masking strategy (M_h and Ms), is a particularly smart mechanism for mitigating the risk of \"Confirmation Bias\" among the different 3DGS models.\n* Practicality and technical innovation of the EMA-GS variant -- authors successfully tackle the increased computational budget of the dual 3DGS framework.\n* Rigorous & comprehensive empirical validation -- authors compare with various prior methods, including the most recent ones. Table 4 is particularly insightful and includes many important ablations.\n\nWeaknesses:\n* The difference and comparison between HybridGS (another dual method) is not articulated well enough.\n* Lack of hyper-parameter sensitivity analysis. Specifically, the method's performance hinges on the careful balancing of several loss terms and update rules, controlled by (\\lambda_m and \\lamba_{mask}). For a method that claims \"robustness,\" it is critical to demonstrate that it is not overly sensitive to its own internal settings.\n* Computational cost of mask generation is unclear (was it included into the run-times?)\n* The limitations section (in the Supplementary) is quite brief.\n\nQUESTIONS:\n* Improve experimental clarity: clarify all ambiguous methodological and experimental details, including the full computational cost of the pipeline (with mask generation), the precise definition of the alternating masking strategy, and provide statistical significance for the main results (e.g., mean and standard deviation over multiple runs)\n* It would be great to include sensitivity analysis for the key hyper-parameters\n* Strengthen Related Work -- a more nuanced discussion of the related literature, explicitly contrasting the proposed conceptual framework with that of HybridGS would be quite beneficial.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-06"}