Robust Neural Rendering in the Wild with Asymmetric Dual 3D Gaussian Splatting
Authors: Chengqi Li
Zhihao Shi
Yangdi Lu
Wenbo He
Xiangyu Xu
Venue: NeurIPS 2025 spotlight ()
Keywords: Novel View Synthesis; 3D Reconstruction; In-the-wild Data
Primary area: applications

Abstract: 3D reconstruction from in-the-wild images remains a challenging task due to inconsistent lighting conditions and transient distractors. Existing methods typically rely on heuristic strategies to handle the low-quality training data, which often struggle to produce stable and consistent reconstructions, frequently resulting in visual artifacts. In this work, we propose Asymmetric Dual 3DGS, a novel framework that leverages the stochastic nature of these artifacts: they tend to vary across different training runs due to minor randomness. Specifically, our method trains two 3D Gaussian Splatting (3DGS) models in parallel, enforcing a consistency constraint that encourages convergence on reliable scene geometry while suppressing inconsistent artifacts. To prevent the two models from collapsing into similar failure modes due to confirmation bias, we introduce a divergent masking strategy that applies two complementary masks: a multi-cue adaptive mask and a self-supervised soft mask, which leads to an asymmetric training process of the two models, reducing shared error modes. In addition, to improve the efficiency of model training, we introduce a lightweight variant called Dynamic EMA Proxy, which replaces one of the two models with a dynamically updated Exponential Moving Average (EMA) proxy, and employs an alternating masking strategy to preserve divergence. Extensive experiments on challenging real-world datasets demonstrate that our method consistently outperforms existing approaches while achieving high efficiency. Codes and trained models will be released.

Robust Neural Rendering in the Wild with Asymmetric Dual 3D Gaussian Splatting
Chengqi Li!
Zhihao Shi
Yangdi Lu
Wenbo He Xiangyu Xu²*
McMaster University
2 an Jiaotong University
Abstract
3D reconstruction from in-the-wild images remains a challenging task due to incon- sistent lighting conditions and transient distractors. Existing methods typically rely heuristic strategies to handle the low-quality training data, which often struggle to produce stable and consistent reconstructions, frequently resulting in visual artifacts. In this work, we propose AsymGS, novel framework that leverages the stochastic nature of these artifacts: they tend to vary across different training runs due minor randomness. Specifically, method trains two 3D Gaussian Splatting (3DGS) models in parallel, enforcing a consistency constraint that en- courages convergence on reliable scene geometry while suppressing inconsistent artifacts. To prevent the two models from collapsing into similar failure modes due confirmation bias, we introduce divergent masking strategy that applies two complementary masks: a multí-cue adaptive mask and self-supervised soft mask, which leads to an asymmetric training process of the two models, reducing shared error modes. In addition, to improve the efficiency of model training, we introduce lightweight variant called Dynamic EMA Proxy, which replaces one of the two models with dynamically updated Exponential Moving Average (EMA) proxy, and employs an alternating masking strategy to preserve divergence. Extensive experiments on challenging real-world datasets demonstrate that our method con- sistently outperforms existing approaches while achieving high efficiency. See the project website at https: httth::://tstee /ttt::://sseeeli88 gi thub. 1o/ AsymGS)
Introduction
3D scene reconstruction from multiple views fundamental problem in computer vision. Recent advances such as Neural Radiance Fields (NcRF) [15] and 3D Gaussian Splatting (3DGS) [8] have achieved impressive rendering quality by learning volumetric or point-based scene representations from posed images. However, these methods typically assume that training images exhibit consistent illumination and minimal occlusion, which are rarely satisfied in real-world settings.
In-the-wild images are often captured under varying lighting conditions and contain transient dis- tractors such pedestrians vehicles; these factors introduce substantial noise into the supervision signal, leading to degraded reconstruction quality and visual artifacts. While several recent works
have attempted address these challenges [14 19 18
2 13), they largely rely on heuristic strategies to suppress the effects of corrupted supervision from low-quality training data, such as
26
I0
20
per-image appearance embeddings that are only weakly or indirectly supervised through photometric losses [1] 26). or hand-crafted rules to filter outlier training signals [19]. As result, such approaches often lack stability and generalizability, which leads artifact- artifact-prone prone reconstructions.
To bridge this gap, we propose new framework, called AsymGS, which is motivated by a key empirical observation: artifacts arising from low-quality in-the-wild training data are typically
"Coreepooning author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Diffrwed Morkak
Onn
(EMA-GS)
IlyhnddS
PSNE
WildGrsssan
Sps-ttittingg
GS.W
3DGS
To-hhr wild Digeass
02
605
6.9
455
Training lns.
Difiree Romis Ran E:4,
Figure 1: Left: The key insight of this work is that artifacts arising from low-quality in in-the-wild inputs are typically stochastic across different runs of the same model (Baseline: Run 1 vs. Run 2). This motivates the design of the Asymmetric Dual 3DGS framework, which enhances true scene structure while suppressing errors through cross-model consistency (w/ consistency). Right: Our method compares favorably against the offte-of-the-- approaches in terms of reconstruction quality while maintaining high training efficiency. Results are on the NeRF On-the-go dataset [18].
stochastic. In other words, the artifacts vary randomly across different runs of the same model with only minor training perturbations, such as data order shuffling (see Figure Left). This suggests that enforcing consistency between independently trained 3DGS can help suppress unreliable or spurious signals in the mi-the-wild training images. To this end, we introduce a dual-model architecture where two 3DGS models are trained concurrently with a consistency constraint, following the intuition that true scene structure should be consistently reconstructed across different model runs, while the artifacts induced by low-quality data tend to diverge.
Nevertheless, naively training two models in parallel can lead to confirmation bias, where both models reinforce the same errors. To encourage more divergent error modes and mitigate confirmation bias, we introduce a divergent masking strategy: applying distinct masks to each model that emphasize complementary factors for filtering out transient or distracting content. One mask is learned in a self-supervised manner based on feature-level similarity between predicted and ground truth images, while the other, called multi-cue adaptive mask, uses ddereo-based correspondence to identify likely distracting regions. These complementary filtering schemes encourage the two 3DGS models to focus on different static aspects of the scene. Consequently, this asymmetric strategy leads to divergent and complementary optimization paths and reduces shared error modes. The final reconstruction is then guided by the agreement between the two models, which reliably captures consistent and accurate scene structures while suppressing artifacts.
While the dual-model framework effectively improves the reconstruction quality, introduces notable computational overhead in the training process. To mitigate this, we introduce a lightweight variant, called Dynamic EMA Proxy, which replaces the second 3DGS model with a dynamic, training-free Exponential Moving Average (EMA) copy of the primary model. Unlike standard EMA AЛ. our Dynamic EMA proxy specifically designed to track the evolving nature of 3DGS representations, accounting for Gaussian densification and pruning. Since only one model is actively trained in this setup, which no longer allows independent masks for two models, we additionally design an alternating masking strategy that alternates between the two masks, maintaining divergent training signals and mitigating confirmation bias.
1) We propose a AsymGS framework for in-the-wild 3D scene reconstruction. By enforcing consistency constraints between two 3DGS models with complementary masks, our framework significantly improves the robustness and accuracy of scene representations. 2) We develop divergent masking strategy by introducing different masking mechanisms for each 3DGS model, which handle various types of distractors and promote divergent optimization paths to mitigate confirmation bias. 3) To address the computational overhead of the dual-model framework, we introduce a Dynamic EMA Proxy, coupled with an alternating masking strategy, which effectively improves training efficiency. 4) We conduct extensive evaluations across a diverse set of in-the-wild 3D scene reconstruction datasets, demonstrating that our method consistently achieves state-of-the-art performance and efficiency, highlighting its robustness and generality.
Our contributions are as follows:
2 Related work
3D Scene Reconstruction. Neural Radiance Fields (NeRF) 15 revolutionizes photorealistic novel view synthesis by modeling scenes as continuous functions that map 3D coordinates to color and density. More recently, 3D Gaussian Splatting (3DGS) 8 25 has gained attention as a real-time

---

G +
G. +V -1
Mutual Consisteney (Eq.
52
g
G2 1 15
F
5
mm1
Lyz
G2
Color Transform (Eq. 3)
Mh (Eq.
(Eq.
1
Mn.
M2
Multi-Cue Adaptive Mask
.....
Self-Supervised
Loos
Rasterization (1q.
Figure 2: Overview of the AsymGS framework. Two 3DGS models G1 and G2 are concurrently optimized with the reconstruction loss E, and 22 (Eq. 4. along with the mutual consistency loss f.m and Lm2 (Eq. (6. In addition, we apply mask loss (Eq. ( for learning soft mask in self-supervised manner. For improved efficiency. we also propose an EMA version of our framework by replacing G2 with a dynamic EMA proxy. Both the mask loss and the EMA proxy have been omitted here for clarity. Note that the color transform in this figure is for illustration purpose, which undergoes a rasterization process in practice as introduced Section 3.1
alternative, representing scenes with optimizable Gaussian primitives. While effective, both methods assume static scenes to enforce multi-view consistency, an assumption often violated in in-the-wild settings due to varying illumination and transient objects, limiting their practical applicability.
3D Scene Reconstruction in the Wild. NeRF-W 14 first addressed 3D scene reconstruction in the wild with modular architecture combining learnable appearance embedding and an uncertainty map to suppress distractors-4a approach that has since become standard. NeRF On-the-go [18] builds on this by using DINOv2 feature [16] residuals construct uncertainty maps. Robust- NeRF 19 tackles noisy training images through robust loss function. GS-W (26), Wild-GS 24 and WildGaussian [1] extend learnable embeddings with both global and per-Gaussian embeddings for fine-gr -meiiia appearance modeling. SpotlessSplats [20] introduces learnable mask based on thresholded and dilated residuals, while SWAG [2] adds view view-dependent opacity term Gaussian to identify transient distractors. HybridGS [3 employs a dual-model setup (3DGS static content and 2DGS S for dynamic distractors) to learn an accurate uncertainty map iter- atively. Despite their contributions, these methods largely rely heuristic strategies to suppress corrupted supervision signals from low-quality training data. For instance, the per-image image appearance embeddings only weakly indirectly supervised through photometric losses [14] 26), and outlier filtering is often governed by hand-crafted rules [19]. As result, these approaches frequently suffer from instability and produce reconstructions with noticeable artifacts. In this work, we propose principled framework, AsymGS, to reduce such artifacts and improve reconstruction stability, based on the key observation that many artifacts exhibit stochastie behavior and not consistent across different training runs. Our method explicitly exploits this property to enhance robustness and achieve high-fidelity reconstructions.
Gaussian local
Method
An overview of the proposed algorithm is shown in Figure 2 Please refer to the caption for details.
3.1 Preliminaries
3D Gaussian Splatting (3DGS) g represents a 3D scene as a set of N 3D anisotropic Gaussians
= [ Fach Gaussian G. is parameterized by a centroid X, a covariance matrix Σ,, an opacity ay and set of spherical harmonic (SH) coefficients 0, for view-dependent color representation. For

---

rendering with viewing camera V,, we project both centroids of Gaussians and the covariance matrix onto the 2D image plane as Xij and E,j. respectively. The projected opacity aijs which is function of 2D image plane coordinate can then be defined as below:
aiy(y) = a; cxp - (y (y- -xy)) Exy)) "lyy ((x))))
Xis)
(I)
The color C of a pixel located y for camera Vj is computed using the a-blending with the following formula:
1-1
v. E Ci (y)
II (1-ax)(y)) axy(y)), Cit = SH (ray,
(2)
where Faj is the ray direction from the j-th camera center to the i-th Gaussian centroid, and Ciý is the corresponding color of the observed Gaussian primitive obtained using spherical harmonic (SH)
function. Performing Eq. 2 for every pixel on the image plane constitutes the rasterization process. which results in the rendered image I Rastcrizo (G, Vj).
View-dependent appearance modeling, To address appearance variations in in-the- wiid data, we follow the approach of WildGaussian [1I] to adaptively adjust the observed color of the Gaussian primitives to account for the view-dependent factors, such as the varying illumination across images captured different times of day.
This adjustment is conditioned on both the per- per-Gaussian appearance embedding p, and the per-view appearance embedding 9/- Specifically, ps. qj, and Cij sent into an MLP to predict affine transformation parameters:
(a, f qj. Ciy, Cر = +
where a and are three-dimensional outputs corresponding to RGB channels. The transformed color Ciy is then used to replace Cij in the blending process described by Eq. 2 and the resulting view-dependent image is denoted as I Rastorizedep (G. V,). During training, Ps. qj- f. and 3DGS parameters are jointly optimized.
3.2 Dual 3DGS
central insight of this work is that artifacts arising from in-the-wild training data are typically stochastic in nature. When two 3DGS models are trained on the same scene but with different view sampling orders, their static scene representations remain consistent, whereas their renderings could diverge in regions affected by outliers. An example is shown in Figure L
Motivated by this observation, we introduce framework with two 3DGS, where each model is trained with a different sampling order, and a consistency constraint is enforced between their renderings. Specifically, maintain two sets of Gaussians, G1 and G2, to represent the same scene. In each training iteration, we independently sample two views from separate training view lists, yielding two viewing cameras V, and V2, along with their corresponding ground-truth images, and Ig-
Similar to 3DGS g we train G1 and G2 with reconstruction objectives defined as:
CM = Crecom (IF) I,,M), CM Crecmm (1 I2, M,M,,
(4)
Lyconn (i, I, M) = A DSSIM(M O M + (1 -
(5)
where F Rasterizedep (G,,, V,) is the rendered image for the n-th 3DGS model from viewpoint V, using Eq. and M is spatial mask to filter out transient distracting regions, such as pedestrians or moving vehicles, which will be detailed in Section 3.3 C denotes element-wise multiplication.
DSSIM represents the structural dissimilarity index measure [23] À is hyperparameter to balance
the DSSIM and the L1 terms.
Mutual consistency. Since G; and G2 represent the same underlying scene, their renderings from the same camera viewpoint should stay close. This motivates us to define mutual consistency regularization as:
Lm
|| i II.
Lm² = 1155 1
(6)

---

where 1y.. Rasterize(Gr, V2) is the view siewddependent rendering obtained via Eq. We emphasize that this consistency constraint is performed over T instead - because captures the intrinsic appearance of the 3D scene, whereas r is affected by dynamic lighting. This strategy provides a principled way to preserve static structures while suppressing spurious signals, which enables more robust and reliable reconstruction.
Note that we only use the L1 loss for consistency regularization in Eq. as incorporating the DSSIM loss adversely affects performance in our experiments. Furthermore, we empirically find that incorporating consistency regularization too early in training can hinder convergence, as both models may still be dominated by noise and unstable geometry. To address this, we adopt progressive strategy: we first allow the two models to be trained independently for a number of warm-up iterations, during which they develop their own estimates of the static scene. Once their reconstructions become sufficiently stable, we introduce the consistency loss to encourage convergence on shared, reliable structures.
3.3 Asymmetrie Dual 3DGS
While the above framework offers improved consistency through mutual supervision, its symmetric design, where both 3DGS models are trained in the same manner using the reconstruction loss in 5q. poses a risk of confirmation bias: both models may converge toward the same reconstruction errors due to their similar optimization signals.
To address this issue, we propose an Asymmetric Dual 3DGS variant, where each model trained with distinct masking strategy that emphasizes complementary criteria for filtering out transient or distracting content. This encourages divergent error patterns, enhances robustness, and mitigates confirmation bias. Specifically, we use a Multi-Cue Adaptive Mask and Self-Supervised Soft Mask.
Multi-Cue Adaptive Mask (M,). As illustrated in Figure 3 MA is hard binary mask (1 indicates static regions and O indicates distractors) that identifies transient and distracting regions by integrating multiple cues, including semantic segmentation, stereo correspondence, pixel-level residuals, and feature-level residuals.
We begin by applying the Segment Anything (SAM) model 9
12 to partition each image into semantically coherent regions. To detect static content, we perform multi-view stereo matching across the training images with COLMAP [2.] Semantic regions are considered static if they contain sufficient number of valid correspondences (we empirically choose a threshold of 3 matches). Among the remaining regions, identify transient distractors by analyzing reconstruction residuals. For each region, we compute pixel-level residuals, iee., the L1 error between the rendered and ground- truth images, and feature-level residuals, i.e., the cosine distance between DINOv2-enceo encoded feature maps [16] of the rendered and ground-truth images. Regions with above-average residuals in both during training. This multi-cue approach offers
metrics are classified as distractors and masked higher robustness than prior methods that rely on single cues 19 [8] 14] 17), which generalizes more effectively across diverse in-the-wil scenes. See Algorithm the supplementary material for full details of the Multi-Cue Adaptive Mask.
Self-Supervies Soft Mask (M.). To complement the rule-based hard mask M,, we introduce a learnable soft mask M,, whose values range between 0 and 1. Unlike the static M,, this soft mask is optimized jointly with the model and adapts throughout training. The objective for M, is derived from the cosine similarity between DINOv2 feature maps of the ground- -tuth image F and the rendered image
Lmask
M, [ineep(CossneSSimilaai ())
where fimeer denotes spatial interpolation to match the training image resolution. This formulation is self-supervised, requiring no ground-trutth masks. We initialize M, as an all-one tensor, allowing the model to gradually refine the mask training progresses (Figure B).
As shown in Figure 3 the hard mask M, is more definitive with clearer boundaries but can be over- confident, potentially missing certain transient objects. In contrast, the soft mask M, is often more sensitive to subtle distractors and better captures ambiguous regions, thus providing complementary information to M,.
5

---

Input
M
M, (Early)
M, (Final)
Figure 3: Comparisons of hard and soft masks. Distractors are highlighted in red boxes in the input.
The right four columns show the evolving of the self-supervised soft mask across different epochs.
By combining the loss terms in Eq.
6 7
the overall objective for our AsymGS can be written as:
M. + CM Am + Lmz) + Amakk Cmask:
where Am and Amak are hyperparameters for balancing loss terms. Since M, dynamically evolves during training and intrinsically differs from the fixed MA. training one model with CMAs and the other with C,M, in our AsymGS framework introduces complementary inductive biases. This asymmetry promotes diverse learning dynamics, making it less likely for the two models to converge on the same reconstruction error, thereby reducing confirmation bias and enhancing overall robustness.
3.4 Dynamic EMA proxy
While the AsymGS framework significantly improves robustness and reconstruction quality, requires simultaneous training of two 3DGS models, which introduces considerable computational overhead and undermines the fast training advantage of 3DGS. To address this issue, we propose a more efficient alternative by replacing one of the models with dynamic EMA proxy. Moreover, an alternating masking strategy is introduced to counteract confirmation bias. This design retains the benefits of dual-model regularization while significantly reducing computation cost.
Let G denote the set of Gaussians actively optimized during training, and GyMa its EMA counterpart, updated each training step by:
(9) where and & denote the current and previous training iterations, respectively. Here, we slightly abuse set notation for simplicity: the weighted summation between G1 and GEMA performed element-wise over corresponding Gaussian attributes, such as the centroids, opacities, and SH coefficients. We then rewrite the consistency regularization in Eq. with the EMA proxy as follows:
GÊma = G - (1 (1- 11--B) B) G GbMA = G..
Cie = ||ffira -if-1 111.
(10)
where ifeaa Rasterize GEMA, Vi) is the rendering of the EMA Gaussians from view Vi. Since only one 3DGS model requires gradient updates, and the E.A update is a simple weighted average, this approach greatly improves training efficiency while preserving the benefits of dual-model consistency.
Dynamic update. Standard EMA is primarily designed for neural networks, where the number of parameters is typically fixed throughout training [22] 4 However, applying it to 3DGS presents unique challenges, as the number of Gaussians dynamically changes during training due operations such as cloning, splitting, and pruning [
To support this dynamic data structure, develop dynamic EMA mechanism by introducing the following rules: 1) Cloning: When a Gaussian is cloned, its EMA attributes are also cloned. 2) Pruning: When Gaussian is pruned, its EMA counterpart is removed as well. 3) Splitting: When a Gaussian splits into two, attributes that undergo discontinuous changes, i.e., the centroids and variances, reinitialized in the EMA according to the values of the split Gaussians. The remaining attributes (e.g., opacities and SH coefficients) are directly inherited from the original EMA representation.
Alternating masking strategy. Since only one model is trainable in our EMA framework, the original asymmetric training strategy used in Eq. g (i.e., M, and M,) is not directly applicable.

---

Table 1: Quantitative results on the NeRF On-the-go dataset [8] of average training hours per scene. The best and second-best results are highlighted in bold and underline, respectively.
Efficiency is reported in terms
Scene
High Occlusion
Medium Occlusion
Low Occlusion
Method
PSNR
SSIM1
LPIPS
PSNR
SSIMI
LPIPS
PSNR
SSIM1
LPIPS]
Hrs.
RobustNeRF 19
20.60
0.602
0.379
21.72
0.741
0.248
16.60
0.407
0,480
NeRF On-the-go
18
22.37
0.753
0.212
22.50
0.780
0.205
20.13
0.627
0.287
43
3DGS 8
19.03
0.649
0.340
19.19
0.709
0.220
19.68
0.649
0.199
0.35
Mip-Splatiing
19.25
0.664
0.333
19.73
0,684
0.279
20.03
0.661
0.195
0.16
GS-W 26
18.52
0.645
0.335
21.04
0.737
0.208
19.75
0.660
0.287
0.55
WildGaussian
10
23.03
0.771
0.172
22.80
0.811
0.092
20.62
0.658
0.235
0.50
SLS-mip 20
21.92
0.710
0.222
22.79
0.817
0.162
20.02
0.59
0.276
HybridGiS
23.05
0.768
0.204
23.51
0.830
0.160
21.42
0.684
0.268
0.18
Ours (GS-GS)
24.34
0.825
0.150
24.56
0.872
0.090
21.91
0.728
0.189
0.28
Ours (EMA-GS)
24.12
0.818
0.154
0.864
0.090
21.77
0.722
0.162
0.18
Mip-Splatting 223
HybridGS 13
Our (EMA-GS)
Ours (GS-GS)
Ground Truth (bottom)
Figure 4: Qualitative results on the NeRF On-the-go
18
(top) and the RobustNeRF
19
datasets.
Instead, propose an alternating masking strategy by switching between the hard mask M,, and the soft mask M, for training G1, which retains the complementary advantages from both decisive, rule-based filtering and adaptive, learned filtering. The final loss for our dynamic EMA framework can be written as:
L -= Ma/> Am.em + Amask Cmask:
(11)
where Ma/s indicates alternating between masks. This strategy essentially injects randomness into the EMA update process, promoting diversity in optimization and reducing overfitting erroneous supervision signals. Note that we also explored other forms of randomness, including randomly mixing up EMA renderings with ground truth and applying random dropout Gaussian primitives. Nevertheless, we empirically find that alternating masking remains the most effective approach.
Discussion. Our approach is related prior works that also leverage EMA, such [22] and 4. which apply EMA to neural networks for tasks like semi-supervised or unsupervised image classification. However,
method diverges in key aspects: unlike these methods that operate in the context of neural networks, we apply EMA to 3DGS, a dynamic representation where the number of Gaussians evolves throughout training. This necessitates our proposed dynamic EMA mechanism, waeeAmeeehhhii which adapts EMA updates to structural changes in the learned scene representation. Additionally, introduce alternating masking strategy preserve the benefits of asymmetric training with single learnable model. These innovations mark significant departures from conventional EMA usage and highlight the contributions of this work.
4 Experiments
We evaluate our method on three in me-the-wild datasets: the NeRF On-the-go dataset [18] the Robust- NeRF dataset [19) and the Photo dataset (6]. NeRF On-the-go and RobustNeRF mainly suffer from transient distractors. Photo'oourism contains both distractors and dynamic lighting. We denote our full dual-model approach as GS-GS, and the efficient variant with the EMA proxy as EMA-GS. Implementation details can be found in Section of the supplementary material.
7

---

Table 2: Quantitative results on the RobustNeRF dataset 19 average training hours per scene. The best and second-best results are highlighted in bold and underline, respectively.
Efficiency is reported in terms of
Scene
Statue
Android
Yoda
Crab
Method
PSNR1
SSIM
LPIPS]
PSNRT
SSIM
LPIPSL
PSNRT
SSIM
LPIPS.
PSNRT
SSIMT
LPIPS)
Hrs.
RobusNNRF [9
20
0.150
0.750
On-the-go
0.770
0.240
23.50
0.750
0.210
29.96
3DG3
8
21.02
0,810
0.160
23.11
0.810
0.130
26.33
0.910
0.140
29.74
Mip-Splatiin
Ab
22.08
0.300
0.135
23.45
0.301
0.106
27.96
0.933
0.136
29.18
0.929
0.129
0.14
GS-W
21.5 11
0.962
0.102
24.23
0.824
0.090
32.74
0.957
0.084
33.22
0.952
0.088
0.37
23.25
0.105
SLS-mlp
22,54
0.840
0.130
25.05
0.850
0.090
33.66
0.960
0.100
34.43
HybridGS [3
22.93
0.770
0.100
25.15
0.850
D.0721
35.32
0.960
0.077
35.17
0.960
D.080
Ours (GS-GG)
23.47
0.894
0.097
25.61
0,857
0.071
37.18
0.969
0,074
36.18
0.964
0
0.31
Ours (EMA-GS)
23,49
0,096
0.077
0,080
0.21
Mip-Splatting
25
WildGaussian
11
Our (EMA-GS)
Ours (GS-GS)
Ground Truth
Figure 5: Qualitative results on the
dataset
4.1 Comparison with SOTA
As shown in Tables T3 the proposed AsymGS framework, in both the GS-GS and EMA-GS setups, consistently outperforms all baselines by significant margin across all evaluation datasets, highlighting the robustness and generality of our approach. Visual comparisons are provided in Figure
In terms of training efficiency, our GS-GS model trains in an average of 30 minutes per scene on the NeRF On-the-go and RobustNeRF datasets, while the EMA proxy further reduces training time by one-third. For the Tourr'ou dataset, which features high-resolution imagery and dynamic lighting, EMA-GS cuts training time from 7.2 hours to 2.9 hours compared the previous SOTA [1] while achieving better reconstruction quality.
4.2 Ablation study
Dual 3DGS framework. As shown in Table 4 the dual 3DGS framework, whether implemented directly or via an EMA proxy, consistently outperforms the single-model baseline across three datasets (L.1 vs. 1.5 & I.12 in Table D Importantly, the mutual consistency loss is crucial component. Removing it reduces the framework to simple ensemble, which is shown to be ineffective, leading to an average drop of 0.5 dB in PSNR and consistent degradation in SSIM and LPIPS (L5 vs. L9, L12 vs. L16).
Additionally, the GS-GS setup achieves better performance than its EMA proxy counterpart (L5 vs. L12). We attribute this to two main factors: first, the GS-GS setup enables both models to be actively updated, effectively doubling the training iterations; second, the EMA proxy introduces confirmation bias due to its model accumulation nature, which limits its ability to correct erroneous predictions.
Distractor modeling. The results in Table further show that applying masks significantly improves reconstruction performance especially when distractors occupy a large portion the input image. (L5 L6, L12 vs. L13).
Either M, or M, independently improves performance over the base model (for M,, L2 vs. L3, L6 vs. L7, L13 vs. L14; for M,, L2 vs. L4, L6 vs. L8, L13 vs. L15), indicating that both masks are effective, although they have different characteristics. This observation is further supported by the visual results in Figure 3 where the hard-selected mask performs well in simpler scenes with clearly defined regions 3 while the self-supervised mask excels more complex scenes containing multiple or ambiguous distractors (Figure 3 Bottom). Moreover, combining both masks leads to additional gains, confirming their complementary roles (I.5 vs. I.7 L8, I.12 vs. L.14 & L15).
8

---

Table 3: Quantitative results on the Photo Tourism dataset [6. Efficiency is reported in terms of average training hours per scene. The best and second-best results are highlighted in bold and underline, respectively. We did not compare with HybridGS [33 on Photo'oourssm, as it does not consider varying illumination, and cannot be customized or retrained due the absence of training code.
Scene
Brandenburg Gate
Sacre Coeur
Trevi Fountain
Method
PSNR1
SSIM
LPIPS..
PSNR1
SSIM
LPIPS..
PSNR1
SSIM
LPIPS!
Hrs.
NeRF 15
18.90
0.882
0.138
15.60
0.846
0.163
16.14
0.696
0.282
3DGS 88
19.37
0.880
0.141
17.44
0.835
0.204
17.58
0.709
0.266
2.2
Mip-Splatiig
25
20.01
0.877
0.166
17.54
0.831
0,203
17.36
0.684
0.319
2.3
NeRF-W [4
24.17
0.891
0.152
19.20
0.803
0.192
18.97
0.698
0.265
164
Ha-NeRI [
24.04
0.887
0.139
20.02
0.801
0.171
20.18
0.691
0.223
452
K-Planes B
25.49
0.879
0.224
20.61
0.774
0.265
22.67
0.714
0.317
0.6
RefinedFields
?
26.64
0.886
22.26
0.817
23.42
0.737
150
GS-W 26
23.51
0.897
0.166
19.39
0.825
0,211
20.06
0.723
0.274
1.2
SWAG 2
26.33
0.929
0.139
21.16
0.860
0,185
23.10
0.815
0.208
WildGaussian
11
27.77
0.927
0.133
22.56
0.859
0.177
23.63
0.766
0,228
7.2
Ours (GS-GS) 28.56
0.938
0.109
23.78
0.887
0.139
24.52
0.790
0.202
Ours (EMA-GS)
28.50
0.937
0.115
23.37
0.882
0.150
23.85
0.775
0.242
2.9
Table 4: Effectiveness of different modules. The first block presents results from a single base model using different mask strategies. The second and third blocks evaluate the EMA-GS and GS-GG setups, respectively. "w/ M,/* indicates alternating between M,, and M, at each training iteration. "w/o M denotes that no mask is applied. The results averaged across scenes within each dataset.
Datasel
PhotoTourism
NeRF On-the-go
RobustNeRI
Line
Method
PSNR1
SSIM
LPIPS.
PSNR
SSIM
LPIPS!
PSNR
SSIM
LPIPS|
Single
Ma/s
24.76
0.864
0.167
22.97
0.798
0.133
29.62
0.914
0.083
Single w/o M
24.68
0.859
0.174
19.67
0.669
0.269
25.67
0.881
0.126
3
Single wo M,
24.82
0.862
0.172
22.97
0.802
0.125
28.97
0.908
0.088
4
Single w/o Ma
24.83
0.862
0.171
22.24
0.783
0.153
29.38
0.912
0.086
5
EMA-GS
25.24
0.864
0.169
23.40
0.801
0.135
30.27
0.917
0.080
EMA-GS w/o M
24.70
0,862
0.170
21.68
0.766
0.172
28.42
0,905
0.092
EMA-GS w/o M,
24.76
0.864
0.166
22.94
0.802
0.126
0,909
0.089
8
EMA-GS w/o Mr
24.84
0.863
0.169
22,15
0.780
0.159
29.29
0.911
0.087
9
EMA-GS w/o C.m
24.73
0.861
0.174
23.10
0.801
0.132
29.72
0.915
0.081
10
EMA-GS w/ Mixup
25.13
0.866
0.162
23.39
0.802
0.132
30.18
0.915
0.081
11
EMA-GS w/ Dropout
25.11
0.863
0.171
23.39
0.804
0.130
30.19
0.913
0.082
12
GS-GS
25.62
0.872
0.150
23.61
0.809
0.143
30.61
0,921
0.080
13
GS-GS w/o M
25.09
0.867
0.160
22.39
0.785
0.164
29.45
0.914
0.087
14
GS-GS w/o M,
25.24
0.867
0.159
23.32
0.812
0.132
30.38
0.920
0.079
15
GS-GS w/o MA.
25.11
0.867
0.161
22.76
0.794
0.157
30.17
0.919
0.082
16
GS-GS w/o Cu
25.08
0.869
0.155
23.13
0.808
0.135
29.92
0.918
0.082
EMA proxy. The effectiveness of the EMA-GS setup heavily depends on the masking strategy (L5 vs. 1.7 & 1.8). Using only single type of mask often leads to performance similar even worse than the single-model baseline (I.3 vs. 1.7, 1.4 vs. 18), indicating that confirmation bias can undermine robustness. This highlights the need for diverse masking signals to fully exploit the benefits of the EMA proxy. Moreover, as introduced in the Alternating Masking Strategy part of Section 3.4| we have also tried other forms of regularization improve the performance of our EMA model, such as random mixup or dropout. However, as shown in L10 and LII of Table X these methods do work as good as our original approach of alternating between the two masking strategies.
Conclusion
In this work, we present AsymGS, a robust and efficient framework for 3D scene reconstruction in unconstrained, in-the-wild environments. Our method employs two 3DGS models guided by distinct masking strategies to enforce cross-model consistency, effectively mitigating artifacts caused by low-quality observations. To further improve training efficiency, we introduce a dynamic EMA proxy that significantly reduces computational cost with minimal impact performance. Extensive experiments on three challenging real-world datasets validate the effectiveness and generality of our approach.

---

Acknowledgments and Disclosure of Funding
This research was supported by the National Natural Science Foundation of China (62302385). Chengqi Li and Wenbo He thank the NSERC Discovery grant on Towards a Robust and Trustworthy Machine Learning for Visual Contents in Data-centric Applications. Chengqi Li also gratefully acknowledges the Neural Information Processing Foundation for the NeurlPS 2025 Scholar Award supporting his travel.
References
[1] Xingyu Chen, Qi Zhang, Xiaoyu Li, Yue Chen, Ying Feng, Xuan Wang. and Jue Wang. Hallucinated Neural Radiance Fields in the Wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12943 12952, June 2022.
[2] Hiba Dahmani, Moussab Bennehar, Nathan Piasco. Luis Roldão, and Dzmitry Tsishkou. SWAG: Splatting in the Wild Images with Ierearance-Conditionn Gaussians. In European Conference on Computer Vision (ECCV), volume 15134, pages 325 325-340, 2024.
[3] Sara Fridovich-Kan Keil, Giacomo Mcanti, Frederik Rahbæk Warburg, Benjamin Recht, and Angjoo Kanazawa. K- K-Planes: Explicit Radiance Fields in Space, Time, and Appearance. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 22479-12483 June 2023.
[4] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum Contrast for Unsupervised Visual Representation Learning. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), June 2020.
[5] Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao. 2D Gaussian Splatting for Geometrically Accurate Radiance Fields. In SIGGRAPH 2024 Conference Papers. Association for Computing Machinery, 2024. doi: 10.1145/3641511.365473
[6] Yuhe Jin, Dmytro Mishkin, Anastasiaa Mishchuk, Jiri Matas, Pascal Fua, Kwang Moo Yi, and Eduard Trulls. Image Matching across Wide Baselines: From Paper to Practice. International Journal of Computer Vision, 2020.
[7] Karim Kassab, Antoine Schnepf, Jean-Yves Franceschi, Laurent Caraffa, Jeremie Mary, and Valérie Gouct-Brunet RefinedFields: Radiance Fields Refinement for Unconstrained Scenes. arXiv preprint arXiv:2312.00639 2023.
[8] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkuehler, and George Drettakis. 3D Gaussian Splatting for Real-Time Radiance Field Rendering. ACM Transactions on Graphics, 42, July 2023.
[9] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollar, and Ross Girshick. Segment Anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 4015-4026, October 2023.
[10] Jonas Kulhanek and Torsten Sattler. NerfBaselines: Consistent and Reproducible Evaluation of Novel View Synthesis Methods. arXiv preprint urXiv:2406. 17345, 2024.
[11] Jonas Kulhanek, Songyou Peng, Zuzana Kukelova, Mare Pollefeys, and Torsten Sattler. WildGaussians: 3D Gaussian Splatting In the Wild. In Advances in Neural Information Processing Systems, volume 37, pages 21271-21288, 2024.
[12] Feng Li, Hao Zhang, Peize Sun, Xueyan Zou, Shilong Liu, Chunyuan Li, Jianwei Yang, Lei Zhang, and Jianfeng Gao. Segment and Recognize Anything at Any Granularity. In European Conference on Computer Vision (ECCV), pages 467-484, Springer Nature Switzerland, 2024.
[13] Jingyu Lin, Jiaqi Gu, Lubin Fan, Bojian Wu, Yujing Lou, Renjie Chen, Ligang Liu, and Jieping Ye. HybridGS: Decoupling Transients and Staties with 2D and 3D Gaussian Splatting. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025.
10

---

[14] Ricardo Brrril- Noha Radwan, Mehdi S. M. Sajjadi, Jonathan T. Barron, Alexey Doso- vitskiy, and Daniel Duckworth. NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7210-7219, June 2021.
[15] Ben Mildenhall, Pratul Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. In Eu- ropean Conference on Computer Vision (ECCV), pages 405-421. Springer Nature Switzerland, 2020.
[16] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V. Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fermandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Mido Assran, Nicolas Ballas, Wojeiech Galuba, Russell Howes, Po- YaoYao Huang, Shang-Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Herve Jegou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski. DINOv2: Learning Robust Visual Features without Supervision. Transactions on Machine Learning Research, 2023.
[17] Konstantinos Rematas, Andrew Liu, Pratul P. Srinivasan, Jonathan T. Barron, Andrea Tagliasac- chi, Thomas Funkhouser, and Vittorio Ferrari. Urban Radiance Fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pages 12932- 12942, June 2022.
[18] Weining Ren, Zihan Zhu, Boyang Sun, Jiaqi Chen, Mare Pollefeys, and Songyou Peng. NeRF On-the-go: Exploiting Uncertainty for Distractor-free NeRFs in the Wild. In Proceedings of the IFEF/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 8931-8940, June 2024.
[19] Sara Sabour, Suhani Vora, Daniel Duckworth, Ivan Krasin, David J. Fleet, and Andrea Tagliasac- chi. RobustNeRF: Ignoring Distractors with Robust Losses. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 60626-20636 June 2023.
[20] Sara Sabour, Lily Goli, George Kopanas, Mark Matthews, Dmitry Lagun, Leonidas Guibas, Alec Jacobson, David Fleet, and Andrea Tagliasacchi. SpotLessSplats: Ignoring Distractors in 3D Gaussian Splatting. ACM Transactions on Graphics, 2025.
[21] Johannes L Schonberger and Jan-Michael Frahm. Structure-from from-motion revisited. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 4104-4113, 2016.
[22] Antti Tarvainen and Harri Valpola. Mean Teachers are Better Role Models: Weight-aweraged Consistency Targets Improve Semi-supervised Deep Learning Results. In Advances in Neural Information Processing Systems, volume 30, 2017.
[23] Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4): 600-612, 2004.
[24] Jiacong Xu, Yiqun Mei, and Vishal M. Patel. Wild-GS: Real-Time Novel View Synthesis from Unconstrained Photo Collections. In Advances in Neural Information Processing Systems, volume 37, pages 103334-103355, 2024.
[25] Zehao Yu. Anpei Chen, Binbin Huang. Torsten Sattler, and Andreas Geiger. Mip-Splattin: Alias-free 3D Gaussian Splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19447-19456, June 2024.
[26] Dongbin Zhang, Chuming Wang, Weitao Wang, Peihao Li, Minghan Qin, and Haoqian Wang. Gaussian in the Wild: 3D Gaussian Splatting for Unconstrained Image Collections. In European Conference on Computer Vision (ECCV), pages 341-359, 2024.
[27] Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The Unrea- sonable Effectiveness of Deep Features as Perceptual Metric. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 586-595, June 2018
11

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly state the paper's focus on novel view synthesis with in-the-wild image collections. The key contributions (including the asym- metric mual-model framework, mutual consistency regularization, and the integration of complementary masking strategies) explicitly outlined in the methodology and validated through extensive experiments.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly State the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. It is fine to include aspirational goals as motivation long it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitations of our approach are discussed in the supplementary material.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
12

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: The paper presents learning baasdd method and does not include theoretical results or formal proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly Stated referenced in the Statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
The paper provides comprehensive details of the proposed method, along with descriptions of the experimental setup. including datasets, training procedures. hyperpa- rameters, and evaluation metrics, ensuring the main results can be reliably reproduced and validated.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often good way to accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
13

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: The datasets used publicly available, and the code will be released upon publication.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ wttppssccpppbllc///eeep/pppppppp
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should State which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
The paper provides all necessary training and testing details, with method- specific settings described in the main paper and 3DGS-related heuristics detailed in the supplementary material.
Justification:
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[No]
Justification: Error bars are not reported, as repeated runs showed no significant variation under identical settings.
14

---

Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly Stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, but one should State it. The authors should prefer- ably report 2-sigma error bar than State that they have a 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
The paper specifies the computational resources used for each experimental setting. Experiments were performed on an RTX 4090. This information, along with training times, is provided in the supplementary.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
The research fully complies with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
15

---

Answer: [Yes]
Justification: By improving robustness under in-the-wild data, we aim advance 3DGS models toward practical real-world applications. Potential risks, such as misuse in privacy- sensitive environments, are acknowledged the broader context of seene reconstruction technologies.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper does not involve the release of high-risk models sensitive datasets. It uses publicly available 3D scene datasets and does not introduce models or data with significant potential for misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All external assets, including datasets and baseline models, are properly credited in the paper.
16

---

Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should State which version of the asset is used and, if possible, include URL.
The name of the license ge.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
The paper introduces new assets that are clearly documented, Details of our method, datasets, and implementation provided in the main paper, and any additional information is included in the supplementary material.
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing research with human subjects, and therefore no participant instructions, screenshots, or compensation details are applicable.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
17

---

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly State this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: Our work focuses computer vision, and LLMs not used in method.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our L.LM policy hatt::/s//eerrppsecc////feeeeee/ for what should or should not be described.
18

---

Figure 6: Randomness of artifacts across training Row shows the target view (with the presence of distractors); Rows and 3 present results from two independent runs Mip-Splatting: Row 4 shows the result of our method with the mutual consistency regularization.
A Random nature of artifacts
As shown Figure [ different runs of 3DGS on the same scene (with only the view order randomized) result in different artifacts, particularly in uncertain regions. The mutual consistency loss helps suppress these artifacts in both models. On one hand, the shared static regions remain consistent and as a strong regularizer. On the other hand, the differing artifacts in uncertain areas provide complementary supervision signals, allowing regions affected by artifacts in one model to be recovered by the other.
For the effect distinct masking strategies, Table in the main paper presents a quantitative comparison. The performance degrades when both models use the same mask, for both GS-GS and EMA-GS settings. This supports our claim that using separate masks helps prevent convergence to the same erroneous reconstruction patterns.
B Comparison with HybridGS
Our Asymmetric Dual 3DGS framework differs fundamentally from HybridGS [13 in both design and training. HybridGS separates static and dynamic content using two models (3DGS for static, 2DGS for dynamic) and requires a staged training process with a learnable blending mask. In contrast, method uses dual 3DGS setup with mutual supervision to improve robustness against dynamic noise, all within the standard 3DGS training pipeline. While both methods use masking. HybridGS blends outputs based on 2DGS-derived -derieed uncertainty, whereas we apply two distinct masking strategies to reduce confirmation bias from a single, potentially inaccurate mask.
C Multi-cue adaptive mask
use residuals between ground ground-truth truth and rendered images to detect distrac- tors, assuming static regions are learned first. However, this can misclassify object boundaries and miss distractors resembling the background. Others [14 17 use pretrained semantic segmentation to mask known distractors, such as people or sky, but these methods rely on task-specific priors and lack generality across diverse scenes. We propose Multi-Cue Adaptive Masking combine the strengths
Some prior works 19 18
19

---

of residual-ber based and segmentation based methods, while also providing complementary hard mask that captures distinct error patterns compared the self-supervised soft mask.
Algorithm 1 Multi-Cue Adaptive Masking
Require: Rendered image I, ground-truth image I, semantic masks {M&} from SAM, stereo correspondence map 223 from COLMAP
1: pix ||t |
D Pixel-level residual
2: F DINOv2(T): F (IT)
DINOv2 features
3: Ereat = 1 - Cosine
Feature-level residual
4: Epix = Σ Epix/ (rea(I)
>> Average residuals over
5: Ereat Σ Brea// (I)
6: 8 Σ22/ (I)
D Stereo correspondence density over
7: for each mask Mr do
8:
Cpix, = M @ / EM,
Average residuals over Mg
9:
E:ear = Σ Mr 3 Efeat/ M
10:
M O S23/ 2 M₁
> Stereo correspondence density over Mę
if epix,& Epix and efeat, éfent and 8 0.1 then
12:
Mark M& as distractor mask
end if
14:
end for
return MA,- = UfM selected
0 for distractor
Here, the stereo- basod correspondence records the number of matches each pixel in the given image has, based on SIFT feature correspondences proposed in COLMAP [2]] A pixel is considered a valid correspondence (with the stereo correspondence map value set to true the pixel location) its match count exceeds a threshold, indicating it likely belongs to a static region. In contrast, distractors typically yield fewer matches due their limited presence across images. In Algorithm [ S23 denotes the stereo correspondence map, where pixel is considered valid correspondence if it has more than three matches.
D Datasets and metrics
We evaluate our method on three in-the-wild datasets with varying challenges, as shown in Table 5 NeRF On-the-go dataset [18] features indoor and outdoor sequences with consistent appearance but varying distractor ratios (5%-30%). RobustNeRF dataset [19] provides indoor scenes with static geometry and controlled distractor placement (from single-type to 150 varied distractors), where training is done on cluttered views and testing on clean, unseen ones. We use the undistorted versions of these datasets, following the protocols of WildGaussian [I]] and HybridGS [13]. The PhotoTourishy dataset [6] includes landmark scenes (Brandenburg Gate, Sacre Coeur, Trevi Fountain) captured under diverse lighting, weather, and viewpoints, with both significant appearance variation and real-world distractors. We report PSNR, SSIM, and LPIPS 27 to assess reconstruction accuracy and perceptual quality.
E
Implementation details
Our base model is built on Mip-Splatting 255.. Following its default settings, we recompute the sampling rate of each Gaussian every 100 iterations, with a 2D Mip filter variance of 0.1 and a 3D smoothing filter variance of 0.2. We train for 30,000 iterations on NeRF On-the-go and RobustNeRF, with densification and pruning every 1,000 steps until iteration 15,000; and for 100,000 iterations on PhotoTourism, with densification and pruning every 1,000 steps until iteration 50,000. We omit the opacity reset and apply a 1,000-step warm-up before the mutual consistency regularization begins. The consistency regularization weight set to 0. The learnable mask is optimized by loss weighted Amask = 1.0 with learning rate of 0. For EMA, we use smoothing factor of B = 0.8. Semantic regions for the multi-cue adaptive mask are generated using Semantic SAM [2 to create instance-level segmentations and apply Algorithm to select distractor regions masks.
20

---

Table 5: In-the-wild 3D reconstruction datasets.
Dataset
Scene
# Train
# Test
Distractor
Appear. change
Patio-high
222
45
~30%
No
Spot
168
10
~300%
No
Patio
98
26
155%~~20%
No
NeRF On-the-go [88
Corner
101
20
15%
No
Fountain
168
50%
No
Mountain
119
12
50%
No
Statue
255
19
I type
No
Android
122
19
type
No
RobustNeRF
Yoda
109
202
100 types
No
Crab
109
194
150 types
No
Brandenburg Gate
763
10
~3.5%
Yes
PhotoTourism
Sacre Coeur
830
21
~3.5%
Yes
Trevi Fountain
1689
19
~3.5%
Yes
Additionally, 32-dimensional per-view appearance embedding and a 24-dimensional per- Gaussian embedding. Color transformation performed using a three-layer MLP with hidden size 128, outputting a scale and bias for each RGB channel. The learning rates per-view embedding, 0.005 for the per-Gaussian embedding, and 0.0005 for the MLP. The other 3DGS-related hyperparameters follow the setup from origin work shown in Table 6
0.001 for the
Table 6: The other 3DGS-related related hyperparameters.
Parameter
Value
position inii
0.00016
position Ir final
0.0000016
position_Ire ______ddddey_ullll
0.01
feature Ir opacity_ Ir scaling
0.0025
0.1
0.005
rotation_Lh
0.001
percent_ _dense
0.01
lambda_dssaa dsssm
0.2
iictificaaiin_nntereeee
1000
opacity_resee interval
No opacity reset
censify_from___
densify _radd_r__________ threshorddd
0.0002
Table 7: The code repo and licenses.
Method
License
3DGS
attpe://githiin www. comm/grapddeoo-ooo www.iaiggaaggiiiaaar.era http://gsnnnaaa-ssaatttnnnn
Custom
Mip-Splatting
25
ittps://github ctt/aaatoooooooaaa111oo//1/1ii hwtp:////w0//////////eaat htppm/aataaappgggggg.go.m
WildGaussians
11
attps://github. com/ www./////aaaaeeeeeeecccc (kuuhannn///ii gaussians/
MIT License
NeriBaselines
10
https:///ithub www.tttuu.ccommaaee www.rr/rbaaaeeeeeeeeeeeeerrraaea
MFT License
COLMAP 21
attps://github https//ooppppppccc/cccoooccc loap
BSD License
Semantic-SAM [12
attpe://sithub com/ www.dee/ooeeeeeoeeeeeeeeeeeeeeee
Apache 2.0 License
NeRF On-the-go datasel
18
https://github. com/ cvg/
Ro-go
Apache 2.0 License
RohustNeRF dataset
http://robustnerr..p
Custom
PhotoTourism datasct
6
ttpp://github.cem//
inage-natching
image- watchingt-gia... benchmark
Apache 2.0 License
21

---

Table 8: Quantitative results on the NeRF On-the-go dataset are highlighted in bold and underline, respectively.
18
The best and second beosdd-bsst results
Sceee
Micanaia
Co:sa
Spoi
Psii-Rigt
Mathed
EP1P5
LHPPS.
LMMPS
Rebeahuer
12:91
15.05
0.578
0.361
0.344
20.65
0.574
0.235
19-93
00311
18:66
0.964
01:85
3040
0.313
0.38
11.08
0.51
0.34
30.77
0.316
11.30
0901
0.30
30:11
01355
30.01
0.667
0.335
0.845
31.81
33.87
13.35
0.35
0756
SUS-ala 220
1534
0.393
11234
20.19
0.92
0.238
2443
08733
0.236
21.55
0333
0.86
21.32
11256
16383
2831
0054
8239
Caiv E30
12.00
11323
11.83
0.717
11:85
36.55
R085
11285
22.07
00083
25.53
R4554
6.335
1317
0726
1154
Ous (EMA--55)
11.11
0.22
0.75
12.02
0.211
22.18
F
More results
F.1
NeRF On-the-go and RobustNeRF
In [able[[and Table our method (GS-GS) outperforms all baseline methods by more than 1 dB in scenes with medium to high occlusion ratios. The margin is smaller low-occlusion scenes, where 3DGS-basee bass-baedddd methods already perform well due to strong geometric priors from the initial point cloud. A similar trend is observed in [ while the proposed method surpasses the SOTA by approximately 0.4 dB in simpler scenes contaning single distractor type (c.g.. Statue and Android), it outperforms others by more than 1 dB in complex scenes with a large number diverse distractors (e.g., Yoda and Crab). The rendering results in Figure 7 and & further demonstrate the superiority of our method, as competing approaches exhibit distractor remains and missing details.
F.2 PhotoTourrim
The Asymmetric Dual 3DGS achieves an average improvement of 0.8 dB on the PhotoTourism dataset (Table 3. demonstrating its effectiveness under challenging appearance variations. Furthermore, proper appearance modeling is essential for handling wn-the-wild data with diverse visual conditions. This is supported by a significant performance gap of more than dB between methods with and without appearance modeling, as shown in Table 3] and further illustrated by the visual differences in Figure Therefore, we apply appearance modeling for the PhotoTourism dataset by default. As the importance of appearance modeling is addressed here. we omit further discussion in the following ablation section and apply appearance modeling by default for the Photo Tourism dataset.
F.3 Statistical significance of the main result
Table Quantitative results on the NeRF On-the-gat the-go dataset. Each experiment is repeated five times, and we report the mean and standard deviation.
Setting
GS-GS
EMA-GS
Scene
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
High Occlusion
24.36 0.02
0.823 + 0.001
0.151 + 0.001
24.11 0.00
0.819 1 0.002
0.152 1 0.004
Medium Occlusion
21.52 0.06
0.871 4 0.001
0.090 1 0.001
2.006
0.864 4: 0.001
0.092 1 0.002
Low Occlusion
21.99 0.04
0.730 + 0.001
0.184 + 0.004
21.81 0.009
0.723 + 0.002
0.166 + 0.007
Table 10: Quantitative results on the RobustNeRF dataset. Each experiment repeated five times, and we report the mean and standard deviation.
Setting
GS-GS
EMA-GS
Scene
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
Statue
23.44
0.05
018933
0.001
0.098 0.001
23.46
0.06
0.890
0.001
0.097 + 0.001
Android
25.58
0.05
0.856
1 0.001
0.070 = 0.003
25.47
+ 0.06
0.849
0.002
0.070 - 0.002
Yoda
37.12
0.09
0.969
+ 0.001
0.074 1774=00 0.001
36.46
4.06
0.967
0.001
0.078 - 0.001
Crab
36.11 + 0.07
0.963 + 0.001
0.079 = 0.001
35.52 0.07
0.961 + 0.001
0.080 = 0.001
We repeated the experiment five times. Based on the results in Table and 10 statistically significant improvements.
our method shows
22

---

E4 Hyperparameters
We perform hyperparameter tuning on the NeRF On-the-go dataset [8] to optimize the performance of our method (GS-GS and EMA-GS). As shown in Table 11 we tune the EMA smoothing factor B and find that / = 0.8 yields the highest PSNR and SSIM with the lowest LPIPS. In Table [2] we evaluate different densification intervals and observe that interval of 1000 offers the best overall performance. Similarly, Table 13 presents the results of tuning the warm-up interval, where 1000 again emerges the optimal choice, outperforming both shorter and longer intervals. Lastly, Table [4 shows that removing opacity reset improves reconstruction quality, suggesting that preserving learned opacity leads more stable and effective training.
Table 11: Tuning the EMA smoothing factor according to the average performance the NeRF On-the-go dataset (18).
8
PSNR
LPIPS. SSIM
0.5
22.80
0.797
0.136
0.6
22.93
0.797
0.137
0.7
23.12
0.799
0.136
0.8
23.40
0.801
0.135
0.9
23.05
0.798
0.136
Table 12:
Tuning the densification interval according to the average performance on the NeRF
On-the-go go dataset [18]
Setting
GS-GS
EMA-GS
Densification Interval
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS./
500
23.60
0.810
0.129
23.00
0.797
0.134
1000
23.61
0.810
0.135
23.40
0.801
0.135
1500
23.58
0.807
0.146
23.15
0.796
0.143
2000
23.56
0.806
0.152
22.96
0.797
0.145
Table 13: Tuning the warm-up interval according to the average performance on the NeRF On-the-go dataset (18).
Setting
GS-GS
EMA-GS
Warm-up Interval
PSNR
SSIM
LPIPS
PSNR1
SSIM1
LPIPS
23.55
0.808
0.137
22.96
0.798
0.135
500
23.55
0,809
0.137
23.08
0.799
0.134
1000
23.61
0.810
0.135
23.40
0.801
0.135
1500
23.58
0.809
0.137
23.10
0.799
0.135
2000
23.60
0.810
0.135
22.88
0.798
0.136
In Table 15 and 16 although the best performance is generally achieved
default setting (Am - 1.D and Amask = 1.0 for GS-GS; Am = 0.1 and Amask = 1.0 for EMA-GS), the differences settings are minimal (less than 0.1 dB). This indicates that the performance is not highly sensitive to the values of Am and Amask-
G Limitations
We adopt the appearance modeling approach from WildGaussian [1]] using vie- appearance embedding to control global appearance and a per-Gaussian embedding to model the appearance of individual Gaussian primitives. However, this model struggles capture fine-grained effects such as object highlights. likely reason is the limited diversity training data. To address this, we plan to introduce data augmentation with randomized illumination variations.
23

---

Table 14: Impact of opacity reset on reconstruction quality, evaluated on the NeRF On-the-go dataset [8]
Setting
GS-GS
EMA-GS
Opacity Reset
PSNR
SSIM
LPIPS|
PSNR
SSIM
LPIPS|
w/o
23.61 22.87
0.810
0.135
23.40
0.801
0.135
w/
0.790
0.176
22.43
0.786
0.158
Table 15: Performance under varying weights of the mutual consistency loss, evaluated on the NeRF On-the-go dataset [8].
GS-GS
EMA-GS
Am
PSNR
SSIM
LPIPS
Am
PSNR
SSIM
LPIPS
0.0
23.13
0.808
0.135
0.0
23.10
0.801
0.132
0.5
23.66
0.810
0.130
0.05
23.39
0.803
0.136
1.0
23.61
0.810
0.135
0.1
23.40
0.801
0.135
1.5
23.54
0.807
0.142
0.2
23.43
0.805
0.133
2.0
23.44
0.803
0.149
0.3
23.47
0.805
0.134
Table 16: Performance under varying weights of the learnable mask loss, evaluated on the NeRF On-the-go dataset [1]]
Setting
GS-GS
EMA-GS
Amas
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
0.5
23.62
0.809
0.136
23.33
0.802
0.134
1.0
23.61
0.810
0.135
23.40
0.801
0.135
1.5
23.59
0.809
0.137
23.41
0.804
0.135
2.0
23.63
0.811
0.135
23.32
0.802
0.134
H
Social impact
Notre-Dame de Paris suffered devastating fire 2019. Although the building was severely damaged, restoration was aided by a 3D model originally created for video game, highlighting the importance of preserving 3D models of cultural landmarks. However, such sites are often crowded with people, and photos taken at different times may exhibit varying lighting conditions. This highlights the broader societal benefit of accessible and robust 3D scene reconstruction technologies. Our method contributes positively by enabling the creation of high-quality 3D models from in-the-wild images, which are often affected by distractors and lighting variations. By making it feasible to reconstruct cultural landmarks from everyday photos, approach supports digital preservation, education, and historical restoration efforts.
There are potential negative impacts, such as misuse in surveillance drivacy-nvvddin applications. In particular, in-the-wild image collections often contain individuals who are unintentionally captured. To mitigate this risk, we recommend removing or anonymizing identifiable information, such as faces or bodies, from the reconstructed scenes. This can be achieved through automated segmentation or masking techniques applied before during training.
24

---

Ground Truth Figure 7: Qualitative results on the NeRF On-the-go dataset 18 The scenes shown are, from top bottom: Patio-high (high occlusion). Spot (high occlusion), Patio (medium occlusion), Corner (medium occlusion), Mountain (low occlusion), and Fountain (low occlusion).
(25
HybridGS 13
Our (EMA-GS)
Ours (GS-GS)
Mip-Splatting 25
HybridGS 13
Our (EMA-GS)
Ours (GS-GS)
Ground Truth
Figure 8: Qualitative results on the RobustNeRF dataset bottom: Statue, Android, Yoda, and Crab.
19
The scenes shown are, from top to
25

---

Mip-Splatting 25
WildGaussian
|
Our (EMA-GS)
Ours (GS-GS)
Ground Truth
Figure 9: Qualitative results on the Photomourism dataset
6 The scenes shown are, from top to
bottom: Sacre Coeur, Brandenburg Gate, and Trevi Fountain.
26


Reviews:
[{"review_id": "Tc2o7k3jTo", "reviewer": "Reviewer_haRb", "summary": "This paper addresses the challenging problem of 3D scene reconstruction from \"in-the-wild\" image collections, which are often plagued by inconsistent lighting and transient objects (e.g., pedestrians, vehicles) that corrupt the training signal and lead to visual artifacts in the final rendering. Existing methods often rely on heuristic strategies such as per-image appearance embeddings or hand-crafted outlier filtering rules, which can lack stability and generalizability.\nAuthors propose Asymmetric Dual 3DGS -- a novel framework that enforces consistency between two concurrently trained 3DGS models.\nThe authors claim state-of-the-art performance in reconstruction quality (PSNR, SSIM, LPIPS) and training efficiency, validated through extensive experiments on the challenging NeRF On-the-go, RobustNeRF, and PhotoTourism datasets. The authors claim state-of-the-art performance in reconstruction quality (PSNR, SSIM, LPIPS) and training efficiency, validated through extensive experiments on the challenging NeRF On-the-go, RobustNeRF, and PhotoTourism datasets.", "questions": "* Improve experimental clarity: clarify all ambiguous methodological and experimental details, including the full computational cost of the pipeline (with mask generation), the precise definition of the alternating masking strategy, and provide statistical significance for the main results (e.g., mean and standard deviation over multiple runs)\n* It would be great to include sensitivity analysis for the key hyper-parameters\n* Strengthen Related Work -- a more nuanced discussion of the related literature, explicitly contrasting the proposed conceptual framework with that of HybridGS would be quite beneficial.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n* Principled novel core Idea. By leveraging consistency between two independent training processes as a supervisory signal, the method learns to distinguish reliable scene structure from spurious, inconsistent information, which is typically present in the in-the-wild data.\n* The asymmetric dual 3DGS framework, with its divergent masking strategy (M_h and Ms), is a particularly smart mechanism for mitigating the risk of \"Confirmation Bias\" among the different 3DGS models.\n* Practicality and technical innovation of the EMA-GS variant -- authors successfully tackle the increased computational budget of the dual 3DGS framework.\n* Rigorous & comprehensive empirical validation -- authors compare with various prior methods, including the most recent ones. Table 4 is particularly insightful and includes many important ablations.\n\nWeaknesses:\n* The difference and comparison between HybridGS (another dual method) is not articulated well enough.\n* Lack of hyper-parameter sensitivity analysis. Specifically, the method's performance hinges on the careful balancing of several loss terms and update rules, controlled by (\\lambda_m and \\lamba_{mask}). For a method that claims \"robustness,\" it is critical to demonstrate that it is not overly sensitive to its own internal settings.\n* Computational cost of mask generation is unclear (was it included into the run-times?)\n* The limitations section (in the Supplementary) is quite brief.", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the detailed and constructive review. We greatly appreciate your recognition of the core ideas, practical improvements, and empirical rigor of our work, as well as your thoughtful suggestions for improving clarity, analysis, and comparisons. Your concerns are addressed below.\n\n---\n**W1: The difference and comparison between HybridGS (another dual method) is not articulated well enough.**\n\nOur approach differs fundamentally from HybridGS in both design and training. HybridGS separates static and dynamic content using two models (3DGS for static, 2DGS for dynamic) and requires a staged training process with a learnable blending mask. In contrast, our method uses a dual 3DGS setup with mutual supervision to improve robustness against dynamic noise, all within the standard 3DGS training pipeline. While both methods use masking, HybridGS blends outputs based on 2DGS-derived uncertainty, whereas we apply two distinct masking strategies to reduce confirmation bias from a single, potentially inaccurate mask.\n\nWe will elaborate more on these differences in the main paper.\n\n---\n**W2: Lack of hyper-parameter sensitivity analysis. Specifically, the method's performance hinges on the careful balancing of several loss terms and update rules, controlled by ($\\lambda_m$ and $\\lambda_{mask}$). For a method that claims \"robustness,\" it is critical to demonstrate that it is not overly sensitive to its own internal settings.**\n\n**Table A: Performance under varying weights of the mutual consistency loss.**\n\n| Dataset | On-the-go  |  |  | RobustNeRF |  |  |\n|---|---|---|---|---|---|---|\n| $\\lambda_m$  | PSNR | SSIM | LPIPS | PSNR | SSIM | LPIPS |\n| 0 | 23.10 | 0.801 | 0.132 | 29.72 | 0.915 | 0.081 |\n| 0.05 | 23.39 | 0.803 | 0.136 | 30.20 | 0.917 | 0.080 |\n| 0.1 | 23.40 | 0.801 | 0.135 | 30.27 | 0.917 | 0.080 |\n| 0.2 | 23.43 | 0.805 | 0.133 | 30.30 | 0.917 | 0.081 |\n| 0.3 | 23.47 | 0.805 | 0.134 | 30.24 | 0.917 | 0.081 |\n\n\n**Table B: Performance under varying weights of the learnable mask loss.**\n| Dataset | On-the-go  |  |  | RobustNeRF |  |  |\n|---|---|---|---|---|---|---|\n| $\\lambda_{mask}$ | PSNR | SSIM | LPIPS | PSNR | SSIM | LPIPS |\n| 0.5 | 23.33 | 0.802 | 0.134 | 30.20 | 0.916 | 0.080 |\n| 1.0 | 23.40 | 0.801 | 0.135 | 30.27 | 0.917 | 0.080 |\n| 1.5 | 23.39 | 0.804 | 0.135 | 30.20 | 0.917 | 0.080 |\n| 2.0 | 23.32 | 0.802 | 0.134 | 30.21 | 0.916 | 0.081 |\n\nAs shown in Table A and B, although the best performance is generally achieved at our default setting ($\\lambda_m = 0.1$ and $\\lambda_{mask} = 1.0$), the differences across settings are minimal (less than 0.1 dB). This indicates that the performance is not highly sensitive to the values of $\\lambda_m$ and $\\lambda_{mask}$. When $\\lambda_m = 0$, mutual consistency regularization is disabled, resulting in a significant performance drop. Due to time constraints, all experiments above were conducted using the EMA-GS setting, and the PhotoTourism dataset was excluded.\n\n---\n**W3: Computational cost of mask generation is unclear (was it included into the run-times?)**\n\n**Table C: Average per-scene running time across different datasets (in hours).**\n\n| Time\\Dataset | On-the-go  | RobustNeRF  | PhotoTourism |\n|---|---|---|---|\n| Mask Generation  | 0.06 | 0.07 | 0.56 |\n| Training (GS-GS)  | 0.28 | 0.31 | 5.30 |\n| Training (EMA-GS)  | 0.18 | 0.21 | 2.90 |\n\nNo, it is not included in the overall runtime. Mask generation is a one-time preprocessing step performed offline, similar to COLMAP in existing 3DGS pipelines, which is also excluded from training time. We present the mask generation times in Table C. Notably, the PhotoTourism dataset requires significantly more time due to its higher image resolution and greater number of views.\n\n---\n**W4: The limitations section (in the Supplementary) is quite brief.**\n\nWe follow WildGaussians in using a per-view appearance embedding to control global appearance, and a per-Gaussian embedding shared across views to model the appearance of individual Gaussian primitives. While this strategy is effective for handling dynamic lighting variations across training frames, it may struggle to capture fine-grained effects such as object highlights. This can happen when the per-view and intrinsic appearances are not well decoupled, where the per-view embedding incorrectly attributes static intrinsic appearance to dynamic lighting, and the per-Gaussian embedding lacks the capacity to model high-frequency details. While our proposed mutual consistency loss helps improve this decoupling, it does not completely resolve the issue. This limitation is common in many existing works [1,2,3] and remains an important direction for future research.\n\n[1] Fridovich-Keil et al. - 2023 - K-Planes: Explicit Radiance Fields in Space, Time, and Appearance\n\n[2] Dahmani et al. - 2025 - SWAG: Splatting in the Wild Images with Appearance-Conditioned Gaussians\n\n[3] Kulhanek et al. - 2024 - WildGaussians: 3D Gaussian Splatting In the Wild\n\n---\n**Q1: Improve experimental clarity: clarify all ambiguous methodological and experimental details, including the full computational cost of the pipeline (with mask generation), the precise definition of the alternating masking strategy, and provide statistical significance for the main results (e.g., mean and standard deviation over multiple runs).**\n\n**(1) The full computational cost of the pipeline is discussed in the response to Weakness 3.**\n\n**(2) Clarification on the alternating masking strategy**\n\nIn the GS-GS setting, we apply distinct masks to each branch of the dual 3DGS models as shown in Eq. (8). However, this is not feasible in the EMA-GS setting, which uses only a single 3DGS model and maintains an exponential moving average for consistency regularization. To encourage diverse supervision, we adopt an alternating masking strategy described in Eq. (11): one mask is used during a given optimization step, and the other is applied in the following step, alternating throughout training. This approach mitigates the bias of relying on a single mask.\n\n**(3) Statistical significance for the main results**\n\n**Table D: Quantitative results on the NeRF On-the-go dataset. Each experiment is repeated five times, and we report the mean and standard deviation.**\n\n| Setting |GS-GS | | | EMA-GS | | |\n|------------------|--------|--------|--------|--------|--------|--------|\n| Scene | PSNR | SSIM | LPIPS | PSNR | SSIM | LPIPS |\n| High Occlusion | 24.36 ± 0.02 | 0.823 ± 0.001 | 0.151 ± 0.001 | 24.11 ± 0.05 | 0.819 ± 0.002 | 0.152 ± 0.004 |\n| Medium Occlusion | 24.52 ± 0.06 | 0.871 ± 0.001 | 0.090 ± 0.001 | 24.26 ± 0.08 | 0.864 ± 0.001 | 0.092 ± 0.002 |\n| Low Occlusion | 21.99 ± 0.04 | 0.730 ± 0.001 | 0.184 ± 0.004 | 21.81 ± 0.09 | 0.723 ± 0.002 | 0.166 ± 0.007 |\n\n**Table E: Quantitative results on the RobustNeRF dataset. Each experiment is repeated five times, and we report the mean and standard deviation.**\n| Setting |GS-GS | | | EMA-GS | | |\n|------------------|--------|--------|--------|--------|--------|--------|\n| Scene | PSNR | SSIM | LPIPS | PSNR | SSIM | LPIPS |\n| Statue | 23.44 ± 0.05 | 0.893 ± 0.001 | 0.098 ± 0.001 | 23.46 ± 0.06 | 0.890 ± 0.001 | 0.097 ± 0.002 |\n| Android| 25.58 ± 0.05 | 0.856 ± 0.001 | 0.070 ± 0.003 | 25.47 ± 0.06 | 0.849 ± 0.002 | 0.070 ± 0.002 |\n| Yoda | 37.12 ± 0.09 | 0.969 ± 0.001 | 0.074 ± 0.001 | 36.46 ± 0.06 | 0.967 ± 0.001 | 0.078 ± 0.001 |\n| Crab | 36.11 ± 0.07 | 0.963 ± 0.001 | 0.079 ± 0.001 | 35.52 ± 0.07 | 0.961 ± 0.001 | 0.080 ± 0.001 |\n\nWe repeated the experiment for five time. We did not include results on the PhotoTourism dataset due to time constraints, but we will include it in the final version. Based on the results in Table D and E, our method shows statistically significant improvements.\n\n---\n**Q2: It would be great to include sensitivity analysis for the key hyperparameters.**\n\nPlease refer to Weakness 2, we will revise it accordingly.\n\n---\n**Q3: Strengthen Related Work -- a more nuanced discussion of the related literature, explicitly contrasting the proposed conceptual framework with that of HybridGS would be quite beneficial.**\n\nPlease refer to Weakness 1, we will revise it accordingly."}, {"review_id": "WlnmIZdvTC", "reviewer": "Reviewer_REjw", "summary": "This paper presents a novel method for outlier suppression in 3DGS training. Their method is motivated by the observation that training the same model twice, but with different sample ordering, results in output stochasticity in uncertain areas. They therefore train two model simultaneously enforcing a cross-consistency loss that encourages model to agree photometrically when rendered from the same view. Furthermore, they give each model separate masking strategies to mitigate confirmation bias. These masking strategies are similar to some previous works, but allow them to combine strengths of each. The method also involves some extra machinery to account for appearance modeling and offers an EMA variant offers efficiency gains. The method achieves state of the art results over most datasets and the design choices are thoroughly ablated.", "questions": "1. Can you elaborate on the technique of the multi-cue adaptive mask? I'm a bit confused - are \"static\" regions always included? note that semantically static objects could still be considered as outliers due to difficult to model specular reflections. Are then only the \"dynamic\" regions considered for the union of the other cues?\n\n2. How is the self-supervised mask regularized against collapsing to zero?", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The paper is generally well written in all aspects, and the technique is sufficiently described. \n\nThe method, though it is somewhat complicated in it's complete implementation, it achieves state of the art results over a sufficient set of the most relevant datasets, making it a significant contribution. This perhaps isn't too surprising, as their method allows them to combine the strengths of previous methods through the use of different masking strategies in the models. All design choices are thoroughly ablated, justifying the architecture.  \n\nDetails are scant in the description of the Multi-Cue Adaptive Mask. I would prefer to see some of this in mathematical notation. The method also sounds somewhat similar to the newly released RoMo [1], in that it combines SAM features with epipolar cues.\n\nIf I further wanted to nitpick at weaknesses, one could say that because the method could be seen as one that combines strengths of previous works, it could be classed as less than original. The authors observation that retrained models tend to disagree in areas of uncertainty is simply empirical and could have been better motivated. Yet this is difficult to do in a limited amount of space.\n\nIt could also be faulted for being somewhat complex - yet there is no other simpler technique that can achieve the same metrics.\n\n[1] https://romosfm.github.io/", "was_revised": true, "final_justification": "I thank the authors for the information provided in their rebuttal. In my opinion the methods in the \"Multi-Cue Adaptive Mask\" deserve a bit more attention in the main text. I can't fault the authors for a simple empirical observation about training image ordering. The exact underlying causes of this I'm sure will be answered by further work. I also thank the authors for their explanation about why the self-supervised mask can't collapse like other strategies I've seen.", "rebuttal": "Thank you for the thoughtful and encouraging review. We truly appreciate your recognition of the paper’s clarity, technical depth, and contributions, as well as your constructive feedback on areas for improvement.\n\n---\n**W1: Details are scant in the description of the Multi-Cue Adaptive Mask.**\n\nDue to space constraints in the main paper and our focus on presenting the dual-model framework and its efficient variant, we chose to include the detailed formulation of the Multi-Cue Adaptive Mask in the supplementary material. For better clarity, we also provide an expanded explanation in our response to Question 1 below.\n\n---\n**W2: The authors observation that retrained models tend to disagree in areas of uncertainty is simply empirical and could have been better motivated.**\n\nWe observe that randomness in the training process, particularly in the order in which views are fed into the 3DGS optimization, can lead to stochastic visual artifacts. While our paper does not provide a formal theoretical justification for this phenomenon, the underlying intuition is reasonable: if a single noisy view is introduced late in training, its influence may not be sufficiently corrected and could result in more prominent artifacts. Conversely, when introduced earlier, the model may better learn to ignore or compensate for the noise. These inconsistencies motivate our design of a dual 3DGS framework, where two models supervise each other. Since they are trained on the same scene, their shared static regions remain consistent, while the differing artifacts in uncertain areas provide complementary supervision signals, allowing regions affected by artifacts in one model to be recovered by the other.\n\n---\n**Q1: Can you elaborate on the technique of the multi-cue adaptive mask? I'm a bit confused - are \"static\" regions always included? note that semantically static objects could still be considered as outliers due to difficult to model specular reflections. Are then only the \"dynamic\" regions considered for the union of the other cues?**\n\n**(1) Multi-Cue Adaptive Mask**\n\nThe Multi-Cue Adaptive Mask is a hard binary mask derived from four cues: semantic segmentation, stereo correspondence, pixel-level residuals, and feature-level residuals.\n\nWe first segment each image into **semantic regions** using Semantic-SAM [1], which serve as units for aggregating cues.\n\nFor **stereo correspondence**, we use SIFT matches from COLMAP [2]. A pixel with matches in more than three views is considered a reliable static point ($\\mathbf{S}_{\\ge 3}$ in Algorithm 1 of the supplementary material). Regions with many such points are unlikely to be distractors.\nWe also compute **pixel-level** ($L_1$ loss) and **feature-level** (DINOv2 cosine distance) residuals between rendered and ground-truth images. High residuals indicate distractors.\n\nEach region is classified as a distractor if: (1) Its average residual exceeds the global mean, and (2) Its static point density is below the global mean. Distractor regions are excluded by assigning a mask value of zero. **We take the union of all such regions and subtract them from an all-one mask.**\n\n**(2) Specular Reflections**\n\nFollowing the original 3DGS, we use spherical harmonics (Eq. (2) of our main paper) to model view-dependent appearance variations, including specular reflections. When specular effects are mild, our method remains effective. In rare cases where specular reflections are severe and cause abrupt appearance changes across nearby viewpoints, such regions may be treated as outliers by our masking mechanism. However, this scenario is uncommon when sufficient training views are available. We acknowledge this limitation and plan to explore more robust handling of such extreme cases in future work.\n\n\n[1] Li et al. - 2025 - Segment and Recognize Anything at Any Granularity\n\n[2] Schonberger and Frahm - 2016 - Structure-From-Motion Revisited\n\n---\n**Q2: How is the self-supervised mask regularized against collapsing to zero?**\n\nThe learnable mask is optimized to reflect the feature residual between the rendered and ground-truth images. This residual only becomes zero when the features are orthogonal, which is a rare occurrence. Therefore, the target generally remains non-zero throughout training, making mask collapse unlikely.\n\nIn addition, we include a warm-up stage where the 3D representation is sufficiently established. This ensures that in static regions, where the rendered and ground-truth features are highly similar, the residual provides a stable and meaningful supervision signal to guide mask learning.\n\nLastly, the mask is learned independently for each view. If a mask temporarily collapses in certain regions for one view, signals from other views can still contribute effectively, since multiple views are jointly used in the 3DGS optimization process.\n\nWhile we did not observe mask collapse in our experiments, we believe including a regularization term is beneficial. We will try an explicit regularization loss that penalizes excessively small mask values (e.g., L2 loss targeting a moderate value), encouraging the mask to remain active and informative throughout training."}, {"review_id": "KOuMCeHoGQ", "reviewer": "Reviewer_b1d5", "summary": "This paper proposes a new method to **reconstruct scenes despite transient objects** being present in only some of the frames (e.g. a passerby in an outdoor scene). Specifically, Asymmetric Dual 3DGS optimizes two 3DGS in parallel and leverages the consistency between the two reconstructions to improve them. This is achieved in two steps: \n* **Enforcing consistency between the models**: their renderings are supervised to be identical.\n* **Preventing the two reconstructions from being identical**: the two 3DGS identify transient objects in different ways, the first 3DGS uses a combination of SAM+COLMAP+DINOv2 and residual error, while the second uses the DINOv2 residual error only (i.e. the difference in DINO features between render and ground-truth).\nThen, the authors apply both transient strategies to the first model, and replace the second model by a moving average of the first, reducing the computational overhead of their technique.\nBoth approaches perform well on a number of datasets with transient objects and varying lighting, comparing against numerous methods such as HybridGS and WildGaussians.", "questions": "- It is a bit counter-intuitive that the dual-models would perform better than a single model combining both masking strategies (Single w/ Mh/s). What is the interpretation of this result?\n- There are many other ways to keep the models different, and using two different losses for the same goal of masking transient objects is not the most intuitive. Have you experimented with other ideas, such as using different initializations or training on different subsets of images?\nMinor points and typos:\n- The term \"frame-independent\" is used with and without dash. It also does not exactly convey the meaning intended.\n- In the EMA section, it is unclear at the beginning that the main model uses both masking strategies. It should be mentioned early.", "limitations": "- yes, limitations are discussed but only in the supplementary material.\nDespite their contributions, these methods largely rely on heuristic strategies to suppress\ncorrupted supervision signals from low-quality training data. [...] In this work, we propose a\nprincipled framework, Asymmetric Dual 3DGS.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "+: The overall observation and philosophy behind the approach are sound and interesting: training two similar 3DGS models to be similar, while making sure they remain different.\n\n-: However in practice, the way these reconstructions are kept different is by introducing more losses to perform the same task of transient mask estimation.\n\n-: The paper does not attempt to meaningfully explain how it differs from previous methods, and instead describes the entire literature of transient NeRF/3DGS as \"heuristic strategies\". This description is very vague and could be used to describe the proposed method too, which is instead called a \"principled framework\". For example, the estimation of multi-cue adaptive mask is very \"heuristic\".\n\n-: In fact, some of these works have a fairly similar idea, such as SimpleNeRF which also uses a simpler model to regularize the main one, or HybridGS which uses an additional 2DGS to separate moving elements. To be clear, these methods are not exactly identical, but the paper would benefit from discussing these differences in detail and why they matter.\n\n+: The experiments compare with several methods for radiance field reconstruction that are robust to transient objects. An extensive ablation study is also performed.", "was_revised": true, "final_justification": "After reading the rebuttal and other reviews, most of my concerns have been resolved. Thus, I retain my rating.", "rebuttal": "Thank you for the insightful review. We’re grateful for your acknowledgment of our key ideas and experimental efforts. We will thoroughly respond to the concerns below.\n\n---\n**W1: However in practice, the way these reconstructions are kept different is by introducing more losses to perform the same task of transient mask estimation.**\n\nWe acknowledge the reviewer’s observation. In our design, we use two distinct masking strategies for each branch of the dual 3DGS model. This explicitly enforces asymmetric supervision over dynamic regions, promoting diversity between the models.\n\nAt the same time, we apply a mutual consistency loss to encourage both models to agree in the static regions. This loss also enables each model to recover information that may be suppressed in the other due to masking, helping to mitigate confirmation bias arising from potentially inaccurate masking strategies. In other words, areas blocked in one branch can often be reconstructed using guidance from the other avoiding same masking error.\n\nOverall, our approach maintains divergence in dynamic regions while enforcing consistency in static regions, enabling both robustness and completeness. In Eq. (8), the first two terms correspond to reconstruction under separate masks, while the middle terms encode cross-model consistency, supporting static region reinforcement and dynamic region recovery.\n\n---\n**W2: The paper does not attempt to meaningfully explain how it differs from previous methods, and instead describes the entire literature of transient NeRF/3DGS as \"heuristic strategies\". This description is very vague and could be used to describe the proposed method too, which is instead called a \"principled framework\". For example, the estimation of multi-cue adaptive mask is very \"heuristic\".**\n\nWe refer to existing methods as heuristic based on the design choices they make and the lack of formal principles guiding their generalization. For example, RobustNeRF introduces custom loss functions to suppress noisy training signals. However, this is achieved using several hand-crafted rules, for instance, treating residuals as inliers only if they fall below a certain percentile within a certain-size local neighborhood. While effective empirically, such rules are ad hoc and task-specific, potentially compromising stability when conditions vary.\n\nIn contrast, our Dual 3DGS framework is grounded in the principle of mutual consistency. It explicitly leverages the assumption that transient artifacts are random across training runs, guiding the training process toward stable and generalizable representations. This principle-driven formulation allows our method to generalize better to diverse scenes.\n\nNevertheless, we acknowledge that our approach still involves certain heuristic components. To improve clarity, we will revise the manuscript to replace vague terms like ''heuristic'' and ''principled'' with more specific and objective descriptions of the methodological differences.\n\n---\n**W3: In fact, some of these works have a fairly similar idea, such as SimpleNeRF which also uses a simpler model to regularize the main one, or HybridGS which uses an additional 2DGS to separate moving elements. To be clear, these methods are not exactly identical, but the paper would benefit from discussing these differences in detail and why they matter.**\n\nThank you for your advice on more discussions about related methods. \n\nOur approach differs fundamentally from HybridGS in both design and training. HybridGS separates static and dynamic content using two models (3DGS for static, 2DGS for dynamic) and requires a staged training process with a learnable blending mask. In contrast, our method uses a dual 3DGS setup with mutual supervision to improve robustness against dynamic noise, all within the standard 3DGS training pipeline. While both methods use masking, HybridGS blends outputs based on 2DGS-derived uncertainty, whereas we apply two distinct masking strategies to reduce confirmation bias from a single, potentially inaccurate mask.\n\nSimpleNeRF focuses on few-shot 3D reconstruction with sparse input views, using multiple augmented NeRF models (parallel and serial) to provide additional supervisory signals. In contrast, our work tackles robust in-the-wild reconstruction by employing two parallel 3DGS models with mutual consistency regularization, which reinforces static regions while suppressing dynamic artifacts through complementary representations. Additionally, SimpleNeRF requires about 22 hours of training per scene, whereas our asymmetric dual 3DGS completes training in under one hour.\n\nWe will elaborate more on these differences in the main paper.\n\n---\n**Q1: It is a bit counter-intuitive that the dual-models would perform better than a single model combining both masking strategies (Single w/ Mh/s). What is the interpretation of this result?**\n\nBoth the dual-model settings (lines 5 and 12 in Table 4) and the single model with alternating masks (line 1) utilize two distinct masks. However, a key difference lies in **the mutual consistency loss** used in the dual-model setup. This loss reinforces reliable signals from static regions and enables one model to help recover regions suppressed by the mask in the other.\n\nWhen the mutual consistency loss is removed (lines 9 and 16 in Table 4), the performance of the dual-model framework drops and approaches that of the single model, effectively becoming a simple ensemble.\n\n---\n**Q2: There are many other ways to keep the models different, and using two different losses for the same goal of masking transient objects is not the most intuitive. Have you experimented with other ideas, such as using different initializations or training on different subsets of images?**\n\nYes, we have explored several strategies to introduce diversity between the models. For instance, we experimented with **random dropout**, where a subset of 3DGS primitives is randomly frozen during optimization, and **random mixup**, where the ground-truth image is blended with the rendered output for a given view. These methods are evaluated in **lines 10 and 11 of Table 4** in the main paper. The results show that, although they introduce randomness, their effectiveness is limited compared to the alternating masking strategy. \n\nWe considered using different initializations or subsets of views but ultimately chose not to pursue these approaches. Because the original 3DGS paper demonstrated that deviating from COLMAP-based initialization leads to significant performance degradation. Similarly, training with only partial data adversely affects reconstruction quality.\n\n---\n**Q3: The term \"frame-independent\" is used with and without dash. It also does not exactly convey the meaning intended.**\n\nThank you for pointing this out. The use of \"frame-independent\" in Line 145 is a typographical error, and we intended to write \"frame-dependent\". In our work, the appearance modeling is actually view-dependent, as we assign a learnable global appearance vector to each input view and jointly optimize it with the rest of the 3DGS model. The term \"frame\" was intended to mean \"view,\" but we agree that \"view-dependent\" is a more accurate and consistent term. We will revise the terminology throughout the paper accordingly.\n\n---\n**Q4: In the EMA section, it is unclear at the beginning that the main model uses both masking strategies. It should be mentioned early.**\n\nIn the EMA section, our primary focus was to highlight the computational efficiency introduced by the EMA proxy design, and the masking strategy was not emphasized upfront. Although the alternating masking strategy is described later in the subsection, we agree that providing a brief mention of it earlier in the section would give readers a clearer overview. We will revise the text to introduce the masking strategy at the beginning for better clarity."}, {"review_id": "4ExLiMv7OO", "reviewer": "Reviewer_8u6B", "summary": "This paper propose a novel and interesting framework to tackle distractors for in-the-wild scene reconstruction. The authors leverage the stochastic nature of artifacts caused by distractors to form a mutual consistency loss for different runs. To avoid both runs converage toward the same reconstruction errors, the authors apply two types of dynamic masking strategy for two models. For computational efficiency, the authors apply EMA strategy during optimization to avoid double gradient descent. In a nutshell, this paper is interesting and somewhat intuitive.", "questions": "For Eq.(7), will it still work in the case that the reconstructed 3D scene overfits to the distractors? If not, how does this method avoid overfitting to distractors with 3D Gaussians floating in the sky?  \nMore can be seen in \"Weakness\"", "limitations": "yes", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "**Strengths:** \n\n1. This paper propose to leverage the stochastic nature of artifacts to tackle distractors, which is insteresting. \n2. To obtain mutual consistency across different runs, this paper trains two models in parallel, and further accelerate the training speed with EMA strategy.\n3. The ablation study is comprehensive and convincing.\n\n**Weakness:**\n\n1. My major concern is that some statements in this paper requires further visual analysis to support, especially the stochastic nature of artifacts, which is the core idea in this paper. \n    - For example, I wanna see more examples across various scenes and different runs to demonstrate the appearance of artifcats is stochastic. \n    - The visual comparison between applying the same mask or different mask on two parallel models is also necessary to show whether this design can avoid converging toward the same reconstruction errors\n    - Does the above conclusion also work for EMA stategy? A visual comparison is also needed here.", "was_revised": true, "final_justification": "I've read all the reviewers' questions and the authors' responses. The authors have addressed most of the reviewers' questions. No further questions. Because most reviewers concern about the relationship between the mask strategy and hybrid-GS, the authors should add more visual comparisons and examples in the camera-ready version / supplementary material according to the reviewers' suggestions.", "rebuttal": "Thank you for your thoughtful feedback and valuable suggestions! We are sorry that we cannot include images showing the stochastic artifacts according to new rebuttal rules, but we are committed to including more visual evidences in the next step.\n\n---\n**W1: My major concern is that some statements in this paper require further visual analysis to support, especially the stochastic nature of artifacts, which is the core idea in this paper. For example, I wanna see more examples across various scenes and different runs to demonstrate the appearance of artifacts is stochastic. The visual comparison between applying the same mask or different masks on two parallel models is also necessary to show whether this design can avoid converging toward the same reconstruction errors. Does the above conclusion also work for EMA strategy? A visual comparison is also needed here.**\n\nWe sincerely appreciate the reviewer’s insightful comments. Unfortunately, unlike NeurIPS last year, this venue does not provide a mechanism to submit a rebuttal PDF with visual evidences.\n\nRegarding **the stochastic nature of artifacts**, we included visual examples in the main paper. As shown in Figure 1, different runs of 3DGS on the same scene (with only the view order randomized) result in different artifacts, particularly in uncertain regions. The mutual consistency loss helps suppress these artifacts in both models. On one hand, the shared static regions remain consistent and act as a strong regularizer. On the other hand, the differing artifacts in uncertain areas provide complementary supervision signals, allowing regions affected by artifacts in one model to be recovered by the other.\n\nFor **effect of distinct masking strategies**, Table 4 in the main paper presents a quantitative comparison. The performance degrades when both models use the same mask, for both GS-GS and EMA-GS settings. This supports our claim that using separate masks helps prevent convergence to the same erroneous reconstruction patterns.\n\nWe agree that linking quantitative results to visual evidence would strengthen our conclusions. Therefore, we are committed to including additional visual comparisons across different scenes, masking strategies, and framework settings in the supplementary materials.\n\n---\n**Q1: For Eq.(7), will it still work in the case that the reconstructed 3D scene overfits to the distractors? If not, how does this method avoid overfitting to distractors with 3D Gaussians floating in the sky?**\n\nYes, Eq.(7) remains effective even if the 3D reconstruction overfits to distractors. \n\nWhen the model overfits to distractors, the rendered regions corresponding to distractors will exhibit noticeable deviations from the ground truth. These discrepancies result in low similarity scores, which serve as targets for the learnable mask as shown in Eq.(7). As a result, the mask is driven toward zero in those regions, effectively suppressing the influence of distractors. \n\nFor floating Gaussians in the sky, we adopt the pruning strategy from the original 3DGS implementation to remove transient or oversized primitives, preventing their accumulation in regions without valid geometry."}]

Author rebuttal:

Meta-review: Based on the discussion on the stochastic nature of training 3DGS by its randomness, they proposed an asymmetric dual 3DGS. Divergent masking techniques, i.e., multi-cue adaptive and self-supervised soft masks, are applied to ensure the asymmetrical models. One clever trick is one of two asymmetric models is replaced with EMA Gaussians to avoid directly training two models and a masking strategy is alternatively applied to give stochasticity. AC thought this interesting approach may need some attention from the community for further developments. Strong experimental validation and ablation studies support this idea.

There are no remaining major issues, and two reviewers recommend a “spotlight” presentation for its novelty. AC also enjoyed reading this paper and glad to recommend it as a spotlight.

AC strongly encourages you to revise the manuscript in accordance with the rebuttal responses and to responsibly prepare the release of the code for the benefit of the community.

Minor comments: 
- You should correctly refer to the corresponding section of Appendix, especially, for Multi-Cue Adaptive Mask in the manuscript. Recommend that every sections, tables, abd figures are appropriately refered in the manuscript. This will greatly help for readers.
- Section titles should be in lower cases except for the first word and other special cases according to the official NeurIPS style guideline. “All headings should be lower case (except for first word and proper nouns)” in Line 55. Major ML/vision conferences adopt this style except some.
Decision: Accept (spotlight)