Novel View Synthesis from A Few Glimpses via Test-Time Natural Video Completion
Authors: Yan Xu
Yixing Wang
Stella X. Yu
Venue: NeurIPS 2025 poster ()
Keywords: Novel View Synthesis
3D Gaussian Splatting
Primary area: applications

Abstract: Given just a few glimpses of a scene, can you imagine the movie playing out as the camera glides through it? That’s the lens we take on sparse-input novel view synthesis, not only as filling spatial gaps between widely spaced views, but also as completing a natural video unfolding through space. We recast the task as test-time natural video completion, using powerful priors from pretrained video diffusion models to hallucinate plausible in-between views. Our zero-shot, generation-guided framework produces pseudo views at novel camera
poses, modulated by an uncertainty-aware mechanism for spatial coherence. These synthesized frames densify supervision for 3D Gaussian Splatting (3D-GS) for scene reconstruction, especially in under-observed regions. An iterative feedback loop lets 3D geometry and 2D view synthesis inform each other, improving both the scene reconstruction and the generated views. The result is coherent, high-fidelity renderings from sparse inputs without any scene-specific training or fine-tuning. On LLFF, DTU, DL3DV, and MipNeRF-360, our method significantly outperforms strong 3D-GS baselines under extreme sparsity. Our project page is at https://decayale.github.io/project/SV2CGS.

Novel View Synthesis from A Few Glimpses via Test-Time Natural Video Completion
Yan Xu
Yixing Wang'
Stella X. Yu1.2
University of Michigan UC Berkeley fyxumich.yixing, stellayu) @umi ch. edu
Abstract
Given just few glimpses of scene, can you imagine the movie playing out the camera glides through it? That's the lens we take on sparse-input novel view synthesis, not only filling spatial gaps between widely spaced views, but also completing a natural video unfolding through space.
We recast the task as test-lime natural video completion, using powerful priors from pretrained video diffusion models hallucinate plausible in-between views. Our zero-shot, generation guided framework produces pseudo views at novel camera poses, modulated by an nncertainty-awan mechanism for spatial coherence. These synthesized frames densify supervision for 3D Gaussian Splatring (3D-GS) for scene reconstruction, especially in under-observed regions. iterative feedback loop lets 3D geometry and 2D view synthesis inform each other, improving both the scene reconstruction and the generated views.
The result is coherent, high-fidelity renderings from sparse inputs without any scene- specific training or fine-tuning. On LLFF, DTU, DL3DV, and MipNeRF-300 our method significantly outperforms strong 3D-GS baselines under extreme sparsity. Our project page is at https: https://deccyalee gi thub io httppooject//Vccc..
1 Introduction
Humans can effortlessly imagine how scene appears from unseen viewpoints by mentally filling in gaps, by drawing prior visual experience infer what's missing. Inspired by this ability, we reinterpret novel view synthesis - long-standing challenge in computer vision and graphics [8, 21, 11, 42, 31, 66, 44, 18, 27] as the task of completing natural video from sparse camera views (Fig. 1). From this perspective, sparse-input novel view synthesis becomes analogous to recovering missing frames in video captured along an unconstrained camera trajectory. This framing naturally invites the use of powerful generative priors learned from large-scale video data. In particular, pretrained video diffusion models [5, 55], which are trained synthesize coherent and realistic scene motions, offer a compelling tool for filling in plausible scene content between widely spaced views.
Recently, NeRF [31, 2, 4, 42] and 3D Gaussian Splatting (3D-GS) [18, 61, 13, 15, 28) have significantly advanced novel view synthesis. Unlike NeRF, which represents scenes using an implicit function, 3D-GS models scenes explicitly with set of 3D Gaussian primitives and renders images through efficient rasterization. 3D-GS achieves photorcalistic rendering with substantially faster inference speed, making it focal point of recent research interest.
However, synthesis from sparse inputs remains difficult. NeRF 3D-GS methods typically rely on dense input views to accurately constrain the optimization process. In sparse-view settings, occlusions and geometric ambiguities [63] often lead to rendering artifacts and degraded quality. Recent efforts [22, 67, 10, 16, 50, 52] focus more constrained camera paths (e.g., object-centric or forward-facine views). contrast, real-world image capture from walking with a handheld smartphone often produces widely spaced, unconstrained views with large occlusions and out-of-view view regions (Fig. 1).
39th Conference on Neural Information Processing Systems (NeurIPS 2025).

---

30 Gausriann Splating
loput View
interpoldeed View
Traising
Frozen
Baselina (wio video diffusion supervision)
wia GS primitive densification
Ground Truth
Figure I: We view sparse-input novel view synthesis temporal-spatic completion of a natural- looking video. Left: Our seneration-guider reconstruction pipeline. With the initialized 3D-GS from sparse input views, T we create guidance images interpolated poses and estimate their uncertainty, based on the currently optimized 3D-GS. @ Using both guidance images and their uncertainties, we modulate the diffusion score function to interpolate between sparse input views. C The interpolated views are used to constrain 3D-GS optimization. Right: With our interaatio-guidee reconstruction, the under-observed regions in the inputs are enhanced by the views generated by the diffusion model.
Motivated by the natural video completion perspective and strong priors in pretrained video diffusion models, we propose a zero-shot, generation-guided reconstruction pipeline integrating video diffusion with 3D-GS. Our approach defines target camera trajectories between sparse input views and uses video diffusion priors to synthesize plausible intermediate pseudo-views. These views provide supervision to better constrain 3D-GS training, especially in the under-observed regions in the inputs.
To recover missing views along a natural video trajectory, we must generate images at specified camera poses. However, existing video diffusion models [5, 6, 19, 45] are typically conditioned only on the initial frame and produce uncontrolled camera motions. While recent methods [49, 60] introduce trajectory conditioning during training, they still lack guarantees of pose alignment at inference and rely heavily on datasets with camera parameters, limiting generalization and scalability.
We propose novel uncertainty- aware modulation mechanism that couples video diffusion with 3D Gaussian Splatting (3D-GS), enabling accurate, controllable frame interpolation under vverse-view settings. In this setup, 3D-GS provides consistent 3D representation to guide view synthesis, while synthesized frames serve as pseudo supervision to further refine the 3D-GS model.
Fig. 1 illustrates our overall workflow. Our method begins by initializing 3D-GS from sparse views. After initialization, we interpolate camera poses between sparse inputs and create corresponding guidance images on the interpolated poses by inversely warping pixels from the nearest input view. The warping process is based on the depth maps rendered by the currently optimized 3D-GS. These guidance images are essential to maintaining the content and structural consistency during view interpolation, but may contain missing parts and artifacts due to imperfect 3D-GS depths and occlusion. We thus further model the uncertainty of these guidance images by assessing cross-vier consistency terms of photometry and geometry, and thereby focus the diffusion process more on correcting high-uncertainty regions, while keeping the reliable parts. Using both the guidance images and their associated uncertaintics, we adaptively modulate the diffusion process to interpolate between the sparse views. The interpolated pseudo views are then added to the training set of 3D-GS. Furthermore, to improve the scene completeness for 3D-GS, we propose a Gaussian primitive densification module to densify the 3D-GS point cloud in under-observed regions using these pseudo views as bridges. The process above is repeated iteratively to refine the 3D-GS reconstruction.
To summarize, our contributions are threefold: 1) We propose zero-shot, generation-guiin 3D-GS pipeline that leverages pretrained video diffusion models to improve novel view synthesis under sparse inputs. 2) We introduce an wncertainty-aware modulation mechanism to integrate 3D-GS with video diffusion for controllable pseudo-vies generation, and Gaussian primitive densification module to enhance scene completeness. 3) Our method achieves tate-of-the-as performance, with

---

over 2.5 dB PSNR gain on DL3DV and strong results on LLFF and DTU, demonstrating robust generalization. While we primarily use Stable Video Diffusion [5], our framework is agnostic the diffusion backhone and compatible with alternatives [55, 19].
2 Related Work
Sparse-input Novel View Synthesis. Sparse-inpun novel view synthesis aims to reconstruct representation for generating novel views of a scene using few input images. Although existing training-based methods, Le. NeRF [31] and 3DGS [18], work well with dense inputs, their performance drops significantly with sparse views due to overfitting [37, 46, 33, 12, 39]. Several recent works explore robust novel view synthesis under sparse inputs. One group 33, 16, 43, 40, 20, 67, 56] focuses on imposing additional regularization on views deviating from the training views. For example, GeoAug [7] randomly samples novel views around input frames and constrains rendering match the input after view warping. Niemeyer et al. [33] introduce smooth depth regularization on unseen views. SPARF [43]. GeCoNeRF [20], and Few ViewGS [56] integrate multi-view correspondence and geometry loss into optimization. However, these methods do not address the fundamental issue of information deficiency in unobserved regions.
Another line of methods explores including priors from pre-trained neural networks [12, 46, 51, 34, 67, 22] for regularization. For example, Jain et al. [16] leverage CLIP [36] features to provide semantic guidance. DSNeRF [12] and SparseNeRF [46] use depth regularization from pre-trained depth estimators on known views guide optimization. More recently, FSGS [67] and DNGaussian [22] extend the similar sprit to 3D-GS training. However, these priors do not directly provide visual supervision for sperse-view NVS like the visual diffusion prior.
To leverage visual priors, several lines of work have emerged. Liu et al. [26] use diffusion models to generate -osur--observations at unseen views, while Wu er al. [50] guide diffusion with NeRF representation [58] to synthesize novel views.
Novel View Synthesis with Diffusion Priors.
To reduce the computational burden of fine-tuning diffusion models, Xiong et al. [52] and Wang al. [47] adopt Score Distillation Sampling (SDS) [35] extract external visual priors. However, these approaches rely on image-based diffusion models and thus fail to fully capture spatiotemporal correlations across views. More recently, Liu et al. [25] fine-tuned Stable Video Diffusion [5] to provide view interpolation capability for guiding 3D-GS reconstruction. While this significantly improves performance, it requires substantial computational resources, limiting practical efficiency,
Despite progress in niew-conditioned generative models [24, 48, 38, 62], existing methods are either object-centric [24] struggle generate photorealistic views [38, 62, 48]. Recent approaches [14, 57, 49, 60] enable coarse camera motion control for video generation from a single frame but lack a consistent 3D representation, which compromises cross-view consistency and reproducibility.
Consequently, how to effectively leverage zero-shot video diffusion priors for novel view synthesis is an important open challenge. The concurrent work [65] is closely related to ours, but it depends video diffusion model trained with camera poses [60], and the code was not publicly available at the time of our submission. In contrast, our method can, in principle, be applied to any video diffusion model trained on raw videos, making it more broadly generalizable.
Preliminaries More Details in Appendix
3D Gaussian Splatting (3D-GS) [18] represents 3D scenes explicitly using Gaussian primitives, each defined by mean f E R and covariance Σ E R3x3, G(z) exp - 11) Σ Σー' 1)). Each Gaussian also includes spherical harmonics coefficients c for nie-ddependent color and an opacity a, enabling expressive appearance modeling. Rendering is performed efficiently via rasterization. After projecting Gaussians to the image plane, pixel colors are computed using alpha compositing:
Cpix = E: c; I1 - az),, where e; and a; denote the color and opacity of the i-th Gaussian, respectively. For depth rendering, c; is replaced by the z-buffer value.
Stable Video Diffusion (SVD) [5] is an image-to-vided diffusion model that generates natural video conditioned on an input image. By default, generation starts from the given image and autonomously evolves, incorporating random camera movements and scene dynamics.
3

---

Iniiilization L
T
Veew Solection
3D Gaussian
Gaussian Primitive Densification
Splatting
4
Refinement bg
Iopul Images
CLIP
%
x Iterations: [- 7...
Cetip
Xe-1
Decodar
X
Uncertainty-Aware Modulation
Stable Video Diffusion
*
Guidance imges 77-4 and Uncertairty Masks
Pacudo View Images 2
predittion SVD
Latent
Adaofive
Figure 2: Our approach leverages uncertainty aware diffusion to synthesize pseudo views from sparse inputs and uses them refine 3D Gaussian Splatting. @ We initialize 3D-GS from sparse input images. @ We generate guidance images and estimate their uncertainties based on the current 3D-GS renderings. modulation, enhancing uncertain regions while preserving reliable parts. + The resulting aseudo-view images are then used to densify Gaussian primitives and constrain the 3D-GS training.
These guidance images guide the diffusion process via uncertainty-awant
Given forward diffusion process expressed by dx f(t)xdf g(f)dw, where is the noisy latent state at timestamp t, w denotes the standard Wiener process, and f(t) and gt)) are scalar functions, its reverse process ODE [41] ] can expressed as dx = [](()x - 1 2 Vx log(g: (x))
] dt. In the case of the variance exploding (VE) diffusion [41] adopted by Stable Video Diffusion (SVD) [5], it can be simplified as: dx x-xo do,, where the noise of the diffusion process is parameterized as Gaussian noise with variance of ar and &o is the currently predicted clean video by the network based on the latent state at the previous step. practice, we can obtain the estimated denoised sample Xt-1 at the previous time step by discretizing the diffusion process above:
a:
X1
20
X X4
(0:-1 - 0t).
(1)
4 Our Test-Time Optimization Approach to Novel View Synthesis
We recast sparse-input novel view synthesis as a test-time natural video completion problem. this end, we propose an iterative optimization framework that integrates 3D Gaussian Splatting with video diffusion priors to enforce geometric consistency and enhance visual fidelity.
Given few input views = and their associated camera poses, we propose a zero-shot, generation- guided reconstruction pipeline that synthesizes novel views by leveraging pretrained video diffusion model [5] (Fig. 2). The framework consists of four main steps: 1) 3D-GS initialization from the sparse input views; Guidance feature creation and their uncertainty estimation via cross- cross-view consistency check based on the current 3D-GS; 3) Uncertainty
modulation of the video diffusion model generating pseudo views, conditioned on the guidance images and uncertainty masks; 4) Refinement of the 3D-GS by densifying the Gaussian primitives using the generated pseudo-views. Steps 2)-4) are iteratively performed to progressively improve both the 3D-GS representation and the quality of the diffusion model outputs.

---

4.1 Pseudo View Generation via Uncertainty-Aware Modulation
Most off-the-shelf video diffusion models lack precise camera control due to the scarcity of datasets with known camera poses. To ensure broad applicability, we design our framework to be compatible with widely available models [5, 55] that are conditioned solely on single image. Moreover, our approach is theoretically agnostic to variance-exploding diffusion backbones [41].
Video diffusion models [5] usually extract CLIP [36] features Celip from the input frame Timp to inform the U-Net of the scene's overall appearance and layout. Simultaneously, the frame is encoded by a VAE encoder to produce contextual features Cvac. which are injected via classifier-free guidance to maintain consistency with the reference frame. At each denoising timestep t, the model denoises a latent video representation X E R xgNxxxxX> using U-Net Ue (x; Colip: Crae: f), where N, C, H, W are the number of frames, feature and spatial dimensions of the latent, respectively. The U-Net predicts clean latent Xy from x, to update x, with Eq. (I), which direct x, toward Xg. The final denoised latent, Xo, is decoded by the VAE decoder into a video clip.
Our method draws inspiration from diffusion- image editing techniques (29,59, 1,53], particularly SDEdit (29) for its efficiency. Specifically, we propose to modify the original clean latent prediction &o using the guidance feature g R NCCx H extracted from the guidance images by the VAE encoder. This modification is formulated as an optimization problem applied to each frame i:
Xo [i]
arg min |x - ko(:]1 + >.. ||x g(i]l
where index [i] denotes the i-th frame channel corresponding the i-th frame of the generated video, and Yr.i >> is a weighting term that controls the influence of the guidance feature. Next, we describe how to obtain the feature map g that guides the diffusion model to generate views desired poses, and detail how to control Ye, to achieve adaptive modulation.
Guidance Feature Creation. The core idea of our approach is to exploit video diffusion priors to infer occluded or missing content from sparse input views. This requires constructing guidance features that are geometrically aligned with the desired target view.
To resolve this, instead of using the 3D-GS to render color images, we create guidance images by inversely warping pixels from their nearest input view, using depth maps rendered by 3D-GS. Concretely, to construct the guidance image 1esid for the i-th video frame, we first project each pixel p E /euid into the nearest input view rinp IimP, using the rendered depth map Di camera intrinsics K, and camera poses pimp E SIE(3) (input view) and p E SE(3) (guidance view), to get its corresponding pixel q in the input image:
= (p -1 (p)
We fill pixel p with the color of pixel q to obtain the guidance image 1puid. The set of guidance images is denoted as I = U1 ( N where N is the length of the video clip generated by the video diffuuion model in single pass. The VAE encoder will encode these guidance images to have the corresponding guidance feature maps g to guide the diffusion process via Eq. (2).
Uncertainty Evaluation from Cross-View Consistency. The constructed guidance images well preserve scene content and structure by adhering to strict multi-view geometric constraints imposed by the 3D-GS representation. However, because 3D-GS is imperfect during training, especially in under-observed regions, the guidance images may contain missing content or artifacts. To assess the reliability of guidance images, we introduce a strict cyclic consistency check, as illustrated in Fig. 3a. Specifically, in the forward pass, we project each pixel p in the guidance image to its corresponding pixel q in the nearest input image using Eq. (3). We then perform backward projection from q to the guidance view using the depth map DiPP rendered by 3D-GS from the nearest input view: p' = KP (Ping) Dine (q) The uncertainty at pixel p is then quantified by evaluating both geometric and photometric consistency:
Up(p) exp LIP-P'lE P'I
(p)
1 (q)
81
$2
where 7.th is the 3D-GS rendered image from the view of the i-th guidance image, Tim denotes the nearest input image, and $1, 82 are bandwidth parameters controlling the sensitivity to geometric and
5

---

Forwar Piquctoo Buckword
(Poudd -1 Buuut (p)K-
Kpinp
Noaroll Ihout
Photonaer Laconterg
Comresk Unsersrky
Unontarty
Kpauta
RE D (q)k-)
Nearast Inpul Image
Guidenca image
30-69 Rendering
Difusion Generation
(a) Illustration of Forward/Backward Projection
(b) Example of the Uncertainty Estimates
Figure Cross-view consistency provides a principled uncertainty estimate for guidance images, enabling targeted refinement by video diffusion. (a) We evaluate cross-view consistency via a warddd projection cycle between guidance view and its nearest input view using depths rendered from the current 3D-GS. (b) Regions exhibiting poor cross-view consistency (boxed regions in the rendering) are identified as eigh-uncertainty areas (brighter in uncertainty maps), and subsequently refined by the video diffusion model.
photometric discrepancies. If the 3D-GS well constrained at pixel p and no occlusion is present, the image pixel color 1 (q) (q) should closely match the color of the 3D-GS rendering 1 (p), and the brokeprojected position p' should lie near the original p. This results in low uncertainty. Otherwise,
discrepancies in color or geometry increase the uncertainty, as captured by Eq. (4).
Uncertainty-Aware Modulation, Using the uncertainty map, we define Yt, for each pixel in Eq. (2):
f0
(p(p) 8 or kr
(p)
(L/(U/() +c)
(5)
otherwise
where 5 and are threshold hyperparameters and is small constant avoid division by zero. The threshold 7 is determined by the overall uncertainty of frame i, defined by r = F( (Z (U)()) where F()) can be defined as function (linearly or quadratically) increasing between 0 and (see supplementary materials for details). This ensures that in uncertain regions, the optimization in Eq. (2) leans towards the diffusion prediction [1] while reliable areas are guided by the features from g fi). For simplicity, we p denote corresponding positions both image and latent space. In practice, U; is downsampled via average pooling to match the latent resolution before computing Ye, After computing Tt,,, we apply Eq, (2) to obtain the fused latent [o[i]. which then used Eq, (1) to update to X This reverse sampling step is repeated until the final latent xo is obtained, which then decoded into pseudo-view images via the VAE decoder (see Fig. 2).
Extending to View Interpolation. The above pipeline supports single-view extrapolation but degrades under large viewpoint changes. We extend it to two-view interpolation by defining camera paths between inputs and running diffusion forward and backward, conditioned on each view. At each denoising step, the two latent sequences are merged: X;-1 = 3xxforward + (1 33) R( (((x/e))rr beckward). backward to
where R(-) is the reverse operation along the frame index dimension to align the latent x
forward the frame dimension. B E R is the blending weight, with B[i] = (N - i)V - 1) for
i 1,2,..., N, where N is number of interpolated frames between two inputs. See supplementary material for the detailed algorithm.
4.2 3D-GS Optimization Guided by Generation
To better constrain the 3D-GS representation, we pair adjacent inputs and define camera trajectories that cover under-observed regions (see supplement). Using the video diffusion model guided by Tauid
(Sec. 4.1), we interpolate between input views to generate pseudo-view images T = {1 1j=1 where p the number of input pairs.
po P.N
Gaussian Primitive Densification. Sparse-input 3D-GS often yields poor geometry in undervobserved regions due to limited supervision. We address this by enhancing the geometry using pseudo-view Thee and dense stereo model [48]. For efficiency, we select a subset T C T with low fnter-frame covisibility to maximize coverage with minimal redundancy. These views form camera graph

---

FreeNeRF
Ours
Figure 4: Our generation-guided reconstruction produces more coherent and photorealistic novel views than prior methods. We compare our results on the DL3DV dataset using input views with those from FreeNeRF, vanilla 3D Gaussian Splatting, and DNGaussian. By leveraging video diffusion priors to complete under-observer regions, our method better preserves scene structure and appearance consistency.
used to reconstruct point cloud from stereo predictions. We analyze the spatial distribution of reconstructed points, and filter out outliers that significantly deviate from the global average distance to neighboring points. We query existing Gaussian primitives within radius of each remaining point and only add new Gaussian primitives at positions without nearby primitives to augment the current set. See appendix for more details.
3D Gaussian Splatting Optimization. After densifying the Gaussian primitive set, we optimize the 3D-GS model using both the original inputs and the generated pseudo-views In each training iteration, one input view and one pscudo-view are sampled for supervision. For the original input views, apply an LI loss and D-SSIM loss, well as depth regularization term Crg with Pearson correlation similar to [47]: L. wi Ci (1 1"P) WyCD-ssIM (1 _DP)) + wy where 1 is the rendered image from 3D-GS and ris denotes the corresponding input image. For the generated pscudo views, we observe that, despite the carefully designed guidance mechanism, some regions still suffer from temporal inconsistency particulaly distant areas with weak geometry those with fine-grained textures, e.g., grass or tree leaves. To mitigate the negative impact of such inconsistencies on 3D-GS training, use the LPIPS [64] instead of L1 loss. The resulting loss is:
bg w4 LIPIIS (185, (185 )+ wa SD. (88, IPSe) + we crefe
5 Experiments
5.1 Experiment Settings
Datasets and Metrics. Weevalue our method on LLFF [30], DL3DV [23], DTU[17], and MipNeRF. 360 [3] datasets. LLFF consists of 8 forward-facing scenes. Following standard practice [54, 22], we train our model using only 3 input views on this dataset. DL33V comprises diverse indoor and outdoor scenes captured by humans walking through scenes, exhibiting complex and dynamic camera motions. The Mip-NeR2 360 datasct consists of real-world indoor and outdoor scenes designed for evaluating novel view synthesis in large, unbounded environments. To verify the generalizability of our methods and compare with the previous methods, we also test our methods on DTU, an object-centric dataset captured in controlled conditions. For the DTU dataset, follow the protocol from RegNeRF [22], using? 3 training views across 15 evaluation scenes. To focus on the object of interest, mask out the background during evaluation using the provided object masks, consistent with [54, 22]. We apply downsampling factor of 8 for LLFF and 4 for DTU, aligning with prior work. The rendering quality is assessed using PSNR, SSIM, and LPIPS metries.
Implementation details. Our pipeline designed to operate iteratively. In each cycle, we train the 3D-GS model for 10K iterations, followed by update of the pseudo-view images using the video
7

---

FreeNeRF
3D-GS
Ours
GT
FreeNeRF
3D-GS
Ours GT
Figure 5: Our method more faithfully recovers scene geometry and fine-grained appearance under sparse inputs. We show comparisons on the LLFF (left) and DTU (right) datasets. Diffusion- generated pseudo views densify supervision in under-observed areas (boxed), enabling better preservation of structural integrity and figh-frequency details.
Table 1:
Our method consistently outperforms prior approaches under sparse-view settings
across multiple datasets. We show comparisons on LLFF, DTU, and DL3DV. We compare our method with standard reconstruction baselines (vanilla NeRF PRRF/DD-GS) sparse- wwww--iiewwiieeee NeRF methods, and sierse-view-ppoee optimized 3D-GS methods. We color each cell as best second best
and third best.
(3 Viewy)
DTU Views)
DL./W Viewy)
(N Views)
D/AM (9 Views)
PSNx]
SS131
LPIPS.
PSNKI
SSIMI
LPIPS.
PSNK
SSIM
LPIFS.
PSNKS
SSUSF
LFIPSL
FSNKE
SSIM
LPIPST
Mip NERE
16.11
(1401
0.504)
8.55
01.571
013355
10:32
0.191
07618
11.56
01.97
12.42
0.218
(1A00
30-68
17.43
0.522
0.321
10.99
01.585
0.315
10:97
0.248
0.567
12.34
0.392
0.598
12.99
0.403
0.546
DieNNRR
1494
0.370
0.896
11:85
0.833
0.314
RogNeRF
19.08
0.587
0.316
18.89
0.745
0.190
11:46
0.214
0.600
12.89
0.266
0.579
12:33
0.210
0.598
FrreNeRF
19.33
0.612
0.306
1999
0.787
0.182
10.91
0.211
0.995
12.13
0.230
0.576
1285
0.241
0.573
SpaneNeRF
19.86
0.624
0.328
19.55
0,769
0,201
SpunsciS
15.39
0,534
01176
FSCIS
20.31
0652
0.268
12.22
0.296
11535
13.73
0.429
0.540
15.52
0,468
0.16
DNGussiia
19.12
0.59]
0.254
18.91
0.790
01176
11:30
0.213
0,579
12.67
0.329
0.547
13.44
0.365
0.539
IPSM
20:44
0.702
0.207
11.70
0.279
0,534
12.82
0.332
0.521
13.41
0.361
0.529
Ours
30.61
0.705
0.300
2051
0.840
0133
14.03
0.471
0.491
17.35
0.565
0.395
19119
0.616
0.335
diffusion model. After each pseudo- view update, we reset the learning rate schedule of 3D-GS before starting the next optimization cycle to avoid overfitting. For the uncertainty estimation in Eq. (4), we the bandwidth parameters 8 100 and 82 0.25. The ö in Eq. (5) is fixed 0.5 across all experiments. The loss weights are configured as follows:
0.8, W2 0.2, Wy = 1.0, 1.0, Ws 0.2, and Wg 1.0. Additional implementation details are provided in supplementary materials.
5.2 Comparison with Other Methods
We compare method against the state-of-thes the-hrt- four benchmark datasets to demonstrate its effectiveness and generalizability across diverse scenarios.
Table Our method outperforms recent diffusion-based feed-forward approaches. Results are reported on MipNeRF-360 using 9 input views. Our approach surpasses feed- forward approaches (e.g. MVSplat 360)
Comparison on LLFF. We evaluate our method on the LLFF dataset captured by swaying face-forward cam- era. Table shows that our method consistently outper- forms NeRF-based approaches across all evaluation met- rics. When compared to 3D-Gaussian Splatting-bar based baselines such as FSGS [67] and DNGaussian [22], our method remains competitive, particularly LPIPS and SSIM scores. This improvement is largely attributed to the additional supervisory signal provided by the pseudo views generated through the video diffusion model. Notably, the LPIPS metric, which correlates more closely with human perceptual similarity than traditional metrics like PSNR, highlights our method's 's ability to produce visually realistic novel views. Qualitative comparisons presented in Fig. 5.
without expensive diffusion finetuning.
MipNeRFF360
PSNRT
SSIM
LPIPS_
RegNeRF [33]
13.73
0.193
0.629
FreeNeRI [54]
13.20
0.198
0.635
DNGaussian [22]
12.51
0.228
0.683
MVSplat 360
14,86
0.321
0.328
ViewCrafter [60]
16.68
0.382
0.551
3DGS-Enhancer [25]
16.22
0.399
0.454
Ours
17.91
0.495
0.435
Comparison on DTU. To further assess the generalizability of our approach, we evaluate and compare its performance on the DTU dataset. DTU is an object-centric dataset in which each scene

---

Table Ablation experiments on the DL3DV test set. (a) Experiments to show the effectiveness of the proposed components in pseudo-view generation step. (b) Experiments to show the effectiveness of the proposed strategies for 3D-GS optimization.
(a) Pscudo-view generation
PSNR
SSIM LPIPS]
3D-GS optimization
PSNRT
SSIM
LPIPS
Baseline 3D-GS
16.59
0.502
0.405
w/o point filtering
19.01
0.615
0.343
w/ GS interpolation
18.59
0,591
0.369
w/o GS densification
18.23
0.567
0.386
w/ warping interpolation (full)
19.19
0.616
0.335
w/o LPIPS loss
18.81
0.597
0.351
w/o geometric
18.21
0,583
0.378
Full model
19.19
0.616
0.335
w/o photometric
18.93
0.612
0.352
MvSplat 360
Ours
GT
Figure 6: Our test-time optimization better preserves visual and geometric consistency than the feed-forward approach, MVSplat360. 9 views taken the input. While forwad methods can produce plausible novel views, they often struggle to maintain fidelity to the original scene, whereas our method achieves higher consistency.
contains a centered object against monotone background. The evaluation results are presented in Table 1 (middle). In this setting, our method still performs well and outperforms other NeRF-based and 3D-GS-based methods. Specifically, our method outperforms the recond-best approach by significant margin in terms of PSNR, SSIM, and LPIPS. While NeRF-based methods also exhibit competitive accuracy in this scenario. they suffer from slow rendering speeds (approximately 0.21 FPS), whereas our 3D-GS based approach supports real-time rendering at around 430 FPS.
Comparison on DL.DV. We compare with other cutting-edge counterparts on the DL3DV dataset under 3, 6, and 9 view settings. Table (right) shows the quantitative comparison results. Apart from the sparse-input 3D-GS methods, we also compare with the non-sparse view methods and NeRF-based methods Table (right). We outperform previous state-of-the-att methods [22, 67, 47] by a significant margin in this challenging setting. We observe that although DNGaussian [22] works well environments with limited scope with limited camera motions, e.g., object-centric scenarios, it has difficulties in reliably reconstructing the open environment due to the lack of constraints in under-observed regions (qualitative results shown in Fig. 4). Similarly, FSGS [67] also struggles in this challenging setting, though it achieves slightly better performance compared with DNGaussian because it uses a sparse point cloud for initialization. The recent work IPSM [47] uses an image diffusion model to constrain the 3D-GS by enhancing Score Distillation Sampling (SDS). As shown in Table (right), this method struggles with extremely sparse inputs. This limitation arises because the image diffusion model lacks access to a global scene context, whereas the video diffusion model able to infer such context from the input reference frame.
Comparison on Mip.NRRF-360. To evaluate our method on unbounded scenes and ensure fair comparison with recent ferd-frwwrrd approaches [9, 60, 25], we further conduct experiments on the Mip-NeRF 360 dataset [3]. As shown in Table 2, our method consistently outperforms reconstruction- based methods [33, 54, 22] and surpasses state-of-the-an feed-forver ceddffrwwrd approaches [9, 60, 25] by notable margin. As shown in Fig. 6, although feed-forward methods can hallucinate novel views from sparse inputs through large-scale data training, they often struggle to maintain geometric consistency, fine details, and color fidelity compared to our approach.
5.3 Ablation Study
To validate the effectiveness of our proposed components in the pseudo-view generation (Sec. 4.1) and the 3D-GS optimization (Sec. 4.2), conduct extensive ablation study on DL3DV.

---

Baseline
b) wi pscuua view superitsion
c) wi pisuuoo view supeniision GS primbtve densificttion
d) Ground Truth
Figure 7: The proposed pseudo view supervision and primitive densification effectively enhance novel view synthesis. We show two scenes with under-observed regions highlighted in boxes, com- paring results with and without these components. Both strategies improve geometric reconstruction and visual fidelity under-observed regions.
Generation Oupud (el (nceemeenn
Figure 8: Uncertainty-awar modulation is critical for reliable pseudo-view generation. We compare diffusion outputs with and without uncertainty modulation, especially in unreliable regions (boxed). Uncertainty modeling effectively prevents artifacts.
Effectiveness of uncertainty-awarr modulation mechanism. Table 3a compares the baseline 3D-GS trained on sparse views using L. with two variants: one using 3D-GS renderings as guidance ("w/ GS interpolation") and one using our warping-basee guidance ("w/ warping interpolation"). While GS interpolation improves over the baseline, underperforms compared our method due to inaccurate color rendering at novel poses during training.
Fig. shows the effect of uncertainty- modulation by comparing diffusion results with and without it, using identical guidance images. We further ablate the geometric and photometric terms in the uncertainty formulation (Eq. (4)), denoted as "w/o geometric" and "w/o photometric." As shown in Table 3a, removing either term noticeably degrades performance.
Effectiveness of Gaussian primitive densification. We ablate the densification step ("w/o GS densification" in Table 3b), observing a significant performance drop, highlighting its role in improving synthesis quality. Fig. 7 shows that densification enhances reconstruction in under-observed regions. Removing the point filtering step ("w/o point filtering") also degrades performance due to depth outliers from the stereo model.
Effectiveness of LPIPS for pseudo view supervision. We replace LPIPS with LI loss ("w/o LPIPS loss") in Eq. (6), observing notable performance drop (Table 3b). Despite our guidance strategy, cross-view inconsistencies especially in distant or textured regions remain challenging. L1 loss used in vanilla 3D-GS [18] is less robust to such inconsistencies in crffusionngeneratee pseudo views.
Conclusion and Limitations
We present a zero-shot, generation- guidedd pipeline that leverages a pretrained vidco diffusion model to improve 3D-GS reconstruction from sparse inputs. The method synthesizes intermediate views guided by warped depth-based images and uncertainty-awarc modulation, while a densification module further improves scene completeness. Our approach enhances photorealism and coverage in sparse settings while preserving the real-time efficiency of 3D-GS.
Despite these gains, the framework has limitations. Its performance depends on the quality of the pretrained video diffusion model, which may introduce artifacts under extreme viewpoints or in complex scenes. The iterative training procedure adds overhead relative to vanilla 3D-GS pipelines, and early-stage 3D-GS depth errors can affect the guidance despite uncertainty modeling, although this impact typically diminishes over time.
10

---

Acknowledgments. This project was supported, in part, by NSF 2215542, NSF 2313151, and Bosch gift funds to S. Yu at UC Berkeley and the University of Michigan, with additional compute support provided by the NAIRR Pilot under CIS240421.
References
[1] Arpit Bansal, Hong-Min Chu, Avi Schwarzschild, Soumyadip Sengupla, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Universal guidance for diffusion models. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 843 852, 2023.
[2] Jonathan T Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brudd and Pratul P Srinivasan. Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields supplemental material.
[3] Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul Srinivasan, and Peter Hedmma. Mip-nerf' 360: Unbounded anti-aliased neural radiance fields. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5470-5479, 2022.
[4] Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Zip-nerf: Anti-aliased grid- grid-based neural radiance fields. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 19697-19705, 2023.
[5] Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Sealing latent video diffusion models to large datasets. arXiv preprint arX1 2023.
[6] Andreas Blattmann, Rohin Rombach, Huan Ling, Tim Dackhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 22563-22575. 2023.
[7] Chen, Liu, Lianghua Huang, Bin Wang, and Pan Pan. Geoaug: Data augmentation for few-shot nerf with geometry constraints. In European Conference on Computer Vision, pages 322-337 Springer, 2022.
[8] Shenchang Eric Chen Lance Williams. View interpolation for image synthesis. Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH '93, page 279- 288. Association for Computing Machinery, 1993.
[9] Yuedong Chen, Chuanxia Zheng. Haofei Xu, Bohan Zhuang, Andrea Vedaldi, Tat-Jen Cham, and Jianfei Cai. Mvsplat360: Feed-forward 360 scene synthesis from sparse views. 2024.
[10] Jacyoung Chung, Jeongtack Oh, and Kyoung Mu Lee. Depth-regularizee optimization for 3d gaussian splatting in few-shot images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 81 111-88 1-000, 2024.
[11] Paul E. Debevec, Camillo J. Taylor, and Jitendra Malik. Modeling and rendering architecture from photographs: A hybrid geometry- and amage-bascl, approach. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 1996, New Orleans, LA, USA, August 4-9, 1996, pages 11-20, ACM, 1996.
[12] Kangle Deng, Andrew Liu, Jun-Yan Zhu, and Deva Ramanan. Depth-supervised nerf: Fewer views and faster training for free. In Proceedings of the IEEE/CVF conference on computer vision and pattern recogmition, pages 12882-12891, 2022.
[13] Antoine Guédon and Vincent Lepetit. Sugar: Surface-aligned gaussian splatting for efficient 3d mesh reconstruction and high-quality mesh rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5354-5363. 2024.
[14] Chen Hou, Guoqiang Wei, Yan Zeng, and Zhibo Chen. Training-free camera control for video generation. arXiv preprint arXiv:2406. 10126, 2024.
[15] Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao. 2d gaussian splatting for geometrically accurate radiance fields. In ACM SIGGRAPH 2024 conference papers, pages 1-11. 2024.
[16] Ajay Jain, Matthew Tancik, and Pieter Abbeel. Putting nerf on a diet: Semantically consistent few-shot view synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5885-5894, 2021.
11

---

[17] Rasmus Jensen, Anders Dahl, George Vogiatzis. Engin Tola, and Henrik Aanæs. Large scale multi-view stereopsis evaluation. In Proceedings of the IEEE conference computer vision and patlern recognition. pages 406-413, 2014.
[18] Bernhard Kerbl, Georgios Kopanas, Thomas Leimküäler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):139-1, 2023.
[19] Weijie Kong. Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large vidco generative models, arXiv preprint 241.:22322.0003, 2024.
[20] Min-Seop Kwak, Jiuhn Song, and Seungryong Kim. Geconerf: Few-shot neural radiance fields via geometric consistency, arXiv preprini arXiv:2301.09441, 2023.
[21] Mare Levoy and Pat Hanrahan. Light field rendering. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH '96, page 31-42. Association for Computing Machinery, 1996.
[22] Jiahe Li, Jiawei Zhang, Xiao Bai, Jin Zheng, Xin Ning. Jun Zhou, and Lin Gu. Dngaussian: Optimizing sparse-view 3d gaussian radiance fields with global-local depth normalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20775-20785, 2024.
[23] Ling, Yichen Sheng. Zhi Tu. Wentian Zhao, Cheng Xin, Kun Wan, Lantao Yu, Qianyu Guo, Zixun Yu. Yawen Lu, et al. 103dv-10k: A Jarge-scale scene dataset for deep learning based 3d vision. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 22160-22169 22669, 2024.
[24] Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey Zakharov, and Carl Vondrick. Zero-I-to-2 Zero-shot one image to 3d object. In Proceedings of the IEEE/CVF international conference computer vision, pages 9298 9309, 2023.
[25] Xi Liu, Chaoyi Zhou, and Siyu Huang, 3dgs-enhancer: Enhancing unbounded 3d gaussian splatting with view-consistent 2d diffusion priors. In The Thirty-eighth Annual Conference on Neural Information Processing Systems.
[26] Xinhang Liu, Jiaben Chen, Shiu- Hong Kao, Yu- Ku-Wing Tai, and Chi-Keung Tang. Deceptive- Ieffusion-generaati pseudd-observations for high-quality sparse-view reconstruction. In European Conference on Computer Vision, pages 337-355. Springer, 2024.
[27] Fan Lu, Yan Xu, Guang Chen, Hongsheng Li, Kwan-Yee Lin, and Changjun Jiang, Urban radiance field representation with deformable neural mesh primitives. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 465 066,476,222 2023.
[28] Tao Lu, Mulin Yu, I. Linning Xu, Yuanbo Xiangli, Limin Wang, Dahua Lin, and Bo Dai. Scaffold-gs: Structured 3d gaussians for view-adaptive rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20654-20664, 2024.
[29] Chenlin Meng, Yutong He, Yang Song. Jiaming Song. Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. In International Conference on Learning Representations.
[30] Ben Mildenhall, Pratul P Srinivasan, Rodrigo Cayon, Nima Khademi Kalantari, Ravi Ramamoorthi, Ren Ng. and Abhishck Kar. Local light field fusion: Practical view synthesis with prescriptive sampling guidelines. ACM Transactions Graphics (ToG), 38(4):1-14, 2019.
[31] Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes neural radiance fields for view synthesis. Communications of the ACM, 65(1):99-106, 2021.
[32] Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with a multiresolution hash encoding. ACM transactions on graphics (TOG), 41(4):1-15, 2022.
[33] Michael Niemeyer, Jonathan T Barron, Ben Mildenhall, Mehdi SM Sajjadi, Andreas Geiger, and Noha Radwan. Regnerf: Regularizing neural radiance fields for view synthesis from sparse inputs. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5480-5490, 2022.
[34] Avinash Paliwal, Wei Ye, Jinhui Xiong, Dmytro Kotovenko, Rakesh Ranjan, Vikas Chandra, and Nima Khademi Kalantari. Coherentgs: Sparse novel view synthesis with coherent 3d gaussians. In European Conference on Computer Vision, pages 19-37. Springer, 2025.
12

---

[35] Ben Poole, Ajay Jain, Jonathan T. Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. arXivv 2022.
[36] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. International conference on machine learning, pages 8748-8788 144888763. PmLR, 2021.
[37] Scunghyeon Sco, Yeonjin Chang, and Nojun Kwak. Flipnerf: Flipped reflection rays for few-shot novel view synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 22883-22893, 2023.
[38] Brandon Smart, Chuanxia Zheng, Iro Laina, and Vietor Adrian Prisacariu. Splatt3r: Zero-shot gaussian splatting from uncalibrated image pairs. 2024.
[39] Nagabhushan Somraj, Adithyan Karanayil, and Rajiv Soundararajan. Simplenert: Regularizing sparse input neural radiance fields with simpler solutions. In SIGGRAPH Asia 2023 Conference Papers, pages 1-11. 2023.
[40] Nagabhushan Somraj and Rajiv Soundararajan, ViP-NeRF: Visibility prior for sparse input neural radiance fields. August 2023.
[41] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Represemtations.
[42] Justus Thies, Michael Zollhöfer, and Matthias NieBner. Deferred neural rendering: Image synthesis using neural textures. Acm Transactions on Graphics (TOG), 1-12, 2019.
[43] Prune Truong, Marie-Julic Rakotosaona, Fabian Manhardt, and Federico Tombari. Sparf: Neural radiance fields from sparse and noisy poses. In Proceedings of the IEEF/CVF Conference on Computer Vision and Puttern Recognition, pages 4190-4200, 2023.
[44] Richard Tucker and Noah Snavely. Single-view view synthesis with multiplane images. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
[45] Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, Jiayu Wang, Jingfeng Zhang, Jingren Zhou, Jinkai Wang. Jixuan Chen, Kai Zhu, Kang Zhao, Keyu Yan, Lianghua Huang, Mengyang Feng, Ningyi Zhang, Pandeng Li, Pingyu Wu, Ruihang Chu, Ruili Feng. Shiwei Zhang, Siyang Sun. Tao Fang. Tianxing Wang. Tianyi Gui. Tingyu Weng. Tong Shen, Wei Lin, Wei Wang, Wei Wang. Wenmeng Zhou, Wente Wang. Wenting Shen, Wenyuan Yu, Xianzhong Shi, Xiaoming Huang, Xin Xu, Yan Kou, Yangyu Lv, Yifei Li, Yijing Liu, Yiming Wang. Yingya Zhang, Yitong Huang, Yong Li, You Wu, Yu Liu, Yulin Pan, Yun Zheng, Yuntao Hong, Yupeng Shi, Yutong Feng. Zeyinzi Jiang, Zhen Han, Zhi- Fan Wu, and Ziyu Liu. Wan: Open and advanced large-scale video generative models. arXiv preprint Xiv:2503.2033111 2025.
[46] Guangcong Wang, Zhaoxi Chen, Chen Change Loy, and Ziwei Liu. Sparsenerf: Distilling depth ranking few-shot novel view synthesis. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9065-9076, 2023.
[47] Qisen Wang, Yifan Zhao, Jiawei Ma, and Jia Li. How to use diffusion priors under sparse views? Advances in Neural Information Processing Systems, 37:30394-30424 2025.
[48] Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud, Dust3r: Geometric 3d vision made easy. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 20697-20709, 2024.
[49] Zhouxia Wang, Ziyang Yuan, Xintao Wang, Yaowei Li, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. Motionetri: A unified and flexible motion controller for vidco generation. In ACM SIGGRAPH 2024 Conference Papers, pages 1-11, 2024.
[50] Rundi Wu, Ben Mildenhall. Philipp Henzler, Keunhong Park. Ruiqi Gao, Daniel Watson, Pratul P Srinivasan, Dor Verbin, Jouathan T Barron, Ben Poole, et al. Reconfusion: reconstruction with diffusion priors. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 21551-21561, 2024.
[51] Jamie Wynn and Daniyar TTuruuhhambeto.. Diffusionerf: Regularizing neural radiance fields with denoising diffusion models, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4180 4189, 2023.
13

---

[52] Haolin Xiong, Sairisheek Muttukuru, Rishi Upadhyay, Pradyumna Chari, and Achuta Kadambi. Sparsegs: Real-time 360 ((deg] sparse view synthesis using gaussian splatting. arXiv preprint arXiv:211..00206, 2023.
[53] Sihan Xu, Yidong Huang. Jiayi Pan. Ziqiao Ma. and Joyce Chai. Inversion-free image editing with language-guided diffusiin models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 9452-944 9461, 2024.
[54] Jiawei Yang, Marco Pavone, Yue Wang. Freenerf: Improving few-shot neural rendering with free frequency regularization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8254 8263, 2023.
[55] Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06072, 2024.
[56] Ruihong Yin, Vladimir Yugay. Yue Li, Sezer Karaoglu, and Theo Gevers. Fewviewgs: Gaussian splatting with few view matching and multi-stage training. arXiv preprint 1.X0x:2411.002233 2024.
[57] Meng You, Zhiyu Zhu, Hui Liu, and Junhui Hou. Nvs-solver: Video diffusion model as zero-shot novel view synthesizer. arXiv preprint arXiv:240. 15364, 2024.
[58] Alex Yu, Vickie Ye, Matthew Tancik, and Angjoo Kanazawa, pixelnerf: Neural radiance fields from one or few images. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 4578 4587, 2021.
[59] Jiwen Yu. Yinhuai Wang, Chen Zhao, Bernard Ghanem, and Jian Zhang. Freedom: Training free energy- guided conditional diffusion model. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 23174-23184 84, 2023.
[60] Wangbo Yu, Jinbo Xing, Li Yuan, Wenbo Hu, Xiaoyu Li, Zhipeng Huang, Xiangjun Gao, Tien-Tsin Wong, Ying Shan, and Yonghong Tian. Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis. arXiv preprint arXiv:2409. 02048, 2024.
[61] Zehao Yu, Anpei Chen, Binbin Huang, Torsten Sattler, and Andreas Geiger. Mip-splatting: Alias-fre free 3d gaussian splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19447-19456, 2024.
[62] Junyi Zhang, Charles Herrmann, Junhwa Hur, Varun Jampani, Trevor Darrell, Forrester Cole, Deqing Sun, and Ming-Hsuan Yang. Monstár: A simple approach for estimating geometry in the presence of motion. arXiv preprint arxiv24100008825, 2024.
[63] Kai Zhang, Gernot Riegler, Noah Snavely, and Vladien Koltun. Nerf++: Analyzing and improving neural radiance fields. arXiv preprint arX14:2010.07492, 2020.
[64] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as perceptual metric. In CVPR, 2018
[65] Yingji Zhong, Zhihao Li, Dave Zhenyu Chen, Lanqing Hong, and Dan Xu. Taming video diffusion prior with scene-grounding guidance for 3d gaussian splatting from sparse inputs. arXiv preprint 253302055, 2025.
[66] Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, and Noah Snavely. Stereo magnification: Learning view synthesis using multiplane images. arXiv preprint 1rXiv:1805.098/1 2018.
[67] Zehao Zhu, Zhiwen Fan, Yifan Jiang, and Zhangyang Wang. Fegs: Real-time few-shot view synthesis using gaussian splatting. European conference on computer vision, pages 145-163. Springer, 2025.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction accurately define the scope and assumptions of the problem the paper addresses, and the main claims match the proposed method described in the Method Sec. 4 as well results presented in Experiment Sec. 5.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer:
[Yes]
Justification:
We discussed the limitations our work in Sec. ??.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or a speech-to-tex. system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
15

---

Answer: [NA]
Justification:
We acknowledge that we do not propose any novel theoretical results that need
proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
The details the proposed method are thoroughly discussed in Sec. and the implementation details are included in Sec. 5.1 and supplementary material. Furthermore, we will release the code upon acceptance.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data provided or not.
the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
be accomplished in various ways. For example, if the contribution a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
Depending on the contribution, reproducibility
While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on nature the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (c.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but should be possible for other researchers have some path reproducing verifying the results.
16

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The paper will provide open access to the data and code with instructions to reproduce all experimental results, in the camera ready version upon acceptance.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cu
jublic/guides/Cdeee http:/oooeesumiiisooooooooe..e ssionPolicy) for details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https://nips https///iipscccpullliee
for more details. The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
gguides/CodeSubmii
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The training and evaluation details, data selection, and hyperparameters disclosed in Sec. 5.1 for readers to understand and reproduce the experimental results.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: The experimental results in the paper are not accompanied by any statistical significance tests, as prior work in this area typically only reports aggregate results without such analysis. To ensure comparability and clarity, we follow the reporting conventions in the related literature.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
17

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (c.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [Yes]
Justification: We disclose the the computation resources and computation time we use in supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer:
[Yes]
Justification: We have reviewed the NeurIPS Code of Ethics. The subject of this paper, the proposed method and experiments do not have ethical concerns.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consideration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer:
[Yes]
Justification:
We discussed the societal impact at the end of our paper.
Guidelines:
The answer NA means that there is socictal impact of the work performed.
18

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
We believe our paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
The data and model backbones we used are cited in the paper. For the code we adopted, we conform with their licenses of use and also properly give credits to them in the corresponding script headers.
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
19

---

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi thcode. wwt.commaasssee datasess has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do not release new assets in the work.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/codeemoodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question:
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification:
Our work does involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
20

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval. you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMSs if an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: Our paper does not involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy for what should or should not be described.
21


Reviews:
[{"review_id": "IkVJZS4umA", "reviewer": "Reviewer_sLmb", "summary": "This paper addresses the challenging problem of novel view synthesis from sparse input images, specifically within the 3D Gaussian Splatting (3DGS) framework. The authors reframe this task as a video completion problem, proposing a novel generation-guided pipeline that leverages a pretrained video diffusion model to fill in the \"gaps\" between sparse views. The core of the method is an iterative process. It begins with an initial 3DGS model. This model is used to generate depth maps, which in turn are used to create \"guidance images\" by warping pixels from the nearest real input views. Crucially, the paper introduces an uncertainty-aware modulation mechanism; it evaluates the reliability of these guidance images via a cross-view consistency check and uses this uncertainty to steer the video diffusion model. The diffusion model is tasked with generating plausible content primarily in high-uncertainty (e.g., occluded or under-observed) regions while preserving reliable content from the guidance images. These generated \"pseudo-views\" are then used as additional supervision to refine and densify the 3DGS representation. This cycle is repeated to progressively improve the final 3D scene. The authors demonstrate great performance on several challenging datasets, most notably DL3DV.", "questions": "1. Are there any failure cases related to the weaknesses discussed in point 1 (Critical reliance on SVD)? Specifically, what happens if the SVD generates undesirable hallucinations that are both appearance- and geometry-inconsistent? If so, are there any proposed solutions to address this issue?\n\n2. Can this proposed method outperform other generative-based methods with similar technical approaches? Such as Viewcrafter [1] (feed-back loop between generated views and 3D representation) and 3DGS-enhancer [2] (frames interpolation by video diffusion).\n\n[1] Yu, Wangbo, et al. \"Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis.\" Open Source Project 2024.\n\n[2] Liu, Xi, Chaoyi Zhou, and Siyu Huang. \"3dgs-enhancer: Enhancing unbounded 3d gaussian splatting with view-consistent 2d diffusion priors.\" NeurIPS 2024.\n\nJustification of the rating: \nI think overall, this paper presents a technically novel approach with good performance, with clear writing and organization. However, as a generative-based method, comparisons with other generative-based baselines are necessary. Therefore, my rating is borderline accept because of limited evaluation.", "limitations": "N/A", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths:\n\n1. The paper is well-written and easy to follow, the writing quality is great.\n\n2. Zero-shot is great. While prior works have used diffusion models, they have largely focused on image-based priors or required fine-tuning. This work's use of a zero-shot, pre-trained video diffusion model is a significant and novel contribution, as it inherently models the spatio-temporal consistency between views. This approach provides a new and effective direction for tackling the ill-posed nature of sparse-view reconstruction.\n\n3. The technical execution of the core idea is excellent. The proposed pipeline is well-designed and sophisticated. The \"uncertainty-aware modulation\" is a particularly clever mechanism. Instead of naively using generated views, it provides a principled way to fuse geometric information from the current 3D-GS reconstruction with the powerful generative prior of the diffusion model.\n\nWeaknesses:\n\n1. (major weakness) Missing comparisons with other generative-based methods: The paper only compares against non-generative sparse-view NeRF or 3DGS approaches, but does not include any baselines that also leverage generative priors. Such comparisons are important for a fair evaluation.\n\n2. (minor weakness) Critical reliance on SVD: The proposed method heavily relies on the capabilities of the underlying video diffusion model (SVD), as the authors acknowledge. As a result, it may struggle with out-of-distribution scenes, where the generative prior can hallucinate plausible but inaccurate details (appearance- and geometry-inconsistent contents), potentially corrupting the resulting 3DGS.", "was_revised": true, "final_justification": "I think overall, this paper presents a technically novel approach with good performance, with clear writing and organization. However, as a generative-based method, comparisons with other generative-based baselines are necessary. Therefore, my intial rating is borderline accept because of limited evaluation. As the authors have resolved my concerns on evaluation and showcased the effectiveness of the proposed method in their rebuttal, I would like to increase my rating to accept.", "rebuttal": "# Reviewer sLmb\n\n**Q1:** \"(major weakness) Missing comparisons with other generative-based methods: The paper only compares against non-generative sparse-view NeRF or 3DGS approaches, but does not include any baselines that also leverage generative priors. Such comparisons are important for a fair evaluation.\" and \"Can this proposed method outperform other generative-based methods with similar technical approaches? Such as Viewcrafter [1] (feed-back loop between generated views and 3D representation) and 3DGS-enhancer [2] (frames interpolation by video diffusion).\"\n\n**A1:** We thank the reviewer for highlighting the importance of comparing against generative-based methods.\nIn our original submission, we focused on zero-shot NeRF and 3D-GS baselines, which, like our approach, do not require computation-intensive training or fine-tuning of the diffusion models. This setup was chosen to emphasize practical deployment scenarios where compute resources and data collection are limited.\n\nTo directly address the reviewer’s concern, we have now included additional comparisons with ViewCrafter [1] and 3DGS-Enhancer [2] on the DL3DV and MipNeRF-360 datasets. These results are provided below. Note that the training set of the released ViewCrafter model includes our test set on DL3DV, and we present the results here for reference only.  \n\n*Table R1-1: Comparing our zero-shot (training-free) method with the recent generative-based methods (that need expensive GPU hours for diffusion training) on the **DL3DV** dataset. Our method achieves comparable or even better performance.* \n\n|  |  | 3 Views |  |  | 6 Views |  |  | 9 Views |  |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |\n|  | PSNR↑ | SSIM↑ | LPIPS↓ | PSNR↑ | SSIM↑ | LPIPS↓ | PSNR↑ | SSIM↑ | LPIPS↓ |\n| ViewCrafter*  | 13.45 | 0.402 | 0.473 | 16.22 | 0.421 | 0.435 | 18.23 | 0.553 | 0.348 |\n| 3DGSEnhancer | 14.33  | 0.424 | **0.464** | 16.94 | 0.565 | **0.356**  | 18.50 | 0.630 | **0.305** |\n| Ours | **14.62** | **0.471**  | 0.491 | **17.35** | **0.566** | 0.396 | **19.16**  | **0.631** | 0.341 |\n\n\n*Table R1-2 Comparison with baselines on **MipNeRF-360** datasets. We follow the convention on DL3DV  to select 9 sparse views for each scene.*\n\n|  | PSNR↑ | SSIM↑ | LPIPS↓ |\n| --- | --- | --- | --- |\n| ViewCrafter | 15.68 | 0.382  | 0.551 |\n| 3DGSEnhancer | 16.22 | 0.399 | 0.454 |\n| Ours | **17.60** | **0.469** | **0.405** |\n\nThe results show that:\n\n- Our method achieves comparable or superior performance to both ViewCrafter and 3DGS-Enhancer, despite being significantly more lightweight and requiring no additional training of the diffusion model.\n- While 3DGS-Enhancer (trained on DL3DV) performs slightly better on DL3DV in some metrics, we observe notable generalization issues when tested on the unseen MipNeRF-360 dataset, where our zero-shot method consistently outperforms it. A similar degradation has also been observed for ViewCrafter.\n\n**Q2** \"(minor weakness) Critical reliance on SVD: The proposed method heavily relies on the capabilities of the underlying video diffusion model (SVD), as the authors acknowledge. As a result, it may struggle with out-of-distribution scenes, where the generative prior can hallucinate plausible but inaccurate details (appearance- and geometry-inconsistent contents), potentially corrupting the resulting 3DGS.\" and \"Are there any failure cases related to the weaknesses discussed in point 1 (Critical reliance on SVD)? Specifically, what happens if the SVD generates undesirable hallucinations that are both appearance-and geometry-inconsistent? If so, are there any proposed solutions to address this issue?”\n\n**A2:** Thank you for your insightful comments. We agree that our method relies on the generative capability of the underlying video diffusion model (SVD), and appreciate the opportunity to clarify how we address potential failure cases and hallucinations. \n\nIn our experiments, we primarily observed two types of inconsistencies:\n\n1. Moderate temporal inconsistency in distant regions with weak geometric support or fine-grained textures (e.g., grass, tree leaves), where the diffusion model may generate visually plausible but slightly inconsistent content.\n2. Occasional generation collapse in individual frames, particularly in early iterations when the 3D-GS guidance is still inaccurate or the input views are extremely sparse.\n\nTo mitigate **Type 1**, as described in Lines 236–238, we replace L1 loss with LPIPS loss during 3D-GS supervision using pseudo-views. This encourages perceptual-level alignment and helps reduce the impact of subtle inconsistencies. The effectiveness of this strategy is demonstrated in Table 3(b) in our paper.\n\nFor **Type 2**, while we acknowledge that hallucinations can occasionally appear, we find that they do not significantly affect final 3DGS convergence. These artifacts are typically isolated, non-persistent, and often suppressed by the global optimization process of 3D-GS, which relies on multi-view consistency.\n\nThat said, we recognize that such hallucinations can degrade reconstruction quality. We consider these unreliable generations as transient distractors, and future work can incorporate recent advances such as “SpotLessSplats: Ignoring Distractors in 3D Gaussian Splatting”, which explicitly addresses the suppression of distractors in 3D Gaussian Splatting. Integrating such filtering mechanisms could further improve the robustness of our pipeline.\n\nFinally, while the capabilities of our method are inherently bounded by the current performance of SVD, we are optimistic that diffusion models will continue to improve significantly with scale. Our pipeline is designed to be modular and compatible with future generative backbones, making it well-positioned to benefit from the ongoing progress in video generation."}, {"review_id": "lA5qQJkvR1", "reviewer": "Reviewer_axbY", "summary": "This paper presents a pipeline for solving sparse-view novel view synthesis (NVS) by leveraging a pretrained stable video diffusion model. Given a guidance image and an uncertainty mask, the model generates pseudo-views, which are then used to iteratively refine a 3D Gaussian Splatting (3DGS) reconstruction, leading to high-quality final results.", "questions": "Please refer to the points in the weakness.", "limitations": "Yes", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n1. Directly using a pretrained diffusion model without finetuning is insightful in terms of computational resources.\n\n**Weaknesses:**\n1. The MipNeRF-360 dataset, which is one of the most challenging benchmarks for outdoor unbounded NVS, is not evaluated in this paper.\n2. Strong baselines that also utilize video diffusion priors, such as **3DGS-Enhancer** [1] and **MVSplat360** [2], are missing from the comparison.\n3. The motivation for using a pretrained video diffusion model is questionable. Since SVD includes generative priors from dynamic and synthetic content, this may conflict with the goal of optimizing a static 3D representation. Although finetuning requires significant computational resources, it can substantially improve performance.\n4. Previous works [1][2] highlight the degradation caused by the VAE component. Without aligning the latent space between the rendered pseudo-views and the ground truth images, quality issues such as color shifts and distortions can arise. This paper does not address this critical point.\n5. Based on both image and video visualizations, noticeable inconsistencies, artifacts, and color shifts are observed. While directly using SVD without finetuning significantly reduces computation, it also inevitably sacrifices performance, particularly because enhancements such as those discussed in points 3 and 4 cannot be incorporated.\n\n**References:**\n[1] *3DGS-Enhancer: Enhancing Unbounded 3D Gaussian Splatting with View-consistent 2D Diffusion Priors*  \n[2] *MVSplat360: Feed-Forward 360 Scene Synthesis from Sparse Views*", "was_revised": true, "final_justification": "I am impressed by the strong performance compared to baseline methods. Rather than following the popular data-driven trend, the proposed approach demonstrates that pretrained models can be highly effective when used with careful design and insight.", "rebuttal": "**Q1+Q2:** \"The MipNeRF-360 dataset, which is one of the most challenging benchmarks for outdoor unbounded NVS, is not evaluated in this paper.\" and \"Strong baselines that also utilize video diffusion priors, such as 3DGS-Enhancer [1] and MVSplat360 [2], are missing from the comparison.\" \n\n**A1+A2:** Thank you for the helpful suggestions. We have extended our experiments to include comparisons with 3DGS-Enhancer and MVSplat360, and provided additional benchmarking results on the challenging MipNeRF-360 dataset.  These results demonstrate that our method exhibits **strong generalization ability** across diverse and unseen scenarios.\n\nAs a lightweight **zero-shot** approach, our method **significantly outperforms** both **3DGS-Enhancer** and **MVSplat360** on the MipNeRF-360 benchmark, despite requiring **no diffusion model training or task-specific finetuning**. On the DL3DV dataset, where **3DGS-Enhancer** was specifically trained, our method still achieves **comparable or even superior performance**.\n\nThe detailed quantitative comparisons are provided below.\n\n(1) **Comparison with 3DGS-Enhancer and DNGaussian (a representative method on sparse-input 3DGS) on DL3DV**\n\n*Table R2-1* shows that our method achieves **comparable or even superior performance** to 3DGS-Enhancer, despite operating in a **zero-shot** manner. In contrast, 3DGS-Enhancer requires **GPU-intensive training** of the diffusion model specifically on the DL3DV dataset.\n\nNote that we do not include **MVSplat360** for fairness, as its training set includes our test set on DL3DV.\n\n*Table R2-1: Results on the DL3DV dataset.* We hold every eighth image as our test split, and\nevenly sample 9 sparse views from the remaining images for training the 3D-GS. (Please refer to the reviewer sLmb's section for more results.)\n\n|  | PSNR↑ | SSIM↑ | LPIPS↓ |\n| --- | --- | --- | --- |\n| DNGaussian | 13.44  | 0.365 |  0.539 |\n| 3DGSEnhancer | 18.50 | 0.630 | 0.305 |\n| Ours | 19.16 | 0.631 | 0.341 |\n\n(2) **Comparisons on MipNeRF-360 Dataset.** \n\nWe further evaluate our method on the challenging MipNeRF-360 dataset, comparing it against three competitive baselines. \n\n*Table R2-2 Results on MipNeRF-360. We hold every eighth image as our test split, and evenly sample 9 sparse views from the remaining for 3D-GS optimization.* \n\n|  | PSNR↑ | SSIM↑ | LPIPS↓ |\n| --- | --- | --- | --- |\n| DNGaussian | 12.51  | 0.228  | 0.683 |\n| 3DGS-Enhancer | 16.22 | 0.399 | 0.454 |\n| MVSplat 360 | 14.86  | 0.321 | 0.528 |\n| Ours | **17.60** | **0.469** | **0.405** |\n\n*Table R2-2* shows that our method **outperforms all baselines across all evaluation metrics**, demonstrating strong generalization and robustness on unseen outdoor scenes. We found that although 3DGS-Enhancer (specifically trained on **DL3DV** dataset) performs comparably to ours on DL3DV (see Table R2-1),  it exhibits significant **performance degradation** on MipNeRF-360, suggesting limited cross-dataset generalization. Similarly, **MVSplat360 suffers on this dataset, where its outputs exhibit notable color shifts and semantic inconsistencies.** We will include qualitative comparisons in the final version of the paper to illustrate these issues.\n\nAdditionally, the feed-forward method MVSplat360 is constrained by computational and memory overhead, and its publicly released model is limited to generating 480p images. In contrast, our method maintains high-resolution synthesis capability while requiring no additional diffusion model training or task-specific finetuning, which is more scalable and efficient for real-world deployment.\n\nWe will update the main paper and supplementary material to include this evaluation and comparative analysis.\n\n**Q3:** \"The motivation for using a pretrained video diffusion model is questionable. Since SVD includes generative priors from dynamic and synthetic content, this may conflict with the goal of optimizing a static 3D representation. Though finetuning requires significant computational resources, it can substantially improve performance.\"\n\n**A3:** Thank you for the insightful comment. While it is true that SVD's training data includes dynamic content, we find its inductive biases generalize well to static camera transitions, especially when guided by our structural constraints.\n\n1. SVD captures strong spatiotemporal priors that promote coherent view transitions and occlusion-aware reasoning, likely due to its text-to-image pretraining.\n2. Our **uncertainty-aware modulation** enforces geometric and photometric consistency, reducing hallucinations and improving alignment, as demonstrated in Table 3(a).\n\nThe supplementary video (2:10–2:20) also shows that the generated views maintain static scene properties, with minimal negative impact on 3D-GS results.\n\nIn summary, while fine-tuning could improve task-specific alignment, our results show that, **with proper guidance**, a pretrained SVD model can already serve as a **practical and effective prior** for static 3D-GS—supporting **lightweight, zero-shot deployment** in resource-constrained settings.\n\n**Q4:** \"Previous works [1,2] highlight the degradation caused by the VAE component. Without aligning the latent space between the rendered pseudo-views and the ground truth images, quality issues such as color shifts and distortions can arise. This paper does not address this critical point.”\n\n**A4:** We thank the reviewer for raising this important point. We fully agree that VAE-induced degradation is a known issue, and we have made a concerted effort to address it through a set of lightweight but effective mechanisms—designed specifically to preserve our zero-shot setting without retraining the diffusion model. While these strategies are already part of our current method, we appreciate the opportunity to better highlight their role in the paper and will revise the manuscript to clarify this contribution.\n\nWhile prior works such as 3DGS-Enhancer [1] and MVSplat360 [2] address this through **explicit training** of the diffusion model to better align the latent space, our work **intentionally avoids diffusion model retraining** to preserve our **zero-shot design**. Instead, we propose specific mechanisms to **robustify the 3D-GS optimization under these non-ideal generative conditions and alleviate color drifts and distortion via the underlying 3D-GS representation:** \n\n- **Perceptual Loss for Robust Supervision:** We replace the standard L1 loss with the **LPIPS loss** in Eq. (6), which encourages perceptual similarity rather than strict pixel-wise alignment. This makes the 3D-GS training **less sensitive to moderate color shifts or spatial inconsistencies** in the generated pseudo views. The effectiveness is confirmed by our ablation in Table 3(b) and the qualitative improvements in the supplementary video.\n- **Feedback via Uncertainty-Aware Modulation:**  Our framework incorporates an **uncertainty-aware modulation mechanism**, which uses the intermediate 3D-GS representation to guide the diffusion model during pseudo-view synthesis. Since the 3D-GS maintains a coherent, geometry-aware representation, this feedback helps suppress inconsistencies and stabilizes the appearance across views.\n- **Latent Space Scaling:** Empirically, we observed that **scaling the conditional latent variables** from the pseudo views (i.e., VAE latents) **helps mitigate color saturation and drift**, likely by regularizing the decoder’s sampling behavior.  While not the core focus of our technical contribution, this adjustment improved the visual consistency and will be more clearly described in the final version.\n\nThe 3D-GS renderings (1:17–2:03) in the supplementary video show limited color drifts and distortions due to our mechanisms.\n\nIn summary, while we do not directly modify the diffusion model, our method introduces **lightweight yet effective solutions** for mitigating VAE-induced degradation, all while maintaining the advantages of **zero-shot inference and generalizability**.\n\n**Q5:** \"Based on both image and video visualizations, noticeable inconsistencies, artifacts, and color shifts are observed. While directly using SVD without fine-tuning **significantly reduces computation**, it also inevitably sacrifices performance, particularly because enhancements such as those discussed in points 3 and 4 cannot be incorporated.\"\n\n**A5:** We appreciate the reviewer’s observation and agree that pseudo-views generated by SVD may exhibit minor artifacts such as color shifts or local inconsistencies, especially in wide-baseline or occluded regions. However, our approach is deliberately designed to address this challenge through lightweight, principled mechanisms that preserve the zero-shot nature of the pipeline—without resorting to costly diffusion model fine-tuning.\n\nOur framework prioritizes generalization, efficiency, and deployment practicality. While some prior methods mitigate such artifacts through scene-specific fine-tuning (e.g., 3DGS-Enhancer [1]), they incur significant computational overhead and require paired supervision or GPU access—making them less viable for real-world applications.\n\nIn contrast:\n\n- **In resource-constrained or edge environments**, retraining large generative models is infeasible. Our inference-only pipeline allows for fast, GPU-friendly execution.\n- **In field scenarios** such as surveying, architecture, or cultural preservation, our method supports quick adaptation to novel scenes without retraining or manual intervention.\n- **Compared to training-heavy approaches** like MVSplat360, which face generalization limitations and are restricted to low-resolution outputs (e.g., 480p), our method supports high-resolution, cross-dataset reconstructions (see A1/A2).\n\nThis trade-off is intentional: our method avoids retraining while still delivering high-resolution 3D reconstructions with reliable consistency and quality across diverse scenes. We will revise the paper to better highlight this design philosophy and its practical benefits."}, {"review_id": "urp2MNKyYT", "reviewer": "Reviewer_xfoR", "summary": "- This paper reframes sparse-input novel view synthesis as a natural video completion task and introduces a zero-shot, generation-guided reconstruction pipeline that integrates pretrained video diffusion models with 3D Gaussian Splatting (3D-GS). \n\n- The method leverages uncertainty-aware modulation to generate pseudo views at intermediate camera poses, which are used to densify and refine the 3D representation. Iterative refinement and primitive densification significantly improve rendering quality, particularly in under-observed regions. \n\n- Experiments on DL3DV, LLFF, and DTU datasets demonstrate strong performance improvements over baseline 3D-GS methods.", "questions": "Mentioned in the weaknesses", "limitations": "Mentioned in the weaknesses", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n- The reinterpretation of novel view synthesis as a video frame completion task is conceptually elegant and allows the use of powerful pretrained video diffusion models. The use of best/second-best/third-best rankings in quantitative evaluation adds credibility and fairness to the reported improvements. (Table 1, Tabel 2)\n\n- Supplementary videos show strong qualitative performance in scene reconstruction, particularly in maintaining structural coherence and reducing rendering artifacts. (video.mp4 01:16-02:21)\n\n- The proposed uncertainty-aware modulation and primitive densification modules demonstrate a well-designed synergy between generative priors and 3D representation learning. (Page 2 - Introduction)\n\n**Weaknesses**\n\n**Major**\n- Despite improvements, the method still shows limitations in rendering fine-grained details, especially in texture-rich regions. For example, like artifacts in the video. (like 01:37 in the supplementary video, GT-Ours)\n\n- The paper claims a significant reduction in rendering artifacts, but visual results—while improved—still show occasional imperfections, which should be acknowledged more explicitly. (like 01:42 / 02:01 in the supplementary video, GT-Ours)\n\n**Minor**\n- While the provided supplementary videos are promising, the number of qualitative examples (Both paper and supplementary video showing 5 examples) looks insufficient to fully validate generalizability. This is non-critical, but adding more diverse scenes would strengthen the experimental evidence.", "was_revised": true, "final_justification": "Thank you for your detailed rebuttal! I will maintain my current rating!", "rebuttal": "# Reviewer xfoR\n\n**Q1:** \"Despite **improvements**, the method still shows limitations in rendering fine-grained details, especially in texture-rich regions. For example, like artifacts in the video. (like 01:37 in the supplementary video, GT-Ours)\"\n\n**A1:** We agree that some loss of fine-grained detail—such as the artifact at 01:37 in the supplementary video (GT vs. Ours)—can occur in texture-rich or underconstrained regions. This is a known challenge in sparse-input, wide-baseline settings, where details like dense foliage or fine textures are not fully supported by geometry alone. Similar limitations have been reported in prior works such as DNGaussian and 3DGS-Enhancer.\n\nMost importantly, our method operates in a **fully zero-shot** setting, using only a few posed input images and a **frozen** diffusion model, without any task- or scene-specific fine-tuning. Despite this, it achieves globally consistent geometry and appearance across most views, as demonstrated throughout the supplementary video.\n\nTo mitigate detail loss, we introduce two key mechanisms:\n- Uncertainty-aware modulation (Sec. 4.1), which stabilizes pseudo-view generation, and\n- Perceptual LPIPS loss (Eq. 6), which improves supervision signal beyond pixel alignment.\n\nThese design choices lead to measurable and visible improvements in reconstruction quality, as shown in Table 3 and the qualitative comparisons.\n\nThat said, we recognize that further improvement in high-frequency texture recovery remains a valuable direction. Future extensions could incorporate semantic priors, generalizable refinement networks, or lightweight super-resolution modules to better capture fine details under limited-view conditions.\n\n**Q2:** \"The paper claims a significant reduction in rendering artifacts, but visual results—while improved—still show **occasional** **imperfections**, which should be acknowledged more explicitly. (like 01:42 / 02:01 in the supplementary video, GT-Ours)\" \n\n**A2:** We thank the reviewer for highlighting these examples. We agree that occasional imperfections—such as those at 01:42 /  02:01 in the supplementary video—can appear, particularly in extreme viewpoints or complex scenes. These limitations stem from the generative bounds of the frozen video diffusion model and the inherent under-constraint in sparse-view inputs.\n\nThat said, our method operates under **strict zero-shot** conditions, using **no task- or scene-specific fine-tuning**, which distinguishes it from most prior approaches. Despite these constraints, our pipeline achieves **strong qualitative and quantitative performance**, with consistent geometry and appearance across views.\n\nTo address these challenges, we introduce two targeted strategies:\n- **Uncertainty-aware modulation**, which stabilizes pseudo-view generation, and\n- **Perceptual supervision via LPIPS**, which improves robustness to generative artifacts.\n\nThese components contribute significantly to performance improvements, as shown in our ablation: PSNR improves from 16.59 (baseline 3D-GS) to 19.16 with our full method (Table 3).\n\nWe will revise the manuscript to explicitly acknowledge these remaining imperfections and position them as valuable opportunities for future enhancement—e.g., through lightweight refinement or adaptive guidance methods—while emphasizing the strength of our results given the zero-shot setting.\n\n**Q3:** \"While the provided supplementary videos are promising, the number of qualitative examples (Both paper and supplementary video showing 5 examples) looks insufficient to fully validate generalizability. This is non-critical, but adding more diverse scenes would strengthen the experimental evidence.\" \n\n**A3:** We thank the reviewer for the helpful suggestion. Due to submission file size constraints, we focused the supplementary video and paper on a concise set of representative scenes -- **8 scenes in DL3DV, 3 scenes in DTU**, and **3 scenes in LLFF** -- chosen to reflect diverse geometry, texture, and baseline conditions.\n\nImportantly, our **quantitative evaluation is conducted over the full test sets of each dataset**, ensuring statistically meaningful assessment of generalizability. The qualitative examples are intended as illustrative highlights rather than comprehensive coverage.\n\nWe will expand the qualitative results in the final version of the paper and supplementary materials to include a broader range of scenes and viewpoints, further supporting the generalization claims."}, {"review_id": "bnLOBoWKdD", "reviewer": "Reviewer_pPRG", "summary": "The paper presents a zero-shot, generation-guided reconstruction framework that leverages pretrained video diffusion models to enhance 3D Gaussian Splatting (3D-GS) for sparse-input novel view synthesis. It reinterprets the task as natural video completion, using diffusion to generate pseudo views that provide dense supervision for 3D-GS training. An uncertainty-aware modulation mechanism and Gaussian primitive densification module are introduced to improve spatial coherence and scene completeness. Experiments on DL3DV, LLFF, and DTU show performance gains over baselines, but concerns remain regarding novelty, baseline comparisons, and visualization details.", "questions": "Please see the weaknesses.", "limitations": "As described in the weaknesses, the major limitation of this paper is missing comparisons with the relevant ideas. From my point of view, a detailed comparison with these methods could further clarify the true contributions of this paper.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1. The uncertainty-aware modulation mechanism effectively guides diffusion for consistent view synthesis.\n\n2. The iterative refinement between 3D-GS and diffusion models enhances scene reconstruction in under-observed regions.\n\nWeaknesses:\n\n1. Lack of novelty: Zero-shot, generation-guided 3D reconstruction has been previously explored in works like ReconX [1], ViewCrafter [2], Wonderland [3], See3D [4], VideoScene [5], and Difix3D+ [6]. The proposed framework does not sufficiently distinguish itself from these prior art methods.\n\n2. More baseline comparisons: The paper claims state-of-the-art performance but omits recent advances such as those referenced in [1]-[6]. Comparing against the latest methods is essential to validate superiority. Additionally, since the approach relies on view interpolation, comparisons with two-view reconstruction methods (e.g., pixelSplat [7], MVSplat [8], NoPosSplat [9]) are suggested to be included into experiment.\n\n3. Missing input condition visualization: Visualization experiments in the main paper lack explicit input sparse views. Including visualization of the sparse input images would better illustrate the method’s interpolation capabilities.\n\n[1] ReconX: Reconstruct Any Scene from Sparse Views with Video Diffusion Model\n\n[2] ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis\n\n[3] Wonderland: Navigating 3D Scenes from a Single Image\n\n[4] You See it, You Got it: Learning 3D Creation on Pose-Free Videos at Scale\n\n[5] VideoScene: Distilling Video Diffusion Model to Generate 3D Scenes in One Step\n\n[6] Difix3D+: Improving 3D Reconstructions with Single-Step Diffusion Models\n\n[7] pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction\n\n[8] MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images\n\n[9] No Pose, No Problem: Surprisingly Simple 3D Gaussian Splats from Sparse Unposed Images", "was_revised": true, "final_justification": "The rebuttal well addressed my concerns on the comparisons with recent works. The authors are encouraged to demonstrate this clearly in the final version. I will raise my final score.", "rebuttal": "# Reviewer pPRG\n\n**Q1:** \"Lack of novelty: Zero-shot, generation-guided 3D reconstruction has been previously explored in works like ReconX [1], ViewCrafter [2], Wonderland [3], See3D [4], VideoScene [5], and Difix3D+ [6]. The proposed framework does not sufficiently distinguish itself from these prior art methods.\"  \n\n**A1:** We thank the reviewer for highlighting recent works such as ReconX [1], ViewCrafter [2], and other concurrent CVPR 2025 papers. \n\nGiven that many of these methods involve 3D reconstruction and diffusion models, we understand the potential for confusion. However, our framework is *fundamentally distinct in two key ways*:\n1. It is truly zero-shot — frozen SVD only, no model parameters trained at any stage, and\n2. It is truly test-time — leveraging a pretrained video diffusion model as is without any task-specific training for novel view synthesis.\n\nIn our manuscript, \"zero-shot\" refers specifically to using a frozen video diffusion model—trained for generic video generation—as a prior to guide 3D Gaussian Splatting (3D-GS) optimization for novel view synthesis. We do so without any fine-tuning or additional training, making our pipeline lightweight, modular, and broadly deployable.\n\nWhile the cited methods also leverage diffusion models, most of them **train those models explicitly for 3D or novel view synthesis**, using domain-specific datasets and large-scale compute. For example, See3D trains on **114 × A100 GPUs for ~25 days**. These models **are not plug-and-play**: they require **substantial pretraining tailored to the task**, and typically operate only within their trained data domain. In contrast, our method uses **off-the-shelf, frozen diffusion models** with **no retraining or adaptation**, enabling broader generalization and ease of use in real-world, low-resource scenarios.\n\nFurthermore, our focus is not on training a diffusion model to regress 3D scenes or render views directly, but on **enhancing 3D-GS reconstruction quality under sparse-view constraints**, while **retaining its real-time rendering advantages** (~400 FPS). Our pipeline also introduces a key technical innovation— uncertainty-aware modulation —that steers the frozen diffusion model to synthesize pseudo-views between sparse inputs, which are then used to supervise 3D-GS in under-observed regions.\n\nThis modulation is part of an **iterative feedback loop** that jointly refines both the 3D representation and synthesized views, yielding significant gains in fidelity and consistency (see Fig. S-3). We are not aware of any prior work that combines frozen diffusion priors with 3D Gaussian Splatting in this manner.\n\nWhile approaches like ReconX or ViewCrafter could hypothetically be paired with 3D-GS, they lack **test-time modulation, iterative supervision, and zero-shot adaptability**—all of which we find to be critical in sparse-input scenarios.\n\nWe will revise the related work section to better clarify these distinctions and ensure our contributions are clearly stated in the final version.\n\n**Q2:** \"More baseline comparisons: The paper claims state-of-the-art performance but omits recent advances such as those referenced in [1]-[6]. Comparing against the latest methods is essential to validate superiority. Additionally, since the approach relies on view interpolation, comparisons with two-view reconstruction methods (e.g., pixelSplat [7], MVSplat [8], NoPosSplat [9]) are suggested to be included into the experiment.\" \n\n**A2:**  Thank you for this valuable suggestion. We are excited to share new experimental results added during the rebuttal, which demonstrate that our method consistently outperforms recent training-based baselines on both seen and unseen datasets— **despite operating in a zero-shot setting with no fine-tuning or task-specific training**.\n\nWhile our primary focus is on enhancing 3D Gaussian Splatting (3D-GS) under sparse-input settings with zero-shot diffusion priors, rather than directly generating scenes via feed-forward diffusion (as clarified in response **A1**), we agree that additional comparisons can further contextualize our contributions.  \n\nWe conducted further experiments against representative feed-forward baselines, including **ReconX, ViewCrafter, and MVSplat360**. Due to time and resource constraints, we focused on methods with public access or reported benchmarks.   ReconX and ViewCrafter as two typical video-generation works for comparison, whereas MVSplat 360 (MVSplat360: Feed-Forward 360 Scene Synthesis from Sparse Views) is a representative feed-forward 3DGS work, one of the latest works in this domain, and it beats pixelSplat [7] and MVSplat [8].  Other concurrent CVPR2025 papers either have not released their code or have adopted different evaluation criteria from ours such that we are unable to fairly reproduce or compare against them due to time constraints and limited accessibility during the rebuttal phase.  We will try to include further comparisons in our final version. \n\n *Table R4-1: Comparing our zero-shot method with the recent generative-based methods on the  DL3DV dataset.* \n\n|  |  | 3 Views |  |  | 6 Views |  |  | 9 Views |  |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |\n|  | PSNR↑ | SSIM↑ | LPIPS↓ | PSNR↑ | SSIM↑ | LPIPS↓ | PSNR↑ | SSIM↑ | LPIPS↓ |\n| ReconX | **14.73** | 0.411 | **0.453** | 17.23 | 0.466 | 0.429  | 18.63 |  0.591  | 0.377 |\n| ViewCrafter | 13.45 | 0.402 | 0.473 | 16.22 | 0.421 | 0.435 | 18.23 | 0.553 | 0.348 |\n| Ours | 14.62 | **0.471** | 0.491 | **17.35** | **0.566** | **0.396** | **19.16** | **0.631** | **0.341** |\n\nDespite relying on **frozen diffusion models and no training**, our method delivers comparable or stronger performance than methods specifically trained on DL3DV.   ReconX results are taken from their paper; ViewCrafter and ours were evaluated directly.\n\n*Note: As ReconX has not released the code and model, we report their performance on the DL3DV dataset based on their paper. Besides, as the training data of MVSplat-360 includes our test set on DL3DV, and retraining their model exceeds our available computational resources, we did not test its performance on DL3DV, and we chose to use the MipNeRF-360 dataset for fair benchmarking (discussed below).* \n\n*Table R4-2 Generalization comparison with baselines on MipNeRF-360 datasets. We follow our DL3DV convention to select 9 sparse views for each scene.* \n\n|  | PSNR↑ | SSIM↑ | LPIPS↓ |\n| --- | --- | --- | --- |\n| MVSplat 360 | 14.86 | 0.321 | 0.528 |\n| ViewCrafter | 15.68 | 0.382 | 0.551 |\n| Ours | **17.60** | **0.469** | **0.405** |\n\nOn this unseen benchmark—unseen by all methods—our framework generalizes significantly better. Training-heavy baselines show color drift and content inconsistencies, while our method remains robust under sparse-view input.\n\nKey observations:\n\n- **Generalization gap**: ViewCrafter performs well on DL3DV (its training domain) but degrades on MipNeRF-360. Our zero-shot pipeline maintains strong performance across both.\n\n- **Resolution limitations**: MVSplat360 (480p) and ViewCrafter (320×512) are constrained by memory usage. In contrast, our pipeline produces high-resolution outputs (576×1024), consistent with the SVD backbone.\n\n- **Practicality**: Training-heavy methods like See3D require **~25 days on 114×A100 GPUs**, and often **take minutes to render a single view**—limiting their usability in real-time or interactive settings. Our method supports **real-time rendering (~400 FPS)**, making it much more suitable for practical deployment.\n\nIn summary, while feed-forward baselines show impressive progress, our method offers a compelling alternative: it leverages frozen generative priors and efficient 3D representations to enable high-quality, real-time novel view synthesis without retraining. We hope this inspires further integration between generative models and structured 3D representations.\n\nWe will update the paper to include these new results and clarify the distinctions in methodology, efficiency, and deployment.\n\n**Q3:** \"Missing input condition visualization: Visualization experiments in the main paper lack explicit input sparse views. Including visualization of the sparse input images would better illustrate the method’s interpolation capabilities.\"\n\n**A3:** Thank you for pointing this out.  Showing the sparse input views alongside the synthesized outputs will help better illustrate the interpolation behavior of our method. In the revised version, we will include the original input images in the main qualitative figures to improve clarity and support visual interpretation."}]

Author rebuttal:

Meta-review: This work makes a valuable contribution to sparse-view novel view synthesis by demonstrating that carefully designed zero-shot approaches can compete with training-intensive methods while offering superior generalization and deployment advantages. The technical innovation, experimental validation, and practical significance justify acceptance. The consistent positive reviewer feedback, successful resolution of all major concerns, and great experimental results across diverse datasets demonstrate the work's quality and impact potential.
Decision: Accept (poster)