Title: Novel View Synthesis from A Few Glimpses via Test-Time Natural Video Completion
Venue: NeurIPS 2025
Accepted: yes
Keywords: Novel View Synthesis

3D Gaussian Splatting
Abstract: Given just a few glimpses of a scene, can you imagine the movie playing out as the camera glides through it? That’s the lens we take on sparse-input novel view synthesis, not only as filling spatial gaps between widely spaced views, but also as completing a natural video unfolding through space. We recast the task as test-time natural video completion, using powerful priors from pretrained video diffusion models to hallucinate plausible in-between views. Our zero-shot, generation-guided framework produces pseudo views at novel camera
poses, modulated by an uncertainty-aware mechanism for spatial coherence. These synthesized frames densify supervision for 3D Gaussian Splatting (3D-GS) for scene reconstruction, especially in under-observed regions. An iterative feedback loop lets 3D geometry and 2D view synthesis inform each other, improving both the scene reconstruction and the generated views. The result is coherent, high-fidelity renderings from sparse inputs without any scene-specific training or fine-tuning. On LLFF, DTU, DL3DV, and MipNeRF-360, our method significantly outperforms strong 3D-GS baselines under extreme sparsity. Our project page is at https://decayale.github.io/project/SV2CGS.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a zero-shot, generation-guided reconstruction framework that leverages pretrained video diffusion models to enhance 3D Gaussian Splatting (3D-GS) for sparse-input novel view synthesis. It reinterprets the task as natural video completion, using diffusion to generate pseudo views that provide dense supervision for 3D-GS training. An uncertainty-aware modulation mechanism and Gaussian primitive densification module are introduced to improve spatial coherence and scene completeness. Experiments on DL3DV, LLFF, and DTU show performance gains over baselines, but concerns remain regarding novelty, baseline comparisons, and visualization details.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The uncertainty-aware modulation mechanism effectively guides diffusion for consistent view synthesis.\n\n2. The iterative refinement between 3D-GS and diffusion models enhances scene reconstruction in under-observed regions.\n\nWeaknesses:\n\n1. Lack of novelty: Zero-shot, generation-guided 3D reconstruction has been previously explored in works like ReconX [1], ViewCrafter [2], Wonderland [3], See3D [4], VideoScene [5], and Difix3D+ [6]. The proposed framework does not sufficiently distinguish itself from these prior art methods.\n\n2. More baseline comparisons: The paper claims state-of-the-art performance but omits recent advances such as those referenced in [1]-[6]. Comparing against the latest methods is essential to validate superiority. Additionally, since the approach relies on view interpolation, comparisons with two-view reconstruction methods (e.g., pixelSplat [7], MVSplat [8], NoPosSplat [9]) are suggested to be included into experiment.\n\n3. Missing input condition visualization: Visualization experiments in the main paper lack explicit input sparse views. Including visualization of the sparse input images would better illustrate the method’s interpolation capabilities.\n\n[1] ReconX: Reconstruct Any Scene from Sparse Views with Video Diffusion Model\n\n[2] ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis\n\n[3] Wonderland: Navigating 3D Scenes from a Single Image\n\n[4] You See it, You Got it: Learning 3D Creation on Pose-Free Videos at Scale\n\n[5] VideoScene: Distilling Video Diffusion Model to Generate 3D Scenes in One Step\n\n[6] Difix3D+: Improving 3D Reconstructions with Single-Step Diffusion Models\n\n[7] pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction\n\n[8] MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images\n\n[9] No Pose, No Problem: Surprisingly Simple 3D Gaussian Splats from Sparse Unposed Images\n\nQUESTIONS:\nPlease see the weaknesses.\n\nLIMITATIONS:\nAs described in the weaknesses, the major limitation of this paper is missing comparisons with the relevant ideas. From my point of view, a detailed comparison with these methods could further clarify the true contributions of this paper.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\n- This paper reframes sparse-input novel view synthesis as a natural video completion task and introduces a zero-shot, generation-guided reconstruction pipeline that integrates pretrained video diffusion models with 3D Gaussian Splatting (3D-GS). \n\n- The method leverages uncertainty-aware modulation to generate pseudo views at intermediate camera poses, which are used to densify and refine the 3D representation. Iterative refinement and primitive densification significantly improve rendering quality, particularly in under-observed regions. \n\n- Experiments on DL3DV, LLFF, and DTU datasets demonstrate strong performance improvements over baseline 3D-GS methods.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n- The reinterpretation of novel view synthesis as a video frame completion task is conceptually elegant and allows the use of powerful pretrained video diffusion models. The use of best/second-best/third-best rankings in quantitative evaluation adds credibility and fairness to the reported improvements. (Table 1, Tabel 2)\n\n- Supplementary videos show strong qualitative performance in scene reconstruction, particularly in maintaining structural coherence and reducing rendering artifacts. (video.mp4 01:16-02:21)\n\n- The proposed uncertainty-aware modulation and primitive densification modules demonstrate a well-designed synergy between generative priors and 3D representation learning. (Page 2 - Introduction)\n\n**Weaknesses**\n\n**Major**\n- Despite improvements, the method still shows limitations in rendering fine-grained details, especially in texture-rich regions. For example, like artifacts in the video. (like 01:37 in the supplementary video, GT-Ours)\n\n- The paper claims a significant reduction in rendering artifacts, but visual results—while improved—still show occasional imperfections, which should be acknowledged more explicitly. (like 01:42 / 02:01 in the supplementary video, GT-Ours)\n\n**Minor**\n- While the provided supplementary videos are promising, the number of qualitative examples (Both paper and supplementary video showing 5 examples) looks insufficient to fully validate generalizability. This is non-critical, but adding more diverse scenes would strengthen the experimental evidence.\n\nQUESTIONS:\nMentioned in the weaknesses\n\nLIMITATIONS:\nMentioned in the weaknesses", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a pipeline for solving sparse-view novel view synthesis (NVS) by leveraging a pretrained stable video diffusion model. Given a guidance image and an uncertainty mask, the model generates pseudo-views, which are then used to iteratively refine a 3D Gaussian Splatting (3DGS) reconstruction, leading to high-quality final results.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n1. Directly using a pretrained diffusion model without finetuning is insightful in terms of computational resources.\n\n**Weaknesses:**\n1. The MipNeRF-360 dataset, which is one of the most challenging benchmarks for outdoor unbounded NVS, is not evaluated in this paper.\n2. Strong baselines that also utilize video diffusion priors, such as **3DGS-Enhancer** [1] and **MVSplat360** [2], are missing from the comparison.\n3. The motivation for using a pretrained video diffusion model is questionable. Since SVD includes generative priors from dynamic and synthetic content, this may conflict with the goal of optimizing a static 3D representation. Although finetuning requires significant computational resources, it can substantially improve performance.\n4. Previous works [1][2] highlight the degradation caused by the VAE component. Without aligning the latent space between the rendered pseudo-views and the ground truth images, quality issues such as color shifts and distortions can arise. This paper does not address this critical point.\n5. Based on both image and video visualizations, noticeable inconsistencies, artifacts, and color shifts are observed. While directly using SVD without finetuning significantly reduces computation, it also inevitably sacrifices performance, particularly because enhancements such as those discussed in points 3 and 4 cannot be incorporated.\n\n**References:**\n[1] *3DGS-Enhancer: Enhancing Unbounded 3D Gaussian Splatting with View-consistent 2D Diffusion Priors*  \n[2] *MVSplat360: Feed-Forward 360 Scene Synthesis from Sparse Views*\n\nQUESTIONS:\nPlease refer to the points in the weakness.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the challenging problem of novel view synthesis from sparse input images, specifically within the 3D Gaussian Splatting (3DGS) framework. The authors reframe this task as a video completion problem, proposing a novel generation-guided pipeline that leverages a pretrained video diffusion model to fill in the \"gaps\" between sparse views. The core of the method is an iterative process. It begins with an initial 3DGS model. This model is used to generate depth maps, which in turn are used to create \"guidance images\" by warping pixels from the nearest real input views. Crucially, the paper introduces an uncertainty-aware modulation mechanism; it evaluates the reliability of these guidance images via a cross-view consistency check and uses this uncertainty to steer the video diffusion model. The diffusion model is tasked with generating plausible content primarily in high-uncertainty (e.g., occluded or under-observed) regions while preserving reliable content from the guidance images. These generated \"pseudo-views\" are then used as additional supervision to refine and densify the 3DGS representation. This cycle is repeated to progressively improve the final 3D scene. The authors demonstrate great performance on several challenging datasets, most notably DL3DV.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The paper is well-written and easy to follow, the writing quality is great.\n\n2. Zero-shot is great. While prior works have used diffusion models, they have largely focused on image-based priors or required fine-tuning. This work's use of a zero-shot, pre-trained video diffusion model is a significant and novel contribution, as it inherently models the spatio-temporal consistency between views. This approach provides a new and effective direction for tackling the ill-posed nature of sparse-view reconstruction.\n\n3. The technical execution of the core idea is excellent. The proposed pipeline is well-designed and sophisticated. The \"uncertainty-aware modulation\" is a particularly clever mechanism. Instead of naively using generated views, it provides a principled way to fuse geometric information from the current 3D-GS reconstruction with the powerful generative prior of the diffusion model.\n\nWeaknesses:\n\n1. (major weakness) Missing comparisons with other generative-based methods: The paper only compares against non-generative sparse-view NeRF or 3DGS approaches, but does not include any baselines that also leverage generative priors. Such comparisons are important for a fair evaluation.\n\n2. (minor weakness) Critical reliance on SVD: The proposed method heavily relies on the capabilities of the underlying video diffusion model (SVD), as the authors acknowledge. As a result, it may struggle with out-of-distribution scenes, where the generative prior can hallucinate plausible but inaccurate details (appearance- and geometry-inconsistent contents), potentially corrupting the resulting 3DGS.\n\nQUESTIONS:\n1. Are there any failure cases related to the weaknesses discussed in point 1 (Critical reliance on SVD)? Specifically, what happens if the SVD generates undesirable hallucinations that are both appearance- and geometry-inconsistent? If so, are there any proposed solutions to address this issue?\n\n2. Can this proposed method outperform other generative-based methods with similar technical approaches? Such as Viewcrafter [1] (feed-back loop between generated views and 3D representation) and 3DGS-enhancer [2] (frames interpolation by video diffusion).\n\n[1] Yu, Wangbo, et al. \"Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis.\" Open Source Project 2024.\n\n[2] Liu, Xi, Chaoyi Zhou, and Siyu Huang. \"3dgs-enhancer: Enhancing unbounded 3d gaussian splatting with view-consistent 2d diffusion priors.\" NeurIPS 2024.\n\nJustification of the rating: \nI think overall, this paper presents a technically novel approach with good performance, with clear writing and organization. However, as a generative-based method, comparisons with other generative-based baselines are necessary. Therefore, my rating is borderline accept because of limited evaluation.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-02"}