Title: ViDAR: Video Diffusion-Aware 4D Reconstruction From Monocular Inputs
Venue: NeurIPS 2025
Accepted: yes
Keywords: monocular reconstruction

diffusion guidance
Abstract: Dynamic Novel View Synthesis aims to generate photorealistic views of moving subjects from arbitrary viewpoints. This task is particularly challenging when relying on monocular video, where disentangling structure from motion is ill-posed and supervision is scarce. We introduce Video Diffusion-Aware Reconstruction (ViDAR), a novel 4D reconstruction framework that leverages personalised diffusion models to synthesise a pseudo multi-view supervision signal for training a Gaussian splatting representation. By conditioning on scene-specific features, ViDAR recovers fine-grained appearance details while mitigating artefacts introduced by monocular ambiguity. To address the spatio-temporal inconsistency of diffusion-based supervision, we propose a diffusion-aware loss function and a camera pose optimisation strategy that aligns synthetic views with the underlying scene geometry. Experiments on DyCheck, a challenging benchmark with extreme viewpoint variation, show that ViDAR outperforms all state-of-the-art baselines in visual quality and geometric consistency. We further highlight ViDAR’s strong improvement over baselines on dynamic regions and provide a new benchmark to compare performance in reconstructing motion-rich parts of the scene.


Peer reviews:
comments: SUMMARY:
The authors propose ViDAR, a monocular-video pipeline that (i) personalises an SDXL diffusion model to the target clip (DreamBooth-style), (ii) enhances renders from a monocular Gaussian-splat baseline (MoSca) to create high-fidelity pseudo-multi-view supervision, and (iii) introduces diffusion-aware reconstruction: losses that mask supervision to dynamic pixels and a joint optimisation of the sampled camera poses. On the DyCheck benchmark, ViDAR reports superior PSNR/SSIM—particularly in dynamic regions—across all 14 scenes and supplies an ablation isolating each contribution.

STRENGTHS AND WEAKNESSES:
Strengths:
Bridges generative diffusion priors with geometrically-consistent monocular 4D reconstruction; supplies a dynamic-mask benchmark.

Weakness:
1. Dependency on Baseline Geometry – Authors acknowledge diffusion cannot fix large geometric errors; performance is still gated by MoSca quality.
2. Ablation Scope – No study on diffusion-mask accuracy; sensitivity to Track-Anything noise is unclear.

QUESTIONS:
Failure Modes – What artefacts remain when baseline geometry is poor (e.g., heavy occlusion)? Examples or quantitative failure analysis would be valuable.
Generalisation – Have you tried ViDAR on truly in-the-wild phone clips beyond DyCheck? Qualitative evidence would help.

LIMITATIONS:
The paper notes diffusion cannot repair major geometric errors and that training is compute-heavy.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-01

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nViDAR is a 4D reconstruction framework that takes a monocular video as input. ViDAR initializes the 4D representation with a prior 4D monocular video reconstruction work, MoSca, which produces a 4D Gaussian splatting representation. Stable Diffusion XL model is finetuned to be personalized for the input monocular video such that the model captures the distribution of the images from the current video better. This personalized model is used to enhance the quality of novel view samples, which are then used together with the original video to enhance the quality of 4D Gaussians. ViDAR demonstrates superior quality over the prior 4D monocular video reconstruction approaches.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- ViDAR presents a pipeline for a challenging and useful 4D monocular video reconstruction task that has a lot of downstream applications, such as simulation and content creation.  \n- ViDAR outperforms previous approaches both qualitatively and quantitatively, and presents results with comparison with multiple other works. It also points out the limitations of metrics and proposes an enhanced metric that focuses more on the dynamic regions of scenes that are more relevant for the task. On top of the summary above, it further proposes techniques to enhance the quality of 4D representation, namely Tracking-based Gaussian Classification, Sampled Camera Optimization, and Dynamic Reconstruction, whose effectiveness is demonstrated.\n- The paper is clearly written and easy to follow.\n\nWeaknesses:\n- The main part of the proposed pipeline is a combination of popular techniques, such as dataset enhancement with Diffusion Model works (ex. Instruct-Nerf2Nerf (Haque et al. 2023), ReconFusion (Wu et al. 2023)) and personalized DM for view reconstruction (ex. RefFusion, Mirzaei et al. 2024), on top of a strong 4D initialization given by MoSca. I believe the novelty of the work comes from the robustness of the pipeline, and the effect of each component needs to be more clearly demonstrated. \n    - Table 4 shows that removing some of the proposed techniques shows degradation in quality that can be worse than the initial MoSca. This could suggest that the personalized DM do not contribute much to the performance of the pipeline, unless more evidences are supported.\n    - Compared to other state-of-the-art methods, such as CAT4D, ViDAR leverages more information such as depth map and 2D tracks. It is not clear if adding one or two proposed components to CAT4D (especially MoSca initialization) could make it perform better. Note that I do not mean that this wouldn't be okay, but it is not clear which component is adding the most benefit.\n- ViDAR uses a personalized image diffusion model, but the name of the method (Video Diffusion-Aware … ) is misleading which suggests a ‘video diffusion’ model is used. In line 160, it is actually mentioned ‘a personalized video diffusion model’ - I believe this is incorrect. \n\nMinor:\n- In line 94, CAT4D is mentioned as a concurrent work, but it appeared on Arxiv on Nov 27, 2024, which is prior to the date (March 1, 2025) NeurIPS considers as concurrent.\n\nQUESTIONS:\n- As I believe the significance of the work comes from building a robust pipeline for 4D monocular video reconstruction, it would be great to have the code released such that the community can benefit from it. Would it be a possibility? \n- Regarding the first point in weaknesses, it would be great to have additional ablations demonstrating the effect of each component more clearly. \n  - Minor: Recent works such as DIFIX3D+ (Wu et al. 2025) suggest that training a general diffusion model for enhancement purposes might be better than a personalized model. This is concurrent work, so it is just a suggestion to include some further discussion on the benefit of personalized DM.\n- Why was a video diffusion model not used, and an image diffusion model was used instead? Apart from the confusion mentioned in the weakness section, there is no clear discussion on this choice.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work aims to reconstruct the dynamic 3D scene from one monocular video and proposes Video Diffusion-Aware Reconstruction (ViDAR), which utilizes diffusion models to generate pseudo-multi-view supervisions for the training of the dynamic Gaussian Splatting. Specifically, this work generates fine-grained details by conditioning the scene-specific features and introduces the diffusion-aware loss function and the strategy of camera pose optimization.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n  * The core idea of this work includes diffusion enhancement and pseudo-novel-view sampling. The proposed approach sounds like it makes sense.\n  * The dynamic region mask for evaluation is valuable since the co-visibility mask metrics of the DyCheck dataset [C] are largely confounded by good reconstructions of static regions. Extended evaluation of dynamic region masks should be widely adopted.\n  * This paper is well-written and is easy to understand.\n\n* Weakness\n  * Although the method proposed in this work is very easy to understand, it is difficult to tell the difference between 4D reconstruction and 3D reconstruction using the diffusion enhancement method. That is, the proposed method can also be inserted into a static 3D reconstruction setting (like sparse views, blurring, etc.). I think the key to using diffusion enhancement in 4D reconstruction is to explain how to solve the problem of how to generate content and ensure the consistency of geometric structure at the same time.\n  * The rendered images of the proposed method tend to be smooth and blurred. For example, in Figure 4, the sleeve textures of \"teddy\", the clothes textures of \"block\", and the pants textures of \"space-out\", are compared to SoM [A] and MoSca [B]. Is this due to diffusion enhancement?\n  * This work only conducts experiments on one dataset, i.e. the DyCheck dataset [C]. It is not sufficient. Meanwhile, as far as I know, there may exist pose errors in the DyCheck dataset [C] as illustrated in Appendix A.3. of MoDGS [D]. Therefore, this work should be supplemented with more quantitative results on dynamic reconstruction datasets.\n\n[A] Wang Q, Ye V, Gao H, et al. Shape of motion: 4d reconstruction from a single video[J]. arXiv preprint arXiv:2407.13764, 2024.\n\n[B] Lei J, Weng Y, Harley A W, et al. Mosca: Dynamic gaussian fusion from casual videos via 4d motion scaffolds[C]//Proceedings of the Computer Vision and Pattern Recognition Conference. 2025: 6165-6177.\n\n[C] Gao H, Li R, Tulsiani S, et al. Monocular dynamic view synthesis: A reality check[J]. Advances in Neural Information Processing Systems, 2022, 35: 33768-33780.\n\n[D] Qingming L I U, Liu Y, Wang J, et al. MoDGS: Dynamic Gaussian Splatting from Casually-captured Monocular Videos with Depth Priors[C]//The Thirteenth International Conference on Learning Representations. 2025.\n\nQUESTIONS:\nSee Weakness. Besides, I think the proposed method can also be inserted into a static 3D reconstruction setting (like sparse views, blurring, etc.).\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

comments: SUMMARY:
The paper presents ViDAR, an algorithm for 4D reconstruction from monocular videos, addressing the challenge of generating high-quality novel views of dynamic scenes with spatio-temporal consistency. ViDAR employs MoSca to create a 4D scene representation and samples novel camera poses to produce degraded multi-view renderings. Leveraging a DreamBooth-adapted diffusion model, it refines these renders into pseudo-supervision signals. To resolve diffusion-induced inconsistencies, the approach incorporates a diffusion-aware loss and camera pose optimization.

STRENGTHS AND WEAKNESSES:
Strengths:
1. The paper features high readability and a well-structured logical flow.
2. Experiments on the DyCheck dataset demonstrate significant advantages over state-of-the-art methods.

Weaknesses:
1. The methodological novelty is constrained, resembling a combination of per-scene optimization (MoSca) and a DreamBooth-adapted diffusion model.
2. There is a lack of theoretical support or prior evidence indicating that DreamBooth or SDXL can ensure view-time consistency. Relying on such pseudo-GTs to achieve spatio-temporal consistency appears tenuous.
3. The computation is excessively time-consuming, with the diffusion enhancement stage alone taking up 12 hours.

QUESTIONS:
As noted in the Weaknesses section, a more detailed explanation of how misaligned diffusion outputs nonetheless enhance result consistency would make the proposed method far more convincing.

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 2
DATE: 2025-07-01