GeoVideo: Introducing Geometric Regularization into Video Generation Model
Authors: Yunpeng Bai
Shaoheng Fang
Chaohui Yu
Fan Wang
Qixing Huang
Venue: NeurIPS 2025 poster ()
Keywords: Video Generation
Geometric Regularization
3D/4D Generation
Primary area: deep_learning

Abstract: Recent advances in video generation have enabled the synthesis of high-quality and visually realistic clips using diffusion transformer models. However, most existing approaches operate purely in the 2D pixel space and lack explicit mechanisms for modeling 3D structures, often resulting in temporally inconsistent geometries, implausible motions, and structural artifacts. In this work, we introduce geometric regularization losses into video generation by augmenting latent diffusion models with per-frame depth prediction. We adopted depth as the geometric representation because of the great progress in depth prediction and its compatibility with image-based latent encoders. Specifically, to enforce structural consistency over time, we propose a multi-view geometric loss that aligns the predicted depth maps across frames within a shared 3D coordinate system. Our method bridges the gap between appearance generation and 3D structure modeling, leading to improved spatio-temporal coherence, shape consistency, and physical plausibility. Experiments across multiple datasets show that our approach produces significantly more stable and geometrically consistent results than existing baselines.

GeoVideor Introducing Geometric Regularization into Video Generation Model
Yunpeng Bai'
Shaoheng Fang'
Chaohui 2
Fan Wang?
Qixing Huang'
The University of Texas at Austin, DAMO Academy, Alibaba Group, 'Hupan Lab
https://geeviddeo github... iieiio/GGeeee GeoViiee
brick building sarrounded by mature trees.
Geo Video
Figure I: GeoVideo introduces a geometric consistency loss using predicted depths and camera poses enhance multi-view consistency of the output frames, leading high-quality 3D reconstruction from the output video frames.
Abstract
Recent advances in video generation have enabled the synthesis of high-quality and visually realistic clips using diffusion transformer models. However, most existing approaches operate purely in the 2D pixel space and lack explicit mechanisms for modeling 3D structures, often resulting in temporally inconsistent geometries, implausible motions, and structural artifacts. In this work, we introduce geometric regularization losses into video generation by augmenting latent diffusion models with ver-frame depth prediction. We adopted depth as the geometric representation because of the great progress in depth prediction and its compatibility with image- based latent encoders. Specifically, to enforce structural consistency over time, we propose a multi-vies geometric loss that aligns the predicted depth maps across frames within a shared 3D coordinate system. Our method bridges the gap between appearance generation and 3D structure modeling, leading improved spatio- temporal coherence, shape consistency, and physical plausibility. Experiments across multiple datasets show that our approach produces significantly more stable and geometrically consistent results than existing baselines.
Introduction
Video generation has recently made significant strides in creating visually impressive and high-quality clips [4,
28 69
29
53
77]. Powered by diffusion transformer models [39 and large-scale training
datasets 63
611, these systems are capable of synthesizing realistic videos conditioned on
various inputs, such as text prompts [26] 35,
26 62) or images [2] 43 57 38). Despite these
successes, current video generation models often fail accurately capture underlying geometry, coherent motions, and physical consistency in dynamic scenes. As result, the generated videos,
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

although plausible at first glance, often exhibit temporal artifacts such as shape deformation, structure flickering, and implausible object interactions.
This limitation stems from the fact that most video generation models operate purely in the 2D) pixel/latent space and rely temporal attention promote cross-frame coherence. Although effective in maintaining short-term consistency, these approaches lack explicit mechanisms to model 3D structures, leading to violations of object permanence, shape integrity, and motion realism.
This shortcoming highlights a deeper insight: realistic video generation demands more than visual coherence it requires structured understanding of the 3D world. After all, videos naturally encode spatio-temporal observations real environments. Viewed through this lens, video generation can be reframed as form of world modeling the construction of continuous, physically grounded
representations of the dynamic world. Emerging research on world generation [4] 5) underscores its
potential in applications such 3D scene synthesis [34 70 44 robotics [64 20 [6). and embodied
AI 41 45 [4]. However, achieving physically plausible world modeling requires the generated
scenes to maintain consistent geometry over time, which current models struggle enforce.
To address these limitations, we propose Geo Video, which introduces geometric regularization into the video generation process. Specifically, we augment the generative model to predict per-frame depth maps alongside RGB frames and enforce cross-frame depth consistency. This regularization encourages the model to maintain coherent 3D geometry throughout the video. By aligning predicted depth across consecutive frames, the model is guided to better capture the underlying scene structure, resulting in enhanced realism, temporal stability, and physical plausibility. Our key insight that depth consistency offers implicit geometric supervision that complements appearance- based learning. This helps bridge the gap between 2D frame-level synthesis and 3D-consistent scene modeling, paving the way toward more structured and physically grounded video generation.
The main contributions of this work are:
Explicit Geometry Modeling in Video Generation. We introduce per-frame depth predic- tion into latent diffusion-based video generation models, enabling explicit modeling of 3D scene structure throughout the generation process.
Geometric Regularization. We propose cross-frame consistency loss that lifts pre- dicted depths into a global 3D point cloud and supervises them via multi-view reprojection alignment, encouraging globally coherent geometry.
Improved Spatiotemporal Coherence. Our approach significantly enhances structural consistency, motion stability, and geometric plausibility generated videos, as demonstrated several benchmarks.
2
Related Work
2.1 Diffusion Models for Video Generation
The remarkable success of diffusion models in image generation 42 47 46
has recently inspired their 65 where they have quickly become the dominant paradigm. has emerged as widely adopted strategy: a VAE [7] module
extension to video generation 25
In particular, latent diffusion [5 46
first encodes video data into a compact latent space, and the diffusion process is performed within this lower-dimensional representation. Current state-of-the-art methods 66 29. 76 utilize 3D
Variational Autoencoder (69) in combination with a Diffusion Transformer (DiT) backbone, achieving highly realistic and high-fidelity video synthesis. Despite these advances, existing diffusion- based video generation models [I8, 3] often struggle to accurately capture geometric structures, coherent temporal motions, and physical consistency. To mitigate these limitations, recent research has explored the introduction of additional priors to guide the generation process toward better alignments with real-world dynamics.
For example, Track4Gen 22 jointly models video generation and point tracking across frames within a single network, providing enhanced spatial supervision over diffusion features and improving both motion and structural consistency. Similarly, VideoJAM (6) learns a joint appearance-motion representation that instills an effective motion prior to the video generator. Other contemporary approaches have also taken advantage of motion representations to improve motion coherence in image-to-vider generation tasks [50] [9]. Meanwhile, OmniVDiff (66) models appearance, depth,

---

Canny edges, and semantic segmentation simultaneously, enabling multi-modal video generation and multi- modai- conditional generation. However, it does not explicitly impose priors on the generated auxiliary signals. IDOL 73 proposes a human-centered joint video-depth generation framework, but it does not introduce explicit priors and is limited to numan-centered scenarios. WVD [75] supports the simultaneous generation of appearance and point representations but is restricted to image-to-video translation in small static environments. Complementary to these efforts, Yue et al. (72) proposed to lift the semantic characteristics of each frame into 3D Gaussian representation. demonstrating that fine-tuning a foundation model with these 3D-aware characteristics leads to better performance across downstream tasks. Building these insights, work seeks to jointly model geometry during video generation and introduce an explicit geometric regularization loss to further improve the quality, consistency, and realism of synthesized videos.
2.2
CeometryyRRlateer Tasks with Pretrained Diffusion Models
Another line of work uses pre-trained generative models 46 3 67] for geometry-related tasks. pioneering effort this direction Marigold (24). which first proposed treating depth as an image-like modality. By encoding depth maps into the same latent space as RGB images using a latent diffusion VAE, Marigold demonstrated that image generation models can be repurposed into depth estimators. This idea has inspired a series of subsequent works [10 [ that exploit the strong priors of diffusion models to estimate geometric properties such as depth and surface normals. Following this direction, DepthCrafter [19] and Depth Any Video [68] adapt video generation models to perform video-based depth estimation, effectively extending Marigold's 's latent-space strategy from images to videos. Similarly, DiffusionRender [32] leverages video generation models for inverse rendering tasks, jointly recovering not only scene geometry but also materials and lighting from video sequences. Building these advancements, Geo4D [23] further expands the use of video generators to tackle 4D scene reconstruction, capturing dynamic scene structures over time.
2.3 3D Scene Generation
The generation of 3D content has also emerged highly active area of research. The early text- to-3D scene generation methods [17 mainly relied on image generation inpainting models and progressively completed a scene through multiple iterations, resulting limited efficiency and quality, Subsequent methods design specialized models for text-to-3D scene generation. Director3D [30] proposes a text-to-3D generation framework capable of synthesizing both real-world 3D scenes and adaptive camera trajectories. Recently, some methods like SplatFlow [1I and Bolt3D (52)
have also leveraged intermediate 3D representations to directly generate scenes in the form of 3D) Gaussian Splatting, either through multiview diffusion or flow matching models. Prometheus [7] introduces a multiview diffusion model based on an RGB-D latent space to generate 3D Gaussian scenes. However, since Prometheus relies primarily on image-based models, the quality and fidelity of the generated 3D content remain limited. Apart from the methods mentioned above, LDM3D [51] is previous work similar to that re- re-trains Stable Diffusion, but its scope is limited the latent space of RGB-D images. Our method incorporates geometric regularization into the video generation model, enabling the direct extraction of high-quality 3D scenes from the generated videos.
3 Approach
We begin by reviewing the preliminaries of the video diffusion models in Section 3.1 Section 3.2 and Section 3.3 then describe the proposed RGB-D video generation model and our novel geometric regularization loss, respectively. Finally, Section 3.4 introduces the training procedure.
3.1 Preliminaries: Video Diffusion Models
Our method is based on latent irftusion-baaeed video generation models, particularly those with 3D VAE backbones such as Cog VideoX (69). and transformer-based denoisers such as DiT (3) These models encode video frames into compact latent space and apply denoising diffusion to synthesize temporally coherent video sequences. Formally, let X denote a video clip of T frames, and let z E(xir) be its latent representation encoded by VAE. Latent diffusion models
define forward noising process over by gradually adding Gaussian noise: (zz(1) zz (11)))
3

---

D
Motion
Probubility
Video
W
DiT
Wast
D
Cameras
Reprojection
& Match
NNoise
West
Da
Depth
Depth
Figure 2: Overview of the proposed method. The orange-yellow modules are additionally added components. pp. 2. and Pa respectively denote the heads for decoding motion probability, camera pose, and depth from the generated features. We use their outputs to defin geometric consistency loss regularize the video generative model.
Niz!") vaiz(tt))
at a,).), where t = ,,..., S indexes the diffusion timestep and is variance schedule. The model learns reverse process to sample denoised latents: (o(2((-1) (2)))
((z((-1); (2(+),, (()), which is parameterized by spatio-temporal transformer (DīT). After obtaining the denoised latent (9), the final RGB video is reconstructed via the VAE decoder.
3.2 RGBD Video Modeling
To incorporate explicit geometric structure into the video generation process, we choose to represent scene geometry using per-frame depth maps. This choice is motivated by two key factors. First, recent advances 19 video depth estimation have yielded robust and high-quality predictions, making it feasible to obtain reliable depth supervision even in unconstrained settings. Second, depth maps have a natural image-like structure and can be efficiently encoded into the same latent space as RGB frames using shared VAE encoder. This design has been validated in prior works such as Marigold [24] and DepthCrafter [9]. Using this modality compatibility, we can extend existing latent video diffusion models to jointly generate RGB and depth with minimal architectural modifications.
As shown Figure given pair of RGB and depth frames (x1.7 shared latent space:
xD xI.r).
we encode them in
[z RGB 2 [E(x
RGB
E(x
z
where F-)) denotes the VAE encoder and | denotes channel-wise concatenation. The diffusion model operates the latent sequence 2 and learns jointly denoise both modalities. The generated latent is then decoded by the VAE decoder into video frames and depth maps.
We model the joint distribution over RGB and depth frames as:
T
RGB P 1:7 xI.T = P(z)
P P(x, II
RGB
x 1z)
1=1
where each pair is generated from the same underlying latent representation. This formulation enables tight coupling between appearance and geometry throughout the generation process.
3.3 Introducing Geometric Regularization
Although per-frame depth maps provide localized 3D cues, they do not guarantee cross-frame consistency. To enforce coherent 3D structure over time, we introduce geometric regularization loss that lifts predicted depths into world coordinates using known camera intrinsics and extrinsics. Since depth and appearance are decoded from the same underlying features, applying supervision on the depth prediction allows us to enhance the geometric consistency of the underlying shape without needing to account for vien-dependent appearance differences across views.

---

Global point cloud construction. Since the built-in 3D VAE in video generation models is computationally heavy, we additionally train lightweight decoder Da to convert the depth latent zD into depth map D E R W of the same resolution as the RGB frame, enabling more fine-grained and accurate geometric regularization.
parallel, and inspired by VGGT [88,, we also predict the camera intrinsics and extrinsics for each frame using a camera head from generated z RGB Let D be the predicted depth map for frame i, and P, E (3) its camera pose. Using the intrinsic matrix K, we backproject depth into the camera space and transform it into the world space:
X, P; (D,, K):
where -² denotes backprojection from depth to 3D coordinates. The global point cloud is then obtained by aggregating:
T
Kglobal
U X X
(2)
i=1
We denoise Kglobal using voxel grid downsampling and statistical outlier removal to improve robust- ness and computational efficiency.
Depth reprojection consistency. To supervise consistency, we reproject the global point cloud back to each frame and compare its depth with the predicted depth map. For frame i, we project:
(u) = (P.1. x), x E Agiebal
(3)
where Ty (-) denotes depth value after projection into image coordinates u. We then compute the loss:
T M
fgo = T
((()(uu) - DD)( < <6) (D)() - DD())): NE
Σ
(4)
where V; is the set of valid pixels and d is tolerance threshold set to 0.05. This encourages global shape consistency by penalizing depth discrepancies only when local reprojections are reliable. When multiple points project the same pixel, dynamic videos, we introduce an additional head Dp that predicts an object movement probability map (31] from the generated video features RRGR, representing pixels that correspond to dynamic content based multi fialte--fmmmeee information. For dynamic pixels identified in the probability map, only align them with points of similar probability in adjacent frames.
the average of these points compute loss. For
3.4 Parameters Initialization and 2-stage Fine-tuning with Geometric Regularization
Parameters Initialization. To enable the pretrained video generation model support dual-modality inputs (RGB and Depth), we modify the input and output projection layers of the transformer. Let Win XCC, be the input projection matrix, where C. the input feature dimension and C2 the transformer token dimension. Inspired by ControlNet [74]. we extend it by vertically concatenating a zero matrix of the same shape, resulting W xC,,C. The associated input bias in E R C, left unchanged. On the output side, let Wo.. E RC.CC. denote the output projection matrix. To accommodate dual outputs, we horizontally concatenate copy of Wout. resulting in W E RR x2C, The output bias bout RRC C. is duplicated form b E 22C,, The initialization summarized as:
Win
w
61 = bin R.C, E R 2C,x
(5)
baut
W out
Waout
Wou] E CCx2
b
E RO.0.
Dout
This initialization ensures that the depth modality is initially zero-influence pathway, allowing the model to begin fine-tuning from the pretrained RGB state without disrupting performance. During fine-tuning, the model progressively learns represent depth channels.
5

---

Cog CogVidexX
CogVideoX -tnned
Geo Video
Igar
N/A
CogVideoX
CogVideoX-tuned
GooVidco
Figure 3: Comparisons the down-stream 3D reconstruction task. The detailed prompt inputs are provided in the supp. materials. The top three rows show samples of video generation results of each method, while the corresponding 3D reconstructions generated by our method offer complete and high-quality 3D reconstructions.
show in the bottom row. The videos
Stage 1: RGB-D Joint Generation. In the first stage, we fine-tune the model to generate RGB and depth frames simultaneously. To ensure stable learning, we apply gradually increasing weight to the depth loss:
dapth (t) = min(1.0,0m 0.1 + at),
(6)
where is the training step and is set to 0.0001. This gradual increase allows the model to adapt to the new depth supervision without destabilizing RGB generation. The loss for this stage is:
Laage
RGB Atepth (1) CD
Here, CRGB and CD are formulated using the v-prediction 48 strategy.
Stage 2: Geometric Regularization. Once the model can generate perceptually and structurally coherent depth maps, we introduce the geometric regularization term Egeo (described in Section 3). This loss encourages cross-frame depth consistency via reprojection-bail based supervision.
The final training objective becomes:
Listal
Liff
RGB
Adepth
Ageo
Legen-
(8)
Here, Agoo is set to 0.5. This staged training process allows the model to gradually learn to incorporate geometry without sacrificing visual fidelity.
Experimental Results
Implementation details. Our experiments are primarily based on Cog VideoX-5B 69 popular and advanced diffusion-bann video generation model built on the DiT architecture. We conducted experiments for both text-to-video (T2V) and image-to-video (12V) generation tasks. The experiments

---

Table Multi-view geometric consisteney evaluation on the DL3DV dataset. We use VGGT to predict multi-frame depth and pose, and evaluate consistency using our proposed Multi- View Consis- tency Score (MVCS) and reprojection error. MVCS measures Trame-to-frame depth consistency, t: higher is better; while Reprojection Error evaluates how well the globally reconstructed 3D structure aligns with original views, 4: lower is better.
Method
MVCS t
Reproj. Error +
CogVideoX-55F (T2V)
61.2
4.58
Cog CogVideoX-SB (T2V, finetuned on DL3DV)
66.4
3.91
Ours w/o Lge (T2V)
71.3
3.36
Ours (T2V)
77.2
2.52
TacidDreamer
SplatFlow
GeoXideo
LucidDrumer
Director3
SplatFlow
GenVideo
Figure 4: Comparisons with text to 3d scene generation methods. The detailed prompt inputs provided in the supp. materials.
categorized into two types: videos of static scenes and videos of dynamic scenes. For static scenes, we train on the DL3DV-10K (33] dataset. For dynamic videos, we collect a large-scale dataset of approximately 200,000 videos from online sources such Pexels [40]. We use 544 and 1000 videos from the two datasets for evaluation, respectively. We train the model with a learning rate of 2e-5 on 8 H20 GPUs for 20,000 steps. The batch size per GPU, with 15K steps for stage and 5K steps for stage 2. The video resolution is set to 768x1366 360 with 81 frames, following the standard configuration supported by Cog VideoX. The depth labels for videos are estimated using Video Depth Anything (7). while for dynamic videos, we estimate camera poses using MegaSaM [31]. For video captions, we use CogVLM 60 as adopted in Video Dp.. De. and Da are distilled from the corresponding outputs of MegaSaM [3I] VGGT [58) and Video Depth Anything 77, respectively. After distillation, their parameters are kept fixed during fine-tuning provide supervision.
Scene generation results. The original video generation model already possesses some ability generate coherent scenes. However, due to the typically small range of viewpoint changes, difficult to extract sufficient multi-view information from the generated videos to reconstruct a
meaningful 3D scene (as shown in the first row of Figure 3). To specialize the model for scene-level video generation, we fine-tune the video generation model on the DL3DV dataset [33]. However, since the base model relies purely on 2D modeling, struggles to maintain geometric consistency under rapid camera viewpoint changes. The second row of Figure 3 shows the results of Cog VideoX after being fine-tuned on DL3DV, where such inconsistencies are still apparent. In contrast, after integrating our proposed geometric modeling framework, we are able to maintain consistent multi- view structures even under complex viewpoint variations (Figure third row). further demonstrate the geometric consistency of the generated results, we perform cructure-from-mottio (SfM) [49 reconstruction on videos generated by different methods. The original Cog VideoX outputs fail for reconstruction due to lack of sufficient multi-viet view cues. The fine-tuned model only achieves partial
7

---

Table Quantitative results on the DL3DV dataset for text-to-3D scene generation. We compare our method against Director3D, LucidDreamer, and SplatFlow across multiple perceptual metrics. 1:
higher better; L: lower is better.
Method
FID
CLIPScore
NIQE
BRISQUE J
Dremmer
79.96
31.25
11.23
44.52
Director3D
90.20
30.04
13.79
51.67
SplatFlow II
86.77
31.42
14.15
48.85
Ours
72.78
33.84
8.53
36.41
Input Image
CogVideoX
Prompt
Geo GeoVidee Video wio Lyco
neewing acress an opon mosdow, flowing in the ward and poworful loes lifhing it riythhiiaally above groand, Thee soft lighting and open empraaia the skeyonte and strougtt of the avlta The
GeoVideo
scave captores
Rooting Homont of
motion and grase
againse serene bododros..
Figure 5: Comparison with original video generation method & ablation study. The original video generation model, as well as naive joint modeling of depth, struggle to maintain geometric consistency of objects throughout their motion.
reconstruction with limited consistency among few consecutive frames. In contrast, our method enables high-quality and structurally complete 3D scene reconstructions.
To further evaluate 3D consistency across frames, introduce metrics based VGGT (58 the Multi-View Consistency Score (MVCS) and the Reprojection Error. Given the predicted depth maps and camera poses from VGGT, we project each frame into shared 3D space compute cross-view consistency. MVCS measures the alignment of depth maps across views by warping each depth map neighboring frames and comparing it against the corresponding predicted depth. In contrast, Reprojection Error evaluates the pixel-wise distance between original image coordinates and those reprojected from the reconstructed global point cloud, serving as direct indicator of geometric alignment accuracy. Table reportss MVCS and Reprojection Error for different models on the DL3DV dataset. Our method significantly outperforms Cog VideoX-SB and its finetuned variant. Notably, removing the geometric loss Lgoo leads to a clear performance drop, demonstrating the effectiveness of our geometric regularization in preserving structural consistency across views.
Furthermore, we compare our method with three representative text-to-3D scene generation baselines: Director3D (30], LucidDreamer (9. and SplatFlow [II]. The qualitative comparison of generation results can be found in Figure 4 We then follow Director3D and use the following metrics for quanti- tative evaluation. We use CLIPScore [15] to assess the alignment between the generated content and the text prompt. For perceptual evaluation, we adopt the Fréchet Inception Distance (F6)) standard metric for assessing visual fidelity and diversity. In addition, we use two no-reference image quality metrics-Natte Natural Image Quality Evaluator (NIQE) 37] and Blind/Referencelee Image Spatial Quality Evaluator (BRISQUE) (36) to directly assess perceptual quality based on image statistics, without relying ground truth references. Table 2 summarizes the quantitative results in DL3DV dataset. Our method consistently outperforms all baselines
metrics. particular, achieve the lowest FID and the highest CLIPScore, demonstrating superior semantic consistency

---

Table 3: Quantitative comparison with CogVideoX-5B on T2V and 12V tasks. We evaluate on CLIPScore, FVD (Fréchet Video Distance), and metrics from VBench: Subject Consistency
(SC), Background Consistency (BC), Motion Smoothness (MS), Spatial Relationship (SR), and
Video-Image Subject Consistency (VISC). 1: higher is better; 4: lower is better.
Method
CLIPScore f FVD 4
SC f
BC
MS f
SR f
VISC
CogVideoX (T2V)
32.30
145.3
93.8
95.1
93.2
79.4
Ours w/o Lgo (T2V)
33.25
134.2
94.3
96.0
95.4
87.4
Ours w/o MP (T2V)
33.83
131.6
95.8
96.3
96.7
88.2
Ours (T2V)
34.25
122.7
97.2
97.8
98.1
90.3
Cog V-B (12V)
33.42
139.8
94.6
96.4
95.9
80.5
95.2
Ours w/o eoo (I2V)
34.13
128.0
95.0
97.1
96.3
86.3
96.3
Ours w/o MP (12V)
34.77
126.5
96.9
97.6
96.9
88.8
96.8
Ours (I2V)
35.02
120.5
98.1
98.5
98.6
91.1
97.6
and perceptual quality. Moreover, NIQE and BRISQUE scores are also lower, indicating closer match to natural image distributions compared to prior methods. Compared to these 3D generation methods, our approach achieves higher visual fidelity by leveraging the strong priors from pretrained video generation models.
Video generation results. Table presents quantitative comparison between method and Cog VideoX-SB on both text-to-video and image-to-video generation using the 1000 evaluation videos. We report CLIPScore [1] to measure semantic alignment, FVD [54] to assess overall visual quality and temporal consistency, and multiple metrics from VBench [2 including Subject Consistency
(SC), Background Consistency (BC), Motion Smoothness (MS), Spatial Relationship (SR), and Video-Image Subject Consistency (VISC). Our method achieves superior performance across all metrics, clearly outperforming Cog VideoX-5B in both settings. Notably, observe substantial improvements in motion quality (MS), subject coherence (SC), and Spatial Relationship (SR).
Ablation Studies. Our method is based on two core components: (1) explicitly modeling depth and (2) applying supervision on the jointly generated depth. We conduct ablation studies targeting these two aspects. As shown in F we present image-to-video (I2V) result demonstrating that simply modeling both depth and appearance already improves the video quality to some extent. This is because the temporal consistency of the depth labels themselves imposes a form of constraint across frames. However, this alone is insufficient to ensure global geometric consistency throughout the video. When proposed geometric loss Egso is added, the generated videos exhibit significantly improved came-to--frame continuity and structural coherence. The corresponding metric improve- ments are shown in Table 3 and Table [ In addition, motion probability (MP) map in dynamic videos. Table shows that ignoring the MP map leads to noticeable performance drops, particularly in the T2V setting.
also study the impact of incorporating the
Conclusions
In this work, we proposed to augment pre-trained video generation models with geometric regulariza- tion by introducing per-frame depth prediction and enforcing cross-frame depth consistency. This approach leverages the natural spatio-temporal cues encoded in video to guide the model toward learning stable and physically grounded scene representations. Our method provides implicit geomet- ric supervision through depth alignment, allowing for more accurate modeling of object permanence, spatial relationships, and scene dynamics. Through extensive experiments, we demonstrate that framework significantly improves the geometric fidelity and temporal stability of generated videos, outperforming prior baselines in both qualitative and quantitative evaluations. We believe that this represents an important step toward bridging video generation and 3D world modeling, opening new possibilities for downstream applications in simulation, robotics, and embodied intelligence.
Acknowledgements. This work was supported by Damo Academy through Damo Academy Innova- tive Research Program. This research has been supported by computing support on the Vista GPU Cluster through the Center for Generative AI (CGAl) and the Texas Advanced Computing Center (TACC) at UT Austin.

---

References
[1] Yunpeng Bai and Qixing Huang. Fiffdepth: Feed-forwart transformation of diffusion-basee generators for detailed depth estimation. arXiv preprint arXiv:2412.0067/ 2024.
[2] Max Bain, Arsha Nagrani, Gül Varol, and Andrew Zisserman. Frozen in time: A joint video and image encoder for end-to-end retrieval. In Proceedings of the IEEE/CVF international conference on computer vision, pages 1728-1738, 2021.
[3] Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Do- minik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:231 15127, 2023.
[4] Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing. David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh. Video generation models as world simulators. 2024.
[5] Jake Bruce, Michael D Dennis, Ashley Edwards, Jack Parker- Holder, Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, et al. Genie: Generative interactive environments. In Forty-first International Conference on Machine Learning, 2024,
[6] Hila Chefer, Uriel Singer, Amit Zohar, Yuval Kirstain, Adam Polyak, Yaniv Taigman, Lior Wolf, and Shelly Sheynin. Videojam: Joint appearance- motion representations for enhanced motion generation in video models. arXiv preprint arXiv:2502.02492, 2025.
[7] Sili Chen, Hengkai Guo, Shengnan Zhu, Feihu Zhang, Zilong Huang, Jiashi Feng, and Bingyi Kang. Video depth anything: Consistent depth estimation for super-long videos. arXiv preprint vrXiv22501.1237. 2025.
[8] Tsai-Shien Chen, Aliaksandr Siarohin, Willi Menapace, Ekaterina Deyneka, Hsiang-wei Chao, Byung Eun Jeon, Yuwei Fang. HisinYiing Lee, Jian Ren, Ming-Hsuan Yang, et al. Panda-70m: Captioning 70m videos with multiple mross-modality teachers. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pages 13320-13331, 2024.
[9] Jacyoung Chung, Suyoung Lee, Hyeongjin Nam, Jaerin Lee, and Kyoung Mu Lee. Lu- ciddreamer: Domain-free generation of 3d gaussian splatting scenes. arXiv preprint arXiv:2311.13388 2023.
[10] Xiao Fu, Wei Yin, Mu Hu, Kaixuan Wang, Yuexin Ma, Ping Tan, Shaojic Shen, Dahua Lin, and Xiaoxiao Long. Geowizard: Unleashing the diffusion priors for 3d geometry estimation from a single image. In European Conference on Computer Vision, pages 241-258. Springer, 2024.
[11] Hyojun Go, Byeongjun Park, Jiho Jang, Jin- Young Kim, Soonwoo Kwon, and Changick Kim. Splatflow: Multi-view rectified flow model for 3d gaussian splatting synthesis. arXiv preprint 2rX1v:2411.16443, 2024.
[12] Xun Guo, Mingwu Zheng, Liang Hou, Yuan Gao, Yufan Deng, Pengfei Wan, Di Zhang, Yufan Liu, Weiming Hu, Zhengjun Zha, et al. 12v-adapter: A general image-to-video adapter for diffusion models. In ACM SIGGRAPH 2024 Conference Papers, pages 1-12, 2024.
[13) Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to-image diffusion models without specific tuning. arXiv preprint 200.::230..04725. 2023.
[14] Haoran He, Yang Zhang, Liang Lin, Zhongwen Xu, and Ling Pan. Pre-trained video generative models as world simulators. arXiv preprint arXiv:2502.07825. 2025.
[15] Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation metric for image captioning. arXiv preprint arXiv:2 18, 2021.
[16] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
10

---

[17] Lukas Höllein, Ang Cao, Andrew Owens, Justin Johnson, and Matthias Nießner. Text2room: Extracting textured 3d meshes from 2d text-to-image models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7909-7920, 2023.
[18] Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. Cogvideo: Large-scale pretraining for text-to-video generation via transformers. arXiv preprint arXiv:2205. 15868,
2022.
[19] Wenbo Hu, Xiangjun Gao, Xiaoyu Li, Sijie Zhao, Xiaodong Cun, Yong Zhang, Long Quan, and Ying Shan. Depthcrafter: Generating consistent long depth sequences for open-world videos. arXiv preprint arXiv:2409.02095, 2024.
[20] Yucheng Hu, Yanjiang Guo, Pengchao Wang, Xiaoyu Chen, Yen-Jen Wang, Jianke Zhang, Koushil Sreenath, Chaochao Lu, and Jianyu Chen. Video prediction policy: A generalist robot policy with predictive visual representations. arXiv preprint arXiv:2412. 14803, 2024.
[21] Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive benchmark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 21 21807-21818 2024.
[22] Hyeonho Jeong, Chun-Hao Paul Huang, Jong Chul Ye, Niloy Mitra, and Duygu Ceylan. Track4gen: Teaching video diffusion models to track points improves video generation. arXiv preprint arXiv:2412.06016, 2024.
[23] Zeren Jiang, Chuanxia Zheng, Iro Laina, Diane Larlus, and Andrea Vedaldi. Geo4d: Leveraging video generators for geometric 4d scene reconstruction. arXiv preprint (0X.x::504007966 2025.
[24] Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Metzger, Rodrigo Caye Daudt, and Kon- rad Schindler. Repurposing diffusion-ber based image generators for monocular depth estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1492-9502, 2024.
[25] Levon Khachatryan, Andranik Movsisyan, Vahram Tadevosyan, Roberto Henschel, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. Text2video-zero: Text-to-image diffusion models are zero-shot video generators. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15954-15964 2023.
[26] Jihwan Kim, Junoh Kang, Jinyoung Choi, and Bohyung Han. Fifo-diffusion: Generating infinite videos from text without training. arXiv preprint arXiv:2405. /1473, 2024.
[27] Diederik P Kingma, Max Welling, al. Auto-encoding variational bayes, 2013.
[28] KlingAI. Kling AI, 2024.
[29] Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.036600 2024.
[30] Xinyang Li, Zhangyu Lai, Linning Xu, Yansong Qu, Liujuan Cao, Shengchuan Zhang, Bo Dai, and Rongrong Ji. Director3d: Real-world camera trajectory and 3d scene generation from text. Advances in Neural Information Processing Systems, 11775122--55111 2024.
[31] Zhengqi Li, Richard Tucker, Forrester Cole, Qianqian Wang. Linyi Jin, Vickie Ye, Angjoo Kanazawa, Aleksander Holynski, and Noah Snavely. Megasam: Accurate fast, and robust structure and motion from casual dynamic videos. arXiv preprint 24X4432412.00446, 2024.
[32] Ruofan Liang, Zan Gojeic, Huan Ling, Jacob Munkberg, Jon Hasselgren, Zhi-Hao Lin, Jun Gao, Alexander Keller, Nandita Vijaykumar, Sanja Fidler, et al. Diffusionrenderer: Neural inverse and forward rendering with video diffusion models. arXiv preprint arXiv:2501.18590 2025.
[33] Lu Ling, Yichen Sheng, Zhi Tu, Wentian Zhao, Cheng Xin, Kun Wan, Lantao Yu, Qianyu Guo, Zixun Yu, Yawen Lu, et al. D13dv-10k: A large-scale scene dataset for deep learning- based 3d vision. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 22160-22169 2024.
11

---

[34] Fangfu Liu, Wenqiang Sun, Hanyang Wang, Yikai Wang. Haowen Sun, Junliang Ye, Jun Zhang, and Yueqi Duan. Reconx: Reconstruct any scene from sparse views with video diffusion model. arXiv preprini arXiv:2408. 2024.
[35] Willi Menapace, Aliaksandr Siarohin, Ivan Skorokhodov, Ekaterina Deyneka, Tsai-Shien Chen, Anil Kag, Yuwei Fang. Aleksei Stoliar, Elisa Ricci, Jian Ren, et al. Snap video: Scaled spatiotemporal transformers for text-to-video synthesis. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 7038-7048, 2024.
[36] Anish Mittal, Anush Krishna Moorthy, and Alan Conrad Bovik. No-reference image quality assessment in the spatial domain. IEEE Transactions on image processing, 21(12):4695-770
2012.
[37] Anish Mittal, Rajiv Soundararajan, and Alan C. Bovik. Making a "completely blind" image quality analyzer. IEEE Signal processing letters, 20(3):209- 212, 2012.
[38] Haomiao Ni, Bernhard Egger, Suhas Lohit, Anoop Cherian, Ye Wang, Toshiaki Koike-Akino Sharon X Huang, and Tim K Marks. Ti2v-zero: Zero-shot image conditioning for text-to-vides diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9015-9025, 2024.
[39] William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4195 4205,, 2023.
[40] Pexels. Pexels - Free Stock Videos and Photos. https: wwwp:///wwwwppppll..... com/
[41] Yiran Qin, Zhelun Shi, Jiwen Yu, Xijun Wang, Enshen Zhou, Lijun Li, Zhenfei Yin, Xihui Liu, Lu Sheng. Jing Shao, et al. Worldsimbench: Towards video generation models as world simulators. arXiv preprint arXiv:2410.188 18072, 2024.
[42] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint 26X 1(2):, 2022.
[43] Weiming Ren, Huan Yang, Ge Zhang, Cong Wei, Xinrun Du, Wenhao Huang, and Wenhu Chen. Consisti2v: Enhancing visual consistency for image-to-video generation. arXiv preprint 2.Xiv:2402.04324., 2024.
[44] Xuanchi Ren, Tianchang Shen, Jiahui Huang, Huan Ling, Yifan Lu, Merlin Nimier-David, Thomas Müller, Alexander Keller, Sanja Fidler, and Jun Gao. Gen3e: 3d-informed world- consistent video generation with precise camera control. arXiv preprint arXiv:2503.03751, 2025.
[45] Zhongwei Ren, Yunchao Wei, Xun Guo, Yao Zhao, Bingyi Kang, Jiashi Feng, and Xiao- jie Jin. Videoworld: Exploring knowledge learning from unlabeled videos. arXiv preprint arXi::2501.0978) 2025.
[46] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022.
[47] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Bureu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image to-image diffusion models with deep language understanding. Advances in neural information processing systems, 35:36499-36494, 2022.
[48] Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. arXiv preprint 220.::2202.00012.... 2022.
[49] Johannes L. Schonberger and Jan-Michael Frahm. Stracture-from-mmoee motionn revisited. In Proceed- ings of the IEFE conference on computer vision and pattern recognition, pages 4113-4113, 2016.
12

---

[50] Xiaoyu Shi, Zhaoyang Huang, Fu- Yun Wang, Weikang Bian, Dasong Li, Yi Zhang, Manyuan Zhang, Ka Chun Cheung. Simon See, Hongwei Qin. al. Motion-i2v: Consistent and controllable image-to-video generation with explicit motion modeling. In ACM SIGGRAPH 2024 Conference Papers, pages 2024.
[51] Gabriela Ben Melech Stan, Diana Wofk, Scottie Fox, Alex Redden, Will Saxton, Jean Yu, Estelle Aflalo, Shao- Yen Tseng, Fabio Nonato, Matthias Muller, et al. Ldm3d: Latent diffusion model for 3d. arXiv preprint arXiv:2305. 10853, 2023.
[52] Stanislaw Szymanowiez. Jason Y Zhang, Pratul Srinivasan, Ruiqi Gao, Arthur Brussee, Alek- sander Holynski, Ricardo Martin-Brualla, Jonathan T Barron, and Philipp Henzler. Bolt3d: Generating 3d scenes in seconds. arXiv preprint arXiv:2503. /4.4503..4445. 2025.
[53] Genmo Team. Mochi 1.
2024.
[54] Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach. Raphaël Marinier. Marein Michalski, and Sylvain Gelly. Fvd: A new metric for video generation. 2019.
[55] Arash Vahdat, Karsten Kreis, and Jan Kautz. Score-based generative modeling in latent space. Advances in neural information processing systems, 34:1 34:11287-111 1287-113022 2021.
[56] Boyang Wang. Nikhil Sridhar, Chao Feng, Mark Van der Merwe, Adam Fishman, Nima Fazeli, and Jeong Joon Park. This&that: sanguage-gesturng controlled video generation for robot planning. arXiv preprint arXiv:2407.085: 005530, 2024.
[57] Cong Wang, Jiaxi Gu, Panwen Hu, Yuanfan Guo, Xiao Dong, Hang Xu, and Xiaodan Liang. Dreamvideo: High-fidelity image-to-video generation with image retention and text guidance. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1-5. IEEE, 2025.
[58] Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025.
[59] Shijie Wang, Samaneh Azadi, Rohit Girdhar, Saketh Rambhatla, Chen Sun, and Xi Yin. Motif: Making text count in image animation with motion focal loss. arXiv preprint 241 2024.
[60] Weihan Wang, Qingsong Lv, Wenmeng Yu, Wenyi Hong, Ji Qi, Yan Wang, Junhui Ji, Zhuoyi Yang, Lei Zhao, Xixuan Song, Jiazheng Xu, Bin Xu, Juanzi Li, Yuxiao Dong, Ming Ding, and Jie Tang. Cogvlm: Visual expert for pretrained language models, 2023.
[61] Wenjing Wang, Huan Yang, Zixi Tuo, Huiguo He, Junchen Zhu, Jianlong Fu, and Jiaying Liu. Videofactory: Swap attention in spatiotemporal diffusions for text-to-video generation. 2023.
[62] Yanhui Wang, Jianmin Bao, Wenming Weng, Ruoyu Feng, Dacheng Yin, Tao Yang, Jingxu Zhang, Qi Dai, Zhiyuan Zhao, Chunyu Wang, et al. Microcinema: A divide-and divide-and-conquer approach for text-to-video generation. In Proceedings the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 8414-8424, 2024.
[63] Yi Wang, Yinan He, Yizhuo Li, Kunchang Li, Jiashuo Yu, Xin Ma, Xinhao Li, Guo Chen, Xinyuan Chen, Yaohui Wang, et al. Internvid: A large-scale video-text dataset for multimodal understanding and generation. arXiv preprint arXivv2307.00944, 2023.
[64] Hongtao WU, Ya Jing, Chilam Cheang, Guangzeng Chen, Jiafeng Xu, Xinghang Li, Minghuan Liu, Hang Li, and Tao Kong. Unleashing large-scale video generative pre-training for visual robot manipulation. arXiv preprini 23/2. 13139, 2023.
[65] Jay Zhangjie Wu, Yixiao Ge, Xintao Wang, Stan Weixian Lei, Yuchao Gu, Yufei Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation. In Proceedings of the IEEF/CVE International Conference Computer Vision, pages 7623-7633, 2023.
13

---

[66] Dianbing Xi, Jiepeng Wang, Yuanzhi Liang, Xi Qiu, Yuchi Huo, Rui Wang, Chi Zhang, and Xuelong Li. Omnivdiff: Omni controllable video diffusion for generation and understanding. arXiv preprini arXiv: 10825, 2025.
[67] Jinbo Xing, Menghan Xia, Yong Zhang, Haoxin Chen, Wangbo Yu, Hanyuan Liu, Gongye Liu, Xintao Wang, Ying Shan, and Tien-Tsin Wong. Dynamicrafter. Animating open-domain images with video diffusion priors. In European Conference on Computer Vision, pages 399-417. Springer, 2024.
[68] Honghui Yang, Di Huang, Wei Yin, Chunhua Shen, Haifeng Liu, Xiaofei He, Binbin Lin, Wanli Ouyang, and Tong He. Depth any video with scalable synthetic data. arXiv preprini arXiv:2410. 10815, 2024.
[69] Zhuoyi Yang. Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang. Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv: 2400:22408,0007,, 2024.
[70] Wangbo Yu, Jinbo Xing, Li Yuan, Wenbo Hu, Xiaoyu Li. Zhipeng Huang, Xiangjun Gao, Tien-Tsin Wong, Ying Shan, and Yonghong Tian. Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis. arXiv preprint arXiv:2409.02045 2024.
[71] Yang Yuanbo, Shao Jiahao, Li Xinyang, Shen Yujun, Geiger Andreas, and Liao Yiyi. Prometheus: 3d-aware latent diffusion models for feed-forward text-to-3d scene generation. 1rxiv:2412.21177, 2024.
[72] Yuanwen Yue, Anurag Das, Francis Engelmann, Siyu Tang, and Jan Eric Lenssen. Improving 2D Feature Representations by 3D-Aware Fine-Tuning. In European Conference on Computer Vision (ECCV), 2024.
[73] Yuanhao Zhai, Kevin Lin, Linjie Li, Chung-Ching Lin, Jianfeng Wang, Zhengyuan Yang, David Doermann, Junsong Yuan, Zicheng Liu, and Lijuan Wang. Idol: Unified dual-modal latent diffusion for human-centric joint video-depth generation. In European Conference on Computer Vision, pages 134-152. Springer, 2024.
[74] Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pages 3836-3844 1836-3847, 2023.
[75] Qihang Zhang, Shuangfei Zhai, Miguel Angel Bautista, Kevin Miao, Alexander Toshev, Joshua Susskind, and Jiatao Gu. World-consistent video diffusion with explicit 3d modeling. arXiv preprint 21X.x:24122.0821.11 2024.
[76] Zangwei Zheng, Xiangyu Peng. Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all. arXiv preprint arXiv:24/2.20444, 2024.
[77] Yuan Zhou, Qiuyue Wang, Yuxuan Cai, and Huan Yang. Allegro: Open the black box of commercial-level video generation model. arXiv preprint arXiv:2410.1542 2024.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our abstract and introduction clearly explain how method introduces geometric regularization into video generation models.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss this issue in Section 5.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
15

---

Justification:
Our method does not make strong theoretical contributions.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We provide detailed implementation information in Section 4.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
16

---

Answer: [NA]
Justification: At this stage, we do not release the code but provide generated video results from our model.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm ec/
for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
//nips. //nips.cc/pll wwcppbblc/guuiicuceccccccoe.oopp /etgguddsse/CeSbbbiiiinn Stt.SSbbbiiiiooooiiiii.iooi for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Yes, these details are thoroughly diseussed in Section 4.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Yes, please refer to the tables in the main text.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
17

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Yes, these details are thoroughly discussed Section 4.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
We have complied with the Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Yes, addressed this in the introduction.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
18

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification:
We do not identify such risk.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Yes, we have added explanations in the relevant sections.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
19

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do release any new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
20

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: Our method does not rely large language models (LLMs) at its core.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy for what should or should not be described.
21


Reviews:
[{"review_id": "IwM7QbZ7FC", "reviewer": "Reviewer_ee6U", "summary": "This paper aims to predict a high-quality 3D reconstruction by introducing geometric regularization into video diffusion models. Specifically, the authors augment VDM with depth prediction, which serves as an implicit form of geometric supervision. In addition, the paper proposes an explicit geometric regularization term that enhances the geometric consistency of the underlying shapes. The proposed approach is validated through experiments on both text-to-video (T2V) and image-to-video (I2V) tasks.", "questions": "The main issues lie in unclear training details (denoising steps) and insufficient experimental comparisons (3D consistency); for details, see the weaknesses.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n1. The proposed geometric regularization provides explicit supervision for video diffusion models, significantly improving both 3D consistency and temporal coherence in generated videos.\n\n2. The paper also introduces an effective model initialization strategy and a multi-stage training framework, which contribute to stable optimization and improved performance.\n\n**Weaknesses**\n\n[MAJOR]:\n\n1. Diffusion models typically require multiple denoising steps to produce clean outputs. During training with geometric regularization, does the model compute the regularization from one-step denoising output or many-step denoising output? What is the rationale behind this design choice?\n\n2. While the method primarily emphasizes 3D consistency in generated videos, a more comprehensive comparison with state-of-the-art approaches like ViewCrafter[1] and DimensionX[2]—specifically in terms of 3D consistency—would further validate its advantages.\n\n[Minor]\n\n3. Is there any quantitative evaluation of the accuracy or performance consistency across different prediction heads? Such an analysis would better demonstrate whether the model effectively leverages geometric regularization.\n\n4. There are several writing errors. For instance, in line 209, “estimate camera poses using MegaSaM” appears to be incorrect.\n\n[1] Yu W, Xing J, Yuan L, et al. Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis[J]. arXiv preprint arXiv:2409.02048, 2024.\n[2] Sun W, Chen S, Liu F, et al. Dimensionx: Create any 3d and 4d scenes from a single image with controllable video diffusion[J]. arXiv preprint arXiv:2411.04928, 2024.", "was_revised": true, "final_justification": "The reply has addressed my concern. I would like to stay positive rating.", "rebuttal": "**Thank you very much for your time and valuable insights. We truly appreciate your recognition of certain aspects of our work—it is highly encouraging. Please find our responses to your comments below.**\n1. **How geometric regularization is incorporated.**\n   At any diffusion step, we can compute the corresponding clean latent representation from the predicted noise using the predefined noise schedule. We apply our geometric regularization directly on computed clean depth latent. Because applying regularization to the clean depth latent is directly effective, and only the clean depth latent can be reliably decoded by our depth decoder into a meaningful depth map. We will provide a more detailed explanation of this mechanism in the revised version of the paper.\n2. **Comparison with DimensionX and ViewCrafter.**\n   Both DimensionX and ViewCrafter focus primarily on improving the **spatial controllability** of video generation. In contrast, our goal is to enable the video model itself to generate **geometry-grounded** videos, which is fundamentally different. Nevertheless, we can still compare the video quality of our method against these approaches, as shown below. (We use WorldScore [1] uniformly for comparison here.) Since these methods introduce explicit camera pose as conditional input, their models exhibit better camera controllability. If necessary, we will include these comparisions in the revised version of the paper.\n\n    | Models        | WorldScore-Static | WorldScore-Dynamic | Camera Ctrl | Object Ctrl | Content Align | 3D Consist | Photo Consist | Style Consist | Subjective Qual | Motion Acc | Motion Mag | Motion Smooth |\n    | ------------- | ----------------- | ------------------ | ----------- | ----------- | ------------- | ---------- | ------------- | ------------- | --------------- | ---------- | ---------- | ------------- |\n    | ViewCrafter   | 45.01             | 11.01              | 88.26       | 48.52       | 58.45         | 78.10      | 69.13         | 66.52         | 59.93           | 2.38       | 1.08       | 17.00         |\n    | AC3D-5B       | 51.14             | 45.12              | 91.01       | 48.14       | 65.08         | 65.48      | 60.61         | 38.61         | 41.17           | 22.34      | 43.74      | 32.41         |\n    | DimensionX    | 59.58             | 47.63              | 93.27       | 39.52       | 36.33         | 82.04      | 82.44         | 81.08         | 61.33           | 63.63      | 25.59      | 54.76         |\n    | CogVideoX-T2V | 54.18             | 48.79              | 40.22       | 51.05       | 68.12         | 68.81      | 64.20         | 42.19         | 44.67           | 25.00      | 47.31      | 36.28         |\n    | Ours-T2V      | 57.38             | 52.03              | 42.84       | 55.49       | 70.61         | 90.88      | 73.64         | 49.71         | 48.88           | 28.66      | 50.63      | 39.33         |\n    | CogVideoX-I2V | 62.15             | 59.12              | 38.27       | 40.07       | 36.73         | 86.21      | 88.12         | 83.22         | 62.44           | 69.56      | 26.42      | 60.15         |\n    | Ours-I2V      | 64.57             | 62.03              | 41.02       | 42.41       | 38.45         | 92.11      | 90.15         | 86.10         | 64.38           | 72.02      | 29.10      | 62.85         |\n\n\n3. **Quantitative evaluation of different prediction heads.**\n\n    3.1 **Camera pose head.**\n    We report the results of our camera head, which predicts camera poses from video latents, on two datasets—CO3Dv2 and RealEstate10K—using the standard AUC@30 metric, which combines Relative Rotation Accuracy (RRA) and Relative Translation Accuracy (RTA). We also compare our results with methods such as VGGT. Since our camera head is distilled from VGGT, it achieves comparable accuracy to the original VGGT estimator.\n\n    | Methods             | Re10K (unseen) AUC@30 ↑ | CO3Dv2 AUC@30 ↑ |\n    | ------------------- | ----------------------- | --------------- |\n    | Colmap+SPSG         | 45.2                    | 25.3            |\n    | DUST3R              | 67.7                    | 76.7            |\n    | VGGT (Feed-Forward) | 85.3                    | 88.2            |\n    | Ours (Feed-Forward) | 84.9                    | 88.9            |\n\n\n    3.2 **Motion probability map head.**\n    Since ground truth motion probability maps are not available, it is difficult to directly assess the correctness of the predicted maps. As our motion probability map design is inspired by MegaSAM and the corresponding head is also distilled from it, we report the MSE difference between the outputs of our motion prediction head and those of MegaSAM’s corresponding head on the DyCheck and Sintel benchmarks, to demonstrate that our predictions are qualitatively similar.\n\n    | Datasets    | MSE       |\n    | ---------- | --------- |\n    | DyCheck    | 0.042     |\n    | MPI Sintel | 0.056     |\n\n\n    3.3 **Depth head.**\n    Our depth head learns a mapping from depth latent to actual depthmaps. As the generated videos do not have ground truth depth, we evaluate the difference between the depth decoded by our depth head and the depth labels predicted by VideoDepthAnything on the DL3DV dataset, as VideoDepthAnything predicted labels are used as training supervision in our method.\n\n\n    | Heads           | MSE   |\n    | --------------- | ----- |\n    | Our depth head | 0.0014 |\n    | Video VAE Decoder | 0.0011 |\n\n4. **Writing errors.**\n\n    Regarding the writing issues, we will carefully revise the manuscript from beginning to end to correct such errors. We sincerely thank you for your careful attention and feedback.\n\n[1] Duan H, Yu H X, Chen S, et al. Worldscore: A unified evaluation benchmark for world generation[J]. arXiv preprint arXiv:2504.00983, 2025."}, {"review_id": "knB3eeNCcP", "reviewer": "Reviewer_8Q1X", "summary": "This paper proposes GeoVideo, a geometry-regularized latent video generation framework that integrates explicit geometric consistency constraints into diffusion-based video synthesis. The key idea is to predict depth and camera pose jointly with RGB frames and enforce multi-frame consistency via a novel cross-frame reprojection loss computed from 3D point clouds. The proposed method is lightweight and modular, making it easy to plug into existing latent diffusion architectures such as CogVideoX and DiT.", "questions": "1. How robust is the geometric supervision when the predicted camera poses are inaccurate? Have you evaluated sensitivity to pose errors?\n\n2. Can the model handle complex dynamic scenes (e.g., fast motion, occlusions)? How does the motion probability map behave in such cases?\n\n3. Have you considered using external pose/depth priors (e.g., from COLMAP or pretrained estimators) to improve supervision or initialization?\n\n4. How does the proposed method compare quantitatively against WVD or UniGeo, which also integrate geometry into diffusion-based pipelines?", "limitations": "- The geometric regularization is limited to self-predicted geometry, which may not be fully reliable under complex motion, low lighting, or ambiguous structures.\n\n- The method assumes rigid camera motion and does not account for object-level 3D dynamics, which limits applicability to highly dynamic scenes.\n\n- The geometric consistency loss is currently applied only as an auxiliary signal and not deeply coupled into the generation process (e.g., via 3D-aware attention mechanisms or geometry-conditioned denoising).", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "--- Strengths\n\n- Well-motivated integration of geometry: Instead of pursuing full 3D scene modeling (e.g., NeRF-like pipelines), the paper proposes a lightweight, practical mechanism to improve structural consistency in video generation using explicit geometric priors.\n\n- Plug-and-play design: The method requires minimal modification to existing latent diffusion models and can be seamlessly integrated into popular T2V frameworks.\n\n- The cross-frame reprojection loss with motion-aware masking is effective in promoting geometric coherence without requiring ground-truth depth or camera pose.\n\n- The paper reports better results on both traditional perceptual metrics (FID, CLIPScore, NIQE) and structural metrics (MVCS, Reprojection Error). The video visualization in the supplemental materials also demonstrate the effectiveness of the proposed method.\n\n--- Weaknesses\n\n- The method relies on predicted depth and camera pose from the same model, which may introduce circular supervision bias. Errors in pose estimation can propagate into the reprojection loss.\n\n- Limited comparison to other recent geometry-aware diffusion models, such as WVD or UniGeo, especially in ablations or direct quantitative analysis.\n\n- No analysis on failure cases or limits of the geometric consistency under highly dynamic or low-texture regions.\n\n--- Reasons of Rating\n\n- Overall, GeoVideo offers a practical, effective, and relatively novel solution to enhancing structural consistency in video diffusion models. While it does not radically redefine the field, its design simplicity, compatibility, and demonstrated gains make it a meaningful contribution. Strengthening comparative analysis and discussing failure cases would significantly increase its impact. Hence, I'm positive for this work.", "was_revised": true, "final_justification": "The rebuttal has addressed my concerns on the robustness of geometry supervision, the performance on complex dynamic scenes, questions on external pose/depth priors, and the comparisons with WVD and UniGeo. I am still positive about this paper after reading the other reviews and the rebuttal. Overall, I think this work offers an effective solution to forster structural consistency in video diffusion models.", "rebuttal": "**Thank you very much for your time and valuable insights. We truly appreciate your recognition of certain aspects of our work—it is highly encouraging. Please find our responses to your comments below.**\n1. **Robustness of geometric supervision.**\n   Yes, our approach can be influenced by the accuracy of the various prediction heads. Although components like our camera head already yield reasonably accurate predictions **(see our response to Question 3)**, the outputs from different heads are inherently challenging to predict with perfect precision. To improve the robustness of our model, we apply geometric regularization **only at locations where the difference between the reprojected depth and the generated depth is sufficiently small**.\n   We can include an ablation showing that removing this tolerance threshold leads to a degradation in generation quality, mainly due to noise from pose errors and other sources. Introducing a simple tolerance threshold proves to be an effective strategy that reveals the strength of our method.\n\n   | Method                   | CLIPScore ↑ | FVD ↓     | SC ↑     | BC ↑     | MS ↑     | SR ↑     | VISC ↑   |\n   | ------------------------ | ----------- | --------- | -------- | -------- | -------- | -------- | -------- |\n   | CogVideoX-5B (T2V)       | 32.30       | 145.3     | 93.8     | 95.1     | 93.2     | 79.4     | -        |\n   | Ours w/o tolerance (T2V) | 32.41       | 141.6     | 94.6     | 95.0     | 95.4     | 86.6     | -        |\n   | **Ours (T2V)**           | **34.25**   | **122.7** | **97.2** | **97.8** | **98.1** | **90.3** | -        |\n   |                          |             |           |          |          |          |          |          |\n   | CogVideoX-5B (I2V)       | 33.42       | 139.8     | 94.6     | 96.4     | 95.9     | 80.5     | 95.2     |\n   | Ours w/o tolerance (I2V) | 33.18       | 136.5     | 95.2     | 96.0     | 95.6     | 87.6     | 95.4     |\n   | **Ours (I2V)**           | **35.02**   | **120.5** | **98.1** | **98.5** | **98.6** | **91.1** | **97.6** |\n\n\n\n2. **Effects of motion probability maps on complex dynamic scenes.**\n   Due to NeurIPS rebuttal policy, we are not allowed to provide visualizations of the motion probability maps during the rebuttal period. As our motion probability head is distilled from MegaSAM, the predictions closely resemble those of MegaSAM. Their motion probability maps have been shown to improve pose accuracy during bundle adjustment in complex dynamic scenes. In our setup, incorporating such motion predictions also helps improve geometric consistency in generated results for complex dynamic scenarios. We believe the dynamic scenes used in our experiments already present a sufficient level of complexity to demonstrate the effectiveness of our method. Quantitative comparison results for dynamic scenes are presented in the main paper, and additional videos illustrating complex dynamic scenes are included in the supplementary material via an anonymous link.\n\n3. **Using external pose/depth priors.**\n   Yes, in fact, we have already used pretrained estimators! Our camera pose decoder is distilled from pretrained estimators (VGGT) and learns to predict poses from generated latents. Here, we add a comparison between our predicted poses from video latents and those estimated by VGGT on two datasets—CO3Dv2 and RealEstate10K—using the standard AUC@30 metric, which combines Relative Rotation Accuracy (RRA) and Relative Translation Accuracy (RTA).  Since our camera head is distilled from VGGT, it achieves comparable accuracy to the original VGGT estimator. Our depth head and motion probability head are also distilled from pretrained estimators—VideoDepthAnything and MegaSAM, respectively—and their predictions achieve accuracy comparable to the original estimators. As for COLMAP, it is too time-consuming to be run jointly with video generation, making it impractical for our pipeline. Moreover, its accuracy is lower than that of the pretrained estimator VGGT we use for distillation.\n\n   | Methods             | Re10K (unseen) AUC@30 ↑ | CO3Dv2 AUC@30 ↑ |\n   | ------------------- | ----------------------- | --------------- |\n   | Colmap+SPSG         | 45.2                    | 25.3            |\n   | DUST3R              | 67.7                    | 76.7            |\n   | VGGT (Feed-Forward) | 85.3                    | 88.2            |\n   | Ours (Feed-Forward) | 84.9                    | 88.9            |\n\n   \n\n4. **Comparison with WVD and UniGeo.**\n   WVD does not provide open-source code, making a comprehensive comparison difficult. Their method only performs joint modeling of additional XYZ channels, without applying any explicit regularization on them. Moreover, their approach is limited to static scenes, cannot generate dynamic videos, and does not support prompt-based video generation. We have cited their work in the *Related Work* section and discussed these differences. Here, we report a comparison on depth estimation metrics as presented in their paper for the NYU, BONN, and ScanNet datasets.\n\n    UniGeo, on the other hand, is a fundamentally different type of method. It focuses on estimating geometry from existing video inputs, while our work is centered on video generation. Although UniGeo has a GitHub repository, they do not provide their model code or pretrained weights, making direct comparison difficult. We instead refer to the depth estimation metrics reported in their paper on the 7Scenes dataset.\n    It is also worth noting that the UniGeo paper was released on arXiv on May 30, 2025—after the NeurIPS submission deadline.\n\n    To enable a fair comparison with the above methods in terms of depth estimation, we made a specific modification to our framework: we concatenate a real video as conditional input with the noise, and let our model reconstruct both the video and the corresponding depth. In this way, our model can function to some extent as a depth estimator. However, it is important to emphasize that our method was **not originally designed for geometry estimation**, but rather for **geometry-aware video generation**. Nevertheless, as shown in the comparison tables below, our method also demonstrates a advantage in depth estimation. That said, with the help of explicit supervision, our model can also learn to predict depth from video input effectively.\n\n\n\n   | Methods | NYU-v2 |         | BONN  |         | ScanNet++ |         |\n   | ------- | ------ | ------- | ----- | ------- | --------- | ------- |\n   |         | Rel ↓  | δ₁.₂₅ ↑ | Rel ↓ | δ₁.₂₅ ↑ | Rel ↓     | δ₁.₀₃ ↑ |\n   | WVD     | 9.7    | 90.8    | 7.0   | 96.4    | 5.0       | 57.2    |\n   | Ours    | 6.1    | 97.2    | 6.8   | 96.7    | 3.2       | 80.6    |\n\n   | Methods | 7Scenes    |      |\n   | ------- | -------- | ---- |\n   |         | AbsRel ↓ | δ1 ↑ |\n   | UniGeo  | 20.69    | 62.2 |\n   | Ours    | 18.42    | 71.1 |"}, {"review_id": "QBCXmPwztd", "reviewer": "Reviewer_6j7J", "summary": "This manuscript introduces geometric regularization into latent video diffusion models to improve spatiotemporal consistency. \nBuilding on a standard DiT backbone, the authors add parallel depth and camera-pose prediction branches so that each generated RGB frame is accompanied by a depth map and estimated camera pose. \nDuring training, they first jointly learn RGB and depth generation, then in a second stage enforce a geometric consistency loss: depth predictions from different frames are back-projected into a shared 3D point cloud and re-projected to each view, penalizing depth misalignment across frames. \nThis two-stage fine-tuning yields videos whose frames not only look realistic but also adhere to a coherent underlying 3D structure. Extensive experiments on both static (DL3DV) and dynamic video datasets show marked improvements in multi-view depth consistency, reprojection error, downstream SfM reconstruction success, and perceptual/video-quality metrics.", "questions": "please refer to weaknesses.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1.By generating depth and poses alongside RGB and enforcing cross-frame depth alignment, GeoVideo addresses temporal shape flicker and structural drift common in 2D-only methods.\n\n2. The manuscript reports improvements on multi-view consistency scores, re-projection error, 3D reconstruction success, and standard perceptual/video metrics, demonstrating both geometric and visual gains.\n\n3. Separating joint RGB-depth learning from geometric regularization stabilizes training and prevents early geometric constraints from harming visual quality.\n\n\nWeaknesses:\n1. The idea of simultaneously generating RGB frames and geometric cues like depth is not entirely new—earlier work such as [1] has already explored joint synthesis of appearance and geometry (e.g., point maps).\n\n2. The evaluation omits several state-of-the-art text-to-3D and video generation methods (e.g., AC3D, Viewcrafter), so it is unclear how GeoVideo compares against the full spectrum of recent 3D-aware approaches.\n\n3. Relying solely on VBench metrics assesses perceptual and temporal coherence but does not directly evaluate 3D world fidelity; incorporating WorldScore [2]  would provide a more complete picture of GeoVideo’s reconstruction capabilities.\n\n\n[1] Zhang Q, Zhai S, Martin M A B, et al. World-consistent video diffusion with explicit 3d modeling[C]//Proceedings of the Computer Vision and Pattern Recognition Conference. 2025: 21685-21695.\n[2] Duan H, Yu H X, Chen S, et al. Worldscore: A unified evaluation benchmark for world generation[J]. arXiv preprint arXiv:2504.00983, 2025.", "was_revised": true, "final_justification": "The authors have solved most of my concern, I stick to the oginiral rating.", "rebuttal": "**Thank you very much for your time and valuable insights. We truly appreciate your recognition of certain aspects of our work—it is highly encouraging. Please find our responses to your comments below.**\n1. **Difference from WVD.**\n   WVD simply models additional XYZ channels jointly, without introducing any specific regularization on those channels. Moreover, their approach is limited to static scenes, cannot generate dynamic videos, and does not support prompt-based video generation. And WVD does not provide open-source code, making a comprehensive comparison difficult. In our paper, we have cited this work in the *Related Work* section and highlighted the differences. The main contribution of our method lies in introducing **geometric regularization** specifically on the additional modeling channels to enhance consistency in video generation.\n\n2. **Comparison with AC3D and ViewCrafter.**\n   Both AC3D and ViewCrafter focus primarily on improving the **spatial controllability** of video generation. In contrast, our goal is to enable the video model itself to generate **geometry-grounded** videos, which is fundamentally different. Nevertheless, we can still compare the video quality of our method against these approaches, as shown below. (See the next question — we use WorldScore uniformly for comparison here.) Since these methods introduce explicit camera pose as conditional input, their models exhibit better camera controllability.\n\n3. **WorldScore Metrics.**\n   WorldScore is a recently proposed benchmark. In fact, some of the metrics we compute—such as MVCS—share the same underlying principles as certain metrics (3D Consist etc.) from WorldScore. Here, we provide a supplementary evaluation of our method on the WorldScore benchmark. If necessary, we will include these metrics in the revised version of the paper.\n\n\n   | Models        | WorldScore-Static | WorldScore-Dynamic | Camera Ctrl | Object Ctrl | Content Align | 3D Consist | Photo Consist | Style Consist | Subjective Qual | Motion Acc | Motion Mag | Motion Smooth |\n   | ------------- | ----------------- | ------------------ | ----------- | ----------- | ------------- | ---------- | ------------- | ------------- | --------------- | ---------- | ---------- | ------------- |\n   | ViewCrafter   | 45.01             | 11.01              | 88.26       | 48.52       | 58.45         | 78.10      | 69.13         | 66.52         | 59.93           | 2.38       | 1.08       | 17.00         |\n   | AC3D-5B       | 51.14             | 45.12              | 91.01       | 48.14       | 65.08         | 65.48      | 60.61         | 38.61         | 41.17           | 22.34      | 43.74      | 32.41         |\n   | DimensionX    | 59.58             | 47.63              | 93.27       | 39.52       | 36.33         | 82.04      | 82.44         | 81.08         | 61.33           | 63.63      | 25.59      | 54.76         |\n   | CogVideoX-T2V | 54.18             | 48.79              | 40.22       | 51.05       | 68.12         | 68.81      | 64.20         | 42.19         | 44.67           | 25.00      | 47.31      | 36.28         |\n   | Ours-T2V      | 57.38             | 52.03              | 42.84       | 55.49       | 70.61         | 90.88      | 73.64         | 49.71         | 48.88           | 28.66      | 50.63      | 39.33         |\n   | CogVideoX-I2V | 62.15             | 59.12              | 38.27       | 40.07       | 36.73         | 86.21      | 88.12         | 83.22         | 62.44           | 69.56      | 26.42      | 60.15         |\n   | Ours-I2V      | 64.57             | 62.03              | 41.02       | 42.41       | 38.45         | 92.11      | 90.15         | 86.10         | 64.38           | 72.02      | 29.10      | 62.85         |"}, {"review_id": "W9j7hqCxWZ", "reviewer": "Reviewer_jMr7", "summary": "this paper proposes a method to predict depth video, camera parameters, and motion probability along with the rgb videos. by using the additional information, an additional loss term that measures the 3d consistency is defined and applied, thus improving the 3d consistency of the generated video.", "questions": "1. [critical] please provide more details on the 1000 test videos in Table 3, which i think is very important but is missing.\n2. [critical] please include more qualitative results to show the effectiveness of proposed method, with various inputs. the cases shown in the paper (as in Figure 3,4,5) are over simple for a practical video generation applications.\n3. [less critical] why does not perform a complete vbench evaluation benchmark? i am curious about the performance of other metrics.\n4. [less critical] i notice that in Table 1 and 3, the performance of \"ours w/o L_geo\" is already better than the baseline. is there any case where \"ours w/o L_geo\" performs better than \"ours\"? is there any case where the baseline performs better than \"ours w/o L_geo\"?", "limitations": "[critical] the Limitation section is quite brief. please include the negative influence of apply such a artificial loss term.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "strengths:\n1. the idea is quite simple and straightforward, and the paper is easy to read.\n\nweaknesses:\n1. the experiments are less convincing. for example, the authors collected a dataset for training and evaluation for dynamic video generation. as the data has a critical influence on the experimental results, i am curious about the fairness of the experimental results as shown in Table 3.\n2. forcing such an loss may introduce some negative influences on the generation performance, but there is no such analyses.\n\nas i have major concerns on the experiments, i would suggest borderline reject for now.", "was_revised": true, "final_justification": "the latest rebuttal resolved most of my concerns, and i raised my score accordingly.", "rebuttal": "**Thank you very much for your time and valuable insights. We truly appreciate your recognition of certain aspects of our work—it is highly encouraging. Please find our responses to your comments below.**\n\n   1.**Details about the test videos.**\n   Due to the lack of high-resolution dynamic scenes in most public video datasets, we collected video data from websites such as Pexels for our experiments. It is difficult for us to list the exact source of each individual video, but we can roughly provide the distribution of the test videos as follows.\n\n| Main Category        | Subcategory                            | % of Total | Notes                                      |\n| -------------------- | -------------------------------------- | ---------- | ------------------------------------------ |\n| Entertainment        | Music videos / performances            | 7.3%       | Official MVs, covers, busking, instruments |\n|                      | Comedy skits /edits                    | 6.6%       | Internet humor, reactions         |\n|                      | Gaming streams / montages              | 3.7%       | FPS, Minecraft, mobile games               |\n|                      | Movie clips / fan edits / reviews      | 6.4%       | Clips, trailers, commentary                |\n| People & Lifestyle   | Daily vlogs / personal storytelling    | 6.4%       | Travel, routines, life updates             |\n|                      | Social challenges / micro-dramas       | 3.8%       | Pranks, street interviews, short skits     |\n|                      | Beauty / fashion / routines            | 4.1%       | Makeup, hauls, get-ready-with-me           |\n|                      | Family / pets / home life              | 5.4%       | Baby clips, couples, dogs and cats         |\n| Education & Tech     | How-to tutorials (tech/art/DIY)        | 5.8%       | DIY, painting, home repairs                |\n|                      | Science / explainer videos             | 2.9%       | Space, biology, psychology                 |\n|                      | Product reviews / unboxings            | 2.6%       | Phones, gear, tech demos                   |\n| Nature & Travel      | Landscape / drone                      | 5.4%       | Mountains, oceans, cityscapes              |\n|                      | Underwater / macro / slow motion       | 2.9%       | Fish, ice, flowers, insects                |\n|                      | Wildlife / animal behavior             | 4.9%       | Birds, tigers, elephants, insects          |\n|                      | Road trips / hidden gems               | 2.4%       | Nature trails, rural areas                 |\n| Urban & Architecture | City walking / night photography       | 2.7%       | POV walks, rain, lights                    |\n|                      | Buildings / design / interiors         | 1.8%       | Real estate, modern design                 |\n|                      | Street food / markets / urban culture  | 2.2%       | Food stalls, crowds, culture               |\n| Fitness & Movement   | Yoga / workouts / wellness             | 2.8%       | Gym sessions, meditation                   |\n|                      | Dance / choreography / freestyle       | 2.3%       | K-pop, street, experimental                |\n|                      | Extreme sports / climbing / skating    | 2.1%       | solo stunts, aerials                |\n| Food & Cooking       | Cooking demos / recipes                | 2.9%       | Fried rice, baking, plating                |\n|                      | Mukbang / ASMR eating                  | 1.7%       | Eating sounds, visual food focus           |\n| Business / Work      | Ad campaigns / product shots           | 2.9%       | Branded short ads, slow-motion products    |\n|                      | Offices / team meetings / interviews   | 1.3%       | Corporate scenes, recruitment              |\n|                      | Exhibition videos                      | 1.4%       | Real estate/auto show videos               |\n| Religion / Rituals   | Weddings / festivals / spiritual clips | 2.2%       | Church, temples, cultural celebrations     |\n| Other / Niche        |                                        | 3.1%       |                                            |\n\n\n\n   2. **More qualitative results.**\n   While you may have already noticed this, we would still like to kindly remind you that, in addition to the figures shown in the main paper, we have also provided additional video results via an **anonymous link in the supplementary material.** Due to NeurIPS's rebuttal policy, we are not allowed to provide further visualizations at this stage, but the results in the supplementary material do include some complex scene cases.\n\n   3. **Complete VBench metrics.**\n   Since many metrics in VBench are not directly related to geometric properties, we primarily showcase those that are geometry-related in the main paper. Here, we provide the complete set of metrics for reference. If necessary, we will include these metrics in the revised version of the paper.\n\n   | Models        | Subject Consistency | Background Consistency | Temporal Flickering | Motion Smoothness | Dynamic Degree | Aesthetic Quality | Imaging Quality | Object Class |\n   | ------------- | ------------------- | ---------------------- | ------------------- | ----------------- | -------------- | ----------------- | --------------- | ------------ |\n   | CogVideoX     | 93.78%              | 95.08%                 | 98.73%              | 93.17%            | 66.13%         | 61.97%            | 64.74%          | 85.12%       |\n   | Ours          | 97.22%              | 97.79%                 | 99.46%              | 98.11%            | 87.74%         | 63.89%            | 68.22%          | 94.34%       |\n\n   | Models        | Multiple Objects | Human Action | Color   | Spatial Relationship | Scene  | Appearance Style | Temporal Style | Overall Consistency |\n   | ------------- | ---------------- | ------------ | ------- | -------------------- | ------ | ---------------- | -------------- | ------------------- |\n   | CogVideoX     | 69.88%           | 96.57%       | 87.30%  | 79.43%               | 51.17% | 22.38%           | 25.38%         | 27.12%              |\n   | Ours          | 85.93%           | 99.20%       | 90.83%  | 90.33%               | 61.42% | 22.53%           | 25.82%         | 28.21%  |          \n\n4. **Cases where geometric regularization performs worse.**\n   Such cases may exist, for example, as mentioned in our limitation section, for phenomena like fire and smoke. These objects can *somehow* be estimated with depth, and are therefore modeled with corresponding depth in our generation process. However, the addition of the geometric loss term $L_{geo}$ might reduce the dynamic characteristics of such videos compared to *ours w/o* $L_{\\text{geo}}$. These non-rigid, volumetric phenomena may require more sophisticated mechanisms, which are currently beyond the scope of this study. Due to current constraints, we are unable to provide visualizations and can only describe the results verbally. We appreciate your understanding. Nevertheless, based on the average quantitative metrics across most scenarios, our geometric regularization still **shows improvement for the majority of video generation scenarios.**\n\n   In most cases, *ours w/o* $L_{\\text{geo}}$ already outperforms baseline methods. This is because it introduces additional channels with minimal interference to the original pretrained model. Even when depth is not semantically essential to the video content, modeling depth as auxiliary information can still enhance temporal consistency, as depth labels exhibit strong temporal coherence.\n\n\n5. **More comprehensive limitations.** While our method incorporates geometric reasoning to improve the consistency of video generation, it still exhibits some limitations. First, the concept of \"video\" spans a wide range of visual content, and our approach struggles in scenarios where geometry is ill-defined or not meaningful—such as smoke, fire, reflections, or 2D animations—which often leads to suboptimal outputs. Second, although the geometric loss term helps enforce spatial coherence, it may also impose rigid constraints that suppress plausible appearance changes or motion patterns inconsistent with the assumed geometry. This can reduce the diversity or realism of generated results in cases that fall outside standard 3D assumptions. Lastly, the model may not generalize well to out-of-distribution inputs, particularly those involving visual phenomena that cannot be effectively captured by depth-based representations. Addressing these challenges may require more flexible or content-aware mechanisms that can selectively apply geometric priors based on scene characteristics."}]

Author rebuttal:

Meta-review: This work makes a valuable contribution to video generation by demonstrating that explicit geometric supervision can significantly improve temporal consistency without sacrificing visual quality. The technical approach is sound, the experimental validation is comprehensive, and the practical benefits are clear. The consistent positive reviewer feedback and successful resolution of all concerns through detailed rebuttals support acceptance. The plug-and-play nature and demonstrated improvements across multiple metrics make this a valuable addition to the video generation toolkit, with broad applicability to existing frameworks.
Decision: Accept (poster)