3D Scene Prompting for Scene-Consistent Camera-Controllable Video Generation
Authors: JoungBin Lee
Jaewoo Jung
Jisang Han
Takuya Narihira
Kazumi Fukuda
Junyoung Seo
Sunghwan Hong
Yuki Mitsufuji
Seungryong Kim
Venue: ICLR 2026 Poster (2026)
Keywords: Scene-Consistent Video Generation; Camera-Controllable Video Generation; Video Diffusion Models;
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: We present 3DScenePrompt, a framework for camera-controllable video generation that maintains scene consistency when extending arbitrary-length input videos along user-specified trajectories. Unlike existing video generative methods limited to conditioning on a single image or just a few frames, we introduce a dual spatio-temporal conditioning strategy that fundamentally rethinks how video models should reference prior content. Our approach conditions on both temporally adjacent frames for motion continuity and spatially adjacent content for scene consistency. However, when generating beyond temporal boundaries, directly using spatially adjacent frames would incorrectly preserve dynamic elements from the past. We address this through introducing a 3D scene memory that represents exclusively the static geometry extracted from the entire input video. To construct this memory, we leverage dynamic SLAM with our newly introduced dynamic masking strategy that explicitly separates static scene geometry from moving elements. The static scene representation can then be projected to any target viewpoint, providing geometrically-consistent warped views that serve as strong spatial prompts while allowing dynamic regions to evolve naturally from temporal context. This enables our model to maintain long-range spatial coherence and precise camera control without sacrificing computational efficiency or motion realism. Extensive experiments demonstrate that our framework significantly outperforms existing methods in scene consistency, camera controllability, and generation quality.

Published as a conference paper at ICLR 2026
3D SCENE PROMPTING FOR SCENE-CONSISTENS CAMERA-CONTROLLABLE VIDEO GENERATION
JoungBin Lee'
Jaewoo Jung Jisang Han' Takuya Narihira²
Kazumi Fukuda²
Junyoung Seo!
Sunghwan Hong Yuki Mitsufuji Seungryong Kim
KAIST AI
Sony Al
'ETH Zürich (CVG, PRS)
*Sony Group Corporation
$ ETH Center
https: /tpp:://cvlaas weoolabbkaiiste wtt.kkaittstt..iii............ wtt.h//b...e/SSeeeeeeeooooo
Input Video
GT Future Video
i
point cloud from static regions
Temporal consistency from last few frames
Spatial consistency from projected point clouds to specified Cam. Traj.
Condition
specified Trai
Generatel
Input
Cam. Trs).
Future Video
Figure 1: Teaser. Our framework generates the next video chunk that follows a user-specified cam- trajectory while maintaining scene consistency. Our dual spatio-temporal conditioning jointly leverages the last few frames to ensure temporal continuity and the rendered point cloud to enforce spatial consistency.
ABSTRACT
We present 3DScenePrompt, a framework that generates the next video chunk from arbitrary-length input video chunk while supporting highly complex and pre- cise camera control and preserving scene consistency. Unlike previous methods conditioned a single image short clip, our approach employs dual spatio- temporal conditioning that reformulates context-view referencing across the en- condition both lemporally adjacent frames
tire input video. Specifically, ensure motion continuity and spatially adjacent content to preserve scene consis- tency, enabled by 3D scene memory that exclusively represents the static geome- try extracted from the full input sequence. To construct this memory, we leverage dynamic SLAM with a newly introduced dynamic masking strategy that explic- itly separates static scene geometry from moving elements. The resulting static representation can then projected to arbitrary target viewpoints, providing ge- ometrically consistent warped views that act strong 3D spatial prompts, while allowing dynamic regions to evolve naturally from temporal context. This design allows model maintain long-range spatial coherence and precise camera control without compromising computational efficiency or motion realism. Exten- sive experiments demonstrate that framework significantly outperforms exist- ing methods in scene consistency, camera controllability, and generation quality.
"qqual contribution. Co-corresponding authors.

---

Published as a conference paper at ICLR 2026
INTRODUCTION
caatoalcontrolllale video generation (He et al. 2024 Wang et al., 2024bb Jin et al. 2025 Hong jet al., (023) aims to synthesize videos following user-specified camera trajectories while maintain- ing visual coherence and temporal consistency. Recent advances have progressed from generating entirely new videos with controllable viewpoints (Bahmani et al.] 2025a) to enabling users to ex- tend single image short video clips along desired camera paths (He et al. 2024 Agarwal al., 2023]. Yet these methods share a fundamental limitation: they can only process extremely short con- ditioning sequences, typically just a few frames, which constrains their ability to understand longer videos and hence fails to preserve the rich scene context present in those longer videos. What if we could provide a model with arbitrary-length video sequences and generate continuations that not follow precise camera controls but also maintain scene consistency with the entire input? Such technology, which we refer to as ccene-consistent
camerr-controllable video generation, has im- mediate applications in film production (Zhang et al., 2025). virtual reality (He et al. [022b Hong al. 2024b; Cho et al. 2024), robotics Yoonn al.
2025). and synthetic data generation (nnapp
& Bohacck 2025).
Scene-consistent camera-controllable video generation poses three intertwined challenges that must be solved jointly. First, static and dynamic elements must be handled differently: while static scene elements should remain consistent throughout generation, dynamic elements such as moving ob- jects and people should evolve naturally from their most recent states rather than rigidly preserving motions from the distant past. Second, camera control demands understanding the underlying 3D geometry of the scene: the generated content must respect physical constraints, properly handle occlusions, and seamlessly compose dynamic elements onto static geometry, while extrapolating plausible content for previously unobserved regions. Third, these capabilities must be achieved within practical computational constraints, as naive approaches that process all input frames quickly become intractable when the input video sequence is long.
How can we tackle this challenging task by leveraging existing video generative models? Our key insight comes from fundamentally rethinking how video models should reference prior content. Cur- rent image-to- viveo (Yang et al. (024) and video-to-future coo-ffuure-videe -viuoo models (Agarwal et al. 2025) achieve realistic generation by conditioning on temporally adjacent frames maintain short-term consistency and motion continuity. However, adjacency in video is not purely temporal-it can also be spatial. When generating scene-consistent videos, the frames we synthesize may be spatially adjacent to frames from much earlier in the input sequence, particularly when the camera revis- its similar viewpoints or explores nearby regions. This dual nature of adjacency suggests a new conditioning paradigm that leverages both temporal and spatial relationships.
Based on these motivations, we propose 3DScenePrompt, a novel video generation framework de- signed for scene-consisten camera-controllabbe video synthesis. It takes arbitrary-eenthh video as context and generates the future video that is consistent with the scene geometry of the context video. The key innovation lies in our dual spatio-temporal conditioning strategy: the model con- ditions on both temporally adjacent frames (for motion continuity) and spatially adjacent frames (for scene consistency). However, an important consideration for spatial conditioning for our task is that it must provide only the persistent static scene structure while excluding dynamic content, as directly conditioning on spatially adjacent frames from the past would incorrectly preserve dynamic elements. To enable this without temporal contradictions, we construct a 3D scene memory that represents exclusively the static geometry extracted from the entire input video.
To construct this 3D scene memory from dynamic videos, we leverage recent advances in dynamic SLAM frameworks (Zhang al., 2022 Han et al., 2025a An et al. 2025 Han et al., 2025b Zhang et al.) 2024] Li et al. (024)) to estimate camera poses and 3D structure from the input video. To extract only the static regions from the estimated 3D structure, we introduce a dynamic masking strategy that explicitly separates static elements and moving objects. The static-only 3D represen- tation can then be projected to target viewpoints, yielding coometrically-coniittee that serve spatial prompts while allowing dynamic elements to evolve naturally from temporal context alone. Surprisingly, the integration of 3D scene memory provides an additional benefit: the geometrically-consitten warped views provide rich visual references that significantly reduce
warped views
'Throughout our paper, fottc--to-uuuur-viiiee models refer to models that are capable of generating the subsequent frames of the given input video (e.g., coamos -oroodi-prddd...o ct2 (Agarwal et al. 2025)
2

---

Published as a conference paper at ICLR 2026
Input video
video video FITTI
Input video
Last frame Caption Traj.
Last few frames Caption
few frames
Tral, Caption Traj.
Future video
Future video
Future video
(a) Camera control video generation
vidu-to-futurre-iide generation
(c) Ours
Figure 2: Comparison to existing architectures. (a) Camera-controlllble video generation meth-
ods condition on single frame and camera trajectory (Wang al.) 2024b He
2024 Jin al.
2025). (b) The diteo-t-ffutrreviin
generation method (Agarwal et al. 2025 leverages the last
few frames of the input video to preserve temporal continuity future video generation: however, it fails to maintain long-term spatial consistency when revisiting previously defined viewpoints that are not visible in the given frames. Unlike these, (c) our approach combines temporal conditioning (last few frames) with spatial conditioning (spatially adjacent frames) to achieve scene-consistent generation with precise camera control.
uncertainty in viewpoint manipulation. enabling precise camera control without any other explicit camera conditioning.
In summary, 3DScenePrompt enables both accurate camera control and long-range spatial consis- tency by treating the static scene representation as a persistent spatial prompt that guides generation across arbitrary timescales. Extensive experiments demonstrate that our framework significantly outperforms existing methods in maintaining scene consistency, achieving precise camera control, and generating high-quality videos from arbitrary-length inputs.
RELATED WORK
Single-frame conditioned camera-controllable video generation.
Building upon the recent suc-
cess of video diffusion models (Blattmann et al. 2023 Guo et al. [023 Yang et al. 2024 Runway 2024 Kim et al.) 2025 Brooks al.] 2024). recent works (He al. 2024 Wang et al. 2024b
Bahmani et al. 2024) have achieved camera-controllable video generation by introducing additional adapters into U-Net-based video diffusion models that accept camera trajectories. For instance, CameraCtrl and VD3D (Bahmani al. 2024, He et al., 2024) incorporate spatiotemporal camera embeddings, such as Plücker coordinates, via ControlNet like mechanisms (Zhang et al., 2023). While these methods enable precise trajectory following, they only condition single starting im- ages, lacking mechanisms to maintain consistency with extended video context. In contrast, our approach enables leveraging entire video sequences as spatial prompts through 3D memory struction, enabling conssscon generation that preserves the rich scene context within arbitrary- length inputs.
Multi-frame conditioned camera-controllable video generation. Recently, CameraCtr12 (He jct al. 2025a) and Seaweed-APT2 (Lin et al. (0255) have proposed to take multiple frames as condition for camera-controllable vidco generation. This allows the generated videos main- tain temporal smoothness with the provided frames, enhancing the motion fidelity of the generated videos. However, these methods only consider temporal adjacency, which restricts the model from maintaining scene-consistencies with long videos due to memory constraints. In contrast, we intro- duce SLAMM process the conditioning video to consider dual spatio-temporal adjacency, enabling the model to maintain scene-consistency with long videos under efficient computation.
Geometry-ee grounded video generation. Recent works (Ren et al., 2025 Yu et al. 2025 Seo
et al.) 2025) have integrated off-the-shelf geometry estimators Li et al. (2024); Wang al. (2025) into video generation pipelines to improve geometric accuracy. Gen3C (Ren et al. 2025). for in- stance, similarly adopts dynamic SLAM to lift videos to 3D representations. However, these meth- ods exclusively address dynamic novel view synthesis gaatheiis-ggneratinng new viewpoints within the same temporal window the input. This constrained setting allows them to simply warp entire scenes
3

---

Published as a conference paper at ICLR 2026
Static Point Cloud
surplinees
Dynamic Video
mounntain frrl brolpack
Cow
Nort Sert Trames
Noise
Wil Decoder
Inpe
DIT
Rendieeed
Point Cloud
VAEE Emcrder
Patchaay
Spotially Arjpcent.rrrmes
Figure 3: Overview of 3DoeenePrompt framework. To generate the next chunk video that remains spatially consistent with the input video, we design dual -patio-temporal conditioning pipeline to extract the most relevant information from the input video. The last few frames are utilized to provide temporal conditioning, ensuring motion continuity between conditioning inputs and the generated frames. In paralicl, for the spatial conditioning, we first select the most representative frames from the input sequence, lift their static regions into 3D point cloud using the dynamic mask, and render it along a user-specified camera trajectory to preserve scene geometry.
without distinguishing static and dynamic elements. Our work fundamentally differs by generat- ing content beyond temporal boundaries, requiring selective masking of dynamic regions during 3D construction-4 critical challenge that emerges only when static geometry must persist while dynamics evolve naturally into the future.
Long-horizon scene-consistent generation. Various approaches attempt scene-consistent long video generation through different strategies. ReCamMaster (Bai et al., 2025) and Trajecto- ryCrafter (Yu et al. 2025) interpolate frames or construct 3D representations but remain confined to the input's spatiotemporal coverage, essentially performing dynamic novel view synthesis. Star- Gen (Zhai et al. 2025) scales to long trajectories but assumes static worlds, eliminating temporal dynamics entirely. DFoT (Song et al. 2025) most closely relates to our work, proposing guidance methods that condition on previous frames for scene consistency. However, DFoT also faces fun- damental memory constraints when processing extended sequences, limiting its ability to maintain long-range spatial coherence. Our dual spatio-temporal strategy with SLAM-based spatial memory overcomes these limitations by selectively retrieving only the most relevant frames, both tempo- rally and spatially, enabling computationally efficient processing of arbitrary-length videos while maintaining both motion continuity and scene consistency.
3 METHODOLOGY
3.1 PROBLEM FORMULATION AND MOTIVATION
We address the task of scene-consistent camera-controllable video generation: given dynamic video Vn E R xxWxxxxxxxxx x3 of arbitrary length L as context with height H and width W, our goal to generate T' subsequent frames Vout E Txx*xxW> that follow desired camera trajectory
C = (C,) while maintaining consistency with the scene captured in the context input:
V F(V i:: T,C),
where C, SB(3) represents camera extrinsic matrices and T a text prompt when a video gener- ator F(.) is based on pretrained text-to-video priors (Yang al. 2024 Bahmani et al. 2025a
Comparison to existing solutions. This task fundamentally differs from existing video generation paradigms. Existing camtralco generation methods (He et al. 2024 Wang al. 2024b Bahmani et al. 2024) synthesize videos following user-specified trajectories but only condition on single image Iet or plain text T (Fig. (a)):
V Vour F(Lrf, T, C), or V = F(T.C),
which is insufficient for our task, where the entire underlying 3D scene of the context video should be considered. In contrast, video-to-futrr-viidee generation methods such as
4

---

Published as a conference paper at ICLR 2026
Cosmos-predic ct-2 (Agarwal al. 2025 g(-) employ temporal sliding windows to generate future frames (Fig. (b)):
V out I(Vm L
where Vin - L] for w & L represents a small overlap window, typically consisting of the last few frames of Vin- Although this design encourages temporal smoothness by providing the last few frames when generating the future video, it often fails to preserve long-term spatial consistency when the camera revisits regions not covered by the small window w.
3.2 TOWARDS SCENE-CONSISTENT CAMERA-CONTROLLINN VIDEO GENERATION
The key challenge of scene-consistent camera-controllable video generation lies in reconciling two competing requirements: maintaining consistency with potentially distant frames that share spatial proximity (when the camera returns similar viewpoints). while evolving dynamic content naturally from the recent temporal context. Ideally, conditioning on all frames Vin would ensure optimal global spatial consistency. However, this quickly becomes impractical as the sequence grows, since standard self-attention incurs quadratic time/memory in the sequence length.
Dual spatio-temporal sliding window strategy. Instead of increasing the temporal window size w of the existing video-to-future-vide generation methods, we introduce a dual sliding window strategy that conditions on frames selected along both temporal and spatial axes (Fig. (c)) Beyond
the standard temporal window that captures recent motion dynamics, we add spatial window that retrieves frames sharing similar 3D viewpoints, regardless of their temporal distance:
V Vour F(Vin. in, TJC,, where V in (Temporal(w)) (S))
(4)
where the model F generates a future sequence V Voar conditioned on Tempora(ww). last w frames of entire input sequence based on viewpoint similarity to the target viewpoint C. This dual conditioning enables the model reference distant frames that observe the same spatial regions, maintaining scene consistency without processing all L input frames.
the input video Vn [L L]. and (patial(T),, the T retrieved frames from the
While this dual conditioning is conceptually appealing, naïvely retrieving and providing spatially adjacent frames directly would be problematic for our task. Since we aim to generate future con- tent beyond the input's temporal boundary, directly conditioning on frames from earlier timestamps would incorrectly preserve dynamic elements (e.g., walking person from frame 50 should not nec- essarily reappear at that same location when generating frame 200). The spatial conditioning must therefore provide only the persistent scene structure while excluding dynamic content. Rather than retrieving individual frames, we introduce a 3D scene memory M that represents exclusively the static geometry extracted from all spatially relevant frames.
3.3
3D SCENE MEMORY CONSTRUCTION
Our 3D scene memory must efficiently encode spatial relationships across all L frames while ex- tracting only persistent static geometry. To construct the 3D scene memory. we leverage dynamic SLAM frameworks et 2024 Zhang et al. 2024) to estimate camera poses and reconstruct 3D) structure:
(C,P) = (vn).
(5)
where Č= (C:H., are the estimated camera poses, represents the aggregated 3D point cloud from the L input frames, and D-)) represents the dynamic SLAM framework. This SLAM integration effective in that it not only estimates the camera parameters of the input frames but also reconstructs the 3D structure of the scene, which can be further utilized to represent the 3D static geometry.
While the camera poses enable efficient spatial retrieval by comparing viewpoint similarity with the target trajectory C, the aggregated 3D point cloud P still contains both static and dynamic regions. Thus, we now explain our full pipeline on how to identify dynamic regions and only maintain the persistent statie geometry of the input video.
5

---

Published as a conference paper at ICLR 2026
(a) w/o mask
(b) w masic
T=0
T-20
T
Figure 4: Illustration of dynamic masking for static scene extraction. When aggregating 3D points across frames, moving objects create ghosting artifacts if not properly masked. (a) Without masking, dynamic elements (horses and riders) appear frozen at multiple positions, severely degrad- ing the warped views. (b) With our dynamic masking pipeline, these elements are identified and excluded, resulting in clean static-only point clouds that can be reliably warped to new viewpoints.
Dynamic masking for static scene extraction. Naively aggregating points across frames creates ghosting artifacts where moving objects appear frozen multiple positions, as shown in Fig. (a). We address this through comprehensive three-stage masking pipeline that identifies and excludes all dynamic content as depicted in Fig.
We begin with pixel-level motion detection following MonST3R ZZang et al. 2024). For each frame pair, we compute optical flow using SEA-RAFT (Wang et al.
2024a) (FloWoptisa) and compare
it against the flow induced by camera motion alone (Flowwarp).
Regions where the L1 difference
exceeds a specific threshold r are marked as potentially dynamic:
gixel
FloWoptical Flowwarpll > T]
(6)
However, pixel-level detection captures motion only at specific instants and misses complete object boundaries. We therefore propagate these sparse detections to full objects using SAM2 (Ravi al. 2024), where we sample points from dynamic pixels in the first frame for prompts. Yet this approach still has limitations: static objects that begin moving in later frames may not be detected if they appear static initially.
Our solution employs backward tracking with CoTracker3 (Karaev et al. 2024) to aggregate motion evidence across the entire sequence. From the sampled points in each frame obtained from our pixel-level motion detection, we track these points from all frames back t 0, capturing motions of objects that move any point. These aggregated points are used to prompt the final SAM2 pass, producing complete object-level masks M, that cleanly separate all dynamic content (Fig. 4 (b)).
With the full dynamic mask, we can now obtain the static-only 3D geometry P Pstatic.
P static (1 ",
From the constructed static-only 3D geometry P Psuati with our proposed dynamic masking strategy, now obtain the 3D scene memory:
M (Č, statie))
where we now explain how this 3D scene memory M can be used for scene-consistent camera- controllable video generation in the following section.
3.4 3D SCENE PROMPTING
Having constructed the static-only representation P Patatie rather than naively retrieving frames from the input video based on viewpoint similarity, we synthesize static-only spatial frames through the projection of P static- For each target camera pose C, E C. we generate the corresponding spatial frame by projecting the static points from the most spatially relevant input frames:
Spatial(t) TTKK- C. [(n)
(9)

---

Published as a conference paper at ICLR 2026
Dynamic thresholding
Dynamic aggregation
Input video
Optical flow
Flow mask
Point sampling
BW tracking
Propagation
Dynamic mask
Figure 5: Dynamic masking strategy. A three-stage pipeline refines dynamic region detection
produce complete object-level masks: (1) optical-flow differences detect pixel-level motion (Dy- namic thresholding); (2) sample points from these regions for all frames and perform backward tracking (BW tracking) with CoTracker3 (Karaev et al. 2024) to aggregate motion evidence across all frames back to 6-0 (dynamic aggregation), capturing objects that move at any time; (3) prop-
agate aggregated points in the first frame to the entire video using SAM2 (Ravi et al. 2024). The resulting dynamic masks cleanly separate moving elements (people, objects) from the static back- ground, enabling construction of the static-only point cloud P static:
where p(n) static P static contains points from the top-n spatially adjacent input frames C, II(() de- notes perspective projection, and K is the camera intrinsic matrix. The complete spatial condition- ing becomes ST"t (Spatial(t))?
R where spatial adjacency is calculated
by field-of-viev overlap.
This projection-basse approach ensures only static content appears in conditioning while providing geometrically consistent views aligned to target poses. Notably, the static point cloud aggregates in- formation from multiple viewpoints, potentially filling regions occluded by dynamic objects. These projected views serve as 3D scene prompts that provide explicit guidance about persistent scene structure, enabling precise camera control without additional encoding modules.
The projected views Spatial(T') serve what term 3D scene prompts-they provide the model with explicit guidance about the persistent scene structure. By conditioning on both Temporal (w) and (T), framework effectively enables scene-consistent camera-controllable video gen- eration with computational efficiency while preserving the prior for high-quality video synthesis.
EXPERIMENTS
4.1
IMPLEMENTATION DETAILS
Model architecture. We build upon CogVideoX-I2V-SB Yangg et al. 2024), extending its single- image conditioning accept dual spatio-temporal inputs with minimal architectural changes. The key modification is repurposing the existing image conditioning channel to accept concatenated latents from both temporal frames and spatial projections. Specifically, we provide the w frames from Vn as temporal conditioning and T projected views from the static point cloud as spatial conditioning. This enables the DiT backbone to remain entirely unchanged, preserving all pretrained video priors. Both conditions are encoded through the frozen 3D VAE and concatenated channel-wise such that Zeund = (Tenoat(Tmmprra((e)) (pptial))
Fine-tuning. We fully fine-tune the model for total of 4K iterations with batch size of 8 using H100 GPUs, which required approximately 48 hours. We used the 16-bit Adam optimizer with a learning rate of x 10 and adopted the same hyperparameter settings those used in the training of CogVideos VideoX (Yang et al. 2024). For the temporal sliding window, we provide the last frames of the input video, setting w For the projection of top-n spatially adjacent views,
7.

---

Published as a conference paper at ICLR 2026
RealEstate IOK
DynPose-1000
Methods
PSNR
SSIM
LPIPS
MEB3
PSNR
SSIMI
LPIPS
MEB3R
DFoT (Song et al. 2025 3DScehePrompt (Ours)
18,3044
0.5960
0.3077
0.181164
12.1471
0.3040
0.4172
0.183202
20.8932
0.7171
0.2120
0.040843
13.0468
0.3666
0.3812
0.124189
Table 1: Evaluation of spatial and geometrie consistency. We compare DFoT and our framework on the RealEstateI0K (Zhou et al.) 2018) and DynPose- 100K (Rockwell et al. 2025) datasets. For spatial consistency, we evaluate PSNR, SSIM, and LPIPS on revisited camera trajectories, while for geometric consistency, report the ME13R (Asim al. 2025 metric.
Experimental settings. We evaluate our method across four key aspects: camera controllability, video quality, scene consistency, and geometric consistency. Since no prior work directly addresses scene-consistent contoaa- video generation, we compare against two categories of base- lines: (I) camera-controllable methods (CameraCtrl (He et al., 2024), MotionCtrl (Wang et al.] 2024b), FloVD (Jin al.) 2025), AC3D (Bahmani et al. (0025a) for camera control and video qual- ity metrics, and (2) DFoT (Song et al. 2025). which attempts scene-consistent camera-controllabl
generation, for spatial and geometric consistency metrics.
We primarily evaluate on 1,000 dynamic videos from DynPose- 100K (Rockwell et al. 2025 For scene consistency evaluation, we additionally test 1,000 static videos from Real 10K ctt al. 2018), as static scenes provide clearer spatial consistency assessment.
T0K (7hou
4.2 SCENE-CONSISTENT VIDEO GENERATION
As mentioned Section 3.1 one of the unique and key challenges in scene- consistent camera-controllable video generation is maintaining spatial consistency extended du- rations. From given input video, we evaluate spatial consistency by generating camera trajectories that revisit the viewpoints in the given video. By matching frames in the generated video and the in- put video that share the same viewpoint, we calculate PSNR, SSIM, and LPIPS. For RealEstate 10K, we evaluate the whole image, whereas we only evaluate the static regions by masking out the dy- namic regions for HonPose-100K, We also assess geometric consistency using Met3R (Asim et al.) 2025), which measures multi-view alignment of generated frames under the recovered camera pose.
Evaluation Protocol.
Results. As shown in Tab. D 3DScene significantly outperforms DFoT across all metrics for both static and dynamic scenes. Most notably, our ME13R evaluation error drops 77% (0.041 vs 0.181), demonstrating superior multi-view geometrie alignment. While DFoT similarly tack- les sceme-consistent camera-contio comera--ontrollable video generation through history guidance, their approach fails to maintain scene-consistency for long sequences due memory constraints. In contrast, our dual spatio-temporal conditioning enables long-term scene-consistency without causing significant computational overhead. The qualitative comparisons shown in Fig. oaaso validate the effectiveness of our approach over DFoT.
4.3 CAMERA-CONTROLLANEE VIDEO GENERATION
Evaluation Protocol. We employ the evaluation protocol of previous meth- ods (He et al. 2024; Zheng et al.) 20244] Jin al. (025) for the camera controllability. We provide an input image along with as-
Table 2: Camera controllability evaluation.
DynPose- 100K TransFerr
Methods
mRodFir
mCamMCJ
MotionCiri Wane el
2024b
3.5654
7,8231
9.7834
CameraChii CHCCCCCCCCCCCCC 2024
3.3273
9,5989
11.2122
FloVD
11.0302
12.6202
sociated camera parameters for 12V mod-
AC3D) Esommann & 5l
2025a
3.0675
9,7044
11.1634
els (He et al.] 2024; Wang et al. [024bb] Jin et al. (025) and solely provide camera parameters for the T2V model (Bahmani
DFoT Stong
2 2.3977
8.0866
9,2330
3DSeendPoompl (w
2.3898
7.7819
8.9755
3DScmnPPoopt
2,3772
7.4174
8.6352
ct al. 2025a). We conduct experiments using two variants our framework, leveraging different numbers of frames (w) for temporal conditioning baselines, w 1 and 9. The w 1 uses only the last single frame as temporal conditioning, whereas the w = 9 model takes the last nine frames temporal conditioning. To evaluate how faithfully the generated video follows the camera condition, we estimate camera parameters from the synthesized vidco using MegaSAM (Li et al. 2024], and compare the estimated camera parameters against the condition camera trajectory C.
8

---

Published as a conference paper at ICLR 2026
Camera
1:97
T+03
Generated
Last Fromes
Condition
History Gulded Camera Control
T:5
TiS11
Te00
Fromes close Traj.
Lost Frames
Condition
Conditions for Generating the Next Video
Figure 6: Visualization of generated videos following trajectories that revisit early frames in the input video. We visualize and compare frames obtained from DFoT (Song et al. 2025 and Ours. We condition both DFoT and ours to generate video that follows the camera trajectory shown in the top row (GT Future Video). which revisits viewpoint in the input (Input Video). The comparison shows that ours shows much more consistent generation, whereas DFoT fails generate scene-consistent frames mainly due to the limited number of frames it can condition on. Also, the comparison of the camera trajectory shows that our method more faithfully follows the given camera condition.
Table 3: Evaluation video generation quality. We assess the quality of generated videos using FVD and VBench+ scores. For FVD, lower values indicate higher video quality. For VBench++ scores, higher values indicate better performance. All VBench+ scores are normalized.
DynPose 100K
Methods
Overall
Subject Consist
Bg Consist
Aesthetic Quality
Imaging Quality
Temporal Flicker
Motion
Dynamic
FVD
Score
Smooth
Degree
MotionCtal WWang al
2024b
1017.4247
0.5625
0.5158
0.7093
0.3157
0.3149
0,8297
0.8432
0.7900
CameraCtrl er
0.6280
0.6775
0.3736
0.3888
0.6837
0.6955
0,9900
HoVD Jin el a. 2013
171.2697
0.7273
0.7964
0,8457
0.4722
0.5546
0.7842
0.8364
0.9900
AC3D
20255a
281.2140
0.7428
0.8360
0.4766
0.8020
0.8673
1,0000
3DScene romptt (Ours)
127.4758
0.7747
0.8669
0.8727
0.4990
0.5964
0.8551
0.9260
1.0000
The comparison between the estimated and input camera parameters quantified using three met- rics: mean rotation error (mRotErr), mean translation еrror (mTrransErr), and mean error in the camera extrinsic matrices (mCamMC). For the generated video, also assess video synthesis per- formance using the Fréchet Video Distance (FVD) (Skorokhodov et al. (002) and seven metrics from VBench++ (Huang et al. (024)) subject consistency, background consistency, aesthetic qual- ity, imaging quality, temporal flickering, motion smoothness, and dynamic degree.
Results. We first evaluate camera controllability and compare method with competitive base- lines. As shown Tab. 2 our approach consistently outperforms existing methods, indicat- ing 3DScenePrompt is capable of generating videos with precise camera control. We also note that the effect of using different temporal conditioning window sizes is minimal for the camera- controllability performance, suggesting that better camera-controllability is achieved from our spa- tial prompts rather than the increased temporal context. We then assess the overall video quality (Tab. 3 and provide qualitative comparisons (Fig. 7) As observed Tab. 3 our method achieves the best generation quality across all metrics for dynamic video generation, which is further sup- ported by the visual results in Fig. 7
4.4 ABLATION STUDIES
We analyze critical components of our framework: the dynamic masking strategy that separates static and dynamic elements, and the number of spatially adjacent frames n retrieved for spatial conditioning. Tab. demonstrates the impact of varying and the necessity of dynamic masking.

---

Published as a conference paper at ICLR 2026
Input Video
Video
Camera
CameraCtel
T+60
Lost Frome
Condition
Fall
RovD
12V control
Condition
man black cop and (-shirt stands the open door luxury car, conversing with onother Inside vehicle, porking lot with severol cars (mooe)
T2V camera control
T-20
Fromes
Condition
Generoting
Figure 7: Visualization of scene-consistent camera-controllable video generation. Comparison of different methods (Wang al. 2024b He et al. 2024; Jin al. 2025 Bahmani al. 2025b) for generating videos from the same input (shown in Input Video) that follow the camera trajectory shown the rightmost column (Camera). Our method best preserves scene consistency with the video. Note the red-boxed regions: while the input video shows a white wall, competing methods either lose scene detail or fail maintain the original scene structure. contrast, our approach ac- curately remembers the white wall and maintains consistent scene elements throughout generation. In addition, when compared with the GT Future Video, ours best follows the camera condition, effectively verifying the strength of our framework for scene-consistent camera-controllable video generation.
Without dynamic masking (4th row), the model suffers significantly across all, showing a large drop of PSNR of approximately 0.8dB and also an increase of ME13R error.
This degradation occurs because unmasksed dynamic objects create ghosting artifacts when warped to new viewpoints, corrupting the spa- tial conditioning. Regarding the number of spa- tially adjacent frames, we find that performance stabilizes around n = 7, with minimal im-
Table 4: Ablation study on varying n.
Methods
Dynamic mask M
DynPose- 100K
PSNRT
SSIM
LPIPS]
ME3R
0.3732
0.124773
Ours (n 4)
13.0382
0.3733
0.3758
0.124893
Ours
/
13.0206
0.3631
0.3810
0.123507
Ours (n= 7)
x
12:2304
0.3063
0.382]
0.134885
provements beyond this point, suggesting that
Ours (a 7)
13.0468
0.3666
0.3812
0.124189
7 frames provide sufficient spatial context while maintaining computational efficiency.
CONCLUSION
In this work, we introduced 3DScenePrompt, framework for scene-consistent camera-controllabll video generation. By combining dual spatio-temporal conditioning with static-only 3D scene memory constructed through dynamic SLAM and our dynamic masking strategy, we enable gen- erating continuations from arbitrary-length videos while preserving scene geometry and allowing natural motion evolution. Extensive experiments demonstrate superior performance in camera con- trollability, scene consistency, and generation quality compared to existing methods. Our approach opens new possibilities for long-fom video synthesis applications where maintaining both spatial consistency and precise camera control is essential.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This research was supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (RS-2015 ASS-201--11190055. RS-2024- 00509279, RS-2025-11212066
RS-2023-00227592, RS-2025-0221 14279, RS-2024-00457882, RS- 2025-2544 838, 25-2025-22544885 RS-2025-0221 04499 RS-2025-022172555 and the Culture, Sports, and Tourism R&D Program through the Korea Creative Content Agency grant funded by the Ministry of Culture, Sports and Tourism (RS-2024-00345025, RS-2024-0033300 00222280, ASS-0033-00266500 and National Research Foundation of Korea (RS-2024-003466997) and the ETH AI Center through ETH AT Center postdoctoral fellowship to Sunghwan Hong.
RS-2023-
REFERENCES
Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chat- topadhyay, Yongxin Chen, Yin Cui, Yifan Ding, al. Cosmos world foundation model platform for physical al. arXiv preprint arXiv:2501. 03555, 2025.
Honggyu An, Jin Hyeon Kim, Seonghoon Park, Jaewoo Jung, Jisang Han, Sunghwan Hong, and Seungryong Kim. Cross-view completion models zero-shot correspondence estimators. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 100--1115, 2025.
Mohammad Asim, Christopher Wewer, Thomas Wimmer, Bernt Schiele, and Jan Eric Lenssen. Met3r: Measuring multi-view consistency in generated images. arXiv preprint arXiv:2501.06336 2025.
Sherwin Bahmani, Ivan Skorokhodov, Aliaksandr Siarohin, Willi Menapace, Guocheng Qian, Michael Vasilkovsky, Hsin-Ying Lee, Chaoyang Wang, Jiaxu Zou, Andrea Tagliasacchi, et al. Vd3d: Taming large video diffusion transformers for 3d camera control. arXiv preprint arXiv:2407.12781, 2024.
Sherwin Bahmani, Ivan Skorokhodov, Guocheng Qian, Aliaksandr Siarohin, Willi Menapace, An- drea Tagliasacchi, David B. Lindell, and Sergey Tulyakov. Ac3d: Analyzing and improving 3d camera control in video diffusion transformers. Proceedings of the IÉEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.
June 2025a.
Sherwin Bahmani, Ivan Skorokhodov, Guocheng Qian, Aliaksandr Siarohin, Willi Menapace, An- drea Tagliasacchi, David B Lindell, and Sergey Tulyakov. Ac3d: Analyzing and improving 3d camera control in video diffusion transformers. In Proceedings of the Computer Vision and Pat- tern Recognition Conference, pp. 22875-22889, 2025b.
Jianhong Bai, Menghan Xia, Xiao Fu, Xintao Wang, Lianrui Mu, Jinwen Cao, Zuozhu Liu, Haoji Hu, Xiang Bai, Pengfei Wan, et al. Recammaster: Camera-controlle generative rendering from single video. arXiv preprint arXiv:2503.1164 2025.
Andreas Blattmann, Tim Dockhom, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models large datasets. arXiv preprint arXiv:2311.1511 2023.
Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng. Ricky Wang, and Aditya Ramesh. Video generation models world simulators, 2024. URL video generation modensoodel as world ld simulators
http://pppaaa.cccrrseeeeeeaaaeee wwt/research//
Scokju Cho, Sunghwan Hong, Sangryul Jeon, Yunsung Lee, Kwanghoon Sohn, and Seungryong Kim. Cats: Cost aggregation transformers for visual correspondence. Advances in Neural Infor- mation Processing Systems, 34:901 1 1-0023, 2021.
Scokju Cho, Sunghwan Hong, and Seungryong Kim. Cats++: Boosting cost aggregation with volutions and transformers. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45 (6):7174-7194, 2022.
11

---

Published as a conference paper at ICLR 2026
Seokju Cho, Heeseong Shin, Sunghwan Hong, Anurag Arnb, Paul Hongsuck Seo, and Seungryong Kim. Cat-seg: Cost aggregation for open vecabuocyy semantic segmentation. In Proceedings of the IEEE/CVF Conference Computer Viston and Pattern Recognition, pp.
2024.
Decart, Julian Quevedo, Quinn McIntyre, Spruce Campbell, Xinlei Chen, and Robert Wachen. Oa- sis: A universe in transformer, 2024. URL
.oo/
Chen Gao, Ayush Saraf, Jia-Bin Huang, and Johannes Kopf. Flow-edge guided video completion. In European Conference on Computer Vision, pp. 713-729. Springer, 2020.
Zekai Gu, Rui Yan, Jiahao Lu, Peng Li, Zhiyang Dou, Chenyang Si, Zhen Dong, Qifeng Liu, Cheng Lin, Ziwei Liu, Wenping Wang, and Yuan Liu. Diffusion as shader: 3d-aware video diffusion for versatile video generation control. arXiv preprint arXiv:2501.03847 2025.
Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang. Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to-image diffu- sion models without specific tuning. arXiv preprint arXiv:2307.04425, 2023.
Jisang Han, Honggyu An, Jaewoo Jung, Takuya Narihira, Junyoung Seo, Kazumi Fukuda, Chae- hyun Kim, Sunghwan Hong, Yuki Mitsufuji, and Seungryong Kim. D' 2ust3r: Enhancing 3d reconstruction with 4d pointmaps for dynamic scenes. arXiv preprint arXiv:250..06264 2025a.
Jisang Han, Sunghwan Hong. Jaewoo Jung, Wooseok Jang, Honggyu An, Qianqian Wang, Se- ungryong Kim, and Chen Feng. Emergent outlier view rejection in visual geometry grounded transformers. arXiv preprint urXiv2251.004012, 2025b.
Hao He, Yinghao Xu, Yuwei Guo, Gordon Wetzstein, Bo Dai, Hongsheng Li, and Ceyuan Yang. Cameractrl: Enabling camera control for text-to-video generation. arXiv preprint arXiv:2404.00 02010 2024.
Hao He, Ceyuan Yang, Shanchuan Lin, Yinghao Xu, Meng Wei, Liangke Gui, Qi Zhao, Gordon Wetzstein, Lu Jiang, and Hongsheng Li. Cameractrl ii: Dynamic scene exploration via camera- 2025a.
controlled video diffusion models. arXiv preprint arXiv:2503.
Haoran He, Yang Zhang, Liang Lin, Zhongwen Xu, and Ling Pan. Pre-trained video generative models as world simulators. arXiv preprint arXiv:2502.078.. 077.25 2025b.
Sunghwan Hong, Seokju Cho, Jisu Nam, Stephen Lin, and Seungryong Kim. Cost aggregation with 4d convolutional swin transformer for few-shot segmentation. In European Conference on Computer Vision, pp. 108-126. Springer, 2022a.
Sunghwan Hong. Jisu Nam, Seokju Cho, Susung Hong, Sangryul Jeon, Dongbo Min, and Seungry- ong Kim. Neural matching fields: Implicit representation of matching fields for visual correspon- dence. Advances in Neural Information Processing Systems, 15112-1323,,, 2022b.
Sunghwan Hong. Seokju Cho, Seungryong Kim, and Stephen Lin. Unifying feature and cost aggregation with transformers for semantic and visual correspondence. arXiv preprint 1rXiv:2403.1/122 2024a.
Sunghwan Hong. Jaewoo Jung, Heeseong Shin, Jisang Han, Jiaolong Yang. Chong Luo, and Seungryong Kim. Pf3plat: Pose-free feed-forward 3d gaussian splatting. arXiv preprint arXiv:2410.22128. 2024b.
Sunghwan Hong, Jaewoo Jung, Heeseong Shin, Jiaolong Yang, Seungryong Kim, and Chong Luo. Unifying correspondence pose and nerf for generalized pose- free novel view synthesis. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20196- 20206, 2024c.
Susung Hong, Junyoung Seo, Heeseong Shin, Sunghwan Hong, and Seungryong Kim. Direct2v: Large language models are frame-level directors for zero-shot text-to-video generation. arXiv preprint arXiv:2305. 14330, 2023.
12

---

Published as a conference paper at ICLR 2026
Ziqi Huang, Fan Zhang. Xiaojie Xu, Yinan He, Jiashuo Yu. Ziyue Dong. Qianli Ma, Nattapol Chan- paisit, Chenyang Si, Yuming Jiang, et al. Vbench++: Comprchensive and versatile benchmark suite for video generative models. arXiv preprint arXhv:2411.13503 2024.
Wonjoon Jin, Qi Dai, Chong Luo, Seung-Hwar Hwan Back, and Sunghyun Cho. Flovd: Optical flow meets video diffusion model for enhanced camera-controlled video synthesis. arXiv preprint arXiv:2502.08244 2025.
Nikita Karaev, Iurii Makarov, Jianyuan Wang, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Cotracker3: Simpler and better point tracking by pseudo-labelling real videos. arXiv preprint arXiv:2410.1 1831, 2024.
Chachyun Kim, Heeseong Shin, Eunbeen Hong, Heeji Yoon, Anurag Amab, Paul Hongsuck Seo, Sunghwan Hong, and Seungryong Kim. Seg4diff: Unveiling voca-vocybblaryy semantic segmen- tation in text-to-image diffusion transformers. The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025.
Václay Knapp and Maty Bohacek. Synthetic human action video data generation with pose transfer. In Synthetic Data for Computer Vision Workshop@ CVPR 2025, 2025.
Zhengqi Li, Richard Tucker, Forrester Cole, Qianqian Wang. Linyi Jin, Vickie Ye, Angjoo Kanazawa, Aleksander Holynski, and Noah Snavely. Megasam: Accurate, fast, and robust struc- ture and motion from casual dynamic videos. arXiv preprint arXiv:2412.0446, 2024.
Haotong Lin, Sili Chen, Jun Hao Liew, Donny Y. Chen, Zhenyu Li, Guang Shi, Jiashi Feng, and Bingyi Kang. Depth anything 3: recovering the visual space from any views. arXiv preprint arXiv:2511.10647 2025a.
Shanchuan Lin, Ceyuan Yang, Hao He, Jianwen Jiang, Yuxi Ren, Xin Xia, Yang Zhao, Xuefeng Xiao, and Lu Jiang. Autoregressive adversarial post-training for real-time interactive video gen- eration. arXiv preprint arXiv:2506.093350 2025b.
Yu Lu, Yuanzhi Liang, Linchao Zhu, and Yi Yang. Freelong: Training- free long video generation with spectralblend temporal attention. arXiv preprint arXiv:2407.199/5 2024.
Kepan Nan, Rui Xie, Penghao Zhou, Tichan Fan, Zhenheng Yang, Zhijic Chen, Xiang Li, Jian Yang, and Ying Tai. Openvid- m: A large-scale high-quality dataset for text-to-video generation. arXiv preprint ar arXiv:2407.023: 2024.
Federico Perazzi, Jordi Pont-Tuset, Brian McWilliass, Luc Van Gool, Markus Gross, and Alexander Sorkine-Hornund A benchmark dataset and evaluation methodology for video object segmen- tation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 724-732, 2016.
Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, al. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.007 14, 2024.
Xuanchi Ren, Tianchang Shen, Jiahui Huang. Huan Ling, Yifan Lu, Merlin Nimier-David Thomas Müller, Alexander Keller, Sanja Fidler, and Jun Gao. Gen3c: 3d-informed world-consistent video generation with precise camera control, arXiv preprini arXiv:2503.0377 033751 2025.
Chris Rockwell, Joseph Tung, Tsung- Yi Lin, Ming- Yu Liu, David F Fouhey, and Chen-Hsuan Lin. Dynamic camera poses and where find them. In Proceedings of the Computer Vision and Pallern Recognition Conference, pp. 12444-12455. 2025.
Runway. Runway. https: Ttttps://ruwaaym..com 2024. Accessed: 2024-11-05.
Junyoung Seo, Jisang Han, Jaewoo Jung, Siyoon Jin, Joungbin Lee, Takuya Narihira, Kazumi Fukuda, Takashi Shibuya, Donghoon Ahn, Shoukang Hu, et al. Vid-camedit: Video cam- era trajectory editing with generative rendering from estimated geometry. arXiv preprint arXiv:2506. 13697, 2025.
13

---

Published as a conference paper at ICLR 2026
Ivan Skorokhodov, Sergey Tulyakov, and Mohamed Elhoseiny. Stylegan-v: A continuous video generator with the price, image quality and perks of stylegan2. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.
2022.
Kiwhan Song, Boyuan Chen, Max Simchowitz, Yilun Du Russ Tedrake, and Vincent Sitzmann. History-guided video diffusion. arXiv preprint
2025.
Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. In Proceedings of the Computer Viston and Pattern Recognition Conference, pp. 5294-5306, 2025.
Yihan Wang, Lahav Lipson, and Jia Deng. Sea-raft: Simple. efficient, accurate raft for optical flow. In European Conference on Computer Vision, pp. 36-54. Springer, 2024a.
Zhouxia Wang, Ziyang Yuan, Xintao Wang, Yaowei Li, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. Motionctrl: A unified and flexible motion controller for video generation. In ACM SIGGRAPH 2024 Conference Papers, pp. 1-11, 2024b.
Tong Wu, Shuai Yang, Ryan Po, Yinghao Xu, Ziwei Liu, Dahua Lin, and Gordon Wetzstein. Video world models with long-term spatial memory. arXiv preprint arXiv:2506.05588 2025.
Zeqi Xiao, Yushi Lan, Yifan Zhou, Wenqi Ouyang, Shuai Yang, Yanhong Zeng, and Xingang
Pan. Worldmem: Long-term consistent world simulation with memory, 2025. URL https:
// 2axxiv.rr/abbs/20000
Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with expert transformer. arXiv preprint arXiv:2408.06072, 2024.
Heeji Yoon, Jaewoo Jung, Junwan Kim, Hyungyu Choi, Heeseong Shin, Sangbeom Lim, Hong- gyu An, Chachyun Kim, Jisang Han, Donghyun Kim, et al. Visual representation aligmment for multimodal large language models. arXiv preprint arXiv:2509.07973 2025.
Mark Yu, Wenbo Hu, Jinbo Xing, and Ying Shan. Trajectorycrafter: Redirecting camera trajec- tory for monocular videos via diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, 2025.
Yuanwen Yue, Damien Robert, Jianyuan Wang, Sunghwan Hong. Jan Dirk Wegner, Christian Rup- precht, and Konrad Schindler. Litept: Lighter yet stronger point transformer. arXiv preprint arXiv:2512.13668 2025.
Shangjin Zhai, Zhichao Ye, Jialin Liu, Weijian Xie, Jiaqi Hu, Zhen Peng, Hua Xue, Danpeng Chen, Xiaomeng Wang, Lei Yang, et al. Stargen: A spatiotemporal autoregression framework with video diffusion model for scalable and controllable scene generation. arXiv preprint arXiv:2501.05763, 2025.
Junyi Zhang, Charles Hermann, Junhwa Hur, Varun Jampani, Trevor Darrell, Forrester Cole, De- qing Sun, and Bing-Hsuan Yang. Monst3r: A simple approach for estimating geometry in the presence of motion. arXiv preprint arXiv:2410.03825. 2024.
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference computer vision, pp. 3836-333333 1836-3847,00 2023.
Ruihan Zhang, Borou Yu, Jiajian Min, Yetong Xin, Zheng Wei, Juncheng Nemo Shi, Mingzhen Huang. Xianghao Kong, Nix Liu Xin, Shanshan Jiang, et al. Generative for film creation: A survey of recent advances. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 6267-6279, 2025.
Zhoutong Zhang, Forrester Cole, Zhenggi Li, Michael Rubinstein, Noah Snavely, and William T Freeman. Structure and motion from casual videos. In European Conference on Computer Vision, pp. 30-37.00 Springer, 2022.
14

---

Published as a conference paper at ICLR 2026
Guangcong Zheng, Teng Li, Rui Jiang, Yehao Lu, Tao Wu, and Li. Cami2v: Camera-controlled image-to-video diffusion model. arXiv preprint arXiv:2410.15957, 2024.
Siyuan Zhou, Yilun Du, Yuncong Yang, Lei Han, Peihao Chen, Din Yeung, and Chuang Gan. Learning 3d persistent embodied world models, 2025. URL fettp::/erxiv..rr///e.
2505. 05495
Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, and Noah Snavely. Stereo magnification: Learning view synthesis using multiplane images. arXiv preprint 1805. 2018.
15

---

Published as a conference paper at ICLR 2026
APPENDIX
NOTATION SUMMARY
Symbol
Meaning
VA REXHIXWX
input arbitrary-length video with /. frames.
Vau T
generated future video with frames.
T
text prompt for video generation for video models based on text-to-video (T2V) generation.
(C.)LL
desired camera trajectory the generated video Van should follow. camera extrinsics parameter where C2 E (3)().
C2
K
camera intrinsics parameter.
F())
camera-controllable video generation framework.
g()
video-to-futter future-video generation framework.
hef
image condition for image-to-vidco (I2V) generation.
Vla vl
indexing operation; samples frames between frame and (y-1).
Temporal (w)
temporally adjacent frames condition.
Sp)
spatially adjacent T frames for condition.
V
conditioning frames for our frumework, includes both Temporal and (p)tial dynamic SLAM frameworks.
D()
II-))
perspective projection operator.
P
aggregated 3D point clouds.
smnii
nggregated 3D point clouds only from static regions.
M
3D scene memory composed of camera extrinsics C, and static point clouds P satii- object-level masks representing dynamic regions of frame i.
M
E()
3D VAE.
A ADDITIONAL EXPERIMENTAL RESULTS
More Qualitative Results. We provide additional qualitative results of our method to further demonstrate its ability to generate high-quality outputs, as shown in 10 and 11.
Long-Video Generation Results. Although our primary interest is building framework capable of generating spatially consistent next-video chunk given an arbitrary video as context, by itera- tively applying our method, one of the potential applications our framework is generating videos of arbitrary length.
To generate longer videos, we extend our method using iterative autoregressive generation, and conduct quantitative evaluations long-video generation using the DAVIS dataset (Perazzi (et al.] 2016), where we evaluate video generation quality (PSNR, SSIM, LPIPS) and camera- controllability (RMSE, MSE, ATE), and compare with previous I2V baselines reported in Tab. 5 However, as there are no publicly available video-to-futrreviddes camera-controllable generation methods for our baselines, for fairer comparison, we adopt three different strategies to generate long videos with our baseline methods. Specifically, we adopt 1) iterative autoregressive genera- tion, 2) latent interpolation with autoregressive generation, and 3) applying training- free long video generation techniques.
For iterative autoregressive generation, we simply provide the last frame of the previously generated video as the input condition when generating the next video. For latent interpolation, we increase the number of input latents through interpolation, which can directly increase the number of gen- erated frames at inference. After applying latent interpolation, we similarly adopt autoregressive generation to generate videos with the desired number of frames. By generating more frames at once, this reduces the number of iterations of autoregressive generation, which can increase overall video quality by reducing еттоr accumulation. Finally, we also adopt FreeLong (Lu et al. 2024), training-fre free approach for long video generation, which introduces frequency blending of latents specifically in Stable Video Diffusion (Blattmann et al.] (023). As we compare with the FloVD (Jin
al. 2025) model fine-tuned from VodeoX (Yang et al. 2024). which is transformer-based
16

---

Published as a conference paper at ICLR 2026
Table 5: Quantitative evaluation of spatially consistent long video generation on DAVIS Per- azzi et al. 2016 dataset.
DAVIS
Methods
PSNR1
SSIM
LPIPS.
RMSE
MSE,
ATT.
CameraCtrl (He et al. latent interpolation
2024
8.64
0.19
0.66
95.45
9332.08
0,1709
14.12
0.49
0.45
53.50
3214.71
0,2225
FreeLong et 2024
13.46
0.44
3314.31
0.2147
FloVD (Jin al.
10.77
0,42
0.55
76.08
6131.95
0.2242
Ours
17.28
0.60
0.35
37.28
1583.77
0.1794
Table 6: Dynamic mask ablation study.
DynPose- 100K
Methods
PSNR
SSIM
LPIPS
ME13R,
mRotErr (0)
mTransErr.
mCamMCJ
(a) w/o dynamic mask
11.9963
0.2898
0.3748
0.104002
3,4142
8,5920
10.5049
(b) (a) difference mask
12.1083
0.3248
0.3690
0.155589
2,9390
7.7819
9.1524
(c) (b) SAM2 propagation
13.5957
0.4036
0.3548
0.157610
2.7188
7.5402
8.9696
(d) (c) point BW tracking
13.7311
0.4112
0.3454
0.122859
2.6103
7.4858
8.9181
model, apply FreeLong only for CameraCtrl (He et al. 2024). Due latent interpolation sig- nificantly increasing the computation in CogVideok we also apply the latent interpolation tech- nique for CameraCtrl only. The qualitative results of generated long videos can be found in the suppl supplementary videos or in Fig. 12
As shown in the results, our method shows its potential for long video generation, significantly out- performing previous baselines. However, as is common in auto-regressive generation, extending our framework to extremely long durations introduces the challenge of temporal error accumulation. While our method provides the critical spatial consistency required for long videos, solving long- term drift (error accumulation) typically requires dedicated refinement modules and techniques. We believe that further extending our framework to mitigate the aforementioned issues is very impor- tant and interesting future direction to explore.
Ablation Study on Dynamic Masking Pipeline. We ablate each stage of our dynamic masking pipeline presented in Sec. 3.3 Stage (a) performs training without any dynamic mask, causing moving objects to interfere with static-scene learning. Stage (b) introduces a pixel-level L1 flow- difference mask, described in Eq. 66 which detects motion per frame but often fails to fully capture dynamic objects due to noisy or fragmented flow estimation. Stage (c) improves object- level consistency by using SAM2 propagation from points sampled in the first frame; however, this stage cannot detect new or displaced dynamic objects appearing in later frames. Stage (d) further adds point tracking across the sequence, enabling backward propagation into the first frame be- fore SAM2 segmentation, which allows the mask to capture all dynamic objects throughout the entire clip. The quantitative comparison for each variant summarized Tab. 6 where we eval- uate the video generation quality (PSNR, SSIM, LPIPS), acene-consistern controllability (mRotErr, ErrnnsErT, mCamMC) in subset of the DynPose- 1ePoo----OOK dataset. The results effectively verify the need for each stage of our proposed dynamic mask generation pipeline.
(Met3r), and camera-
Ablation study on the number of temporal condition images. As CogVideo: generates 49 frames simultancously, the temporal window size can selected within the range 1 < < 49, meaning (49 w) frames are newly generated. In our framework, setting the window size to w = 9 (providing the last few frames) one of the key contributions and design choices ensure sufficient temporal context to maintain motion continuity and coherency with the input video, while still gen- crating a sufficient number of frames (40). To better validate this choice, we conduct experiments with varying window sizes (w 1 and w 5). As summarized in Tab. 7 the results demon- strate that while the window size has a negligible effect on camera controllability metrics (mRotErr, ETransEr, mCamMC), providing a larger context window (w 9) yields superior motion smooth- ness and temporal coherence, as evidenced by improved VBench metrics (Temporal Flicker and Motion Smoothness). which is aligned with intentions.
17

---

Published as a conference paper at ICLR 2026
Table 7: Ablation study on the size of temporal window w. We conduct an ablation study on camera controllability with respect to the number of conditioned images w, and evaluate motion co- herence by measuring VBench (Huang et al. 2024)'s Temporal Flicker and Motion Smooth metrics. VBench metrics are normalized.
DynPose- 100K
Methods
mirransirr mRotErr.
mCamMCJ,
Temporal Flickert
Motion Smooth
Ours (w
2,3898
7.7819
8.9785
0.8379
0.9253
Ours (w 5)
2,3837
7.5512
8.6233
0.8508
0.9262
Ours (w 9)
2.3772
7.4174
8.6352
0.8561
0.9335
Table 8: Ablation study on the number of projected images We conduct an ablation study on scene consistency and camera controllability with respect to the number spatially adjacent frames retrieved for spatial conditioning the DynPose- 100K (Rockwell et al. 2025) dataset.
PonPos--O0KK
Methods
PSNR
SSIM
LPIPS
mRotEr)
mTransErr]
mCamMCL
Ours (n 0)
11.9555
0.3370
0,4512
3.4142
8,5920
10.5049
Ours (n 4)
13.0382
0,3733
0.3758
2.3739
7.4278
8.6488
Ours (n 7)
13.0468
0.3666
0,3812
2.3772
7.4174
8.6352
Ablation study on the number of projected images. We further evaluate the impact of using different numbers of projected views as spatial conditions. To do this, we compare our method against a baseline setting (n 0), where n denotes the number of projected frames. In this baseline, the model is conditioned solely on the temporal frames without any projected spatial views. The results, presented in Tab. [] show that without the incorporation of our spatial prompts, the baseline framework fails to maintam scene consistency (resulting in significantly lower PSNR and SSIM scores) and struggles projected images (n 4 and 7) significantly outperform the n = O baseline across all metrics. This effectively verifies the critical importance of our spatial prompts for ensuring both high-fidelity scene consistency and precise camera control.
adhere to the target camera trajectory.
contrast, the variants utilizing
DepthAnything v3 for 3D memory. One of the advantages of our framework is that do not have any special architecture designs tailored to MegaSAM and can always replace MegaSAM AM with any geometry or correspondence models Hong et al. (2022a) 2024c a): Wang al. (2025); Cho al. [2021] 2022). typically a more robust and powerful model to resolve the current limitations. Here, show an experiment where we replace MegaSAM with the recently released DepthAnything v3 (Lin et al. 2025a) for inference. The results, summarized in Tab. 9 show improvements across metries, together with reduced inference time. This confirms that our method directly benefits from stronger priors and highlights& a key advantage of our design: the components can be swapped at inference time without changing the overall pipeline, allowing users to flexibly choose different pretrained models.
Inference Time. Tab. 10 presents the end-to-end inference latency of our method in comparison to existing camera-controllable video diffusion models. Approaches such as CameraCtrl (He et al.) 2024), MotionCtrl (Wang al. 2024b), and FloVD (Jin al. 2025) rely exclusively on diffusion- based synthesis, where inference time is primarily determined by the denoising process. In con- trast, our pipeline incorporates additional stages for SLAM-based 3D reconstruction and dynamic masking. With MegaSAM (Li et al., 2024) employed for dynamic SLAM, the preprocessing stage requires approximately minutes, resulting in a total inference time of over minutes. However, re- placing MegaSAM with more lightweight and advanced method, such as DepthAnything v3 (Lin) ct al.] 2025a), significantly reduces the SLAM processing time to roughly 10 seconds. This sim- ple replacement reduces the total latency to approximately 5 minutes, where the overall runtime is dominated by the diffusion- based video generation itself. Consequently, our framework achieves in- ference speeds comparable to CameraCtrl and MotionCtrl, which leverage more lightweight video generation backbone, demonstrating that the proposed method introduces negligible computational overhead while enabling spatially consistent and camera-controllabbe generation.
18

---

Published as a conference paper at ICLR 2026
Table 9: Using various methods for generating 3D memory.
DynPose- 100K
Methods
PSNRT
SSIM
LPIPS..
Ours w/ MegaSAM (Li al. 2024)
13.0468
0.3666 0.3980
0.3812
Ours w/ DepthAnything v3 (Lin et al. 2025a)
13.4534
0.3637
Table 10: Inference time comparison of different methods.
Dynamic Masking
Method
HLAMMPrrocsstie
Video Generation
Inference Time
Depth Warping
CumeraCurl
al. 2024
min sec min 9.5 8 min 5.32 sec
min sec 2 min 9.5
CationCtrl (Wang 2024h
FloVD Jin et ar. 2025
8 min 5.32 see
Ours MegaSAM Li al 2024
4 min 18.813 see
58.88 sec
4 min 3.62 sec
min 21.31 31 sec min 12.53 sec
Ours DepthAnything [Ln al.
2025a
10.031 see
sec
4 min 3.62 sec
B TRAINING DATASET CURATION PIPELINE.
Our training data is curated from two primary sources: RealEstate10K (Zhou et al. 2018), which consists of static indoor scenes, and Open Vid- IM (Nan et al. 2024). which features diverse dynamic content. For static videos from RealEstate we directly extract 3D scene geometry without applying dynamic masking, these scenes contain negligible motion. In contrast, for dynamic videos from Open Vid- we conduct extensive data filtering and preprocessing: we remove game- like or low-resolution videos, exclude samples with excessive camera motion, and apply the dynamic masking pipeline described in Section 3.3 to separate static and dynamic regions. We further select long video sequences with lengths of at least L>100 100 frames to ensure sufficient input video length. To construct the 3D scene memory, we employ VGGT (Wang et al.) (202)) for static scenes and MegaSAM (Li et al. 2024) for dynamic scenes. After filtering and processing, final dataset comprises approximately 30K and 20K high-quality long videos from RealEstate 10K and Open Vid- 1M, respectively.
C DISCUSSION AND COMPARISON WITH RELATED WORKS
In this section, we discuss the differences between our proposed framework and recent relevant works, focusing 3D memory integration, camera-controlllable generation, and dynamic object handling.
Explicit 3D Memory and World Models Several recent works have explored utilizing 3D mem- ory for video generation. Persistent embodied world model (Zhou et al. 2025) introduces 3D mem- ory but relics on implicit memory representations Hong et al. (2022b) and targets static scenes. In contrast, work leverages explicit 3DD memory achieve scene-consistent generation specifically for real-world dynamic videos. Similarly, WorldMem (Xiao et al. 2025) builds upon the Oasis (De- cart al. 2024) architecture and is trained primarily on the Minecraft dataset. It addresses static scenes (RealEstate 10K) or synthetic domains and requires key frames as conditions by increasing the sequence size. Our approach differs by leveraging the learned priors of DiTs generate scene- consistent, camera- conerraconttolllall videos in complex real-world dynamic scenarios. Furthermore, by projecting constructed static-only 3D point clouds the target trajectory, we condition on long videos efficiently without increasing computational costs.
We also differentiate our work with SPMem (Wu al., 2025), a concurrent work conceptually sim- ilar to ours. However, our method distinguishes itself in two key aspects. First, regarding dynamic object handling, while SPMem computes static components via naïve TSDF fusion, we introduce a concise and accurate dynamic mask generation pipeline to effectively remove dynamic regions. Sec- ond, in terms of efficiency, SPMem employs an additional ControlNee-styyle Diffusion-ar aonn---SSaaerr (Gu et al. 2025) architecture, necessitating architecture search, such the number of sufficient layers for the Contro NentrolNet-stylee block for different diffusion backbones, and the introduction the additional
model leads to extra computational resources for training. Conversely, we achieve scene-consistent
19

---

Published as a conference paper at ICLR 2026
T100
T
=30
T=40
3D Scene Memory
Ours
Figure 8: Visualization of the 3D scene memory and the generated results.
generation without any architectural changes by injecting spatial and temporal conditioning into zero-padding slots, enabling efficient training and inference. While a direct comparison in perfor- mance and efficiency would better highlight these differences, it is currently not possible due to the unavailability of public code.
Camera-Controolabbe Video Generation Works such CameraCtr12 (He al. 2025a) and APT2 (Lin et al. 2025b) explore camera conditioning using Plücker embeddings and autoregressive generation. While effective for short sequences, these methods typically model consistent video gen- eration by taking only previous images as conditions without explicitly modeling 3D scene structure or considering spatial adjacency. Consequently, they fail incorporate all previous frames due to computational constraints.
In contrast, our approach constructs an explicit 3D static memory using point clouds Yue et al. 2225),, enabling precise pixel-wise spatial alignment across views. Instead of relying autore- gressive propagation or feature-level Plücker conditioning, our 3D memory directly integrated into the diffusion transformer via 3D-warped point clouds, providing scene-consistent constraints that do not accumulate drift over time. Additionally, our design explicitly handles dynamic objects by separating static and dynamic components, challenge not addressed by these prior methods.
Dynamic Object Removal and Inpainting Regarding the removal of dynamic objects, our spatial conditioning shares surface-level similarities with video inpainting techniques such FGVC (ao et al. 2020). However, a fundamental difference lies in the role of the condition, as shown in Fig. standard inpainting, the model operates under hard constraints Tramework, spatial prompts serve as a soft spatial guide. This allows the diffusion model to fill holes and refine details while respecting the geometric layout where reliable projections exist. Crucially, this flexibility ensures the model can generate dynamic details on top of the condition, blending geometric consistency with realistic temporal dynamics, rather than merely filling static holes.
fill masked regions. In our
D LIMITATION
Our framework, 3DScenePrompt, enables scene-consistent, camera-controllable video generation and demonstrates improved performance over existing methods. Nonetheless, it has few limita- tions. To obtain point clouds, we rely on MegaSAM (Li et al., 2024) to process the input videos, which introduces additional computational overhead and increases inference time. This overhead could be reduced by adopting a more advanced module, e.g., DepthAnything3 (Lin et al., (025a)). In addition, the quality of the dynamic masks directly affects the fidelity of the generated videos. Thanks to our general and modular design, replacing the current component with a more advanced dynamic masking strategy that benefits from stronger models could further improve overall perfor- mance.
20

---

Published as a conference paper at ICLR 2026
Input Video
CameraCtri
MotionCtri
FlovD
Fummr
Ours
GT Video
Figure 9; More qualitative results.
21

---

Published as a conference paper at ICLR 2026
Input Video
CameraCtri
MotionCtrl
FlovD
Ours
GT video
Figure 10: More qualitative results.
22

---

Published as a conference paper at ICLR 2026
Input Video
Ours
GT Video
Input Video
Ours
GT Video
Input Video
Ours
GT Video
Figure 11: More qualitative results.
23

---

Published as a conference paper at ICLR 2026
Input Video
T= 15
T= 30
T= 45
T= 60
75
T=
T= 105
T= 120
Ours Long Video
T= 135
T= 150
T= 165
T= 180
Input Video
T=
T=
45
60
T= 75
T= 90
T= 105
T= 120
Long Video
T= 135
T= 150
T= 165
T= 180
Figure 12: Long-viden generation results.
24

---

Published as a conference paper at ICLR 2026
E REPRODUCIBILITY STATEMENT
As mentioned in Section 3.1 our model builds upon the open-sourced CogVideoX-I2V-5B (Yang al., 2024) model, where each of the processes for dynamic masking also detailedly explained. We will also make all the codes publicly available.
F USE OF LARGE LANGUAGE MODELS
In accordance with the ICLR 2026 submission policy, we disclose that we used Large Language Models assist in grammar correction for the writing in this manuscript.
25


Reviews:
[{"review_id": "Ykl9BPCKii", "reviewer": "Reviewer_3iY4", "summary": "The paper proposes 3DScenePrompt, a framework for scene-consistent, camera-controllable video generation from an arbitrary-length input video. The core idea is a dual spatio-temporal conditioning scheme: (i) a short temporal window of recent frames to preserve motion continuity, and (ii) spatial window obtained by projecting a static-only 3D scene memory built from the entire input via dynamic SLAM + dynamic masking. The static memory is rendered to the target viewpoints and concatenated with the temporal window to condition a CogVideoX-I2V backbone. Experiments on RealEstate10K and DynPose-100K report improved spatial/geometric consistency, camera controllability, and overall video quality over baselines including DFoT and recent camera-control methods.", "questions": "In addition to the weakness section listed above, I have the following questions:\n\n- How exactly is the pose similarity is computed and what are the thresholds?\n\n- The paper argues benefits for consistency, but doesn’t show what happens when the planned camera explores regions poorly covered by SLAM. What happens there?\n\n- How are colors/features assigned when projecting? How do you handle holes or depth ordering?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- Dual spatio-temporal conditioning: retrieving spatially adjacent content by pose similarity, not just the most recent frames.\n\n- Static-only 3D scene memory: A practical masking pipeline (flow residuals -> backward point tracking -> SAM2 object masks) to construct a static-only 3D scene memory by excluding dynamics before projecting to target viewpoints.\n\n- Experimental results show that the method performs favorably against baselines. There is also a nice spread of different metrics: geometric consistency, spatial revisitation metrics, camera-following errors, FVD, and VBench.\n\n- Ablations isolate the value of dynamic masking and the number of spatial frames.", "weaknesses": "- Fairness of controllability comparisons: Competing camera-control baselines are evaluated with their standard inputs (single image or text + camera), whereas your method consumes last 9 frames + spatial prompts, which is strictly richer conditioning. This can advantage camera trajectory adherence and quality metrics. Consider a variant that is restricted to the same conditioning budget (e.g., single frame + spatial prompts vs single frame) to isolate the benefit of spatial prompts rather than extra temporal frames.\n\n- Running the entire pipeline seems computationally expensive. What is the end-to-end run time during inference compared to the baselines?\n\n- The spatial retrieval is defined by FOV overlap to the planned camera. It’s less clear how the method performs when the planned trajectory explores previously unseen regions or when SLAM coverage is sparse/erroneous.\n\n- How sensitive is the model w.r.t. the output errors/bias of the pretrained models? What are the failure modes? What happens when we use different pretrained models?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WH4Mg9qs4y", "reviewer": "Reviewer_hP5a", "summary": "This paper presents 3DScenePrompt, a framework for generating camera-controllable videos from arbitrary-length input videos while maintaining scene consistency. The key innovation is a dual spatio-temporal conditioning strategy: the model conditions on both temporally adjacent frames (last 9 frames) for motion continuity and spatially adjacent frames for scene consistency. To handle dynamic elements correctly, the authors construct a 3D scene memory using dynamic SLAM that represents only static geometry, extracted via a three-stage dynamic masking pipeline (pixel-level motion detection, SAM2 propagation, and backward tracking with CoTracker3). This static-only 3D representation is projected to target viewpoints to provide spatial prompts while allowing dynamics to evolve naturally. Experiments on RealEstate10K and DynPose-100K show improvements over baselines in scene consistency, camera control, and video quality.", "questions": "1. What happens when SLAM fails (texture-less scenes, fast motion, dynamic-dominated scenes)?\n2. The paper shows only success cases. When does the method fail?\n3. Table 4 only compares with/without the full mask. What's the contribution of each stage (pixel detection, SAM2, backward tracking)?\n4. The temporal window size is fixed at 9 with no justification or ablation. \n5. There should be a baseline method, that the model does not condition on the projected views, instead the original chosen temporal frames.\n5. Does the method work on other video diffusion models beyond CogVideoX?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The motivation of the paper is clear: when generating the videos consistent with previous frames, the static and dynamic element must be handled differently, and the underlying 3D geometry must be kept during the camera controlling.\n\n2.  The dual spatio-temporal conditioning strategy is intuitive yet powerful. The recognition that spatial conditioning must provide only static geometry while temporal conditioning handles dynamics is key.\n\n3. Building on frozen CogVideoX preserves pretrained priors and suggests the approach could generalize to other video models.", "weaknesses": "1. The ablation study is not sufficient, see the question parts.\n2. There are some recent works try to address the scene-consistent camera controlled video generation (Line 336), for example, Cameractrl 2 and apt2.\n3. The method only tested on CogVideoX.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AvWtmJQhea", "reviewer": "Reviewer_iskH", "summary": "The paper proposes a method to maintain geometric and temporal consistency for camera-controlled video extension generation. For temporal consistency, the last 9 frames of the original video are given to the model as a condition. For spatial consistency, the method extracts a 3D point cloud representation of the static scene from the original video and projects the point cloud to corresponding views, and provides the model as a condition, hence the name ScenePrompting. The 3D point cloud only encodes the static scene by masking out the dynamic objects. The paper's evaluation shows improved performance compared to prior methods, and the ablation studies validate the effectiveness of the design choice.", "questions": "1. Regarding weakness 1, can the authors explain why competing explicit 3D memory works are not mentioned and compared against? I think this is critical for the paper's acceptance.\n\n2. Regarding weakness 2, can the author explain the relationship between the proposed dynamic object removal pipeline and existing works?\n\n3. The paper's introduction section inspires the development of the proposed method for long-duration generation. Can the author clarify the maximum duration the model supports? And also, how long were the generated videos used for evaluation? The supplementary material only shows very short example videos, where the original videos are so short and can completely be fitted to the attention as a condition, which eliminates the need for such complex spatial prompting design, and the generated videos are also short, which does not showcase the enhancement using the method.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The paper is clearly written. The idea of using explicit 3D point cloud for long-term video generation memory is sound.", "weaknesses": "1. Missing comparisons to competing works [1, 2, 3]. The paper completely does not mention or compare with these prior works that also utilize explicit 3D memory for video generation.\n\n2. Missing related works: CameraCtrl2 [4] explores improved camera conditioning and training on a large-scale foundational model. APT2 [5] explores long-duration and real-time camera-controlled generation. These works can be included in the related works section.\n\n3. The paper proposes methods to remove dynamic objects from video. However, erasing and inpainting dynamic objects from video is an established research topic [5]. The paper does not mention such previous work.\n\n4. The model in the paper can only generate short videos of a few seconds, which limits the actual use case of long-duration memory. The videos in the supplementary materials do not show significant improvement. Also, the method only compares against old works (CameraCtrl1 instead of CameraCtrl2).\n\n[1] Video World Models with Long-term Spatial Memory\n\n[2] Learning 3D Persistent Embodied World Models\n\n[3] WORLDMEM: Long-term Consistent World Simulation with Memory\n\n[4] CameraCtrl II: Dynamic Scene Exploration via Camera-controlled Video Diffusion Models\n\n[5] Autoregressive Adversarial Post-Training for Real-Time Interactive Video Generation\n\n[6] Flow-edge Guided Video Completion", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fpZshnfsEO", "reviewer": "Reviewer_xUs1", "summary": "The method conditions a video diffusion backbone on both the last few temporal frames for motion continuity and projected views from a static‑only 3D scene memory for spatial consistency, where the memory is created by dynamic SLAM plus a masking pipeline that removes moving objects using optical‑vs‑warped flow differencing, backward point tracking, and SAM2 propagation to yield clean static point clouds. Static‑only point clouds are projected from spatially adjacent viewpoints to the target camera poses and concatenated with temporal latents through a frozen 3D VAE channel, enabling camera‑controllable generation without modifying the DiT backbone.", "questions": "I have highlighted the major questions in the weaknesses already, and I am summing them up below(I have summarised them shortly so that it is easier for the reviewer to quote the exact question they are answering. Please refer to the Weakness for the detailed problem and question asked.\n1. Why is the spatial structure not preserved in the first supplementary video where two chairs were supposed to be side-by-side, but instead a white pillar or cupboard appears?\n2. Is there any error in camera pose estimation that, when back-projected to image space, causes inaccurate projections and leads to inconsistency?\n3. Are the comparisons fair, given that your method takes both the initial few frames and the last few frames as input, while the other methods only use the last frame as prior?\n4. Are the better qualitative results mainly due to conditioning on more input frames rather than the proposed architectural changes?\n5. Why didn’t the authors train a baseline like CameraCtrl or another strong model with minimal modifications to take more input frames and verify whether improvements come from more inputs or architecture changes?\n6. What are the limitations of the proposed work?\n7. What are the potential future directions or improvements for this work?\n8. Why are there so few video results in the supplementary, only a handful instead of 15–20 videos as expected for a video generation paper?\n9. Are the generated videos coherent with each other when given the same initial video but different trajectories, especially in the static regions?\n10. Why hasn’t the paper compared its method with ReCamMaster, given that its evaluation code has been available since before the ICLR deadline?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "1. Proposed new problem statement which is relevant to Video Generation which the previous camera controllable methods which conditions only on the last few frames\n2. Dual spatio‑temporal conditioning addresses both motion continuity and scene consistency, overcoming the short‑window limitation of prior future‑video models.\n3. A static‑only 3D scene memory cleanly separates persistent geometry from dynamics, avoiding the leakage of outdated moving objects into future frames.\n4. Three‑stage dynamic masking (optical‑vs‑warped flow differencing, backward point tracking, SAM2 propagation) yields clean static point clouds without ghosting.", "weaknesses": "1. In the first video in the supplementary video, I see that the spatial structure is not preserved. In the initial video, there were supposed to be 2 chairs side-by-side but I don’t see them in the generated video even though the 3D scene memory captures it. Instead a white pillor / cupboard seems to appear which is wrong. Why is it so? Is there any error in camera pose estimation that when back projected on image space, they are not projected accurately and may lead to inconsistency?\n2. I feel like the comparisons are not fair. The video comparisons are all against methods which only do camera control video generation with only the last frame as prior. But your method takes in the few initial videos frames along with the last few frames for video generation and so automatically your method would be more consistent since it is taking more frames as input. This is an unfair comparison in my opinion. The good qualitative results maybe just because you are conditioning your model with more priors and hence more consistent results or is it because of your proposed new architectural changes that enabled better consistent results. I feel the paper didn’t differentiate these two effectively. I agree that it is a novel problem statement and hence no baselines exist but the authors could have trained CameraCtrl or any of the best model to take in more inputs and trained the model with very minimal changes to effectively clarify if the difference in clarity is from more data inputs or actual model architectural changes. \n3. There are no limitations mentioned in the main paper or the appendix. I request the authors to propose the limitations of the work and the future work. It is important for the reviewers and the community to know the limitations so that there could be more work/discussions in that direction to resolve it. \n4. Minimal video results. For a paper proposing video generation, I would expect a minimum of 15-20 videos to be shown in the supplementary. It is difficult for the reviewer to judge if the results are cherry picked or does it actually generalise well on many videos and different camera trajectory. \n5. There are no results shown in the case where given an initial video and 2-3 different trajectories, are all 2-3 videos coherent to the initial video atleast in the region of static part. Are the generated videos coherent with each other in the common regions?\n6. What about a comparison against the newly released ReCamMaster? It is an ICCV paper I assume and hence the evaluation code has been released long before ICLR deadline. I would request the authors to compare this work with their method and show few comparison.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)