Video World Models with Long-term Spatial Memory
Authors: Tong Wu
Shuai Yang
Ryan Po
Yinghao Xu
Ziwei Liu
Dahua Lin
Gordon Wetzstein
Venue: NeurIPS 2025 poster ()
Keywords: diffusion model
world model
Primary area: applications
TL;DR: A memory framework for video world models that improves long-term consistency

Abstract: Emerging world models autoregressively generate video frames in response to actions, such as camera movements and text prompts, among other control signals. Due to limited temporal context window sizes, these models often struggle to maintain scene consistency during revisits, leading to severe forgetting of previously generated environments. Inspired by the mechanisms of human memory, we introduce a novel framework to enhancing long-term consistency of video world models through a geometry-grounded long-term spatial memory. Our framework includes mechanisms to store and retrieve information from the long-term spatial memory and we curate custom datasets to train and evaluate world models with explicitly stored 3D memory mechanisms. Our evaluations show improved quality, consistency, and context length compared to relevant baselines, paving the way towards long-term consistent world generation.

Video World Models with Long-term Spatial Memory
Tong Wu*!, Shuai Yang*2.4 Ryan Po', Yinghao Xu', Ziwei Liu", Dahua Lin Gordon Wetzstein'
Stanford University
Shanghai Jiao Tong University Thhe Chinese University of Hong Kong Shanghai Artificial Intelligence Laboratory
6 S-L.ab, Nanyang Technological University
httpa://sppent http///mmm.githuu..... github. io/
headed anle through nosit courtyard Autuen frees goiden stone while stotue and old architectune evoke sense hultory mystery.
Global Static Point Cloud
Dynamic Video Generation
Spatial meem
moves forword horseback siong or empty city street, buildege shiiit
Global Statie Point Cloud
Markinseue
lihtt
Video Generation
Spotial-tres
Figure 1: We augment video world models with memory. In this context, we consider the conven- tional approach conditioning autoregressively generated frames with few recent context frames as short-term working memory. We explore two additional mechanisms modeling different types of long-term memory: spatial and episodic memory. The former is represented as a point map that is autoregressively generated along with the video frames and fused into the spatial memory by extracting only its static scene parts. To remember visual detail and identities for long time horizons, also store sparse set of historical reference frames episodic memory. Together, our memory mechanisms significantly improve the long-term consistency of emerging video world models.
Abstract
Emerging world models autoregressively generate video frames in response actions, such as camera movements and text prompts, among other control signals. Due to limited temporal context window sizes, these models often struggle maintain scene consistency during revisits, leading to severe forgetting of previously generated environments. Inspired by the mechanisms of human memory, introduce a novel framework to enhancing long-term consistency of video world models through geometry-grounce
long-term spatial memory. Our framework includes mechanisms to store and retrieve information from the long-term spatial memory and curate custom datasets to train and evaluate world models with explicitly stored 3D memory mechanisms. Our evaluations show improved quality, consistency, and context length compared relevant baselines, paving the way towards long-term consistent world generation.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Introduction
World models are generative systems that learn predict an environment response actions, making them well suited for simulating complex, interactive settings [28]
2
30
74
[0]
Video
diffusion models T 37 79 (55 have emerged as powerful approach to architecting world models, especially when used with autoregressive next-frame prediction [1
12 I8 22
41
53
60
65
[3 35]. Existing video generation models, however, often struggle with long- honi-horizon consistency due to limited temporal context windows, frequently forgetting previously seen scenes during revisits. This is due to the relatively small number of previously generated context frames that the model can consider when generating new frames-a problem primarily caused by the quadratic growth of computational complexity in the attention module of the underlying diffusion transformers.
To address this challenge, current world models simply keep the number of context frames low to maintain computational feasibility. Several very recent approaches explore progressive downsampling of temporally more distant frames to increase the temporal context window size [25] 86]. Yet, all these approaches rely on image-basede representations of the past and lack a persistent 3D understanding of the world, limiting spatial consistency.
Inspired by the mechanisms of human memory B], we propose new framework to enhance the long- term consistency of video world models through long-term spatial memory grounded in geometry. Drawing from cognitive theories, our approach incorporates three distinct forms of caamooyy-patiien working, and episodic-eaa each modeled through a dedicated representation. Similar to existing models, our framework relies on a set of recently generated context frames. We consider this a short-term working memory mechanism, making both static and dynamic aspects of the most recent past accessible in the form of pixel data. To help remember long-term spatial relationships, we introduce an additional long-term spatial memory. This mechanism primarily relies on an explicit 3D representation of the generated world, augmented by sparse episodic memory in the form of a set of keyframes from the past. We implement the spatial memory using geometry geometry--rounded point cloud representation. Before storing newly generated information into this memory mechanism, we filter out dynamic parts of the world to primarily remember the static parts of the work in this memory.
The primary contribution of work is the design of our memory mechanism, combining short-term working memory with long-term spatial and sparse episodic memory, as illustrated in Figure [] We develop approaches store newly generated information into the spatial and episodic memory bank as well as retrieve information from it to effectively condition the generation of new video frames. Moreover, we curate a custom dataset to train and evaluate proof-of proof-of-principle implementation of the proposed mechanisms. Our evaluations show that the quality and 3D consistency of our approach surpasses that of relevant baselines. With this work, we hope to contribute to the growing community effort of unlocking infinite-length, consistent world generation capabilities for computer graphics, robotics, and other interactive applications.
2 Related work
We consider world model to be a generative system that autoregressively generates image or video frames, conditioned on actions or camera pose. This work builds on several important concepts the generative AI literature, which we briefly review in the following.
Image and video generation. In recent years, diffusion models have emerged as the sfate-of-the-art paradigm for image generation [3I) surpassing the performance of BAN-based methods in both fidelity and diversity by modeling complex distributions through iterative denoising 45
67
20 These successes have naturally extended to the task of video generation, as diffusion-based architectures have been adapted to include the temporal domain, enabling the generation of high- quality video clips spanning the order of tens to hundreds of frames [89] [8] 78, [56] [1 14 8)
Additional advancements in efficiency and controllability have also followed, through techniques
such improved tokenization (63] 75 and flow-matching objectives 49 40)
Autoregressive video generation. To support generation of longer videos with online inference capabilities, autoregressive (AR) approaches and architectures have been introduced in the diffusion framework (62
76 71. The autoregressive regime is also particularly intuitive, given the natural temporal ordering of video data. Taking inspiration from LLMs, state-of state-of-the-ar methods model

---

video data by processing frames into spatio- centiottempoal tokens [48 89 78 63 21 56 14, learning to autoregressively generate new tokens until full video formed. This in stark contrast with conventional diffusion-based frameworks, which choose to iteratively denoise an entire image/video Recent approaches include training conditional diffusion models that generate new frames given a set of previous clean frames, allowing AR generation simply by passing newly generated frames as context for future frames [37 34 16
at once [9] 37
29 52
88 24 [2] Others modify the diffusion objective by assigning independent noise levels for each frame during training, which also supports AR inference [13 80 60]. In practice, these methods can be used for infinite-lnngth generations by utilizing sliding window context, but suffers from limited memory and drift.
41
Controlled video generation. Controlled or conditional video generation is core component of world simulators. A series of works have explored explicit camera control, including [72 32 33
33 5 4 which enable novel-view multi-view generation by disentangling camera trajectories from dynamic content. Most of these approaches take either a single image or a text prompt as input, and often struggle maintain long-term consistency, especially in dynamiic scenes. More recent efforts 88 7 [ focus on re-filming or generating synchronized views in dynamic settings, further advancing controllability. Beyond direct camera injection, structural conditioning via point clouds, tracking. or 3D-aware priors has proven effective for improving spatial consistency and trajectory alignment 83 57 26 82 [3]. In addition spatial and motion-level control, some models support action-based or scene-level conditioning [54] 19], where either structured action vocabularies or segmented text prompts serve as high-level drivers of video progression.
Long-context video generation. While advances in autoregressive methods have allowed for generation of longer, and even potentially iniftnite-length videos (14), the reliance sliding context windows limit their effective memory. The straightforward approach of increasing memory by training with longer context lengths is effective but computationally demanding. Recent works have explored efficient architectural alternatives to attention-based transformers, such as state-space models and linear attention 669 50 While efficient, such methods often perform worse than conventional diffusion transformer architectures. Other concurrent works have proposed compressing prior frames lower context length [86] 25], subsequently lowering computational demand the cost of information lost in context frames. A separate branch of works ground previously generated frames in a 3D representation (e-g.. point clouds X3
26
82
57 and query this representation
based on the camera position of future frames. These methods allow for precise camera control, but struggle to handle dynamic scenes with complex motions. SlowFast- FGen [8] introduces a dual-branch slow-fast learning paradigm to explicitly disentangle long-term temporal dependencies from short-term motion dynamics, enabling more coherent and action-aware long video generation. Complementarily, (17) leverages continual adaptation at inference to maintain temporal consistency over extremely long sequences, demonstrating that lightweight test-time optimization can substantially extend generation duration without retraining.
In this work, we adopt the conventional approach of concatenating recent context frames as working memory in diffusion based generation to autoregressively generate multiple future frames. Meanwhile, we iteratively predict and filter the static point maps of newly generated frames to update global spatial memory, which serves geometric guidance for long-term generation. We further incorporate a sparse set of historical reference frames as an episodic memory.
3 Method
Our framework includes mechanisms for storing new observations into memory and retrieving information from it. Drawing inspiration from three distinct forms of human memory- spatial, working, and episodic-we introduce a memory storage mechanism that models each type using a dedicated representation. In the following section, we describe how each memory type is constructed and maintained (Sec. (.2)) how is integrated into the model as a conditional signal (Sec. 3.3). 3 and how curate the appropriate data to facilitate efficient learning of these mechanisms (Sec. 3.4
3

---

Statie point cloud
Ceadition
readering
DVT STTSSAA
DNT NITSSAK
VAE
Ene.
Dynamic Video Generation
Pachulty
DH Bioc
Bi Clus
VAE
sere-gaiddig
DIT
OIT
Action Text
13
Statie Point Cloud
Densising BiT
Pachify
VAE line
Historical
Network.rchitteetnn Architecture
reference
frames
update
Figure 2: Overview of our system. A latent video generation model, implemented by a diffusion transformer (DiT), is conditioned three different memory mechanisms when autoregressively generating new frames. First, recent context frames model a short-term working memory. Second, a point cloud representation (left) is autoregressively generated along with the video frames. This long-term spatial memory contains the static parts of the world. Third, set of historical reference frames (lower left) is stored as a sparse, long-term episodic memory. Together, these memory mechanisms enable consistent long-term video generation.
3.1 Preliminaries
61 learn model data distribution by reversing forward diffusion process denoted by 0g + o2C, where the positive scalar parameters a, and determine the signal-to-noise ratio based on a predefined noise schedule, and is drawn from a standard Gaussian distribution. The diffusion model then learns to predict the noise added through the following denoising objective:
Diffusion models 559
36
(()) = Eo,.. (|Eo( t) -el12-
(1)
Video diffusion models commonly operate in latent space, following two-stage process: first, input videos are encoded using 3D variational autoencoder (VAE) [66] (7 I0], and then a diffusion model trained to model the resulting latent representations. Generation proceeds by sampling within the latent space and decoding the results back to pixel space. Our prototype implementation builds on Cog VideoX [79], which adopts this two-stage framework. Specifically, Cog VideoX employs diffusion transformer with 3D attention blocks to capture the distribution of the latent space. Our model improves upon this architecture with additional control signals for enabling long-ten memory.
3.2 Spatial Memory Storage
Coherent Static Structure for Spatial Guidance. Human spatial memory refers to our ability to encode, store, and retrieve information relating to the physical layout and structure of our environment. To this end, we construct a persistent spatial memory in the form of a long-term static point map that captures high-confidence, temporally consistent 3D structures. To further separate static elements like buildings from dynamic elements like characters and animals, we adopt truncated signed distance function (TSDF) fusion (84). We denote the TSDF value and associated weight of voxel v as (vv) and W (u). respectively. Given new observation from frame i, the voxel update rule follows the standard weighted averaging:
D'(v) W(v) D(v) + wi di(v) W(v) + w;
W'(v) W(r) + wi,
where dv(v) is the truncated signed distance between voxel and the observed surface in frame i, and w; is a rreeeddepnn confidence weight (typically set to 1).
This fusion process inherently filters out dynamic elements in the scene: due to inconsistent depth observations across frames, such voxels accumulate low-confidence, noisy TSDF values and are naturally suppressed in the final fused volume.

---

During the autoregressive generation process, spatial memory is incrementally updated with newly observed static maps, which are reconstructed in an online recurrent manner by CUT3R (70] and filtered by TSDF-Fusion to eliminate the dynamic parts.
Recent Frames for Dynamic Context Guidance. In humans, working memory is in charge of temporarily holding information needed for performing reasoning and comprehension tasks. Similarly, our model requires knowledge of nearby previous frames to generate temporally coherent future frames. Drawing on this concept, we incorporate short-ferm memory stream based recent frames, which provides motion continuity atop the static scene structure. We adopt a simple autoregressive generation strategy, where the model generates N future frames by conditioning each step on the most recent & + latent frames. This procedure can be iteratively applied, enabling open-ended video generation with consistent temporal dynamics.
Representative Historical Slots to Enhance Details. Human episodic memory is a type of explicit memory that stores specific important events from the past, allowing us to "recall" the relevant experiences when needed. While the fused static point cloud captures stable scene geometry, the fused static point cloud often too sparse to preserve detailed visual cues from the past. To compensate for this, we maintain a set of representative historical frames as auxiliary references. Specifically, during generation, we monitor the size of newly revealed unknown regions via mask- based visibility checks. When the revealed area exceeds a predefined threshold, the corresponding frame is selected and added to the memory set in an incremental fashion.
3.3 Memory-guided Video Generation
Different from those video generation models that focus on camera control under the same temporal sequences 82 6 83 26), we mainly focus the temporal progression. For example, for driving on the road, when the camera moves to the left, the car should complete reasonable dynamic motion in accordance with the hints of the camera language and the input prompt. while maintaining the consistency of the static part to achieve the simulation of the interactive world model. Based on this requirement, we carefully design several key modules to achieve dynamic and static decoupling.
As illustrated in Figure first, we introduce static point cloud rendering as an additional conditioning input to our video diffusion model. The condition video is rendered from the current static spatial memory along the input trajectory, with background regions lacking point clouds set to black. We then utilize the pre-trained 3DVAE 77] to encode the static point cloud rendering into condition latents. We follow a similar design as the ControlNET [87 to add the static point clouds rendering to guide the camera movement and keep the static areas consistency. We copy the first 18 pre- pre-trained DiT blocks from Cog VideoX as the condition DiT to process the condition latents. In the condition DIT, we process the output feature from each main DiT block through a zero-initialized linear layer before adding it to the corresponding feature map in the main DIT. Second, to support the generation of new dynamic elements and the temporal extension of existing ones, propose to concatenate the last five frames of source video tokens with the target video tokens along the frame dimension for dynamic context guidance. In addition, the target condition tokens are also combined with recent context tokens as mentioned above to ensure frame-level correspondence. Third, for modeling information exchange between memory frames and the frames currently being generated, select the representative historical slots frames as auxiliary reference frames. This reference frames are also encoded by 3DVAE and patchify them as reference tokens. we add a historical cross attention to guide information exchange between the frames currently being generated and memory frames. Specifically, the video tokens act as queries and the reference tokens serve as keys and values.
3.4 Geometry-groundee Video Dataset Creation
To train and evaluate our geometry-grounded video generation model, require custom dataset as shown in Figure 3 which is described in the following.
Dataset Construction. We build our dataset from raw videos collected from MiraData [42] segmenting each video into multiple 97-frame clips. For each clip, the first 49 frames serve as the source sequence and the remaining 48 as the target, with a shared transition frame between the source and target sequence to preserve temporal continuity. To recover scene geometry, perform
5

---

Source video
Target video
Whole Clip
MegaSa
Dynamic Point Cloud
Action Annotation
character juays our panes and polferrione wailt artty der coummakk, set agalisst citeeratic arhav backdros.
map
TSDF Fusion
Guidance
Dynamic Prediction
Static Point Cloud
Figure 3: Dataset construction pipeline. We use Mega-SaM [47] to extract camera poses and dynamic point maps from the full video clip. For the source part, dynamic regions are erased via TSDF-Fusion, and the point cloud is rendered along the target trajectory to to serve as static geometry guidance for the target part. Qwen 77 generates annotations for actions in future target frames.
Table I: Quantitative evaluation. We evaluate method and baselines using FVD, view recall, and a user study. For view recall, we use standard metrics to compare the consistency of revisiting parts of scene. The user study provides relative average human ranking scores for three different criteria of all baselines. Our method outperforms these baselines all cases.
View Recall Consistency
User Study
Method
FVD
PSNR
SSIM
LPIPS |
Cam-Ace
Stat-Cons
Dyn-Plaus f
TrajectoryCrafter
355.23
11.71
0.4380
0.5996
1,6320
1.7802
1.6255
DaS
363.36
12.01
0.4512
0.5874
2.5660
2.4396
2.7033
Wan2. Innpaininn
280.06
12.16
0.4506
0.5875
2.1760
2.3956
2.2701
Ours
157.11
19.10
0.6471
0.3069
3.6260
3.3846
3.4011
4D reconstruction using Mega-SaM 47 depth maps. We apply TSDF-Fusion to the source frames, integrating RGB-D observations into volumetric grid. This process suppresses inconsistent depth caused by dynamic objects, yielding a clean reconstruction of the static scene.
extracting camera intrinsics, extrinsics, and per-frame
Paired Training Data. Given the fused geometry, we project the target camera poses to render visibility masks and static-region reconstructions via point-based rendering. The full RGB frames of the target sequence are retained as future supervision, containing dynamic elements beyond the static scene memory. Our final dataset comprises 90K structured video samples, each paired with explicit 3D spatial memory and future observations.
Our dataset can be downloaded here Additional details memory storage and retrieval mechanisms as well as dataset creation are found in the supplement.
Experiments
Implementation Details. We implement our conditional video diffusion model based on VegViiXo-SB-BPP [79] architecture, pretrained from DaS (26). During training, we set the video length to 49 frames with resolution of 480 720. We trained for 6,000 iterations with a learning rate x 10 10-5, using a mini-batch size of 8 and are conducted on eight NVIDIA-A10M GPUs. At
inference time, we adopt the latest 5 historical frames from the recent sequence to enable smooth

---

Table 2: Quantitative evaluation on VBench. Our model achieves top overall performance among
relevant baselines, as measured by VBench metrics.
Aesthetic Quality 1
Imaging Quality f
Temporal Flickering
Motion Smoothness t
Subject Consistency t
Background Consistency 1
Method
TrajectoryCrafter
0,5255
0.6428
0.6160
0,9843
0,8830
0.9227
DaS
0,5635
0.6617
0.7520
0,9856
0,9325
0,9494
GEN3C
0.5203
0,5654
0.7179
0,9882
0.9178
0,9433
Wan2. -Inpainting
0,5661
0.6788
0.6433
0.9868
0.9357
0.9513
Ours
0.5835
0.6701
0.7580
0.9886
0.9359
0.9506
motion prediction. At each auto-regressive iteration, the point map of the newly generated frames are predicted, aligned, and fused into the historical global point clouds, where we create new point rendering sequence given the aimed camera trajectory.
Metries and Baselines. Our evaluations primarily focus on baseline methods that use point-map- based conditioning. This includes TrajetoryCrafter [82), DiffusionASShader (DaS) [26]. GEN3C (57), and also the statc-of-the-ar video generative model 1 [68]. Our test set includes 500 randomly selected video sequences from MiraData, which are not seen during training. We evaluate each method on FVD, view recall consistency, and general video quality on multiple dimensions. Specifically, 1) For static spatial information, the view recall consistency evaluated using image reconstruction metrics (i.e., PSNR, SSIM, and LPIPS) for paired frames at the same camera location within video sequence generated with forward and reversed camera trajectory. 2) For general video quality evaluation, we use the standard FVD and six metrics from Vbench [39] for the visual quality, motion smoothness, and consistency. Moreover, we conduct a user study to further validate the criteria above.
4.1 Quantitative Evaluation
Evaluating View Recall Consistency and Camera Accuracy. To verify that our method benefits from the spatial memory mechanism by maintaining high consistency and accurate camera control when revisiting previously generated parts of the world, we conduct experiments on reversed tra- jectories, where the same camera pose is visited twice, enabling the construction of paired data for reconstruction metrics. As shown in Table [ terms of FVD, PSNR, SSIM, and L.PIS compared to baselines, owing to our memory mechanisms. It is worth noting, however, that even the PSNR of our method is far from perfect, indicating that remembering each and every visual detail of a complex scene a very challenging task.
our method achieves significantly improved scores in
Evaluation on Video Quality. We further utilize VBench to evaluate general video quality across multiple dimensions, as shown Table Compared with baseline methods, our approach demon- strates better performance in aesthetic quality. reduced temporal flickering, smoother motion, and improved subject consistency. Methods primarily designed for 3D statie NVS 4D NVS within the same temporal sequence perform less effectively on our benchmark, likely due to the sparsity of the retained static point cloud and the presence large spatial holes. While Wan2.1 surpasses our method in imaging quality due to its advanced backbone, and also achieves higher score in background consistency, we note that the Wan2. inpainting model often fails to follow geometric guidance and tends to generate relatively static scenes, which makes it easier to maintain high background consistency scores.
4.2 Qualitative Evaluation
We conduct qualitative comparisons with other criteria, as illustrated in Figure 4 First, our method demonstrates superior performance in accurately following camera trajectories. guided by point map rendering. In contrast, the Wan2.1 inpainting model and DaS model often fail, especially under significant camera motion. Second, for view recall consistency, we present pairwise comparisons between two frames generated at the same camera pose but different points in the sequence. Compared with the baselines, our results exhibit significantly higher consistency in static regions during scene revisits, thanks to the static memory mechanism. Finally, we evaluate the ability to generate new actions based on instructions while incorporating static memory. We focus on the harmonious integration of static and dynamic elements, as well as
methods, focusing on three key
7

---

Wan2.1
UinjettryyCrafter
DAS
Ours
Camera Accuracy
Camera
Cam-3
Cam-1
Camera 2
Look
First-time time
View Recall Consistency
Recall
Frame 0
Action
Accuracy
"Crouching carefully, cressing a norrow log bridge"
Frame
Qualitative evaluation. We compare our approach to relevant baselines in several conditions. Baselines cannot accurately generate significant camera pose changes while maintaining a consistent scene (top). When revisiting a previously seen camera pose, baselines fail to complete sparse point clouds forget details, resulting in inconsistency (center). The accuracy of prompted actions often low, and sometimes the character disappears during the generation for the baselines (hottom). Our approach successfully handles these challenging scenarios.
Figure 4:
how closely the dynamic components follow the instructions. The comparison shows that our method performs well in action prediction, whereas the others either fail to accurately follow the instructions or suffer from action drifting, severe deformation, or even character disappearance.
4.3 User Study
We selected 14 representative use cases, including novel-view synthesis of static scenes, novel-view synthesis of dynamic scenes with temporal progression of dynamic subjects in first-person and person perspectives, and scene styles covering realistic and game style. We conduct a user study, evaluating baselines and our methods from three perspectives: camera CecuuracyCCamAAaeee (attssttencyyt-CCCoooaa and dynamic plausibility (Dyn-Plaus). We invited 20 subjects, each with least one year of experience in video/3D/4D generation, to rank the results generated by the four DaS, Wan2.1 1- Inpainting, and ours). Following ControlNet, we evaluate the results using the Average Human Ranking (AHR) metric, where participants rated each output on scale of to 4 (with lower scores indicating poorer quality). The average rankings in Table [] (right) show that our method achieves clear and consistent improvements over the baselines across all metrics.
static
methods (TrajectoryCraffes
4.4 Ablation Study
To verify the effectiveness of each memory component in our video generation framework, we conduct comprehensive ablation studies, as shown in Table 3 Experimental results on VBench metrics indicate that each component consistently contributes performance improvements. Unsurprisingly,

---

Static Point Map
w/o Working Memory
w/o Episodic Memory
Full Model
Ground Truth
Figure 5: Ablation of different memory mechanisms. We evaluate several variants of our model: w/o short-term working memory: the full model without recent context frames; w/o long-term episodic memory: the full model without sparse historical keyframes; full model including short- term working and long-term spatial and episodic memory. Unsurprisingly, the working memory is required for smooth and plausible motions of dynamic objects. The episodic memory is crucial in helping remember visual details from the past, including previously seen characters or objects.
Table 3: Ablation of memory mechanisms. Using all three types, i.e., short-term working and terg-term spatial and episodic memory, leads to the best results measured by VBench metrics.
Aesthetic Quality
Imaging Quality f
Temporal Flickering
Motion
Subject Consistency t
Background Consistency 1
Method
Smoothness t
w/o episodic
0.5603
0.6485 0.6384
0.7260
0.9870
0.9326
0.9489
w/o working mem
0.5551
0.6740
0.9862 0.9886
0.9331
0.9453
Full model
0.5835
0.6701
0.7580
0.9359
0.9506
the context frames play crucial role in enhancing short-term motion coherence. During the autoregressive generation process, the context frames enable smooth transitions in dynamic regions and help produce more plausible motions consistent with the preceding frames. The sparse set of historical reference frames enable the model to better retain and utilize temporally distant details. This episodic memory improves long-term consistency for static regions and subjects, and further enhances the plausibility and continuity motions involving moving entities. The best results are achieved with both of these mechanisms as well as our long-term spatial memory. This is evidenced by both Table and Figure showing that each our memory mechanisms significantly improves the model's quality in terms of motion smoothness, static consistency, and overall visual quality.
5 Discussion
Inspired by the mechanisms of human memory, we introduce soometry-grounder long-term spatial memory mechanism for video world models. This mechanism improves quality, spatial consistency, and context length compared to relevant baselines.
Limitations and Future Work. The TSDF-Fusion algorithm we use for storing newly generated information into the spatial memory is far from perfect. Specifically, artifacts are introduced when looking at previously generated content from camera poses that are very different from those of the previous observations, illustrated in Figure 6 Our memory mechanism is primarily designed to en- able spatial consistency whereas frame packing strategies for extending the temporal context window size [86] primarily focus character consistency. Future work may combine these mechanisms to achieve both types of consistency. The forgetting problem we tackle is just one of several challenges of video world models. Drift, or image quality degradation due to error accumulation over time, is another challenge that we do not address.
Societal Impacts. Video generation models provide significant benefits for content creation but could be adapted for DeepFake generation. Such applications pose significant societal risks and we strongly oppose the use of our work to create deceptive content intended to mislead or spread misinformation.

---

Statie Point Map
Ours
Ground Truth
Figure 6: Failure case. When the distance between consecutive camera poses is too large and the trajectory exhibits overly abrupt angles, the 4D) reconstruction may fail, resulting in significant ghosting artifacts between frames. Consequently, TSDF-Fusion will filter out a large portion of point clouds that should belong to static regions, ultimately leading to extremely sparse spatial memory and loss critical information. For example, Spiderman rapidly traversing between skyscrapers illustrates how such challenging camera trajectory can cause omissions in spatial memory storage, resulting in imprecise camera control and inconsistencies.
Conclusion. Video world models play a crucial role for content creation or creating training data for agents robots. Enabling long-term consistency through memory mechanisms like makes these models more effective.
Acknowledgment
We thank Google and Shanghai Artificial Intelligence Laboratory for their support. Ryan Po is supported by a Stanford Graduate Fellowship.
References
[1] Eloi Alonso, Adam Jelley, Vincent Micheli, Anssi Kanervisto, Amos Storkey, Tim Pearce, and François Fleuret. Diffusion for world modeling: Visual details matter in atari. Advances in Neural Information Processing Systems, 37:58757-58791 2024.
[2] Genesis Authors. Genesis: A universal and generative physics engine for robotics and beyond.
URL http:://github. com/Cennsts-Embbottt----CCC
2024.
[3] Alan Baddeley. Essentials of human memory. Psychology Press, 2013.
[4] Sherwin Bahmani, Ivan Skorokhodov, Guocheng Qian, Aliaksandr Siarohin, Willi Menapace, Andrea Tagliasacchi, David B. Lindell, and Sergey Tulyakov. Ac3d: Analyzing and improving 3d camera control in video diffusion transformers. Proc. CVPR, 2025.
[5] Sherwin Bahmani, Ivan Skorokhodov, Aliaksandr Siarohin, Willi Menapace, Guocheng Qian, Michael Vasilkovsky, Hsin- Ying Lee, Chaoyang Wang, Jiaxu Zou, Andrea Tagliasacchi, David B. Lindell, and Sergey Tulyakov. VD3d: Taming large video diffusion transformers for 3d camera control. In The Thirteenth International Conference on Learning Representations, 2025, URL https::/ httpas:///ppeeeviieee www.ne/ooooomdddnnndadaoaao
[6] Jianhong Bai, Menghan Xia, Xiao Fu, Xintao Wang, Lianrui Mu, Jinwen Cao, Zuozhu Liu, Haoji Hu, Xiang Bai. Pengfei Wan, and Di Zhang. Recammaster: Camera- generative rendering from single video, 2025. URL Rtl.hhtts://arrte wtwpe/vv.rr/abbb//0o000 11647
[7] Jianhong Bai, Menghan Xia, Xintao Wang, Ziyang Yuan, Zuozhu Liu, Haoji Hu, Pengfei Wan, and Di ZHANG. Syncammaster: Synchronizing multi-camera video generation from diverse viewpoints. In The Thirteenth International Conference on Learning Representations, 2025.
URL httphttp//pppereeiiiieeeee hwtpaarrweeeeeeeeeffoorrrroaoaaa
10

---

[8] A. Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch. Maciej Kilian, and Do- minik Lorenz. Stable video diffusion: Scaling latent video diffusion models large datasets. ArXiv, abs/2311.15127, 2023. 265312551
URI. ktttps:/aat semant org/ CorpusID
[9] Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Do- minik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint wrXi:2331..15222 2023.
[10] Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh. Video generation models as world simulators. 2024. URL httpssss://open com com/research/ Dideo-generatee cenneration-models-eet -ode-s-dd-llleeeeeeee world- simulators
[11] Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor. Troy Luhman, Eric Luhman, et al. Video generation models as world simulators. OpenAl Blog, 1:8, 2024.
[12] Haoxuan Che, Xuanhua He, Quande Liu, Cheng Jin, and Hao Chen. Gamegen-x; Interactive open-world game video generation. arXiv preprint arXiv:2411.00769 2024.
[13] Boyuan Chen, Diego Martí Monsó, Yilun Du Max Simchowitz, Russ Tedrake, and Vincent Sitzmann. Diffusion foreing: Next-token prediction meets full-sequence diffusion. In NeurIPS, 2024.
[14] Guibin Chen, Dixuan Lin, Jiangping Yang, Chunze Lin, Juncheng Zhu, Mingyuan Fan, Hao Zhang. Sheng Chen, Zheng Chen, Chengchen Ma, Weiming Xiong, Wei Wang, Nuo Pang, Kang Kang. Zhi-Xin Xu, Yuzhe Jin, Yupeng Liang, Yu-Ning Song, Peng Zhao, Bo Xu, Di Qiu, Debang Li, Zhengcong Fei, Yang Li, and Yahui Zhou. Skyreels-v2: Infinite-length film generative model. 2025. URL httph::t///ppppp www.eeiicaannnlll........e.
org/CorpusID 277856899
[15] Luxi Chen, Zihan Zhou, Min Zhao, Yikai Wang, Ge Zhang, Wenhao Huang, Hao Sun, Ji-Rong Wen, and Chongxuan Li. Flexworld: Progressively expanding 3d scenes for flexiable-view synthesis. arXiv preprint arXiv:2503. 13265, 2025.
[16] Xinyuan Chen, Yaohui Wang, Lingjun Zhang, Shaobin Zhuang, Xin Ma. Jiashuo Yu, Yali Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. Seine: Short-to-long video diffusion model for generative transition and prediction. In ICLR, 2023.
[17) Karan Dalal, Daniel Koceja, Gashon Hussein, Jiarui Xu, Yue Zhao, Youjin Song, Shihao Han, Ka Chun Cheung, Jan Kautz, Carlos Guestrin, Tatsunori Hashimoto, Sanmi Koyejo, Yejin Choi, Yu Sun, and Xiaolong Wang. One-minute video generation with test-time training. 2025. URL https://arxiv org/ abs//b///22 abse///s/220.... 252220024408888
[18) Etched Decart, Q McIntyre, S Campbell, Xinlei Chen, and R Wachen. Oasis: A universe in transformer. URL: https://oasis-model github. io, 2024.
[19] Julian Decart, Quinn Quevedo, Spruce McIntyre, Xinlei Campbell, Robert Chen, and Wachen. Oasis: A universe in a transformer. 2024. URL https:://aaii-mmmdaa .ei..gitt.u....... io/
[20] Prafulla Dhariwal and Alex Nichol. Diffusson models beat gans on image synthesis. ArXiv, abs/2105.05233, 2021. URL httt::::/app.ssmaatiiccattee httpaaatiiiicccooaaoooooooo/roap 234357997
[21] Zhengcong Fei, Debang Li, Di Qiu, Jiahua Wang. Yikun Dou, Rui Wang. Jingtao Xu, Mingyuan Fan, Guibin Chen, Yang Li, and Yahui Zhou. Skyreels-a2: Compose anything in video diffusion transformers. 2025. URL httpp://ppp http:/ppp...mmancscchollllaaa .rr. wwwpCoopoo/oocDD///.... 277750989
[22] Ruili Feng, Han Zhang, Zhantao Yang. Jie Xiao, Zhilei Shu, Zhiheng Liu, Andy Zheng, Yukun Huang, Yu Liu, and Hongyang Zhang The matrix: diriite-horizon world generation with real-time moving control. arXiv preprint arXiv:2412.03368, 2024.
11

---

[23] Kaifeng Gao, Jiaxin Shi, Hanwang Zhang, Chunping Wang, and Jun Xiao. Vid-gpt: Introducing gpt-style autoregressive generation video diffusion models. arXiv preprint arXiv:2406. 10981, 2024.
[24] Kaifeng Gao, Jiaxin Shi, Hanwang Zhang, Chunping Wang, Jun Xiao, and Long Chen. Ca2- vdm: Efficient autoregressive video diffusion model with causal generation and cache sharing. arXiv preprint arXiv:2411.16375 2024.
[25] Yuchao Gu, Weijia Mao, and Mike Zheng Shou. Long-context autoregressive video mod- eling with next-frame prediction. ArXiv, 1bs/2503.19325, 2025. URL httpss:/apt semanticscholar wrg/Corpus 2773133337
[26] Zekai Gu, Rui Yan, Jiahao Lu, Peng Li, Zhiyang Dou, Chenyang Si, Zhen Dong, Qifeng Liu, Cheng Lin, Ziwei Liu, Wenping Wang, and Yuan Liu. Diffusion as shader: 3d-aware video diffusion for versatile video generation control. arXiv preprini arXiv: 0035000,337,,0000
[27] Agrim Gupta, Lijun Yu, Kihyuk Sohn, Xiuye Gu, Meera Hahn, Fei-Fei Li, Irfan Essa, Lu Jiang, and José Lezama. Photorealistic video generation with diffusion models. In ECCV, 2024.
[28] David Ha and Jürgen Schmidhuber. Recurrent world models facilitate policy evolution. In Advances in Neural Information Processing Systems 31, pages 2451-2463. Curran Associates, Inc., 2018. URL https://papers: wwws.cccpccc...mc cc/ wwdeloddeellle-ee
paper//612-recurrent--oo httpr/d-mmmdd---aallattttteeeeee hat.eaaacdiaaateppoo...eeo.oaaaa worldmodels: perlddoodess.iithub..o
https://
[29] Yoav HaCohen, Nisan Chiprut, Benny Brazowski, Daniel Shalem, Dudu Moshe, Eitan Richard- son, Eran Levin, Guy Shiran, Nir Zabari, Ori Gordon, et al. Ltx-video: Realtime video latent diffusion. arXiv preprint arXiv:2501.00103 2024.
[30] Danijar Hafner, Timothy Lillicrap, Jimmy Ba, and Mohammad Norouzi. Dream control: Learning behaviors by latent imagination. arXiv preprint arXiv: 1912.0160, 2019.
2019.
[31] Boyu Han, Qianqian Xu, Shilong Bao, Zhiyong Yang, Kangli Zi, and Qingming Huang. Lightfair: Towards an efficient alternative for fair 12i diffusion via debiasing pre-trained text encoders. In Advances in Neural Information Processing Systems, 2025.
[32] Hao He, Yinghao Xu, Yuwei Guo, Gordon Wetzstein, Bo Dai. Hongsheng Li, and Ceyuan Yang. Cameractrl: Enabling camera control for video diffusion models. In The Thirteenth International Conference on Learning Representations, 2025. URL httttss://ppereeee//
net/forum? -aaiimiiddd--iiiiie 11v24400r
[33] Hao He, Ceyuan Yang, Shanchuan Lin, Yinghao Xu, Meng Wei, Liangke Gui, Qi Zhao, Gordon Wetzstein, Lu Jiang, and Hongsheng Li. Cameractrl ii: Dynamic scene exploration via camera- controlled video diffusion models. arXiv preprint arxiv:2503.1059) 2025.
[34] Yingqing He, Tianyu Yang. Yong Zhang, Ying Shan, and Qifeng Chen. Latent video diffusion models for high-fidelity long video generation. arXiv preprint arXiv:2211.1322) 2022.
[35] Roberto Henschel, Levon Khachatryan, Daniil Hayrapetyan, Hayk Poghosyan, Vahram Tade- vosyan, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. Streamingt2v: Consistent, dynamic, and extendable long video generation from text. arXiv preprint 24X3v:2403.1477. 2024.
[36] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In NeurlPS, 2020.
[37] Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. In NeurlPS, 2022.
[38] Yining Hong, Beide Liu, Maxine Wu, Yuanhao Zhai Kai-Wei Chang, Linjie Li, Kevin Lin, Chung-Ching Lin, Jianfeng Wang, Zhengyuan Yang. Yingnian Wu, and Lijuan Wang. Slowfast- vgen: Slow-fast learning for action-driven long video generation, 2024. URL. httph:tt///// tattps://arxiy
htg/abss24100 23277
12

---

[39] Ziqi Huang, Yinan He, Jiashuo Yu. Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive benchmark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 21807-21818, 2024.
[40] Yang Jin, Zhicheng Sun, Ningyuan Li, Kun Xu, Hao Jiang, Zhuang Nan. Quzhe Huang, Yang Song, Yadong Mu, and Zhouchen Lin. Pyramidal flow matching for efficient video generative modeling. ArXiv, abs/2410.05954 2024. URLhttps::/ http:://aap....mmmtccccooooo www.nticscoolla org/CorpusID: 273228937
[41] Yang Jin, Zhicheng Sun, Ningyuan Li, Kun Xu, Hao Jiang, Nan Zhuang, Quzhe Huang, Yang Song. Yadong Mu, and Zhouchen Lin. Pyramidal flow matching for efficient video generative modeling. In ICLR, 2025.
[42] Xuan Ju, Yiming Gao, Zhaoyang Zhang, Ziyang Yuan, Xintao Wang, Ailing Zeng, Yu Xiong, Qiang Xu, and Ying Shan. Miradata: large-scale video dataset with long durations and structured captions, 2024. URL htttps://arxiii http::::/aaaii..o//bbb////////// 20/pb///400006588
[43] Dan Kondratyuk, Lijun Yu, Xiuye Gu, Jose Lezama, Jonathan Huang, Grant Schindler, Rachel Hornung, Vighnesh Birodkar, Jimmy Yan, Ming-Chang Chiu, al. Videopoet: A large language model for zero-shot video generation. In ICMI., 2024.
[44] Weijie Kong. Qi Tian, Zijian Zhang, Rox Min. Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:24/2.03360) 2024.
[45] Alex X. Lee, Richard Zhang, Frederik Ebert, P. Abbeel, Chelsea Finn, and Sergey Levine. Stochastic adversarial video prediction. ArXiv, abs/1804.01523,, 2018. URL https://api semanticscholar org :rg/CorpusIID:400 4591836
[46] Muyang Li, Ji Lin, Chenlin Meng, Stefano Ermon, Song Han, and Jun-Yan Zhu. Efficient spatially sparse inference for conditional gans and diffusion models. Advances in neural information processing systems, 35:28858-28873, 2022.
[47] Zhengqi Li, Richard Tucker, Forrester Cole, Qianqian Wang, Linyi Jin, Vickie Ye, Angjoo Kanazawa, Aleksander Holynski, and Noah Snavely. Megasam: Accurate, fast and robust structure and motion from casual dynamic videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025.
[48] Bin Lin, Yunyang Ge, Xinhua Cheng, Zongjian Li, Bin Zhu, Shaodong Wang. Xianyi He, Yang Ye, Shenghai Yuan, Liuhan Chen, Tanghui Jia, Junwu Zhang, Zhenyu Tang. Yatian Pang. Bin She, Cen Yan, Zhiheng Hu, Xiao wen Dong, Lin Chen, Zhang Pan, Xing Zhou, Shaoling Dong, Yonghong Tian, and Li Yuan. Open-sora plan: Open-source large video generation model. ArXiv, abs/2412.001311, 2024. URL https:// htstp:://ap..semanticchheee www.nticscholla..... Drg//oorpuII:: 274436278
[49] Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. ArXiv, abs/2210.027477, 2022. URL https://api
semanticscholes argg/CorpssD:25273555
[50] Songhua Liu, Weihao Yu, Zhenxiong Tan, and Xinchao Wang. Linfusion: 1 gpu, 1 minute, 16k image. ArXiv, abs/2409.020977, 2024. URL htttps://api..smmatiiccss /ttppp//////////ee..eattattaaaaa CorpusID:27236668)
org/
[51] Michaël Mathieu, Camille Couprie, and Yann LeCun. Deep multi-scale video prediction beyond mean square еттоr. CoRR, abs/1511 1.004400, 2015. URL
org/Corpus ID: 2005144
[52] Yuta Oshima, Shohei Taniguchi, Masahiro Suzuki, and Yutaka Matsuo. Ssm meets video diffusion models: Efficient long-term video generation with structured state spaces. arXiv preprint 203.722403.07711, 2024.
13

---

[53] J Parker-Holder, P Ball, J Bruce. V Dasagi. K Holsheimer, C Kaplanis, A Moufarek, G Scully, J Shar, J Shi, al. Genie 2: A large-scale foundation world model. URL: http://deepmind. 2024.
gonggeediccoeer/bog///geeeee haa-aagggggniii----r-ee-eeee
[54] Jack Parker-Holder, Philip Ball, Jake Bruce. Vibhavari Dasagi, Kristian Holsheimer, Christos Kaplanis, Alexandre Moufarek, Guy Scully, Jeremy Shar, Jimmy Shi, Stephen Spencer, Jessica Yung, Michael Dennis, Sultan Kenjeyev, Shangbang Long, Vlad Mnih, Harris Chan, Maxime Gazeau, Bonnie Li, Fabio Pardo, Luyu Wang, Lei Zhang, Fred- eric Besse, Tim Harley, Anna Mitenkova, Jane Wang. Jeff Clune, Demis Hassabis, Raia Hadsell, Adrian Bolton, Satinder Singh, and Tim Rocktäschel. Genie 2: large-scale foundation world model. 2024. URL. http:://eemmm..oooogo///eaaeerer genie -arg-arree-seee fonoll--funnntiioon world modeddmmmmdd///--
[55] Ryan Po, Wang Yifan, Vladislav Golyanik, Kfir Aberman, Jonathan Barron, Amit Bermano, Eric Chan, Tali Dekel, Aleksander Holynski, Angjoo Kanazawa, et al. State of the art on diffusion models for visual computing. Computer Graphics Forum, 43(2):e 2024.
[56] Di Qiu, Zhengcong Fei, Rui Wang, Jialin Bai, Changqian Yu, Mingyuan Fan, Guibin Chen, and Xiang Wen. Skyreels-al: Expressive portrait animation in video diffusion trans- formers. abs/2502.10841 2025. URL lhttps://app semanticscholas Org/CorrpssIl 276409428
[57] Xuanchi Ren, Tianchang Shen, Jiahui Huang, Huan Ling, Yifan Lu, Merlin Nimier-Davin Thomas Müller, Alexander Keller, Sanja Fidler, and Jun Gao. Gen3c: 3d-informed world- consistent video generation with precise camera control. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025.
[58] Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang. Oron Ashual, Oran Gafni, et al. Make-a-videes Text-to-video generation without text-video data. In ICLR, 2023.
[59] Jascha Dick-Diccst,in, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynamics. 2015.
[60] Kiwhan Song, Boyuan Chen, Max Simchowitz, Yilun Du Russ Tedrake, and Vincent Sitzmann. History-guided video diffusion. arXiv preprint 2rXiv:2502.06764 2025.
[61] Yang Song, Jascha Dickksies,,,,, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In ICLR, 2021.
[62] Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autoregressive model beats diffusion: Llama for scalable image generation. arXiv preprint arXiv:2406.06555, 2024.
[63] Anni Tang, Tianyu He, Junliang Guo, Xinle Cheng, Li Song, and Jiang Bian. Vidtok: A versatile and open- source video tokenizer. ArXiv, 13061122... 2024. URL httttt/////pp
semanticscholar
:
[64] S. Tulyakov, Ming- Yu Liu, Xiaodong Yang, and Jan Kautz. Mocogan: Decomposing motion and content for video generation. 2018 IEEE/CVF Conference on Computer Vision and Pattem Recognition, pages 1526 1535, 2017. URL http:://app..smmaticcccs CorpusID:4475365
org/
[65] Dani Valevski, Yaniv Leviathan, Moab Arar, and Shlomi Fruchter. Diffusion models are real-time game engines. arXiv preprint arXiv:2408. 14837, 2024.
[66] R Villegas, H Moraldo, S Castro, M Babaeizadeh, H Zhang, Kunze, PJ Kindermans, MT Saffar, and D Erhan. Phenaki: Variable length video generation from open domain textual descriptions. In ICLR, 2023.
[67] Carl Vondrick and Antonio Torralba. Generating the future with adversarial transformers. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2992-3000, 2017. URL https:: h/ppipppppiaaaa www.nticschoolrr..c.mmmmm .trellarrrrrrgoooo Org/CorpusID 8234308
14

---

[68] Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen- Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang. et al. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.2033 14, 2025.
[69] Hongjie Wang, Chih- Yao Ma, Yen-Cheng Liu, Hou, Tao Xu, Jialiang Wang, Felix Juefei-Xuu -u, Yaqiao Luo, Peizhao Zhang. Tingbo Hou, Peter Vajda, Niraj Kumar Jha, and Xiaoliang Dai. Lingen: Towards high-resolution minute-length text-to-video generation with linear compu- tational complexity. ArXiv, abs/2412.09856, 2024. htl.http::/eet api. semanticscholar
[70] Qianqian Wang*, Yifei Zhang*, Aleksander Holynski, Alexci A. Efros, and Angjoo Kanazawa. Continuous 3d perception model with persistent state. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025.
[71] Yuqing Wang, Tianwei Xiong, Daquan Zhou, Zhijie Lin, Yang Zhao, Bingyi Kang, Jiashi Feng, and Xihui Liu. Loong: Generating minute-level long videos with autoregressive language models, arXiv preprint arXiv:2410.02757, 2024.
[72] Zhouxia Wang, Ziyang Yuan, Xintao Wang, Yaowei Li, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. Motionctrl: A unified and flexible motion controller for video generation. In ACM SIGGRAPH 2024 Conference Papers, pages 1-11, 2024.
[73] Wenming Weng, Ruoyu Feng. Yanhui Wang, Qi Dai, Chunyu Wang, Dacheng Yin, Zhiyuan Zhao, Kai Qiu, Jianmin Bao, Yuhui Yuan, et al. Art-v: Auto-regressive text-to-video generation with diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7395-7405, 2024.
[74] Philipp Wu, Alejandro Escontrela, Danijar Hafner, Pieter Abbeel, and Ken Goldberg. Day- dreamer: World models for physical robot learning. In Conference on robot learning, pages 2226-2244 PMLR, 2023.
[75] Enze Xie, Junsong Chen, Junyu Chen, Han Cai, Haotian Tang, Yujun Lin, Zhekai Zhang, Muyang Li, Ligeng Zhu, Yao Lu, and Song Han. Sana: Efficient high-resolution image synthesis with linear diffusion transformers. ArXiv, abs/2410. 10629, 2024. URL https:
/appi.seaantccccooe httpa/aiiieiitla....oooo/raaaaaa orr//oorpueD::233446004
[76] Wilson Yan, Yunzhi Zhang, Pieter Abbeel, and Aravind Srinivas. Videogpt: Video generation using vq-vae and transformers. arXiv preprini SiNi:2204.10155: 2021.
[77] An Yang. Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412. 15115, 2024.
[78] Zhuoyi Yang. Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, Da Yin, Xiaotao Gu, Yuxuan Zhang, Weihan Wang. Yean Cheng, Ting Liu, Bin Xu, Yuxiao Dong, and Jie Tang. Cogvideox: Text- to-video diffusion models with an expert transformer. ArXiv, abs/2408.06072. 2024. URL https://api. http:///aap.eeemmtccccccoooooooo
[79] Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-vided diffusion models with an expert transformer. arXiv preprint N4iv:24408.06777 2024.
[80] Tianwei Yin, Qiang Zhang, Richard Zhang, William T Freeman, Fredo Durand, Eli Shechtman, and Xun Huang. From slow bidirectional to fast autoregressive video diffusion models. In CVPR, 2025.
[81] Jiwen Yu, Yiran Qin, Xintao Wang. Pengfei Wan, Di Zhang, and Xihui Liu. Gamefactory: Creating new games with generative interactive vidcos. arXiv preprint arXiv:2501.08325, 2025.
[82] Mark YU, Wenbo Hu, Jinbo Xing, and Ying Shan. Trajectorycrafter: Redirecting camera trajectory for monocular videos via diffusion models. arXiv preprint uxXiv25503.56638 2025.
15

---

[83] Wangbo Yu, Jinbo Xing, Li Yuan, Wenbo Hu, Xiaoyu Li, Zhipeng Huang, Xiangjun Gao, Tien-Tsin Wong, Ying Shan, and Yonghong Tian. Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis. arXiv preprint arXiv:2409.02045 2024.
[84] Andy Zeng, Shuran Song, Matthias Nießner, Matthew Fisher, Jianxiong Xiao, and Thomas Funkhouser. 3dmatch: Learning local geometric descriptors from rgb-d reconstructions. In CVPR, 2017.
[85] David Junhao Zhang, Roni Paiss, Shiran Zada, Nikhil Karndd, David E Jacobs, Yael Pritch, Inbar Mosseri, Mike Zheng Shou, Neal Wadhwa, and Nataniel Ruiz. Recapture: Generative video camera controls for user-provided videos using masked video fine-tuning. arXiv preprint arXiv:2411 0..4/1.05000 1.05033, 2024.
[86] Lvmin Zhang and Maneesh Agrawala. Packing input frame context in next-frame prediction models for video generation. arXiv preprint PrXiv:2504.12623 2025.
[87] Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffuusin models. In Proceedings of the IEEE/CVF international conference computer vision, pages 3836-3847, 2023.
[88] Zhicheng Zhang, Junyao Hu, Wentao Cheng, Danda Paudel, and Jufeng Yang. Extdm: Distribu- tion extrapolation diffusion model for video prediction. In CVPR, 2024.
[89] Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all. ArXiv, abs/2412.20404, 2024. URL https://api: semanticscholar Crrp/Dop
275138898
[90] Zheng Zhu, Xiaofeng Wang. Wangbo Zhao, Chen Min, Nianchen Deng, Min Dou, Yuqi Wang, Botian Shi, Kai Wang, Chi Zhang, al. Is sora world simulator? comprehensive survey on general world models and beyond. arXiv preprint arXiv:2405.03520, 2024.
16

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the
paper's contributions and scope?
Answer: [Yes]
Justification: We claim the introduction of long-term memory mechanism, including a point-cloud-based spatial memory and episodic memory comprised of sparse set of keyframes along with memory storage and retrieval techniques. Our evaluations show that this method outperforms the baselines.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: See Section 5, paragraph on "Limitations and Future Work".
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
17

---

Answer: [Yes]
Justification: The assumptions on our different memory mechanisms are: (1) short-term working memory: models static and dynamic parts of the scene, but is limited short context window size due to quadratic growth of computational of attention; (2) long-tern long-term spatial memory; models the static parts of the scene, but not dynamic aspects and limited in the visual detail point cloud can store via TSDF fusion; (3) long-term episodic memory: sparse image "snapshots" of past events.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provided implementation details as best as possible within the page limit of the paper, and additional details in the supplement. Code and trained model will be made public.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
18

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
We will make code and the pretained model available.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
//nips.ccpputt wwtpcccccppccccceee coroolicgguideee /wwedeeeeeeeebmmeeeeeeee ceiomisssonPPa ssioccllccy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: These details are clarified in the paper and supplement and will also be easily reproducible from the provided code.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [NA]
Justification: Our metrics use standard benchmarks for the tasks we evaluate. These metrics do not include statistical significance.
Guidelines:
19

---

The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi-
dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots. The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: See text in paper.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics atttps://neuriig httphhwwwwss...pppceeeeeeeeeeeee ctw.ihheeeecuiieiieeeennee
Answer: [Yes]
Justification:
We reviewed the Code of Ethics and confirm that this research conforms to it.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: See Sec. 5.
20

---

Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [Yes]
Justification: Our training and evaluation data is publicly available. Safeguards of the base model, i.e., Cog VideoX, are inherited.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: See text.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
21

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperssi thood cww datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: All code and data we will release will contain the required information.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [Yes]
Justification:
See text and supplement.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [Yes]
Justification: We followed the standards of the country where studies were performed and personal data analyzed.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
22

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification: LLMs were exclusively used for writing, editing, or formatting purposes.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should not be described.
23


Reviews:
[{"review_id": "rI7Wo2hsmS", "reviewer": "Reviewer_arRQ", "summary": "The paper introduces a geometry‑grounded memory framework for autoregressive video‑diffusion “world models” that fuses three complementary memory streams including short‑term working memory (recent latent frames), long‑term spatial memory (TSDF‑filtered global point cloud of static elements), and long‑term episodic memory (sparse key reference frames). It details algorithms for writing to and reading from these memories and integrates them into a DiT‑based latent diffusion backbone. Experiments show large gains in view‑recall consistency, VBench video quality, and human preference over leading geometry‑guided baselines. Extensive ablations confirm that each memory component contributes distinct benefits to long‑horizon coherence.", "questions": "1. Broader baseline set: please include experiments against recent long‑context compression/linear‑attention methods (e.g., LinGen, StreamingT2V, History‑Guided Video Diffusion). This would clarify whether geometric memory is strictly superior or complementary.\n2. Dynamic‑scene analysis: construct a subset where more (e.g., >50%) pixels are dynamic (crowds, traffic). Report whether spatial memory degrades and, if so, propose possible mitigation solution.\n3. Ablation on memory size: explore trade‑off between point‑cloud resolution / key‑frame buffer length and compute. Present memory foot‑print and generation FPS to justify practicality.", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. Technically sound. The memory design is well‑motivated, the TSDF fusion and key‑frame selection are precise, and ablations isolate each subsystem’s impact.\n2. Experimental analysis. Quantitative, qualitative and human studies all align with claims, demonstrating state‑of‑the‑art long‑term consistency while maintaining video quality.\n3. Significance. Bridging 3D perception with video diffusion offers a reusable path toward infinite‑length, scene‑consistent generation for graphics, robotics and simulation communities.\n\nWeaknesses:\n1. Baseline coverage is narrow—recent long‑context or state‑space video diffusion models are absent, leaving open whether geometric memory outperforms purely temporal approaches. \n2. The method is trained only 6 k iterations on 8×A100 GPUs; scalability to larger backbones or diverse datasets is unverified.\n3. The proprietary dataset is promised but not yet released, and spatial memory struggles in highly dynamic or drastic‑camera‑motion scenes (failure case, Figure 6).", "was_revised": true, "final_justification": "The authors addressed most of my concerns during the rebuttal phase. I believe the work is novel and provides valuable insights.", "rebuttal": "Thank you for your detailed review and insightful comments. We will address your questions and concerns below and in the revised paper.\n\n---\n\n### W1&Q1: Broader baseline set.\n\nThank you for the thoughtful suggestions. As LinGen only has an unofficial implementation and does not provide a pretrained checkpoint, it is difficult to include in the current rebuttal phase. Instead, we conducted experiments on StreamingT2V and History-Guided Video Diffusion (Dfot, trained on RE10K) using the RE10K testset for comparison. The results are shown below:\n\n| Method        | Aesthetic Quality | Temporal Flickering | Motion Smoothness | Subject Consistency | Background Consistency |\n| ------------- | ----------------- | ------------------- | ----------------- | ------------------- | ---------------------- |\n| Dfot          | 0.4275            | 0.6800              | 0.9877            | 0.8630              | 0.9190                 |\n| Streaming_t2v | **0.4683**        | **0.8510**          | _0.9926_            | _0.8706_              | _0.9136_                 |\n| Ours          | _0.4630_            | _0.7838_              | **0.9936**        | **0.9583**          | **0.9576**             |\n\nWe achieve second-best results in terms of aesthetic quality and temporal flickering, while significantly outperforming other methods on subject consistency and background consistency, thanks to the incorporation of geometric memory. These improvements are well aligned with the core motivation behind our model design.\n\n---\n\n### W2: Scalability to larger backbones or diverse datasets.\n\nThank you for the thoughtful feedback. We conducted our experiments on CogVideoX and the current dataset due to limited computational resources, yet the results are already highly encouraging. We would like to clarify that scaling up from the 5B CogVideoX model to larger backbones, such as the 14B Wan2.1 model, is feasible, as they share similar DiT architectures and the parameter gap is not substantial. Furthermore, although our task focuses on camera-guided video generation, the training data pairs we use ultimately fall under the caption-video category, similar to those employed in I2V and T2V tasks. Therefore, scaling up the dataset is entirely reasonable. Due to the limited time for rebuttal, we plan to explore larger backbones and a more diverse dataset than the current version for the final version.\n\n---\n\n### W3: Dataset release.\n\nWhile we are unable to include external links in the response due to the rebuttal policy, the dataset has been fully prepared. We will ensure it is released promptly upon paper acceptance and include the link in the final version.\n\n---\n\n### W3&Q2: Dynamic‑scene analysis.\n\nThanks for the insightful comment. \n\n**First**, to handle highly dynamic scenes with drastic camera motion, our strategy involves scaling the depth within the current range and reducing the voxel volume size. This adjustment yields a denser, filtered static point cloud, mitigating the issue of incomplete static-dynamic decoupling caused by extreme motion. By refining the static representation, we prevent excessive sparsity in the static spatial memory, thereby improving overall generation quality.\n\n**Second**, We construct a new subset of 100 highly dynamic scenes covering both real-world and game-engine domains, including rainy and snowy weather as well as day and night environments from Sekai [1], a large-scale video world exploration dataset. Specifically, for the reconstruction metrics, we far surpass other methods, which reflect our ability to keep 3D consistency and control camera accuracy. In terms of vbench metrics, we show superior video quality, although aesthetic quality and imaging quality lag slightly behind Wan2.1-inpainting, as mentioned in Sec.4.1 of our paper, this is due to their advanced backbone. Overall, our method still shows robust performance in these difficult cases.\n\n[1] Li, Zhen, et al. \"Sekai: A Video Dataset towards World Exploration.\" *arXiv preprint arXiv:2506.15675* (2025).\n\n| Method            | PSNR↑       | SSIM↑      | LPIPS↓     | FVD↓       | Aesthetic Quality | Imaging Quality | Temporal Flickering | Motion Smoothness | Subject Consistency | Background Consistency |\n| ----------------- | ----------- | ---------- | ---------- | ---------- | ----------------- | --------------- | ------------------- | ----------------- | ------------------- | ---------------------- |\n| TrajectoryCrafter | 9.8791      | 0.3497     | 0.5892     | 862.73     | 0.4846            | 0.5907          | _0.7158_              | 0.9866            | 0.9533              | 0.9425                 |\n| DaS               | _10.0074_     | _0.3849_     | _0.5719_     | 962.46     | 0.5102            | 0.6272          | 0.6421              | 0.9849            | 0.9565              | 0.9508                 |\n| Wan2.1-Inpainting | 10.0013     | 0.3811     | 0.5797     | _834.30_     | **0.5151**        | **0.6709**      | 0.6276              | _0.9893_            | **0.9583**          | **0.9668**             |\n| Ours              | **19.0984** | **0.6789** | **0.2228** | **256.18** | _0.5123_            | _0.6347_          | **0.8526**          | **0.9903**        | _0.9570_              | _0.9602_                 |\n\n(Bold is best, italic is the second.)\n\n---\n\n### Q3: Ablation on memory size; Present memory foot‑print and generation FPS.\n\n1) Since we use point cloud renderings as the conditioning input, the resolution of the point cloud does not directly impact the memory footprint. Additionally, the key-frame buffer is relatively small compared to the overall model size, so there is no significant trade-off within the practical range.\n2) Our memory footprint is approximately 25GB, and generation FPS is 3.2 minutes for 49 frames on an Nvidia A800. We would like to clarify that the primary focus of our paper is on incorporating long-term and spatial memory into video models, and we do not claim real-time performance. However, we fully agree that bridging academic research and practical deployment is an important direction. In future work, we plan to explore lightweight alternatives and acceleration strategies such as Distribution Matching Distillation to further reduce memory usage and improve inference speed."}, {"review_id": "JbqgqWQHjh", "reviewer": "Reviewer_ddFV", "summary": "The paper proposes Video World Models with Long-term Spatial Memory, combining \n\n1. Short-term “working-memory” (recent frames), \n\n2.  A geometry-grounded static point-cloud map updated online with TSDF fusion, and \n\n3.  A sparse episodic key-frame buffer. The authors build a 90 K-clip “geometry-grounded” dataset derived from MiraData and show gains over TrajectoryCrafter, DaS, and Wan 2.1-Inpainting on view-recall metrics, VBench scores, and a 20-person user study.", "questions": "1. Please address the weaknesses mentioned.\n\n2. Will the full 90 K-clip dataset (videos, depth, poses) be publicly released?  \n\n3. Did you retrain the baselines on your dataset? \n\n4. Can you please report FVD for your method and compare against baselines? \n\n5. What is the wall-clock time per generated 1-second video (e.g., 30 frames) and GPU memory footprint?  \n\n6. How does performance degrade when the depth estimator (CUT3R) produces noisy maps (e.g., night scenes, rain)?", "limitations": "The authors have adequately discussed limitations.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n• The paper poses a timely problem: long-horizon drift/forgetting is a real blocker for autoregressive video world models.  \n\n• Straight-forward implementation– integrating point-cloud renders via a ControlNet-style branch is practical. \n\n• The architectural diagram is clear. Clear writing style.\n\nWeaknesses:\n\n• Limited comparisons with recent works like Gen3C (Ren et al. CVPR 2025). \n\n•  Metric choice biases towards static consistency. The main quantitative gain is PSNR/SSIM on revisited static frames; dynamic realism and action accuracy are mostly scored by the same subjective study. No perceptual metrics such as FVD.", "was_revised": true, "final_justification": "The authors have addressed all my concerns, and I am satisfied. Hence I have increased my rating.", "rebuttal": "Thank you for your detailed review and insightful comments. We will address your questions and concerns below and in the revised paper.\n\n---\n\n### W1: Comparisons with recent works.\n\nThank you for the valuable suggestion. We will include Gen3C and more recent works as additional baselines in our final version.\n\n---\n\n### W2&Q4: Metric choice biases and to report FVD.\n\nThank you for the constructive feedback. For evaluating dynamics and actions, we follow prior works by reporting metrics such as motion smoothness from VBench, which is one of the most widely used indicators of motion quality. We would be happy to include additional metrics if the reviewer could kindly provide suggestions. As requested, we also report the FVD results below, where our method demonstrates superior performance compared to the baselines.\n\n| Method | TrajectoryCrafter | DaS    | Wan2.1-Inpainting | Ours       |\n| ------ | ----------------- | ----------------- | ----------------- | ---------- |\n| FVD↓   | 355.23      | 363.36           | 280.06           | **157.11** |\n\n---\n\n\n### Q2: Dataset release.\n\nWhile we are unable to include external links in the response due to the rebuttal policy, the dataset has been fully prepared. We will ensure it is released promptly upon paper acceptance and include the link in the final version.\n\n---\n\n### Q3: Did you retrain the baselines on your dataset?\n\nThank you for the thoughtful question. We have attempted to retrain the baseline models on our dataset, but their performance deteriorates as training progresses. This is because prior works rely on input conditions and outputs that are temporally aligned, whereas our task involves predicting future observations with temporally misaligned inputs and outputs. The data pairs are constructed accordingly, making the problem more challenging. Without tailored architectural modifications, baseline models struggle to adapt. Moreover, since all baselines are pretrained on general video datasets such as OpenVid and MiraData, which share a similar domain with ours, the domain gap is minimal.\n\n---\n\n### Q5: Wall-clock time per generated 30-frame video and GPU memory footprint.\n\nOur memory footprint is approximately 25GB, and generation FPS is 1.8 minutes for 30 frames on an Nvidia A800.\n\n---\n\n### Q6: Performance degrades when the depth estimator produces noisy maps.\n\nWe construct a new subset of 100 highly dynamic scenes covering both real-world and game-engine domains, including rainy and snowy weather as well as day and night environments from Sekai [1], a large-scale video world exploration dataset. Our evaluation reveal depth estimator perform well for night scenes and rainy/snowy environments, though the TSDF fusion stage may not fully filter out dynamic elements, leading to artifacts. However, our video generation backbone leverages static point cloud rendering as conditioning and integrates clean context into the video latent space. This design allows the self-attention layers to effectively capture cross-view dependencies, enabling the system to *repair distortions* while maintaining 3D consistency. As shown below, our results on this challenging subset remain competitive compared to other methods.\n\n| Method            | PSNR↑       | SSIM↑      | LPIPS↓     | FVD↓       | Aesthetic Quality | Imaging Quality | Temporal Flickering | Motion Smoothness | Subject Consistency | Background Consistency |\n| ----------------- | ----------- | ---------- | ---------- | ---------- | ----------------- | --------------- | ------------------- | ----------------- | ------------------- | ---------------------- |\n| TrajectoryCrafter | 9.8791      | 0.3497     | 0.5892     | 862.73     | 0.4846            | 0.5907          | _0.7158_              | 0.9866            | 0.9533              | 0.9425                 |\n| DaS               | _10.0074_     | _0.3849_     | _0.5719_     | 962.46     | 0.5102            | 0.6272          | 0.6421              | 0.9849            | 0.9565              | 0.9508                 |\n| Wan2.1-Inpainting | 10.0013     | 0.3811     | 0.5797     | _834.30_     | **0.5151**        | **0.6709**      | 0.6276              | _0.9893_            | **0.9583**          | **0.9668**             |\n| Ours              | **19.0984** | **0.6789** | **0.2228** | **256.18** | _0.5123_            | _0.6347_          | **0.8526**          | **0.9903**        | _0.9570_              | _0.9602_                 |\n\n(Bold is best, italic is the second.)\n\n[1] Li, Zhen, et al. \"Sekai: A Video Dataset towards World Exploration.\" *arXiv preprint arXiv:2506.15675* (2025)."}, {"review_id": "dHp2WzRubn", "reviewer": "Reviewer_Kmfd", "summary": "This paper proposes a video-gen method that can condition on reconstructed point map and historical frames, thereby maintaining episodic, working and spatial memory for long video generation.", "questions": "Please see the above.", "limitations": "Please see the above.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Thanks for the good paper. \n* Memory is always an important perspective for video-gen\n* Good formulation that involves spatial, working and episodic memory.\n* Results and ablative experiments are good.\n\nSeveral concerns and questions:\n\n* **Related Works** Some related papers [1,2] use test-time training to maintain memory. Please cite. [1] also mentioned episodic memory in the paper. \n\n[1] SlowFast-VGen: Slow-Fast Learning for Action-Driven Long Video Generation\n\n[2] One-Minute Video Generation with Test-Time Training\n\n* **Explicit VS Implicit 3D representations** The paper mentioned the spatial memory of human beings. However, the place cells and grid cells are more implicit representations than registered pointmap. Do you think in the future we can instead utilize more implicit 3D representations (e.g., similar to [1] and [2] where the latents might potentially already encode part of the 3D representations?) Explicit 3D reconstruction can be accurate but also quite heavy sometimes, especially when the scene gets very large. What do you think are the pros & cons of both? No experiments needed just hope to hear the authors' opinions. \n* **3D reconstruction method** The authors use TSDF for 3D reconstruction. Have you experimented with VGGT or other 3D reconstruction methods? Can you share why did you choose TSDF among other, to give us some insights? No experiments needed just hope to hear the authors' opinions. \n* **Dynamic humans / other agents in static scenes** In practice, when we utilize 3D reconstruction methods such as VGGT on dynamic scenes, the moving humans can be a big problem, which can result in scene distortion or occlusion. Have you come up with any specific methods to deal with this, or do you think it's not a problem?\n* **More related works** These papers are relevant and worth reading, though not citing them is okay as they are not in the same field. Specifically, WonderPlay deals with the dynamic actions mentioned in the last limitation. [4] also talks about 3D working and episodic memory.\n\n[3] The wonder series (WonderJourney, WonderWorld, WonderPlay)\n\n[4] 3DLLM-Mem: Long-Term Spatial-Temporal Memory for Embodied 3D Large Language Model", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed review and insightful comments. We will address your questions and concerns below and in the revised paper.\n\n---\n\n### Q1&Q5: Include more related works.\n\nThank you for the thoughtful suggestions. We will cite the related works [1-4] and include proper discussions in the final version. \n\n---\n\n### Q2: Explicit VS Implicit 3D representations.\n\nThank you for the insightful and valuable feedback. Both explicit and implicit 3D representations have their respective advantages and limitations. Explicit representations (e.g, point clouds) offer high precision, easy editability, and strong compatibility, making them ideal for scenarios requiring strict geometric control, though they come with higher storage and computational costs than implicit representation. In contrast, implicit representations excel in efficient compression and scalability, but face challenges in interpretability and training complexity. We believe the future lies in hybrid representations, for instance, using implicit encoding for global semantics while employing explicit structures for local detail processing.  We will continue to explore this direction in our future research.\n\n---\n\n### Q3: 3D reconstruction method.\n\nWe leverage Mega-SaM during dataset construction and CUT3R during inference for 3D reconstruction. Since their outputs are frame-wise point maps, we further apply TSDF-Fusion to aggregate them into a global point cloud and filter out dynamic elements such as characters and animals. This combination of state-of-the-art 3D reconstruction methods with the traditional TSDF-Fusion approach enables efficient and accurate camera pose estimation and global point cloud reconstruction.\n\n---\n\n### Q4: Dynamic humans / other agents in static scenes.\n\nThank you for the thoughtful comment. Dynamic scenes indeed pose significant challenges. As mentioned in Q3, we chose Mega-SaM and CUT3R in place of VGGT as our main reconstruction tools, as both are specifically designed and trained for dynamic scene reconstruction to ensure accurate camera poses and point maps. However, directly combining per-frame point maps leads to artifacts and misalignment of dynamic elements. To address this, we further incorporate TSDF-Fusion to filter out dynamic content and obtain a clean, static global point cloud."}, {"review_id": "U0bUnRuw9J", "reviewer": "Reviewer_e9qv", "summary": "Paper presents a world model system (predicts video frames) that is incorporated with spatial and keypoint memory to address challenges with persistent memory with world model image generation. The presented method outperforms baselines both in quantitative metrics (video quality, user studies, consistency) as well as in qualitative sense.", "questions": "1) What are the upper/lower bounds of the metrics in the Table 1, Table 2 and Table 3, and what is their rough meaning? For Table 1, you could consider including baseline results (e.g., only repeating static image) and oracle/upper bound (e.g., the original video clip but passed through the image encoder/decoder). This way you can could tell how much of the sequence error comes from the image encoding/decoding.\n\n2) Did you repeat the experiments multiple times and then averaged over to report statistics of results? It is unclear how sensitive the model is to training, and if the results are meaningful, primarly in Table 2 and Table 3, where the differences seem very minute.\n\n3) How was the testing data sampled? L211 states it is \"500 randomly selected video sequences from MiraData\", but it is unclear if these samples are from unseen video clips. Were some raw videos (as used in L186) left out for test set, or are test set clips from same set as training clips? If test clip is from a middle of recording, having the immediate past and future in training set might make the task easier. It would be interesting to see the results on unseen videos, or test samples taken from the very and or beginning of the raw videos.\n\n4) How were human rankings collected, exactly? Could you share the forms you used, and more information of the participants of the study (e.g., how were they recruited). The way questions are posed can have a big influence on the results (e.g., do you show oracle/groundtruth results [should get 4], do you only show one video at a time, etc).\n\n5) Would it be possible to see view recall consistency quantitative results (Table 1) for the ablation? I think these are important metrics for the ablations, especially the one studying impact of the episodic memory.", "limitations": "See strengths and weaknesses.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "I vote for a low confidence accept (hence borderline). The paper presents positive results, both quantitative and qualitative, and the method is novel/interesting enough to be of interest for NeurIPS community, and I can see other works building on this. However, the main weaknesses of the paper are lack of theoretical insight/analysis (now answers to \"why\" questions), and the fact that the results are intuitively expected (enforcing memory in video generation process improves video generation consistency).\n\nI have not kept up with the latest improvements in the field, so I may have missed something. In addition, I have some questions (see Questions) that should be addressed for a more confident vote.\n\n\n**Quality**\n\n- +Reported performance is higher than baselines, in both quantitative (image quality and consistency) and qualitative metrics (human judges)\n- +The work contributes a custom dataset\n- /Comparison against three baselines, but Wan2.1 is not directly comparable (does not have action conditioning)\n- -No theoretical/insightful analysis. While I appreciate the ablation studies on the importance of the provided parts, they do not provide answers to \"why\" they improve results, apart from the intuition.\n- -Method was tested with custom dataset/benchmark (data created from MiraData), which is required for the data method uses, but the data may be out of the distribution of baselines (or the data they were originally presented in). The results would be more convincing if the benchmark was established/defined in a prior work.\n\n\n**Clarity**\n\n- -The text contains numerous cases of vague/unfounded adjectives. These do not bring more value to the scientific text as is, or may even confuse, so they should be revised. For example:\n  - L253. \"Comprehensive\" (what makes it comprehensive?)\n  - L260. \"Large margin\". What is a large margin? The more scientific method is to perform statistical test on the results.\n  - Figure 6 caption: \"_swiftly_ swinging\"\n  - Section 4.2: while qualitative, this section too has many vague adjectives (\"fail to _closely_\", \"significantly higher consistency\", \"harmonious\").\n- L188. What is \"shared transition frame\"?\n\n\n\n**Significance**\n\n- +Dataset and the approach can be useful building blocks/comparisons for future work. I am particularly interested in the evolution of different approaches (e.g., raw image-to-image generation vs. point-cloud generation like here).\n- -While promising results, they are not very surprising: the problem is long term video generation, which requires remembering objects in the scene. The paper proposes very explicit way of handling this (storing keyframes, explicit mapping to point cloud and back), which improves results. I do not expect the \"insight\" of the work to be very valuable to the NeurIPS community.\n\n**Originality**\n\nI am not deeply familiar with the literature, but to me, I not seen major works with similar approach. However, the method seems to be a continuation of TrajectoryCrafter, both relying on point clouds to have consistency, but this proposed method takes this further by encoding the memory.", "was_revised": true, "final_justification": "Authors have addressed my concerns and I have increased my original rating 4 to 5 (Accept) to signal my vote to accept the paper. \n\nPaper solution is strong over the baselines across several sets of data, and the experiments were conducted well. The main limitations of paper are the core methodology being slightly complex with multiple parts, and the lack of deeper understanding what helps and where.", "rebuttal": "Thank you for your detailed review and insightful comments. We will address your questions and concerns below and in the revised paper.\n\n---\n\n### W1: Wan2.1 is not directly comparable.\n\nThanks for the feedback. We make slight adaptations to the Wan2.1-I2V so that it could be comparable in the context. Specifically, we use static point cloud rendering to initialize video latents to implement camera control like trajectorycrafter, and add a low level of noise on top. To ensure the motion coherence of dynamic elements, we select the recent frames as the reference frames. Based on the original backbone of Wan2.1-I2V,  we perform dual encoding on reference frames through Clip visual encoder and 3D-VAE respectively: the semantic features encoded by Clip-vision are concatenated with text prompts to maintain content consistency, and the spatial features extracted by 3D-VAE are concatenated with video latents to enhance geometric continuity, finally achieving a temporal progressive NVS task similar to ours. \n\n---\n\n### W2: Lack of theoretical/insightful analysis. \n\nThank you for the thoughtful comment. The working memory primarily derives from recent context frames. These clean reference contexts are concatenated with noisy video latents at the frame level, enabling us to adapt the self-attention layers into reference mixing layers that effectively capture cross-view dependencies. This architecture not only enhances 3D consistency but also improves short-term motion coherence; The episodic memory system mainly derives from historical reference key frames. Compared to working memory, these views originate from more distant camera positions, endowing the model with stronger robustness. Particularly when user-input trajectories exhibit \"revisit\" (overlaps with previously generated regions), our designed historical cross-attention mechanism can effectively retrieve and leverage these stored details. This approach ensures long-term consistency while preventing memory forgetting, as the system maintains access to relevant historical information even during novel view synthesis. We will add the analysis to the final version.\n\n---\n\n### W3: More convincing testset setup.\n\nThank you for the insightful feedback. We are unable to directly adopt the benchmark established in prior work, as the task definitions are not fully aligned and their dataset is incompatible with the requirements of our setting. But in order to mitigate the impact of the domain gap, we construct a new subset of 100 highly dynamic scenes covering both real-world and game-engine domains, including rainy and snowy weather as well as day and night environments from Sekai [1], a large-scale video world exploration dataset. Specifically, for the reconstruction metrics, we far surpass other methods, which reflect our ability to keep 3D consistency and control camera accuracy. In terms of vbench metrics, we show superior video quality, although aesthetic quality and imaging quality lag slightly behind Wan2.1-inpainting, as mentioned in Sec.4.1 of our paper, this is due to their advanced backbone. Overall, our method still shows robust performance in these difficult and OOD cases.\n\n[1] Li, Zhen, et al. \"Sekai: A Video Dataset towards World Exploration.\" *arXiv preprint arXiv:2506.15675* (2025).\n\n| Method            | PSNR↑       | SSIM↑      | LPIPS↓     | FVD↓       | Aesthetic Quality | Imaging Quality | Temporal Flickering | Motion Smoothness | Subject Consistency | Background Consistency |\n| ----------------- | ----------- | ---------- | ---------- | ---------- | ----------------- | --------------- | ------------------- | ----------------- | ------------------- | ---------------------- |\n| TrajectoryCrafter | 9.8791      | 0.3497     | 0.5892     | 862.73     | 0.4846            | 0.5907          | _0.7158_              | 0.9866            | 0.9533              | 0.9425                 |\n| DaS               | _10.0074_     | _0.3849_     | _0.5719_     | 962.46     | 0.5102            | 0.6272          | 0.6421              | 0.9849            | 0.9565              | 0.9508                 |\n| Wan2.1-Inpainting | 10.0013     | 0.3811     | 0.5797     | _834.30_     | **0.5151**        | **0.6709**      | 0.6276              | _0.9893_            | **0.9583**          | **0.9668**             |\n| Ours              | **19.0984** | **0.6789** | **0.2228** | **256.18** | _0.5123_            | _0.6347_          | **0.8526**          | **0.9903**        | _0.9570_              | 0.9602                 |\n\n(Bold is best, italic is the second.)\n\n---\n\n\n### W4: Issues in clarity.\n\nThank you for pointing this out. We will carefully review the writing and revise the final version to avoid vague or unsupported adjectives.\n\n---\n\n\n### W5: Insight and originality of the work.\n\nBeyond the spatial memory mechanism, another major difference from TrajectoryCrafter and other point cloud-conditioned methods is our ability to handle temporally misaligned inputs and outputs. Specifically, while TrajectoryCrafter performs dynamic novel view synthesis with temporally aligned inputs and outputs, our method predicts future observations based on static conditions and dynamic references stored in working memory. This goes beyond a simple inpainting task. We believe that the problem setup, the construction of corresponding data pairs, and the proposed model design will be inspiring to the community.\n\n---\n\n\n### Q1: Metric clarification. \n\nIn Table 1, user study scores range from 1 to 4, while reconstruction metrics (PSNR, SSIM, LPIPS) mainly assess camera accuracy and 3D consistency in static regions. Because our model uses a 3D VAE for video encoding/decoding, these metrics are particularly sensitive—compression-like processing can introduce measurable deviations even when static regions appear nearly identical to human observers. Following your suggestion, we tested repeating static videos and found approximate ranges: PSNR ↑ (0–22), SSIM ↑ (−1 to 0.75), LPIPS ↓ ( 0.15 to +∞).\n\nWhile the metrics in Tables 2 and 3 are from VBench, a widely used video generation benchmark. According to the VBench paper, all metrics are normalized to 0–100%, where 100% is ideal and lower bounds indicate worst case. Aesthetic Quality (0–1.0) measures artistic merit, Imaging Quality (0–1.0) detects blur/noise, Temporal Flickering (0.6293–1.0) gauges frame consistency, Motion Smoothness (0.7060–0.9975) checks physics-compliant motion, Subject Consistency (0.1462–1.0) tracks foreground stability, and Background Consistency(0.2615–1.0) evaluates scene coherence.\n\n---\n\n\n### Q2: Additional experimental setup.\n\nFollowing the widely adopted experimental setup in previous video generation benchmarks, we fix a single random seed for all experiments and report the corresponding results. We also observe that running the experiments with different seeds and averaging the results over the full 500-sample test set leads to only negligible variations.\n\n---\n\n\n### Q3: Testset setup.\n\nThe 500 test cases are sampled entirely from unseen videos. Specifically, the raw videos have been pre-split into disjoint training and testing sets, ensuring that test clips and training clips never come from the same original video. \n\n---\n\n\n### Q4: User study setup.\n\nIn the final version, we will include screenshots of the user study questionnaire and provide detailed user statistics for clarification. Briefly, we invited 13 graduate students working on video generation research to participate in the user study. Each participant was presented with ground truth and generated videos for evaluation. We report the average human ranking scores based on their responses.\n\n---\n\n\n### Q5: Quantitative results of view recall consistency for the ablation.\n\nThanks for the insightful suggestion. We report the quantitative results of view recall consistency for the ablation as below. Our full model, which uses short-term working and long-term spatial and episodic memory, performs best on reconstruction metrics, demonstrating the effectiveness of the memory mechanism in keeping long-term 3D consistency.\n\n| Method           | PSNR ↑      | SSIM ↑     | LPIPS ↓    |\n| ---------------- | ----------- | ---------- | ---------- |\n| w/o episodic mem | 18.8665     | 0.6628     | 0.2898     |\n| w/o working mem  | 10.1379     | 0.4008     | 0.5682     |\n| Full model       | **19.0984** | **0.6789** | **0.2228** |"}]

Author rebuttal:

Meta-review: This paper introduces a geometry-grounded long-term spatial memory for video world models, combining working, episodic, and spatial memory to improve long-horizon consistency. The approach is technically sound, clearly presented, and shows strong gains over recent baselines with thorough ablations and additional results provided in rebuttal.

Pros:
* Novel memory framework with clear motivation.
* Strong empirical results (quantitative, qualitative, human studies, FVD).
* Practical integration into diffusion backbones; useful for multiple domains.
* Rebuttal addressed reviewer concerns with extra baselines and dynamic-scene results.

Cons:
* Baseline coverage remains somewhat limited.
* Scalability to larger models/datasets not yet demonstrated.

Despite some open issues, the paper makes a clear and impactful contribution to long-term consistent video generation. Therefore, I'd recommend acceptance.
Decision: Accept (poster)