MotionStream: Real-Time Video Generation with Interactive Motion Controls
Authors: Joonghyuk Shin
Zhengqi Li
Richard Zhang
Jun-Yan Zhu
Jaesik Park
Eli Shechtman
Xun Huang
Venue: ICLR 2026 Oral (2026)
Keywords: Interactive Video Generation
Motion Control
Real-Time Generation
Causal Generation
Primary area: generative models
TL;DR: We present MotionStream, a streaming (real-time, infinite length) video generation system with motion controls, unlocking new possibilities for interactive content creation.

Abstract: Current motion-conditioned video generation methods suffer from prohibitive latency (minutes per video) and non-causal processing that prevents real-time interaction. We present MotionStream, enabling sub-second latency with up to 29 FPS streaming generation on a single GPU. Our approach begins by augmenting a text-to-video model with motion control, which generates high-quality videos that adhere to the global text prompt and local motion guidance, but does not perform inference on the fly. As such, we distill this bidirectional teacher into a causal student through Self Forcing with Distribution Matching Distillation, enabling real-time streaming inference. Several key challenges arise when generating videos of long, potentially infinite time-horizons -- (1) bridging the domain gap from training on finite length and extrapolating to infinite horizons, (2) sustaining high quality by preventing error accumulation, and (3) maintaining fast inference, without incurring growth in computational cost due to increasing context windows. A key to our approach is introducing carefully designed sliding-window causal attention, combined with attention sinks. By incorporating self-rollout with attention sinks and KV cache rolling during training, we properly simulate inference-time extrapolations with a fixed context window, enabling constant-speed generation of arbitrarily long videos. Our models achieve state-of-the-art results in motion following and video quality while being two orders of magnitude faster, uniquely enabling infinite-length streaming. With MotionStream, users can paint trajectories, control cameras, or transfer motion, and see results unfold in real-time, delivering a truly interactive experience.

Published as a conference paper at ICLR 2026
MOTIONSTREAM: REAL-TIME VIDEO GENERATION WITH INTERACTIVE MOTION CONTROLS
Joonghyuk Shin
Zhengqi Li
Richard Zhang?
Jun- Yan Zhu?
Jaesik Park'
Eli Shechtman
Xun Huang
Scoul National University, Adobe Research, CCamegie Mellon University, *Morpheus Al
Fixed konih Lngnut trado
generted
Irpua fromne
Bidirectional video diffosion model (0.79 FPS, Latency 1036)
Real time antoregressive diffusion model Latency 0.39s)
ball Leriin dark roon
Stecuming Irpat mrdos
Infinite-lenghh generated video
Real dime drag
Real-time cimera controls
Figure 1: Prior motion-controlled video diffusion models typically operate offline to generate fixed-length sequences in parallel (top left). In contrast, our Mot LonStreaa enables streaming long-video generation from single image with track control at interactive speed (bottom left). Mot Moti onSt ream can be applied to a variety of online downstream applications, such as real-time motion transfer, user drag operations, and 3D camera control (right).
ABSTRACT
Current motion-condition video generation methods suffer from prohibitive la- tency (minutes per video) and non-causal processing that prevents real-time inter- action. We present Mo t. onSt ream, enabling sub-second latency with up to 29 FPS streaming generation on single GPU, Our approach begins by augmenting a text-to-video model with motion control, which generates high-quality videos that adhere to the global text prompt and local motion guidance, but does not perform inference the fly. As such, we distill this bidirectional teacher into causal stu- dent through Self Forcing with Distribution Matching Distillation, enabling real- time streaming inference. Several key challenges arise when generating videos of long, potentially infinite tmee-horizons (1) bridging the domain gap from train- ing on finite length and extrapolating to infinite horizons, (2) sustaining high qual- ity by preventing error accumulation, and (3) maintaining fast inference, without incurring growth computational
due increasing context windows. key to our approach is introducing carefully designed sliding-window window causal atten- tion, combined with attention sinks. By incorporating self-rollout with attention sinks and KV cache rolling during training, we properly simulate inference-time extrapolations with a fixed context window, enabling constant-speed generation of arbitrarily long videos. Our models achieve fftee-o-thhe-e art results in motion following and vidco quality while being two orders of magnitude faster, uniquely enabling infinite-length streaming. With Mot Stttionntrean users paint jectories, control cameras, or transfer motion, and see results unfold in real-time, delivering truly interactive experience.

---

Published as a conference paper at ICLR 2026
INTRODUCTION
The ultimate goal of motion-controlled video synthesis is to give creators the power of director's chair, allowing them to intuitively guide digital actors, objects, and cameras in real time. Although
recent video diffusion models have made impressive strides toward this goal (Wang et al. 2024b
Geng et al. 2025: Burgert et al. 2025 Zhang et al. 2025b Li et al. 2025b oj Niu et al., 2024
Shi et al.) 2024a Zhou al.) 2025a; Lei al. 2025); J. generating high-fidelity videos following
user-specifid motion trajectories, the experience remains far from interactive.
The promise of interactive control is currently hindered by several fundamental constraints. First, generation is too slow for interaction. For example, synthesizing 5-second video clip with Mo- tion Prompting (Geng et al. 2025) takes 12 minutes, trapping users in frustrating "render-and-we wwitt"" cycles. Second, the process is inherently non-causal, since diffusion models process the entire se- quence in parallel with bidirectional attention. A user cannot see any partial results until the entire motion specification is complete. Finally, the inability to generate more than few seconds of video severely limits the scope for any meaningful or extended creative expression. Together, these constraints (slow, non-causal, and short-duration generation) undermine the potential for truly interactive creative experience.
To overcome these challenges, we introduce Mot notionStream. method designed specifically for an interactive creative experience. Unlike conventional diffusion models that operate on the entire video sequence in parallel, Mot inst ream is an autoregressive model that synthesizes video in streaming manner, reacting to user-drawn motion trajectories on-the-fly.
Our approach starts with motion-controlled teacher model that uses lightweight sinusoidal embed- dings with channel-wise concatenation for trajectory conditioning, avoiding the computational over- head of ControlNet-style (Zhang al., 2023) architectures. Trained on both text and motion condi- tions, we introduce joint text-motion guidance that balances precise trajectory adherence with natural secondary motions enabled by text prompts. We then distill this teacher into causal student through Self Forcing-style self-rollout (Huang et al. 2025b). While effective for short sequences, standard approaches drift during extended generation. Our analysis of attention patterns reveals persistent fo- cus on initial frames alongside local temporal dependencies, similar to Streaming (Xiao al.) 2023]. This insight drives our attention sinking mechanism with rolling KV caches, which we incor- porate directly into training properly simulate inference-ting -ime extrapolation distributions, ensuring stable, indefinite-length generation at constant latency through fixed context windows.
Mot MotionStan ream achieves 17 FPS 480P and 10 FPS 720P resolutions with sub-second latency on a single H100 GPU, reaching 29 FPS when optimized with efficient VAF decoders that we specifi- cally train for streaming applications. Through extensive experiments and ablations, we demonstrate state-of-the-arr performance across diverse motion control tasks including camera control, where our approach outperforms recent 3D methods while being more than 20x faster. Mot ionSt transforms video generation from a passive waiting experience into an active creative process, where users can continuously interact with and guide the generation in real-time.
Our key contributions are:
1. We present the first streaming monion-conditioned video generation pipeline capable of running at 29.5 FPS on single H100 GPU, enabling real-time interactive applications.
2. We propose a synergistic system harmonizing efficient architectural designs, including a lightweight track head and conditioning modules, with distillation process that integrates joint text-motion guidance into the training objective, further accelerated by a Tiny VAE.
3. We introduce a distillation strategy for long video generation that systematically explores attention sinks and local attention with extrapolation-awe aware training for the first time, ef- fectively preventing drift during long-term streaming.
4. Our approach achieves state-of-the-an results on motion transfer and camera control at orders of magnitude faster speeds, robustly generalizing to diverse interactive cases.
2

---

Published as a conference paper at ICLR 2026
2 RELATED WORK
Controllable Video Generation. Enabling precise user control is essential for applying video gen- erative models to diverse downstream applications (Li et 2025d Tu et al. 2025 Bahmani al.) 2024a Gao et al.] 2024c Wu ct al.] 2025b] Fu ct al.
2025 p. To this end, a large body of recent
research has explored various types of control signals for video generation, such as structure con- trol (Xing et al. 2024, Yang et al. 2025a Jiang et al., 2025 Pang et al.] [2024] Xing et al.] 2025
camera control Gao et al. 2024b Zheng et al.
2024;
He et al. 2024, Bai et al. 2025| Wu et al,
2025a Yu et al. 2025b Bahmani et al. 2024b
Yang
al. [2024b] Zheng et al.
[024]] (225), sub-
ject control (Huang et al. 2025a] Liu et al.
2025b; Fei et al. 2025 Liu et al. 2025a), and audio
control (Tian et al.] 2024 Gao al.
2025 Peng al. 2024).
As a unique modality that captures underlying vidco dynamics, motion has become a key condi- tioning signal for recent video diffusion models. Recent video diffusion models often condition
generated videos on diverse forms of motion representations, including optical flow, 2D/3D motion
trajectories, bounding boxes, and semantic segmentation (Goldman et al. 2008 Niu et al. 2024 Li
it al. 2024 Wu et al. 2024b Geng et al. 2025 /Zhagg et al. [2025b] Gillman et al. 2025 Shi et al.)
2024a Wu al.) 2024a| Gu al. 2025b Burgert et al. 2025 Tanveer et al. (024). Despite their
impressive quality, these methods are fundamentally limited to offline processing because they rely
on diffusion models with full bidirectional attention, which requires the entire control signal to be known in advance This constraint prevents their use in real-time, interactive applications.
Autoregressive Video Models. Early work adopted generative adversarial networks (GANs) for
autoregressive or parallel video synthesis (Vondrick et al.) [2016] Brooks et al. 2022 Villegas et al.)
2017 Denton al. [017]] Tulyakov et al.) [218]] Liu et al. [021]
al.
2022). More recently,
there has been a paradigm shift towards using diffusion models trained with denoising objectives (Ho
al. 2022 Blattmann et al. 2023b Yang et al. 2025b Kong et al. 2024; Polyak et al., 2024
Blattmann et al. [203aa] Villegas et al.
2023 Deng et al. 22255 Gupta al. [024 Team Wan et al.
2025), or autoregressive (AR) models trained with next-token prediction Weiisenborn et al. 2020
Kondratyuk et al.
2024 Yan al. 2021 1 Wang et al.
2024a Bruce et al. 2024 Ren al. 2025).
Another line of research integrates AR and diffusion to enable causal, high-quality video genera-
tion (Ruhe et al.
2024 Kim et al. 2024 Xie et al.
2024: ZZhang & Agrawala 2025 Sunn et al.
2025
Weng et al.,
2024 Liu et al.] 2024 Chen et al.
2024 Guo et al.][0255b|] Hu et al. [2024]]]] Jin
al.
2025 Gu et al.| [025a Gao et al.
2024a] Li al.] [025f Zhang et al.) 2025a). Our work is
inspired by the recent paradigm that distills a slow teacher model into a fast AR student for real-time
performance (Yin et al. 2025 Huang al.) [0225b Lin et al.,
2025). However, these approaches ei-
ther exhibit severe color drifts beyond the training horizon or require complex long-video finetuning, which poses challenges for controllable video generation.
Interactive Video World Model. Our work also belongs to interactive video world models, which aim to simulate environments for real-time interaction. This area has recently gained significant attention, as several recent works have shown impressive real-time, user-driven interaction (Ball
jet al. 2025 Li et al., 2025a Yan Team 2025 He et al., 2025 Bar et al., 2025 Po et al. 2025). However, most existing approaches either require substantial compute for inference (Ball et al.)
2025 Parker-Holder et al. 2024). or are limited to closed-domain or synthetic environments Yu
et al. 2025a Guo et al. 2025a; Yang et al., 2024a). In contrast, our work demonstrates that real- time, interactive interactivegeneraation for open-domain, photorealistic videos can be achieved on single GPU.
MOTTONSTREAM; STREAMING GENERATION MEETS MOTION CONTROLS
Existing motion-conditioned video generation methods achieve strong motion-video alignment, but cannot support streaming interaction since bidirectional attention requires all future control signals upfront. Our proposed MotionStrean addresses this through carefully designed causal distil- lation techniques, illustrated in Figure 2 We first describe how to equip a pretrained video diffusion model with notionn-conroo capability (Sec. 3.1) to serve as our bidirectional teacher, uti- lizing lightweight track head and control modules designed to minimize architectural overhead. We then introduce our causal distillation pipeline, which performs extrapolation-us aware training with attention sinks and local windows for long video generation, while integrating expensive joint text-
3

---

Published as a conference paper at ICLR 2026
Stage Bidirectional Teacher Training
Alts
"Aninsted scune features clase- peanpt
HXYl J000
Thining Video
Text enooder
Creas
Diffusion Loss
G Greacher
Trieks (B0x 58 pall
Track Heud
Track Emboodding
Diffusion Transformer
Ganedian Natiz
Self Forcing-Sty Distillation (DMD)
cache
adiitiaal areo pudbeed chunnels for Win 2.1)
Stage 2: Causal Distillation
Automgrresdive relling KV tsdar soal allentio sink
Diverse atlention
Gstudent
Full caoeal
Local släding window
Local alkding window with sink
Figure 2: Model architecture and training pipeline. To build a teacher motion-controlled video model, we extract and randomly sample 2D tracks from the input video and encode them using lightweight track head. The resulting track embeddings are combined with the input image, noisy video latents, and text embeddings as input to the diffusion transformer with bidirectional attention, which then trained with a flow matching loss (top). We then distill a few-step causal diffusion model from the teacher through Self Forcing-style DMD distillation, integrating joint text-motion guidance into the objective, where autoregressive rollout with rolling KV cache and attention sink applied during both training and inference (bottom).
motion guidance directly into the distillation objective for efficiency. Combined with our Tiny VAE, these joint efforts enable highly responsive streaming experience.
3.1 ADDING MOTION CONTROLS TO BIDIRECTIONAL TEACHER MODELS
Training high-quality motion-condition teacher model is important, as determines both the quality upper bound and the architectural efficiency of the final distilled system. The teacher must also support diverse motion modalities, from complex real-world object dynamics to camera motions and user drags, which achieve through the following design. We build our motion-guided teacher model on top the Wan DiT family (Team Wan et al. 2025).
Track Representation and Track Head Design. Following MotionPrompting (Geng al. 2025 each 2D track is assigned unique d-dimensional embedding vector én, derived from a randomly sampled ID number through sinusoidal positional encoding. While encoding tracks an RGB video and processing through the CausalVAE is possible, we find that representing them with sinusoidal embeddings with learnable track head achieves superior track adherence, video quality, and faster speed. We validate this our experiments in Table 3 Given N tracks ((*]. vt ) across 7 temporal frames, the input vrackcoonditioning signal cm E BBI''//>xW/axx is constructed by placing visible track embeddings at spatially downsampled locations, where is the VAE spatial downsampling rate and v/(, n) E {0, 1} indicates track visibility:
Cm 1..... J. = ult, n] On.
Our lightweight track-encoding head performs 4x temporal compression followed by 1aala x 1 x convolution. Prior methods adopt a ControlNet-stylee architecture (Zhang et al. 2023 Geng et al. 2025), which doubles FLOPs by duplicating network blocks. Instead, directly concatenate the processed track embeddings with video latents, requiring only minor channel adjustments in the patchifying layer while leaving the core DiT architecture unchanged.

---

Published as a conference paper at ICLR 2026
Training. We train the motion- guided teacher model through rectified flow matching objective (Lu
et al. 2022 Lipman et al., 2022). ). where the forward process linearly interpolates between data 20 [0, 1]. The model is trained to predict the expected velocity fields with conditional flow matching loss LrM- One important limitation to note is that the model cannot inherently distinguish between occluded (non-visible) tracks and unspecified tracks, as both are represented by zero values. When a user releases controls during interaction, the model cannot determine whether the sudden zero values indicate occlusion simply the absence of specification. This ambiguity occasionally leads to artifacts where objects abruptly appear or disap- pear. To address this issue, we introduce stochastic mid-frame masking with probability Pmask 0.2, where Cm. im
and Gaussian noise 21 N (O, I]: 2 (1 [1-t)=a +t2,,
O, for randomly selected mid-frame chunks frand- In practice, we first train the model without masking to establish strong track-following capability, and then fine-tune with stochastic masking to preserve coherence when track signals change intermittently.
Joint Guidance with Text and Motion Conditions. Classifier-fres guidance is an effective tech- nique for steering diffusion models. We use both text and motion guidance and observe that they are complementary to each other. Text guidance generates natural dynamics but fails to maintain tra- jectory adherence. In contrast, track guidance enforces strict trajectory alignment but can produce overly simplistic and rigid motions, such as pure 2D planar translations real-time drag scenarios. Therefore, introduce a joint combination for simultaneous text and motion guidance:
Ubase + W/ (v(ce, Cm) v00 cm)) + wm (v(ce, Cm) v(ce. Ø)),
(2)
where Vbase = a v(Ø ,02,0 Cm) + (1- ul,,,, Ø) and = wt/(wr + Wm) (we omit for brevity). We find that the joint guidance weights (wi, = 3.0, W/m = 1.5) provide a good balance: text condi- tioning enables realistic dynamics, even with sparse, flat-grid inputs, while track guidance preserves trajectories and maintains shape fidelity. We further analyze these effects Sec. [4.3] Although this increases sampling cost from 2 to 3 function evaluations (NFE) per denoising step in the teacher model, our causal distillation (described in the next section) embeds all guidance into single NFE, thereby eliminating this overhead in the student model.
3.2 CAUSAL DISTILLATION
Existing motion-controlled video diffusion models require around 50 denoising steps to generate high-quality video and also presume that the entire input motion trajectory known before generation. In this section, we dis- till the slow teacher model into a causal video diffusion model, enabling real-time stream- ing of long videos with motion control. Our training pipeline starts from the Self Forcing paradigm (Huang et al., 2025b), but off-the- shelf, we find this exhibits large latency fluc- tuations due to varying attention window sizes and performs well only within the teacher's training horizon (i.e., 81 frames), with quality quickly degrading when extrapolating to longer sequences. This section presents several key technical innovations to address these issues.
Tayer 0, Head 10
Tayer 19, Hrad
Tayer 25, Head
Bidrevetoral
Canoll
Spuren Caneal i
Figure 3: Visualization of self attention prob- ability map. We visualize attention probability maps for bidirectional, full causal, and causal slid- ing window attentions. Several attention heads focus on the tokens corresponding to the initial frame throughout denoising generation.
Attention sink in video diffusion model. Au- toregressive generation with sliding-windowd window at- tention, as employed in Self Forcing (Huang al., 2025b). prone to quality degradation and drift during long-video extrapolation. To understand this failure, we visualize the self-
attention maps in Figure 3 for both bidirectional and causal attention. Notably, many attention heads focus on the initial tokens corresponding to the input image. This phenomenon mirrors the observations in large language models Xiao et al. (2023), where initial tokens play crucial role in stable streaming generation. Inspired by this, we adapt the attention sink concept to our video model by maintaining local attention window while preserving the initial frame's tokens fixed anchor
5

---

Published as a conference paper at ICLR 2026
during both training and inference. In our chunk-wiss autoregressive setup, this sinking mechanism prevents the model from drifting during long-video extrapolation, even when trained only on short sequences limited by the teacher model.
Causal Adaptation. Following the initialization protocol from CausVid (Yin et al. 2025). we start student model with the weights of the motion-guided teacher diffusion model and adapt it for causal attention architecture and few-step trajectory, using regression on ODE solution pairs sampled from the teacher. We train the model using attention masks with varying context window size and attention sink size for better generalization.
Self Sareiigg-Sylee Distillation. Following Self Foreing (Huang et al. 2025b), perform temporal
autoregressive roll-out with distribution matching distillation (DMD) (Yin et al. 2022bb a). During roll-out, video chunks are generated sequentially, conditioned on previously self celf-generated outputs rather than ground truth using KV cache.
We first define a video latent divided into L chunks as {2 where is the timestep. The sampling process of the i-th chunk can attend to its own noisy tokens and previously generated clean key and value tokens: C, = { U {=))sss U (=0) m)xx(..-))))scc where S denotes the number of sink chunks, and W the attention window. Setting it to {=]] U {=d]i<s would correspond to causal case with full history context window, matching that of Self Forcing. The generator Go produces each chunk autoregressively through the factorization po(=)- I po (=) C.). Sampling each z/ involves K step iterative denoising on its latent 2/, starting from pure noise 2 I).
In our approach, because the KV cache is continuously updated, RoPE values are assigned based on cache position rather than absolute temporal index. We apply KV cache rolling with attention sinks during both training and inference, implementing local attention without explicit attention masking and fully bridging the train gap, even in extrapolation scenarios. While TalkingMachines (Low & WWnn 2025) also explores attention sinks, our method differs in two key aspects that enhance long-video stability. First, we eliminate the train-inference mismatch by explicitly simulating the extrapolation process during training, using self-rollout with a rolling KV cache and attention sinks. In contrast, TalkingMachines employs synchronized denoising with causal attention masks, which does not fully replicate the dynamics of autoregressive inference. Second, our training process en- sures the teacher model always evaluates continuous video frames. The setup in TalkingMachines introduces temporal discontinuity between the sink frame and subsequent frames, which can push the input outside the teacher's 's pre-training distribution. By maintaining continuity, our teacher pro- vides more robust scores for distillation.
After generating all chunks through self-rollout, we obtain the complete video én = (=d...... # We then apply the DMD objective to this entire sequence, which minimizes the reverse KL divergence between the generator's output distribution and the data distribution: LDMD = E, [DLL (Fy
gen 1 ). The gradient with respect to the generator parameters becomes:
02o
DDLDMD = -Er,ão (S (2o, Spate (2o, t))
(3)
00
where Steal is the score function for real data (approximated by the frozen bidirectional teacher) and Stake is the score function trained on the generator's s outputs.
Intuitively, the driving gradient for the few-step causal generator Go comes from the difference between the estimated real and fake seores. To transfer the high-fidelity control of our joint guidance into the student without inference overhead, define the target real score Sreal using the frozen teacher fo with joint guidance (omitting 2, for brevity):
Syeal
Sbase + we (fo(c,, cm. fo (0, cm.)) um (folce, cm) (c, 0)).
(4)
where Sbase follows the weighting defined in Eq. [ In contrast, we parameterize the fake score Stake without using any CFG through a trainable critic fo (which approximates the generator's is score): Stake (C, ). This configuration effectively "bakes" the computational cost of the teacher's multi-term guidance into the distillation objective, allowing the student generator to replicate the high-quality -uit-g distribution with single function evaluation.
We update the generator (G₂) and fake score estimator (f() with a ratio of 1:5 (Yin et al., (024a)), allowing the critic to better approximate generated distributions. To reduce the memory usage during

---

Published as a conference paper at ICLR 2026
Table 1: Benchmark on Motion Transfer (Reconstruction).
DAVIS Validation Set
Sora Domo Subsct
Method
Backbone & Resolution
FPS
PSNR
SSIM
LPIPS
EPE
PSNR
SSIM
LPIPS
EPE
Image Conductor (Licta) al 2025c
Difm (256P)
2.98
11.30
0.214
0.664
91.64
10.29
0.192
0.644
31.22
Go- With- With-The--Haeee The-Hlow Burget CL a1
2025
CogVideoX-SB (480P)
0.60
15.62
0.392
0.490
41.99
14.59
0.410
0.425
10.27
Diffusion-As SartisionnAs-hhaaee As-Shader ct Al
20256
Cog VideoX-BB (480P)
15.80
0.372
0.483
40.23
14.51
0.382
0.437
18.76
AT1 Wang et al. 2025)
Wan 2.1-14B (480P)
0.23
15.33
0.374
0.473
17.41
16.04
0.502
0.366
6.12
Ours Teacher (Joint CFG)
Wan 2. 1.1..B (480P)
0.79
16.61
0.477
0.427
5.35
17.82
0.586
0.333
2.71
Ours Causal (Distilled)
Wan 2.1-1.38 (480P)
16.7
16.20
0.447
0.443
7.80
16.67
0.531
0,360
4.21
Teacher (Joint CFG)
2:2-5B (720P)
0.456
0.427
7.86
17.18
0.571
0.331
3.16
Ours Causal (Distilled)
Wan 2:2-58 (720P)
10.4
16.30
0.456
0.438
11.18
16.62
0.545
0.343
4.30
self-rollout, we adopt Self Forcing's gradient truncation strategy: randomly sampling denoising step k from [1, K] and using the denoised output at k-th step as the final output, enabling gradient backpropagation only through that step. By additionally treating the cached KV from previous frames stop-gradien context (detached), we track gradients only for the selected denoising step of the current block, effectively reducing the memory usage.
Inference. Our inference procedure identically follows the training process to ensure perfect train-test match. We maintain a KV cache composed of chunks from the initial frames and a fixed-size local window of recent W chunks. As new tokens are generated, this local window is "rolled" to maintain a constant size. For positional encoding, we store the final, RoPE-applied values for the static sink tokens, while tokens in the rolling window store Pre-RoPE activations and receive positional indices dynamically based on their current cache location. Because this entire mechanism is simulated during training, the model seamlessly handles the discontinuity between sink and window tokens. This approach yields two key advantages over full-context methods: (1) initial image anchoring prevents drift during long rollouts, and (2) throughput and latency remain constant regardless of generated video length. We analyze the effects of attention sink and window size in Sec. 4.3 and describe our streaming pipeline in Sec. 4.4
EXPERIMENTS
4.1 IMPLEMENTATION DETAILS
We build upon image-to-video (I2V) variants of Wan 2.1 (1.3B) and Wan 2.2 (5B) (Team Wan al. 2025). We train teacher models on Open Vid- (Nan al. 2024) and synthetic data gen- erated by Wan text-to-video (T2V) model: Wan 2.1 at 832x480 (70K synthetic samples) and Wan 2.2 at 1280x704 (30K synthetic samples). For causal adaptation and Self Forcing distillation, we sample input images, text prompts and 2D tracks from the synthetic datasets described above. For computational efficiency, we track all real and synthetic videos from a 50x50 uniform grid with CoTracker3 (Karaev et al. 2024). We refer readers to Appendix for additional details.
4.2
QUANTITATIVE EVALUATIONS
Motion Transfer.
We evaluate motion-following capability on two datasets: the DAVIS (Perazzi al. 2016) validation set (30 videos) and 20 curated videos from the Sora webpage (Brooks al., 2024 We use both because DAVIS presents challenging sequences with significant occlusions, while the Sora set provides clean examples with consistent visibility, ensuring a comprehensive evaluation. We directly compare the synthesized results with the corresponding ground-truth video frames. Visual fidelity is measured using PSNR, SSIM, and LPIPS (Zhang et al., 2018), while motion accuracy is assessed via End-Point Error (EPE), computed as the 1.2 distance between visible input tracks and the tracks extracted from the generated videos. All models are evaluated in their optimal configuration, and evaluations are performed at 832 x 480 resolution after resizing. Speed measures are based on a single H100 GPU. We refer readers to Appendix for detailed protocols.
Camera Control. To assess our model's ability to generate videos following camera con- trols, we evaluate its zero-shot performance on single-image 3D novel view synthesis. We compare it against several recent diffusion- and feed-forward-bate
view synthesis base- lines (Zhou et al.] 20256] Yu et al.) [024] Xu et al.) (202)) on the LLFF dataset (Milden- To adapt 2D track-controlled model for this task, first
hall al. 2019).
7

---

Published as a conference paper at ICLR 2026
Input
Modoo Onle
Hytes (S)
Tex
3.0.
0.36
LPPS
2.
0.54
Moxion Gnibaase Stald (ple)
Motion Gradenes Stale (1r))
Gradance
1111
9y-22
vy -30
Figure 4: Quantitative ablation on guidance.
Figure 5: Qualitative ablation on guidance. Pure motion guidance produces rigid movements while text guidance enables natural motion and shape preservation even with imperfect tracks. Our Hybrid joint guidance balances these two.
We use Sora subset to ablate guidance strate- gies. Higher text guidance reduces overall met- rics while motion guidance improves trajectory
accuracy at the cost of visual quality (LPIPS).
timate scene geometry using monocular depth network (Wang et al., 2025b) and com- pute single scale factor to aligm the predicted depth with the reconstructed scene point cloud from COLMAP (Schonberger
Frahm 2016). We then derive 2D motion trajecto-
ries by interpolating between the input and target frame using depth and camera parameters. For all methods, the input image as- pect ratio is fixed at 16:9, and all syn- thesized frames are resized to 512 x 288 before evaluation with PSNR, SSIM, and LPIPS metrics. As shown track-conditioned
Table 2: Evaluation on Novel View Synthesis.
LLFF
Method
Resolution
FPS
PSNR
SSIM
LPIPS
in Table [ video generation models outperform other 3D novel view synthesis base- lines by large margin, even though they are not specifically designed for this task. Moreover, our causal mod- els achieve significantly higher gen- eration throughput compared with both the baselines and their bidirectional counterparts.
DepthSplat Xu el.
2025
576P
1.40
13.9
0.28
0.30
ViewCrafier ef et 2024
576P
0.26
14.0
0.30
0.30
SEVA (Zou at 20256
576P
0.20
14.1
0.30
0.29
Ours Teacher (1.38)
480P
0.79
16.0
0.42
0.21
Cansal (1,38)
480P
16.7
15.7
0.23
Ours Teacher (5B)
720P
0.74
14.0
0.40
0.22
Ours Causal (5B)
720P
10.4
15.0
0.39
0.23
4.3 ABLATION EXPERIMENTS
Track Representation. We compare our sinusoidal position encoding with a learnable track head against the RGB encoding strategy using frozen VAE, following prior work (Gu ct al.) 2025b). where each 2D track is assigned unique RGB color vector and placed onto a canvas before being fed into the VAE. Table 3 shows that our method (PE-Head) outperforms RGB-VAE in both efficiency and quality. Specifically, our lightweight PE-based encoding achieves better motion alignment while being two orders of magnitude faster than the VAE-based approach. We hypothe- size sinusoidal encoding preserves stronger identification signals compared to RGB encoding due to richer expressive dimensions.
Table 3: Comparing track representation methods. Our sinusoidal PE with learnable track head outperforms RGB-VAE in both quality and efficiency, achieving 40x faster encoding critical for real-time streaming.
Time (ms)
DAVIS / Sora
Method
PSNR
LPIPS
EPE
RGB-VAE
1053
16.03
16.99
0.433
0.544
0.463 0.336 8.57 3.96
PE-Head
24.8
16.29
17.15
0.452/ 0.559
0.456 0.359
6.54/3.13
While pure motion guidance (w, = O, Wmm > 0) achieves the highest trajectory accuracy, as shown in Figure 4 text guidance provides additional benefits for generating more diverse and realistic results. For example, text captions enable dynamics beyond trajectories alone, such as weather changes or object appear- ances as shown in the first of Figure which illustrates dragging "rainbow appears in background". Our empirical setting (w, = 3.0, W,, = 1.5) balances motion fidelity with natural dynamics, adapting equally well both precise and imperfect trajectories.
Guidance Strategies. Here we ablate our joint guidance approach from Sec 3.1
elephant while prompting
8

---

Published as a conference paper at ICLR 2026
Joday
12UPS
(Local Wandzw-1)
Maxhow
(Locd Winkre-11
Winder (Sai-1))
Framz
Frasce
(2) Impact of Sink Size & Local Window Size
(b) LPIPS Over Time Actoss Diverse Sctups
(c) EPE Over Across
Across Diverse Setups
Figure 6: Impact of Sparse Attention Patterns. Using longer clips (up to 241 frames) from the Sora subset, we ablate attention sink size and local window size in extrapolation scenarios. Having at least single sink chunk is crucial, but more provides marginal benefit, while larger window sizes degrade performance as attending to long-pat pag-ppst history allows errors to accumulate in context tokens.
Impact of Chunk Size, Attention Sink, and Window Size. We investigate three key design choices that govern streaming quality and interactivity: the latent chunk size, the attention sink size, and the local context window size.
The chunk size, which defines how many video latents are processed in parallel, presents a critical trade-off between quality, latency, and throughput. As illustrated in the latency-throughput analysis in Figure A2] small chunks lead to lower throughput, while large chunks (>3) introduce prohibitive latency for real-time interaction. Additionally, shown in Tab. 4 chunk size of causes signifi- cant quality degradation. We, therefore, select a chunk size of as our optimal configuration, with further analysis provided in Appendix Sec. C
To assess the impact of attention sinks and window size on model per- formance, we train a single model with randomly sampled sink and win- dow sizes and then generate videos under different configuration com- binations. We evaluate long-video extrapolation on Sora videos up to 241 frames (average 194 frames). Surprisingly, we find that the min- imal configuration (a single-chunk sink with a single-chunk window) achieves the best performance. Figure a(a) shows that additional sinks provide only marginal gains while increasing latency, and expanded windows actually degrade performance. We hypothesize that this phenomenon arises because restricting the context immediate predecessors, rather than long-past history. prevents error accumulation and thus reduces drift for long-video generation.
Table 4: Ablation study on Sora Extended. c3slwl maintains high visual quality with small latency & through- put fluctuations, while vanilla sliding window exhibits large fluctuations degrading streaming stability.
Sora Extended
Contig
LPIPS
EPG
Latency (s)
Throughpul (FPSS
Ours base (c3s1w1)
0.464
25.34
0.70 + 0.01
16.92 +: 0.80
Remove sink (0380wL)
0.501
26.64
+ 0.005
17.43 0.88
Chunk Size (clslw1)
0.597
76.21
0,30
0.01
13.26 + 1,36
Sliding window
0.480
28.09
0.80 0.08
14.96 +: 1.42
We additionally show reconstruction accuracy and throughput as well as their time evolution in Table A and Figure bbb)c) respectively. We denote our configuration as chunk-3, sink-1, window-1 (abbreviated c3slw1), while sliding window approach used in Self Forcing c3s0w6, attending maximum of 6 previous chunks without sink tokens, with unbounded RoPE positions that scale with temporal frame count. Although removing sink tokens (c3s0w1) yields marginal speed improvements, this comes at the cost of degraded long-term generation stability.
4.4 STREAMING DEMO AND QUALITATIVE RESULTT
For streaming demos, we further optimize the pipeline with an efficient Tiny VAE. Inspired by Boer Bohan (2025), we design and train a smaller VAE decoder with adversarial and LPIPS losses, regressing to the original VAE's latent space. Tiny VAE removes the VAE bottleneck by reducing the decoding time over 10x, improving Wan 2.1 from 16.7 FPS with 0.69s latency to 29.5 FPS with 0.39s latency, and Wan 2.2 from 10.4 FPS with I.Is latency to 23.9 FPS with 0.49s la- tency on a single H100 GPU. We provide details for the tiny VAE in Appendix A noting that the performance trade-off from switching to the tiny VAE is marginal in streaming scenarios, shown

---

Published as a conference paper at ICLR 2026
Long Video Motion Transfer
Streaming Motion Transfer with Online Trackers (Human Pose)
Control
Drag Control
Figure 7: Qualitative Results. Mot i onSt ream can perform diverse downstream applications, in- cluding long video motion transfer (from offline or online sources), drag-based controls, and precise camera control with depth estimation. We showcase a few examples here.
Figure 8: Streaming Demo. An example of real-time streaming generation; more details in Sec.
Leveraging track representation, our method enables causal, real-time execution of capabilities typically found in motion control approaches, such mouse-based drag control and motion transfer using tracks from online trackers. Figure 7 demonstrates several these appli- cations. Additionally, develop interactive demo that enables real-time user control during streaming generation. Figure shows an example, with further details Sec. C and Figure A5
Table A2
5 CONCLUSION
We propose MotionStream a framework for infinite-lnngth video generation with interactive motion control, maintaining a stable 29 FPS a single GPU. Our contributions span from training a motion-guided teacher with efficient track head and joint text-motion guidance, to distilling it into causal student via self-rollout with attention sink and rolling KV caches. MotionSt z amm achieves state-of-the afttth-arrt results across diverse motion nottio-conditiooed generation tasks, while being significantly faster than prior methods. Limitations and future directions, as well ethics and reproducibility statements are discussed in Appendix and G
Acknowledgments. This work was done at Adobe Research, during Joonghyuk Shin's 's internship and while Xun Huang a Research Scientist. Jacsik Park was supported by the ITT grant No.
1-2022-1121134 (Artificial Intelligence Graduate School Program at Scoul National University, 5%). No. 252--22544238 (AI Star Fellowship Support Program, 35%). and the NRF grant No. RS-2024-004058857 (60%) funded by the Korea government (MSIT). Jun- Yan Zhu was supported by the Packard Fellowship.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
AIGC-Apps and Alibaba PAI Team. VideoX-Fun: A Flexible Framework for Video Generation at 2024.
Any Resolution.
Sherwin Bahmani, Ivan Skorokhodov, Victor Rong, Gordon Wetzstein, Leonidas Guibas, Peter Wonka, Sergey Tulyakov, Jeong Joon Park, Andrea Tagliasacchi, and David B Lindell. 4d-fy: Text-to-4d generation using hybrid score distillation sampling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 7996-8996099 8006, 2024a.
Sherwin Bahmani, Ivan Skorokhodov, Aliaksandr Siarohin, Willi Menapace, Guocheng Qian, Michael Vasilkovsky, Hsin- Ying Lee, Chaoyang Wang, Jiaxu Zou, Andrea Tagliasacchi, et al. Vd3d: Taming large video diffusion transformers for 3d camera control. arXiv preprint arXiv:2407.12781, 2024b.
Jianhong Bai, Menghan Xia, Xiao Fu, Xintao Wang, Lianrui Mu, Jinwen Cao. Zuozhu Liu, Haoji Hu, Xiang Bai, Pengfei Wan, et al. Recammaster: Camera-controlled generative rendering from single video. arXiv preprint arXiv:2503.
2025.
Philip Ball, Bauer, F Belletti, et al. Genie 3: A new frontier for world models, 2025.
Amir Bar, Gaoyue Zhou, Danny Tran, Trevor Darrell, and Yann LeCun. Navigation world models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 15791-15801, 2025.
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15122 2023a.
Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent diffusion models. In CVPR, 2023b.
Ollin Boer Bohan. Tachv: Tiny autoencoder for hunyuan video. https: com/ mmaebyollin/taehi 2025.
Tim Brooks, Janne Hellsten, Miika Aittala, Ting-Chus Wang, Timo Aila, Jaakko Lehtinen, Ming- Yu Liu, Alexei Efros, and Tero Karras. Generating long videos dynamic scenes. NeurlPS, 2022.
Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng. Ricky Wang, and Aditya Ramesh. Video generation models world simulators, 2024. URL ptttt:::/oppnall. www. reem/reeearch//
iiteeo-geeerationn snneratioommmdels as wor weerrddsiimnne simulators
Jake Bruce, Michael Dennis, Ashley Edwards, Jack Parker-Holder, Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, et al. Genie: Generative inter- active environments. ICML, 2024.
Ryan Burgert, Yuancheng Xu, Wenqi Xian, Oliver Pilarski, Pascal Clausen, Mingming He, Li Ma, Yitong Deng. Lingxiao Li, Mohsen Mousavi, et al. wit-wwthhhhfooww Notion-conttolllabe video diffusion models using real-time warped noise. In CVPR, 2025.
Boyuan Chen, Diego Martí Monsó, Yilun Du, Max Simchowitz, Russ Tedrake, and Vincent Sitz- mann. Diffusion forcing: Next-token prediction meets full-sequence diffusion. In NeurIPS, 2024.
June Suk Choi, Kyungmin Lee, Sihyun Yu, Yisol Choi, Jinwoo Shin, and Kimin Lee, Enhanc- ing motion dynamics of image-to-video models via adaptive low-pass guidance. arXiv preprint arXiv:2506.0844 08456. 2025.
Haoge Deng, Ting Pan, Haiwen Diao, Zhengxiong Luo, Yufeng Cui, Huchuan Lu, Shiguang Shan, Yonggang Qi, and Xinlong Wang. Autoregressive video generation without vector quantization. In ICLR, 2025.
11

---

Published as a conference paper at ICLR 2026
Emily L Denton et al. Unsupervised learning of disentangled representations from video. In NeurIPS, 2017.
Zhengcong Fei, Debang Li, Di Qiu, Jiahua Wang, Yikun Dou, Rui Wang, Jingtao Xu, Mingyuan Fan, Guibin Chen, Yang Li, et al. Skyreels-a2: Compose anything in video diffusion transformers. arXiv preprint WrXx::2504.02436 2025.
Xiao Fu, Xintao Wang, Xian Liu, Jianhong Bai, Runsen Xu, Pengfei Wan, Zhang, and Dahua Lin. Learning video generation for robotic manipulation with collaborative trajectory control. arXiv preprint wwX.i:25066009440 2025.
Kaifeng Gao, Jiaxin Shi, Hanwang Zhang, Chumping Wang, Jun Xiao, and Long Chen. Ca2-vdm: Efficient autoregressive video diffusion model with causal generation and cache sharing. arXiv preprint arXiv:2411.16377 2024a.
Ruiqi Gao, Aleksander Holynski, Philipp Henzler, Arthur Brussee, Ricardo Martin Brualla, Pratul Srinivasan, Jonathan T Barron, and Ben Poole. Cat3d: Create anything in 3d with multi-view diffusion models. arXiv preprint arXiv:2405. 10314, 2024b.
Shenyuan Gao, Jiazhi Yang, Li Chen, Kashyap Chitta, Yihang Qiu, Andreas Geiger, Jun Zhang, and Hongyang Li. Vista: A generalizable driving world model with high fidelity and versatile controllability. Advances in Neural Information Processing Systems, 777915500-91999 2024c.
Xin Gao, Li Hu, Siqi Hu, Mingyang Huang, Chaonan Ji, Dechao Meng. Jinwei Qi, Penchong Qiao, Zhen Shen, Yafei Song, Ke Sun, Linrui Tian, Guangyuan Wang. Qi Wang. Zhongjian Wang, Jiayu Xiao, Sheng Xu, Bang Zhang, Peng Zhang, Xindi Zhang, Zhe Zhang, Jingren Zhou, and Lian Zhuo Wan-s2v: Audio-driven cinematic video generation, 2025. URL httthttps://arat
org/ /t///bb/2250 18621
Daniel Geng, Charles Hermman, Junhwa Hur, Forrester Cole, Serena Zhang, Tobias Pfaff, Tatiana Lopez-Guevare Yusuf Aytar, Michael Rubinstein, Chen Sun, et al. Motion prompting: Control- ling video generation with motion trajectories. In CVPR, 2025.
Nate Gillman, Charles Herrmann, Michael Freeman, Daksh Aggarwal, Evan Luo, Deqing Sun, and Chen Sun. Foree prompting: Video generation models learn and generalize physics-based based control signals. arXiv preprint arXiv:2505.1 19386, 2025.
Dan B Goldman, Chris Gonterman, Brian Curless, David Salesin, and Steven M Seitz. Video object annotation, navigation, and composition. In Proceedings of the 21 annual ACM symposium on User interface software and technology, pp. 3-12, 2008
Yuchao Gu, Weijia Mao, and Mike Zheng Shou. Long-contexs autoregressive video modeling with next-frame prediction. arXiv preprint arXiv:2503. 19325, 2025a.
Zekai Gu, Rui Yan, Jiahao Lu, Peng Li, Zhiyang Dou, Chenyang Si, Zhen Dong, Qifeng Liu, Cheng Lin, Ziwei Liu, et al. Diffusion shader: 3d-aware video diffusion for versatile video generation control. In Proceedings of the Special Interes! Group on Computer Graphics and Interactive Techniques Conference Conference Pupers, pp. 1-12, 2025b.
Junliang Guo, Yang Ye, Tianyu He, Haoyu Wu, Yushu Jiang, Tim Pearce, and Jiang Bian. Mineworld: a real-time and open-source interactive world model on minecraft. arXiv preprint arXiv:2504.08388, 2025a.
Yuwei Guo, Ceyuan Yang, Ziyan Yang. Zhibei Ma, Zhijie Lin, Zhenheng Yang, Dahua Lin, and Lu Jiang. Long context tuning for video generation. arXiv preprint arXiv:2503. 10589, 2025b.
Agrim Gupta, Lijun Yu, Kihyuk Sohn. Xiuye Gu, Meera Hahn, Fei-Fei Li, Irfan Essa, Lu Jiang, and José Lezama. Photorealistic video generation with diffusion models. In ECCV, 2024.
Hao He, Yinghao Xu, Yuwei Guo, Gordon Wetzstein, Bo Dai, Hongsheng Li, and Ceyuan Yang. Cameractrl: Enabling camera control for text-to-video generation. arXiv preprint a0Xiv:2444.02101, 2024.
12

---

Published as a conference paper at ICLR 2026
Xianglong He, Chunli Peng, Zexiang Liu, Boyang Wang. Yifan Zhang, Qi Cui, Fei Kang. Biao Jiang. Mengyin An, Yangyang Ren, et al. Matrix Matrix-game game 2.0: An open-source, real-time, and streaming interactive world model. arXiv preprint arXiv:2508. 2025.
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David Fleet. Video diffusion models. In NeurlPS, 2022.
Jinyi Hu, Shengding Hu, Yuxuan Song, Yufei Huang. Mingxuan Wang. Hao Zhou, Zhiyuan Liu, Wei-Ying Ma, and Maosong Sun. Acdit: Interpolating autoregressive conditional modeling and diffusion transformer. arXiv preprint arXiv:24/2.07728, 2024.
Chi-Pin Huang, Yen-Siang Wu, Hung-Kai Chung, Kai-Po Chang, Fu-En Yang, and Yu- Yu-Chiang Frank Wang. Videomage: Multi-subject and motion customization of text-to-video diffusion models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 17603-17612, 2025a.
Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman. Self forcing: Bridging the train-test gap in autoregressive video diffusion. arXiv preprint arXiv:2506. 08009, 2025b.
Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianx- ing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive benchmark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 21807-218 2024.
Lifan Jiang, Shuang Chen, Boxi Wu, Xiaotong Guan, and Jiahui Zhang. Vidsketch: Hand-drawn sketch-driven video generation with diffusion control. arXiv preprint arXiv:2502.01101, 2025.
Yang Jin, Zhicheng Sun, Ningyuan Li, Kun Xu, Hao Jiang, Nan Zhuang, Quzhe Huang, Yang Song, Yadong Mu, and Zhouchen Lin. Pyramidal flow matching for efficient video generative modeling. In ICLR, 2025.
Nikita Karaev, lurii Makarov, Jianyuan Wang, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Cotracker3: Simpler and better point tracking by pseudo-labelling real videos. arXiv preprint arXiv:2410.1188 2024.
Jihwan Kim, Junoh Kang. Jinyoung Choi, and Bohyung Han. Fifo-diffusion: Generating infinite videos from text without training. Advances in Neural Information Processing Systems, 37: 89834-89868, 2024.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015.
Dan Kondratyuk, Lijun Yu, Xiuye Gu, Jose Lezama, Jonathan Huang, Grant Schindler, Rachel Hornung, Vighnesh Birodkar, Jimmy Yan, Ming-Chang Chiu, et al. Videopoet: A large language model for zero-shot video generation. ICML, 2024.
Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603,, 2024.
Guojun Lei, Chi Wang, Rong Zhang, Yikai Wang, Hong Li, and Weiwei Xu. Animateanything: Consistent and controllable animation for video generation. In CVPR, 2025.
Jiaqi Li, Junshu Tang, Zhiyong Xu, Longhuang Wu, Yuan Zhou, Shuai Shao, Tianbao Yu, Zhiguo High-dynamic interactive game video generation with
Cao, and Qinglin Lu. Munyuan-gamectn hybrid history condition. arXiv preprint ar X50: 2022206661722, 2025a.
Quanhao Li, Zhen Xing, Rui Wang, Hui Zhang, Qi Dai, and Zuxuan Wu. Magicmotion: Controllable video generation with dense-to-syee to-sea trajectory guidance. In ICCV, 2025b.
Ruining Li. Chuanxia Zheng, Christian Rupprecht, and Andrea Vedaldi. Puppet-master, Scaling interactive video generation as a motion prior for part-level dynamics. In Proceedings of the IEFF/CVF International Conference Computer Vision pp. 13405-13415, 2025c.
13

---

Published as a conference paper at ICLR 2026
Xiaowen Li, Haolan Xue, Peiran Ren, and Liefeng Bo. Diffueraser: A diffusion model for video inpainting, arXiv preprint arXiv:2501.10018 2025d.
Yaowei Li, Xintao Wang. Zhaoyang Zhang, Zhouxia Wang. Ziyang Yuan, Liangbin Xie, Ying Shan, and Yuexian Zou, Image conductor: Precision control for interactive video synthesis. In AAAI, 2025e.
Zhengqi Li, Qianqian Wang, Noah Snavely, and Angjoo Kanazawa. Infinitenature-zroo: Learning perpetual view generation of natural scenes from single images. In ECCV, 2022.
Zhengqi Li, Richard Tucker, Noah Snavely, and Aleksander Holynski. Generative image dynamics. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 24142-24153, 2024.
Zongyi Li, Shujie Hu, Shujie Liu, Long Zhou, Jeongsoo Choi, Lingwei Meng, Xun Guo, Jinyu Li, Hefei Ling, and Furu Wei. Arlon: Boosting diffusion transformers with autoregressive models for long video generation. In ICLR, 2025f.
Shanchuan Lin, Xin Xia, Yuxi Ren, Ceyuan Yang, Xuefeng Xiao, and Lu Jiang. Diffusion adversar- ial post-training for one-step video generation. arXiv preprint arXiv:2501.08116 2025.
Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamm Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747, 2022.
Andrew Liu, Richard Tucker, Varun Jampani, Ameesh Makadia, Noah Snavely, and Angjoo Kanazawa. Infinite nature: Perpetual view generation natural scenes from a single image. In ICCV, 2021.
Lijie Liu, Tianxiang Ma, Bingchuan Li, Zhuowei Chen, Jiawei Liu, Gen Li, Siyu Zhou, Qian He, and Xinglong Wu. Phantom: Subject-consistent video generation via cross-modal alignment. arXiv preprint 10X. 2025a.
Xiaoyu Liu, Mingshuai Yao, Yabo Zhang, Xianhui Lin, Peiran Ren, Xiaoming Li, Ming Liu, and Wangmeng Zuo. Animateanywherr where: Rouse the background in human image animation. arXiv preprint arXiv:2504. 19834, 2025b.
Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. ArXiv, abs/2209.03000 2022. URL hhttss:/aapi.
semant cccholarrcc...oo.oooooooco www.errrr/ooorrreeoeee /Crppus ID: 252111177
Yaofang Liu, Yumeng Ren, Xiaodong Cun, Aitor Artola, Yang Liu, Tieyong Zeng, Raymond H Chan, and Jean-michel Morel. Redefining temporal modeling video diffusion: The vectorized timestep approach. arXiv preprint arXiv:2410.03160, 2024.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In ICLR, 2019.
Chetwin Low and Weimin Wang. Talkingmachines: Real-time audio-driven facetime-style video via autoregressive diffusion models. arXiv preprint arXiv:2506. 03099, 2025.
Ben Mildenhall, Pratul P Srinivasan, Rodrigo Ortiz-Cayon, Nima Khademi Kalantari, Ravi Ra- mamoorthi, Ren Ng, and Abhishek Kar. Local light field fusion: Practical view synthesis with prescriptive sampling guidelines. ACM Transactions on Graphics (ToG), 38(4):1-14, 2019.
Kepan Nan, Rui Xie, Penghao Zhou, Tichan Fan, Zhenheng Yang, Zhijic Chen, Xiang Li, Jian Yang. and Ying Tai. Openvid- 1m:ddmmmmmmmmmmm I:: A large-scale high-quality dataset for text-to-video generation. arXiv preprint 2rXiv:2407.02371. 2024.
Shen Nie, Hanzhong Allan Guo, Cheng Lu, Yuhao Zhou, Chenyu Zheng, and Chongxuan Li. The blessing of randomness: Sde beats ode general diffusion-based image editing. arXiv preprint
arXiv:2311.01410, 2023.
Muyao Niu, Xiaodong Cun, Xintao Wang. Yong Zhang, Ying Shan, and Yingiang Zheng, Mofa- video: Controllable image animation via generative motion field adaptions in frozen image-to- video diffusion model. In ECCV, 2024.
14

---

Published as a conference paper at ICLR 2026
Xingang Pan, Ayush Tewari, Thomas Leimküihler, Lingjie Liu, Abhimitra Meka, and Christian Theobalt. Drag your gan: Interactive point-based manipulation on the generative image mani- fold. In ACM SIGGRAPH 2023 conference proceedings, pp. 1-11, 2023.
Yatian Pang, Bin Zhu, Bin Lin, Mingzhe Zheng, Francis EH Tay, Ser-Nam Lim, Harry Yang, and Li Yuan. Dreamdance: Animating human images by enriching 3d geometry cues from 2d poses. arXiv preprint arXiv:2412.00330 2024.
Jack Parker-Holder, Philip Ball, Jake Bruce, Vibhavari Dasagi, Kristian Holsheimer, Chris- tos Kaplanis, Alexandre Moufarek, Guy Scully, Jeremy Shar, Jimmy Shi, Stephen Spencer, Jessica Yung. Michael Dennis, Sultan Kenjeyev, Shangbang Long. Vlad Mnih, Harris Chan, Maxime Gazeau, Bonnic Li, Fabio Pardo, Luyu Wang. Lei Zhang, Frederic Besse, Tim Harley, Anna Mitenkova, Jane Wang, Jeff Clune, Demis Hassabis, Raia Hadsell, Adrian Bolton, Satinder Singh, and Tim Rocktäschel. Genie 2: A large-scale foun- dation world model. 2024. URL https: /ttp:ss://eeeeennnnn// google//iscoveer////e gerie-2-ar a- a-ragge scale e- foundation world world-model/ model/
Ziqiao Peng. Wentao Hu, Yue Shi, Xiangyu Zhu, Xiaomei Zhang, Hao Zhao, Jun He, Hongyan Liu, and Zhaoxin Fan. Synctalk: The devil is the synchronization for talking head synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
666-676, 2024.
F. Perazzi, J. Pont-Tuset, B. McWilliams, L. Van Gool, M. Gross, and A. Corkine-Hornung A benchmark dataset and evaluation methodology for video object segmentation. In Computer Vi- sion and Pattern Recognition, 2016.
Ryan Po, Yotam Nitzan, Richard Zhang, Berlin Chen, Tri Dao, Eli Shechtman, Gordon Wetzstein, and Xun Huang. Long-conted state-space video world models. ICCV, 2025.
Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih- Yao Ma, Ching- Yao Chuang, et al. Movie gen: A cast of media founda- tion models. arXiv preprint arXiv:2410.137200 2024.
Shuhuai Ren, Shuming Ma, Xu Sun, and Furu Wei. Next block prediction: Video generation via semi-autooregressive modeling. arXiv preprint arXiv:2502.0773/1 2025.
David Ruhe, Jonathan Heek, Tim Salimans, and Emiel Hoogeboom. Rolling diffusion models. In ICML, 2024.
Johannes L Schonberger and Jan-Michael Frahm. Structureffrommmotion revisited. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4104 4113, 2016.
Jay Shah, Ganesh Bikshandi, Ying Zhang, Vijay Thakkar, Pradeep Ramani, and Tri Dao. Flashattention-3:: Fast and accurate attention with asynchrony and low-precision. Advances in Neural Information Processing Systems, 37:68658 666888668888.088 2024.
Xiaoyu Shi, Zhaoyang Huang, Fu- Yun Wang. Weikang Bian, Dasong Li, Yi Zhang, Manyuan Zhang, Ka Chun Cheung, Simon See, Hongwei Qin, et al. Motion-i2y: Consistent and controllable image-to-video generation with explicit motion modeling. ACM SIGGRAPHI 2024 Conference Papers, pp. 1-11, 2024a.
Yujun Shi, Chuhui Xue. Jun Hao Liew, Jiachun Pan, Hanshu Yan, Wenqing Zhang, Vincent YF Tan, and Song Bai. Dragdiffusion: Harnessing diffusion models for interactive point-based image edit- ing. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 1839-88499,2 2024b.
Joonghyuk Shin, Dachyeon Choi, and Jaesik Park. Instantdrag: Improving interactivity in drag- based image editing. In SIGGRAPH Asia 2024 Conference Papers, pp. 1-10, 2024.
Mingzhen Sun, Weining Wang, Gen Li, Jiawei Liu, Jiahui Sun, Wanquan Feng, Shanshan Lao, Yi Zhou, Qian He, and Jing Liu. Ar-diffusion: Asynchronous video generation with auto-regressive diffusion. In CVPR, 2025.
15

---

Published as a conference paper at ICLR 2026
Maham Tanveer, Yang Zhou, Simon Niklaus, Ali Mahdavi Amiri, Hao Zhang, Krishna Kumar Singh, and Nanxuan Zhao. Motionbridge: Dynamic video inbetweening with flexible controls.
arXiv preprint ar arXhv:2412.13191 2024.
Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang. Jianyuan Zeng, Jiayu Wang, Jingfeng Zhang. Jingren Zhou, Jinkai Wang, Jixuan Chen, Kai Zhu, Kang Zhao, Keyu Yan, Lianghua Huang. Mengyang Feng. Ningyi Zhang. Pandeng Li, Pingyu Wu, Ruihang Chu, Ruili Feng. Shiwei Zhang, Siyang Sun, Tao Fang, Tianxing Wang. Tianyi Gui, Tingyu Weng. Tong Shen, Wei Lin, Wei Wang, Wei Wang. Wenmeng Zhou, Wente Wang, Wenting Shen, Wenyuan Yu, Xianzhong Shi, Xiaoming Huang, Xin Xu, Yan Kou, Yangyu Lv, Yifei Li, Yijing Liu, Yiming Wang. Yingya Zhang, Yitong Huang, Yong Li, You Wu, Liu, Yulin Pan, Yun Zheng, Yuntao Hong, Yupeng Shi, Yutong Feng, Zeyinzi Jiang, Zhen Han, Zhi-Fan Wu, and Ziyu Liu. Wan: Open and advanced large-scale scale video generative models. 2025.
arXiv preprint
The FastVideo Team. Github hao-ai-lab/fastvideo: A unified inference and post-training framework for accelerated video generation. URL https://github. com ai lab/
FastVideo
Linrui Tian, Qi Wang, Bang Zhang, and Liefeng Bo. Emo: Emote portrait alive generating ex- pressive portrait videos with audio2video diffusion model under weak conditions. European Conference Computer Vision, pp. 244-260. Springer, 2024.
Yuanpeng Tu, Hao Luo, Xi Chen, Sihui Ji, Xiang Bai, and Hengshuang Zhao. Videoanydoor: High- fidelity video object insertion with precise motion control. In Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers, pp. 2025.
Sergey Tulyakov, Ming-Yo Yu Liu, Xiaodong Yang, and Jan Kautz. Mocogan: Decomposing motion and content for video generation. In CVPR, 2018.
R Villegas, H Moraldo, S Castro, M Babacizadeh, H Zhang. J Kunze, PJ Kindermans, MT Saffar, and Erhan. Phenaki: Variable length video generation from open domain textual descriptions. In ICLR, 2023.
Ruben Villegas, Jimei Yang. Seunghoon Hong, Xunyu Lin, and Honglak Lee. Decomposing motion and content for natural video sequence prediction. ICLR, 2017.
Carl Vondrick, Hamed Pirsiavash, and Antonio Torralba. Generating videos with scene dynamics. NeurIPS, 2016.
Angtian Wang, Haibin Huang, Jacob Zhiyuan Fang, Yiding Yang, and Chongyang Ma. Ati: Any trajectory instruction for controllable video generation. arXiv preprint arXiv:2505.22944 2025a.
Ruicheng Wang, Sicheng Xu, Yue Dong, Yu Deng, Jianfeng Xiang, Zelong Lv, Guangzhong Sun, Xin Tong, and Jiaolong Yang. Moge-2: Accurate monocular geometry with metric scale and sharp details. arXiv preprint arXiv:2507.02546 2025b.
Wenhao Wang and Yi Yang. Vidprom: A million-scale real prompt-gallery dataset for text-to-video diffusion models. Advances in Neural Information Processing Systems, 37:65618 --6618,,,,,,, 2024.
Yuqing Wang, Tianwei Xiong, Daquan Zhou, Zhijie Lin, Yang Zhao, Bingyi Kang, Jiashi Feng, and Xihui Liu. Loong: Generating minute-level long videos with autoregressive language models. arXiv preprint arX1::2410.02757, 2024a.
Zhouxia Wang, Ziyang Yuan, Xintao Wang, Yaowei Li, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. Motionctrl: unified and flexible motion controller for video generation. In SIGGRAPH, 2024b.
Dirk Weissenborn, Oscar Täckström, and Jakob Uszkoreit. Scaling autoregressive video models. In ICLR, 2020.
16

---

Published as a conference paper at ICLR 2026
Wenming Weng, Ruoyu Feng, Yanhui Wang, Qi Dai, Chunyu Wang, Dacheng Yin, Zhiyuan Zhao, Kai Qiu, Jianmin Bao, Yuhui Yuan, et al. Art-v: Auto-regressive text-to-video generation with diffusion models. In CVPR, 2024.
Jianzong Wu, Xiangtai Li, Yanhong Zeng, Jiangning Zhang, Qianyu Zhou, Yining Li, Yunhai Tong, and Kai Chen. Motionbooth: Motion-aware customized text-to-video generation. Advances in Neural Information Processing Systems, 37:34322-344 -4248, 2024a.
Rundi Wu, Ruiqi Gao, Ben Poole, Alex Trevithick, Changxi Zheng, Jonathan T Barron, and Alek- sander Holynski. Cat4d: Create anything in 4d with multi-view video diffusion models. In Pro- ceedings of the Computer Vision and Puttern Recognition Conference, pp. 26057-26068, 2025a.
Tong Wu, Shuai Yang, Ryan Po, Yinghao Xu, Ziwei Liu, Dahua Lin, and Gordon Wetzstein. Video world models with long-term spatial memory. arXiv preprint arX1i:2256.002222 2025b.
Weijia Wu, Zhuang Li, Yuchao Gu, Rui Zhao, Yefei He, David Junhao Zhang, Mike Zheng Shou, Yan Li, Tingting Gao, and Di Zhang. Draganything: Motion control for anything using entity representation. In European Conference on Computer Vision, pp. 331-348, Springer, 2024b.
Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming language models with attention sinks. arXiv preprint arXiv:2309. /7453, 2023.
Yuxi Xiao, Qianqian Wang, Shangzhan Zhang, Nan Xue, Sida Peng. Yujun Shen, and Xiaowei Zhou. Spatialtracker: Tracking any 2d pixels in 3d space. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 30406-20411 2024.
Desai Xie, Zhan Xu, Yicong Hong. Hao Tan, Difan Liu, Feng Liu, Arie Kaufman, and Yang Zhou. Progressive autoregressive video diffusion models. arXiv preprini arXiv:2410.08/51, 2024.
Jinbo Xing, Hanyuan Liu, Menghan Xia, Yong Zhang, Xintao Wang, Ying Shan, and Tien-Tsin Wong. Tooncrafter: Generative cartoon interpolation. ACM Transactions on Graphics (TOG), 43 (6):1-11, 2024.
Jinbo Xing, Long Mai, Cusuh Ham, Jiahui Huang. Aniruddha Mahapatra, Chi-Wing Fu, Tien-Tsin Wong, and Feng Liu. Motioncanvas: Cinematic shot design with controllable image-to- video generation. In Proceedings of the Special Interest Group Computer Graphics and Interactive Techniques Conference Conference Pupers, pp. 1-11, 2025.
Haofei Xu, Songyou Peng, Fangjinhua Wang, Hermann Blum, Daniel Barath, Andreas Geiger, and Mare Pollefeys. Depthsplat: Connecting gaussian splatting and depth. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 16453-16463, 2025.
Wilson Yan, Yunzhi Zhang, Pieter Abbeel, and Aravind Srinivas. Videogpt: Video generation using vq-vae and transformers. arXiv preprint arXiv:2104.10155 2021.
Yan Team. Yan: Foundational interactive video generation. arXiv preprint arXiv:2508.0 08601, 2025.
Mingyu Yang, Junyou Li, Zhongbin Fang, Sheng Chen, Yangbin Yu, Qiang Fu, Wei Yang, and Deheng Ye. Playable game generation. arXiv preprint 21x0v22412.00887, 2024a.
Shiyuan Yang. Liang Hou, Haibin Huang, Chongyang Ma, Pengfei Wan, Di Zhang, Xiaodong Chen, and Jing Liao. Direct-a-video: Customized video generation with user-directed camera movement and object motion. In ACM SIGGRAPII 2024 Conference Papers, pp. 1-12, 2024b.
Yuxue Yang, Lue Fan, Zuzeng Lin, Feng Wang, and Zhaoxiang Zhang. Layeranimate: Layer-level control for animation. arXiv preprint arXiv:2501.0820 2025a.
Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, al. Cogvideox: Text-to-video diffusion models with an expert transformer. In ICLR, 2025b.
Tianwei Yin, Michaël Gharbi, Taesung Park, Richard Zhang, Eli Shechtman, Fredo Durand, and Bill Freeman. Improved distribution matching distillation for fast image synthesis. Advances in neural information processing systems, 37:47455-47493 2024a.
17

---

Published as a conference paper at ICLR 2026
Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Fredo Durand, William T Freeman, and Taesung Park. One-step diffusion with distribution matching distillation. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 6613-6623, 2024b.
Tianwei Yin, Qiang Zhang, Richard Zhang, William T Freeman, Fredo Durand, Eli Shechtman, and Xun Huang. From slow bidirectional to fast autoregressive video diffusion models. In CVPR, 2025.
Jiwen Yu, Yiran Qin, Xintao Wang. Pengfei Wan, Zhang, and Xihui Liu. Gamefactory: Creating new games with generative interactive videos. arXiv preprint arXiv:2501.08825, 2025a.
Mark Yu, Wenbo Hu, Jinbo Xing, and Ying Shan. Trajectorycrafter: Redirecting camera trajec- tory for monocular videos via diffusion models. In Proceedings of the IEEE/CVF International Conference Computer Vision (ICCV), pp. 100-111, October 2025b.
Wangbo Yu, Jinbo Xing, Li Yuan, Wenbo Hu, Xiaoyu Li, Zhipeng Huang, Xiangjun Gao, Tien- Tsin Wong, Ying Shan, and Yonghong Tian. Viewerafter: Taming video diffusion models for high-fidelity novel view synthesis. arXiv preprint arXiv:2409. 02048, 2024.
Lymin Zhang and Maneesh Agrawala. Packing input frame context fext-frame prediction models for video generation. arXiv preprint arXiv:2504. 12626, 2025.
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control text-to-image diffusion models. In Proceedings of the IEEF/CVF international conference on computer vision, pp. 3836-3847, 2023.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 586-595, 2018
Tianyuan Zhang, Sai Bi, Yicong Hong, Kai Zhang, Fujun Luan, Songlin Yang, Kalyan Sunkavalli, William T Freeman, and Hao Tan. Test-time training done right. arXiv preprint arXiv:250S.23884 2025a.
Zhenghao Zhang, Junchao Liao, Menghao Li, Zuozhuo Dai, Bingxue Qiu, Siyu Zhu, Long Qin, and Weizhi Wang. Tora: Trajectory-oriented diffusion transformer for video generation. In CVPR, 2025b.
Xuanjia Zhao, Jian Guan, Congyi Fan, Dongli Xu, Youtian Lin, Haiwei Pan, and Pengming Feng. Fastdrag: Manipulate anything in one step. Advances Neural Information Processing Systems, 37.74439-74460, 2024.
Guangcong Zheng. Teng Li, Rui Jiang, Yehao Lu, Tao Wu, and Xi Li. Cami2v: Camera-controlled image-to-video diffusion model. arXiv preprint arXiv:2410. 15957, 2024.
Sixiao Zheng, Zimian Peng, Yanpeng Zhou, Yi Zhu, Hang Xu, Xiangru Huang, and Yanwei Fu. Vidcraft3: Camera, object, and lighting control for image-to-video generation. arXiv preprint arXiv:2502.0755/, 2025.
Haitao Zhou, Chuang Wang, Rui Nie, Jinlin Liu, Dongdong Yu, Qian Yu, and Changhu Wang. Trackgo: A flexible and efficient method for controllable video generation. AAAI, 2025a.
Jensen Jinghao Zhou, Hang Gao, Vikram Voleti, Aaryaman Vasishta, Chun-Han Yao, Mark Boss, Philip Torr, Christian Rupprecht, and Varun Jampani. Stable virtual camera: Generative view synthesis with diffusion models. arXiv preprint arXiv:2503.1 /4489, 2025b.
18

---

Published as a conference paper at ICLR 2026
A TRAINING EFFICIENT TINY VAE
Table Al: Comparison of Causal VAE Models. We evaluate reconstruction quality the Sora demo samples (resized to 8180832x480) by encoding videos with the Full VAE encoder and de- coding with different VAE variants. Our Tiny VAE achieves order-of-magnitude faster decoding than Full VAEs while outperforming existing community implementations in reconstruction quality.
Decoder Params
Compression Rate
Decoding Timers) (81x852x480)
Model
PSNR
SSIM
LPIPS
Full VAE (Wan 2.1)
73. 33.3M
8xc8x4
1.67
31.43
0,934
0.069
Tiny VAE (Wan 2.1
[Bor Boban 2025
9.84M
8x8x4
0.12
28.85
0.899
0.168
Tiny VAE (Wan 2.1. Ours)
9.84M
8x8x4
0.12
29.27
0.904
0,107
Full VAB (Wan 2.2, 5B)
555M
16x164
1.75
31.87
0.938
0.065
Tiny VAE (Wan 2.2. Ours)
56.7M
16x16x1
0.23
28.43
0.883
0.126
As discussed in the main paper, the full Causal VAE becomes bottleneck in streaming pipelines. Wan 2.1's CausalVAE performs 8x spatial and 4x temporal compression, while Wan 2.2 performs 16x spatial and 4x temporal compression. VAE decoding using the full CausalVAE takes 47% of chunk generation wall time for Wan 2.1 (1.3B), and 35% for Wan 2.2 (5B) in our base setup. Although Wan 2.2 (5B) VAE's higher compression rate enables Wan 2.2 to generate 720P videos in real time, it increases the decoding time and memory footprint.
Inspired by community implementations of Tiny VAE (Boer Bohan 2025 we train compact decoder from scratch with larger data, cleaner training pipelines, and better loss designs. While Boer Bohan 2025) trains Tiny VAE by regressing outputs to the original Wan VAE using adversarial loss from a PatchGAN discriminator and reconstruction loss with replay buffer, we extend this design by incorporating LPIPS loss with proper data scaling, and hyperparameter selections. We use a random subset of videos from Open Vid- IM and synthetic Wan videos (total 280K samples), training for 200K steps with a learning rate of 3 x 10 batch size of 16, and AdamW optimizer. We train at a lower resolution of 144 x 144 and frame length of 21, but the model scales well to larger video dimensions.
As shown in Table AT. Tiny VAE achieves substantially faster decoding with significantly fewer parameters compared to the Full VAE. While Tiny VAEs typically produce slightly lower recon- struction quality than Full VAEs, our implementation substantially outperforms existing community versions. Importantly, when used jointly with our distilled student for latent decoding, we observe minimal quality differences in practice (Table A2). as most quality degradation and drift originate from the diffusion model itself rather than VAE reconstruction. For consistency when evaluating the performance of distilled diffusion model, we use results from Full VAE and report corresponding speed, and adopt Tiny VAE for the streaming demo
Table A2: Evaluating Tiny VAE in Streaming Generation Setup. Using the same distilled stu- dent model, we ablate the impact of switching VAE from original Full VAE to Tiny VAE in Sora demo subset. It's important to note that even after changing to Tiny VAF, our distilled models still outperform all other baselines and quality differences compared to Full VAEs are marginal while achieving 1.75x and 2.3x higher throughput.
Model
Throughput (FPS)
Latency (s)
PSNR
SSIM
LPIPS
Full VAE (Wan 2.1)
16.7
0.69
16.67
0,531
0.360
Tiny VAE (Wan 2. Ours)
29.5
0.39
16.68
0.528
0.365
Full VAE (Wan 2.2, 5B)
10.4
1.14
16.62
0.545
0.343
Tiny VAE (Wan 2.2, Ours)
23.9
0.49
16.62
0.543
0.349
B VBENCH RESULTS AND USER STUDY
We additionally evaluate MotionStra using VBench-12V (uang et al. 2024) and conduct user studies on 20 samples from the Sora demo subset. In Vbench-12V, exclude camera motion
19

---

Published as a conference paper at ICLR 2026
Table A3: VBench-12V Results. We evaluate other baselines using VBench-12V on Sora subset. While the results primarily depend on the choice of backbone, our models generally achieve high performance across all dimensions.
Method
12v
12v hackground
subject consistency
background consistency
motion smoothness
aesthetic quality
imaging quality
subject
Image Conductor ci
2025e
0.847
0.868
0.791
0.889
0.906
0.505
0.689
GWTF BBrrert
0.957
0.974
0.933
0.944
0.981
0.620
0.675
DAS Guu a. 20246
0.972
0,987
0,953
0.958
0,988
0.634
0.695
Wang
Ours Teacher (1.3B)
0.984
0,988
0.948
0.943
0.987
0.625
0.698
Ours Distilled (1.38)
0.982
0.967
0,940
0.941
0.985
0.618
0.684
Ours Teacher (5R)
0.981
0.968
0.947
0.959
0.982
0.637
0.707
Ours Distilled
0.984
0.990
0.945
0.959
0.987
0.630
0.708
and dynamic degree metrics since these dimensions are already constrained by the input trajectory conditions rather than text prompts. VBench results show strong correlation with the underlying backbone model, favoring recent Wan-based architectures. The provision of image and trajectory conditions leads uniformly high scores across methods, reducing discriminative power between models. Nonetheless, both our teacher and distilled models consistently achieve competitive perfor- mance across all evaluated dimensions.
For user study, we collected 2,800 responses evaluating video quality of generated videos using our Wan 2.1 (I.3B) variants. Since accurately assessing track-following capability from thousands of grid point trajectories is challenging for participants, we focused solely video quality assessment, with results shown Figure[A]] As with VBench, video quality correlates strongly with backbone capacity. Notably, ATI (Wang al. (2225)), which leverages Wan 2.1 14B (10x larger than our 1.3B model), generally produces more visually favorable videos. However, despite ATT's aesthetic quality, observe it often lacks precise trajectory adherence. Both teacher and student models outperform other basclines in quality, with the teacher being slightly preferred over the student.
Teweber
Ours Teacher
52.13%
41.07%
GWTF
Ouns Teaeber
48.37%
51.63%
AT1
Distilled
Ours
$1.64%
48.30%
GWTF
Distiled
Ours
42.0%%
57.12%
ATI
Distilled
100
Preference Rate (%)
Figure A1: User Study Results. We evaluate video quality through pairwise comparisons 20 Sora samples. terms of pure video quality, our models outperform all baselines except ATI, which uses 10x larger backbone (Wan 2.1-14B), producing visually favorable videos.
C ADDITIONAL ABLATION EXPERIMENTS AND QUALITATIVE RESULTS
Impact Chunk Size and Sampling Steps. In the main paper, we primarily focused on evaluating the efficiency of diverse sparse attention patterns with attention sink size and attention window size. As shown Table 4 and Figure 6 reducing the chunk size increases the number of autoregressive rollouts required to generate the same length of video while reducing the amount of bidirectional attention, leading to worse performance. Since chunk sizes beyond 3 induce high latency, which makes them less useful for our interactive case, report only speed metrics for these configu-
20

---

Published as a conference paper at ICLR 2026
24
0.3750
Thoughput (PPS)
0.3725
0.3675
16
0.3625
12
200
0.3600
1000
Latenny
Sampling Steps
(a) Impact of Chunk Size and Sampling Steps in Throughput and Latency
(b) LPIPS Over Varying Sampling Stops
Figure A2: Speed and Quality Tradeoffs with Chunk Sizes and Sampling Steps. We visualize the latency-throughput relationship across varying chunk sizes and sampling steps (left), and image quality (LPIPS) across different sampling steps, using our default setup of c3s w1 (right).
Frame
Frame 80
Frame 161
C350W1
C3S0w6
C3S1W1
Figure A3: Long video extrapolation with and without attention sink. Models without attention sink (top two rows) exhibit cumulative drift over time, while our approach with attention sink (bot- tom) maintains stable quality throughout extrapolation.
rations. While we did not perform controlled training ablations for larger chunk sizes, we observe similar trends with Taching (Low & Wang 2025), where quality generally improves with larger chunks at the cost of latency. Figure[A2]c visualizes the latency and throughput given differ- chunk sizes and sampling steps. Note that since all benchmarks are conducted with sink size and a window size of I, the model with chunk size 7 has lower FPS than the one with chunk size because its attention sequence length is longer.
The number of sampling steps per chunk also affects this balance. We train our student model for 3-step generation, which we find to be optimal for our motion-controlled setup; increasing beyond 3 steps yields only marginal quality gains, while reducing to 2 steps causes a noticeable drop in quality, as shown in Figure A2 (b). Although trained for 3-step inference, the DMD framework allows for flexible sampling configurations at test time. Based on these findings, we use a chunk size of 3 with sampling steps as our primary configuration to achieve a strong balance between interactivity and high-quality video generation.
Visualization of Attention Sink's Impact on Long Video Extrapolation. We present long video extrapolation results comparing generation with and without attention sinks. As shown in Figure A3 incorporating at least one sink chunk proves crucial for preventing drift during extended generation. Without this, the model exhibits increasing degradation over time, while the attention sink enables stable quality maintenance throughout the video sequence. Please refer to the videos in the supple- mentary materials for additional results.
Impact of Motion Control on Generative Capability. To assess whether injecting motion con- trol compromises the base model's generative capability, we compare our 1.3B models against the
21

---

Published as a conference paper at ICLR 2026
Table A4: Impact of Motion Control on Generative Quality. We evaluate whether injecting motion control degrades the pretrained model's capability by comparing against larger, dedicated 12V baseline (Wan 2.1 14B 12V). We also report performance when motion conditions are dropped. Results indicate that adding motion conditioning does not significantly degrade the base model's generative quality. While removing motion conditions introduces a slight quality drop as our models were not optimized for this setting, the output still adheres to the given text and image inputs.
Model
12v
12v
subj. consis.
bg consis.
motion
aesth.
imag. qual.
FVD (P)
subject
bg
smooth.
qual.
Wan 12V (14B)
0.979
0.987
0.947
0.953
0.988
0.619
0.711
1274.6
Ours Teacher (1.3B) w. Motion
0,984
0.988
0.948
0.945
0.987
0.625
0.698
578.2
Ours Teacher (1.38) w/o Motion
0.965
0.976
0.889
0.926
0.990
0.613
0.707
1550.4
Ours Student (I.3B) Motion
0.982
0.987
0.940
0.941
0.985
0.618
0.684
745.8
Ours Student (1.3B) w/o Motion
0.973
0.982
0.923
0.943
0.986
0.597
0.688
1532.5
larger Wan 2.1 14B 12V baseline, which serves as a high-quality upper bound (as Wan 1.3B 12V exists) on the Sora dataset. As shown in Table (A4, we observe no noticeable degradation; despite the capacity gap and open-source training data, performance remains robust. In fact, few demo samples demonstrate that motion control enables the model to generate dynamic motions that naive 12V models often struggle produce due static bias (Choi et al. 2025). Consequently, FVD scores are lower with motion conditions, as the generated videos better mimic the target distribution through explicit motion constraints.
We also investigate the model's behavior when motion conditions are not provided. Since our models are incentivized to accurately follow motion and are not explicitly trained for an I2V (or "motion- less") setup, providing empty motion inputs indeed results in quality degradation. Nonetheless, while the quality is slightly lower compared to the fully conditioned setting, the model still fol- lows text prompts effectively without severe visual collapse. One interesting observation is that the teacher model without motion conditions rarely produces sudden scene changes, yet results in lower subject consistency metric. We hypothesize that applying CFG with empty motion condi- tions leads to unstable outputs where text prompts dominate. We did not observe this behavior with siidance-distilles student models. Please refer to the supplementary videos for detailed results.
Qualitative Comparison. We also provide a qualitative comparison between basclines in Fig- ure A4 Please refer to the supplementary videos for detailed results.
Streaming Demo. We show some examples of our streaming demo in Figure A5 The demo starts by accepting an input image and text prompts, which can help generate effects that not achievable through mouse drags. Users can then choose the specific spacing/size of the track grids and start controlling objects in scene, or move the camera. Due to its autoregressive nature, users can pause or resume the streaming generation process. Users can start/end or pause/resume the generation process with Enter and Space keyboard input, which is especially useful for dynamically adding static grids to specify unmoving regions or multiple moving grids to control different motions during streaming. As a video generative model, our method naturally supports drag-basee image editing, generating intermediate transition frames as a bonus, while being faster than most dedicated drag- based image editing methods (Pan et al. 2023 Shi et al. 2024b Nie et al.) 2023; Shin et al. [024] Zhao et al.) 2024).. To further support diverse downstream tasks, we will continuously update our front-end Ul with additional features.
D TRAINING DETAILS
Data Preprocessing. We train our models on two primary data sources: OpenVid-IM (Nan et al.) 2024) and synthetic videos generated by larger Wan text-to-video models. For OpenVid-INM we filter the dataset to 0.6M videos by requiring minimum of 81 frames and 16:9 aspect ratio, sam- pling at 16 FPS. For synthetic data, we use 70K samples for Wan 2.1 (81 frames, 480P resolution, generated by Wan 2.1 14B using text prompts from VidProm (Wang & Yang 2024 and 30K pub- liely available samples for Wan 2.2 (121 frames, 720P resolution, generated by Wan 2.2 5B from the Team)).
Fast Video team (The
22

---

Published as a conference paper at ICLR 2026
imagr Conincter
GWTT
DXS
ATI
Ours 130
Teacber
Oue 150 Cansall
Ours 30 Textter
Oues 5B
Figure A4: Qualitative Comparison. Generated videos from the Sora subset. As seen on the left, our model successfully reconstructs the flower blooming motion, while GWTF captures the motion but suffers quality degradation. ATI produces high-quality videos but with reduced trajectory adherence motion transfer scenarios.
We extract motion trajectories from all videos using CoTracker3 (Karaev et al. 2024), tracking points on a 50x50 uniform grid. Our training follows a two-stage design: initial training on Open Vid- 1M establish general motion conditioning capability, followed by fine-tuning on cleaner synthetic data to improve trajectory adherence and reduce artifacts from noisy real-world videos. For the distillation phase, and requires less data.
only synthetic samples since the distillation process converges quickly
For causal adaptation, we generate 4,000 training samples using our joint guidance strategy (w; 3.0, Wm = 1.5). Notably, Self Forcing-style distillation with DMD objectives does not require complete video sequences during training due the nature of distribution matching loss design, requiring only the first frame, text prompt, and corresponding motion tracks.
Teacher Model Training. We initialize our teacher models from partial weights of VideoXFun's Wan variants Apps & Alibaba PAI Team 2024), which extend Wan 12V models with ad- ditional control channels. This initialization accelerates convergence compared to training from scratch. Both Wan 2.1 and Wan 2.2 undergo two-stage training: (1) initial training on filtered Ope-Vid-IMe (0.6M videos) for 4.8K steps, followed by (2) fine-tuning on cleaner synthetic data for 800 steps (Wan 2.1) and 400 steps (Wan 2.2), approximately one epoch each. During training, we randomly sample 1,000~2,500 tracks and assign a sinusoidal positional embedding of = 64 dimensions. Stochastic track masking (described in Sec. 3.1) is applied during the fine-tune fine-tuning stage. We use batch size 128 with learning rates x 10 55 and x 10-6 -6 for stages and 2, respectively. The track head remains frozen after initial training it already operates chunk-wise.
23

---

Published as a conference paper at ICLR 2026
Figure A5: Streaming Demo. We show some examples of our streaming demo. Green grids indicate the points that are being dragged (online), red grids indicate set of points that should remain static, and blue grids indicate user's pre-drawn trajectories for moving multiple points simultaneously (see chameleon example). More examples can be found on our supplementary website.
We train the motion-guided teacher model through rectified flow matching objective, where the forward process linearly interpolates between data 2n and Gaussian noise 21 ~ I),
= (1 - 1)=0 + LZ1, LE [0, 1].
(A1)
with timestep shifting (Team Wan et
2025). defined as t' (k(k = 1+ 1)44 1000. where 6.0.
kt 10/0000
The model is trained to predict the expected velocity fields conditioned on noisy latents, denoising time steps, text prompts e and motion embedding Cm. with timestep-dependent weighting wt:
PrM Ez..s.. We [[vo,
- (21 - -20)13]
(A2)
Causal Architecture Adaptation. Following Caus Yin et al. 2025). we adapt to few-step causal inference through ODE trajectory regression. We generate 4,000 ODE trajectories from the teacher model and train for 2,000 steps. While the transition from bidirectional to causal attention re- quires significant adaptation, are similar enough to learn jointly with single model. We therefore train with diverse sparse causal attention masks, creating a unified initialization that supports flexible self-rollout configurations. We maintain a batch size of 128 with learning rate 2 x 10-6
find variations within causal patterns (e.g., different window sizes)
Self Forcing-Style Distillation with DMD. Self Foreing distillation converges quickly at around 400 steps with batch size of 64. We set learning rates to 2 x 10 for the generator and 4 x 10-7 for the critic (fake score function), with a 1:5 update ratio and gradient truncation as described in Sec. 3.2
We employ Adam optimizer (Kingma & Ba 2015 Loshchilov & Hutter 2019) with mixed preci- sion (bfloat16) and Py Torch's FSDP. Exponential moving average (EMA) applied during teacher training and DMD distillation. Wan 2.1 variants train at 832 x 480 resolution while Wan 2.2 trains 1280 x 704. With 32 1100 GPUs, training the Wan 2.1 teacher model takes roughly 3 days with
24

---

Published as a conference paper at ICLR 2026
causal adaptation and distillation completing in 20 hours, while Wan 2.2 requires slightly longer training times.
E EVALUATION PROTOCOLS
Since different methods employ various backbone models with different spatial and temporal resolu- tions, we optimize the evaluation setup for each method by matching their primary spatial/temporal dimensions. For DAVIS evaluation, when the number of frames exceeds model's default temporal length, we retain the first and last frames while uniformly subsampling intermediate frames, then compare against correspondingly subsampled ground truth frames.
All Sora demo videos were limited to 81 frames at 16 FPS for standard experiments. except for the extrapolation experiments in Table and Figure which use up 241 frames (average 194 frames, approximately 15 seconds). Similar subsampling was applied for models with shorter temporal contexts (16 frames for AnimateDiff, 49 for CogVideoX).
We report the best scores for each method using its optimal configuration. For Image Conductor, we tested with 1, 10, 100, 1000, 2500 tracks and report results from 100 tracks, which performed best. Go-With-The-Flow requires dense optical flow as input, so we provide flow estimated by Jiffusion-As-Shale
RAFT following their reference implementation. which we provide tracks from SpatialTracker (Xiao et al., 2024) on 70 x 70 grid (their default setting). ATI was tested with 40, 2500 tracks, and we report results from 40 tracks (their default), which performed better. Our models consistently use 2D trajectories from 50 x 50 initial grid points tracked by CoTracker3.
uses 3D tracks as imput, for
After generation, all results are resized to 832 480 resolution to ensure a consistent scale for
metrics, particularly EPE which calculates 1.2 distance between track coordinates. All latency and throughput are measured using a single H100 GPU in bíloat16 precision with Flash Attention 3 (Shah et al. 2024)..
For the camera control benchmark on the LLFF dataset, we derive 50 x 50 grid tracks using the method described in Sec. To minimize unintended object motion and focus the evaluation purely on camera movement, we use the prompt template: "static scene, only camera motion, no object is moving, {scene name))" )", where {scene name) } is replaced with the specific LLFF scene name.
F LIMITATION AND FUTURE WORK
While MotionStream achieves real-time motion control for long-range video generation, we identify several limitations. First, the fixed attention sink mechanism, while ensuring stable long-term gener- ation, constrains the model's 's ability to handle scenarios with complete scene changes. Our approach maintains strong anchoring to the initial chunk, which works well for most motion-controlled gen- eration scenarios where cameras and objects move within consistent environments. However, when presented with trajectories from game engines or other sources where environments change tinuously, the model exhibits a tendency to preserve the initial scene rather than adapting to new contexts. This limitation also inherent to current 2D tracking systems, which cannot meaningfully track and encode complete scene transitions. Future work could explore dynamic attention sinking strategies that adaptively refresh anchor frames for world modeling applications.
Second, we observe artifacts when motion trajectories are extremely rapid or physically implausible, manifesting as temporal inconsistencies or distortions in object uppearance. One good approach for future work would be exploring effective track augmentation strategies during training to better simulate imperfect user inputs and scaling larger backbone models, which generally exhibit more robust visual quality.
Lastly, our pipeline sometimes struggles to preserve source details when scenes, text prompts, or in- tended motions are highly complex. This primarily stems from backbone capacity limitations. While motion conditioning with text prompts can enforce movements beyond what the base model gener- ates from text alone, quality may be unsatisfactory in such cases. We also note that different image conditioning mechanisms across different backbones can affect the robustness of the model in han- dling imperfect motion cues. Interestingly, we empirically observed that the smaller Wan 2.1 (1.3B)
25

---

Published as a conference paper at ICLR 2026
Figure A6: Failure Cases. In the cat and turtle examples, the intention was to bring the cat out of the box and make the turtle hatch from the egg. However, due to limitations in hand-drawn trajectories for expressing complex motions and the backbone model's generalization capacity, the outputs are not physically plausible, and objects are either deformed or simply translated along the tracks. In complex scenes with multiple human identities (second example), the model often loses identity information and produces artifacts. The last row shows our model's output on the world exploration task. Since track representation struggles to capture complete scene transitions and the attention sink prioritizes preserving source features, our pipeline faces difficulty in scenarios where new objects appear or scenes continuously change. More videos can be found on supplementary website.
usually outperforms Wan 2.2 (5B) in preserving source structures, particularly with user-drawn flat- grid imperfect point trajectories. We attribute this Wan 2.1's input image cross-attentin design which helps maintain the original structure throughout, while Wan 2.2's TI2V structure is slightly more experimental. As our models are relatively modest in scale, we expect larger base models to provide improved performance and stability under challenging scenarios. Some of our failure cases can be found in Figure A6
G ETHICS STATEMENT
As video generative models become increasingly capable of producing realistie content that mimics world dynamics, we recognize the potential for misuse. While MotionStrean advances inter- active content creation with intuitive controls, our approach naturally inherits potential risks from the underlying generative technology, including the creation of deceptive media. We emphasize the critical need for parallel development of safeguarding techniques such as watermarking, content au- thentication, and controlled access mechanisms. We encourage prioritizing responsible deployment strategies alongside capability improvements to ensure these tools benefit society while minimizing harms.
Reproducibility. Our models are built upon publicly available Wan model variants and datasets (Open Vid- 1M and synthetic Wan videos). Training was conducted in a well-controlled environment, with all training details, hyperparameters, and implementation specifics provided in Sec. D and the supplementary to ensure reproducibility.
LLM Usage, We used LLMs to help polish the writing and presentation of this manuscript. LLMs were not used for research ideation, experimental design, or scientific discovery in this work.
26


Reviews:
[{"review_id": "p99SiPu7Xm", "reviewer": "Reviewer_EzdD", "summary": "This paper proposes a method named Motionstream, which helps interactive video  generation with flexible motion control and efficient inference speed. Specifically, Motionstream trains a teacher video generation model with both text and motion control and Motionstream distills the teacher with Diffusion strategy to the student with auto-regressive strategy. The student with auto-regressive strategy can generate the interactive video at constant high speed.  To further bridging the domain gap from training on finite length and extrapolate, and preventing error accumulations, MotionStream incorporates self-rollout with attention sinks and KV cache rolling during training. Finally, MotionStream is evaluated from respective of the motion following, video quality and inference efficiency.", "questions": "I find the overall approach of MotionStream to be very similar to that of Matrix-Game 2.0, which leads me to question the novelty of this work's contribution.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. Complete \"Teacher-Student\" Framework: The entire pipeline, from designing an efficient motion-controlled teacher model to distilling a causal student model, is rigorously and systematically designed.\n2. Lightweight Track Encoder: The use of sinusoidal positional encoding with a learnable track head, compared to the VAE-based RGB encoding method, achieves a 40× speedup in encoding while maintaining high track adherence accuracy, which is crucial for real-time systems.\n3. Attention Sinking with Rolling KV Cache: This is one of the core innovations of the paper. The authors adapt the \"attention sink\" concept from LLMs and successfully apply it to video diffusion models. By using the initial frame as a fixed \"anchor\" combined with a local rolling context window, it effectively solves the drift problem in long-video generation, ensuring long-term stability. The paper explicitly highlights that its method eliminates the train-test discrepancy by simulating inference-time extrapolation during training, which offers an advantage over contemporaneous work.", "weaknesses": "1. It is worth noting that utilizing distillation to enhance model inference efficiency is an established technique in model optimization, including the video generation domain. Several contemporaneous works, such as Hunyuan-Gamecraft and notably Matrix-Game 2.0, have adopted a highly similar paradigm—combining a distillation framework with an autoregressive student model and Self-Forcing training. While the integration presented here is non-trivial, the core methodological concept bears a strong resemblance to these existing approaches, which consequently lowers the perceived novelty of the contribution\n2. Furthermore, while the paper highlights motion control as a key contribution alongside the distillation framework, the synergistic relationship between the two remains unclear. They are presented as somewhat decoupled components. The authors should articulate a more cohesive narrative that explicitly links the design of the motion control mechanism to the requirements and success of the causal, distilled student model.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rbb5jWmK7b", "reviewer": "Reviewer_g9qk", "summary": "This paper introduces *MotionStream*, a real-time video generation framework with interactive motion control. The authors distill a bidirectional diffusion teacher into a causal autoregressive model, achieving streaming generation up to 24 FPS on a single GPU. The paper further proposes attention sinks and rolling KV caches to enable stable long-sequence generation without increasing latency. Experiments demonstrate competitive visual quality and strong speedups across several motion-controlled generation tasks.", "questions": "See  Weakness", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Addresses a timely and important problem: bringing real-time interaction to diffusion-based video models.  \n\n2. The proposed attention sink and KV cache mechanisms are intuitively sound and practically effective.  \n\n3. Extensive experiments and ablations support the framework's stability and efficiency.  \n\n4. Overall writing and presentation are polished and easy to follow.", "weaknesses": "1. The evaluation datasets appear limited to short human-action or camera-move clips; it's unclear how the model performs on scenes with large physical transformations (e.g., object deformation, rapid perspective change). \n\n2. Limited theoretical analysis; most findings are empirical.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NDdTQSNHlf", "reviewer": "Reviewer_5Qbe", "summary": "This paper presents MotionStream, a real-time, motion-controlled video generator that achieves sub-second latency and up to 24 FPS on a single GPU, enabling interactive dragging, motion transfer, and camera control with effectively unbounded video length. The model conditions on sparse 2D tracks using lightweight sinusoidal embeddings and a learnable track head, and employs joint classifier-free guidance to balance strict trajectory adherence with text-prompted natural dynamics. Long-horizon stability is maintained via sliding-window causal attention supplemented by attention sinks and rolling KV caches, which anchor early frames while keeping computation constant. The authors substantiate the approach with extensive comparisons, ablations, and a user study, demonstrating clear advantages for efficient motion-controlled video generation.", "questions": "1. In the guidance ablation, why do the text-only generations still exhibit a clear motion trajectory in the visualizations? Is this because the training data contain many similar side-to-side sway patterns, or is there another cause?\n2. In Table 1, models with more parameters perform worse than smaller ones. The manuscript does not explain this.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The manuscript’s motivation is clear and compelling, and the proposed solution appears technically sound and promising.\n2. The authors provide thorough comparative evaluations and ablation studies that isolate the contributions of each component.\n3. The presented demos are persuasive, showcasing the method’s practical impact.", "weaknesses": "1. The manuscript now reads like a composition of prior work—self-forcing, attention sinks, and motion conditioned video generation—assembled to achieve real-time, controllable video generation. However, specific contributions are not clearly presented.\n2. For motion conditioning, the method employs channel-wise concatenated embeddings and fine-tunes pretrained image-to-video models. While computationally efficient, this design may degrade the pretrained model’s general generative capabilities. The manuscript lacks analysis or experiments that quantify the trade-offs introduced by this specialization.\n3. The manuscript does not include comparisons of general image-to-video quality with the original pretrained backbone. Such an evaluation would provide a more complete picture of the method’s impact on baseline capabilities.\n4. The evaluation omits challenging corner cases—such as abrupt, large-amplitude motion changes or trajectories that poorly match the source subject. Moreover, because the approach combats long-horizon self-forcing with attention sinks and a sliding-window scheme tied to the teacher’s training horizon (e.g., position embeddings capped at ~81 frames), the model may be constrained in realizing large or sudden motion transitions. The authors should explicitly test and discuss these scenarios to clarify whether the architectural choices impose limits on motion magnitude or adaptability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Bq7xELC1Zj", "reviewer": "Reviewer_RuEW", "summary": "This paper presents MotionStream, a real-time motion-controlled video generation system that achieves sub-second latency at 24 FPS on a single GPU, representing a two-order-magnitude speedup over existing methods. The approach distills a bidirectional motion-conditioned text-to-video teacher model into a causal student model using Self Forcing with distribution matching loss, enabling streaming inference. To address the challenges of infinite-horizon generation, the system introduces sliding window causal attention combined with attention sinks and KV cache rolling during training, which allows it to generate arbitrarily long videos at constant speed while maintaining quality and preventing error accumulation. This enables truly interactive experiences where users can control videos through trajectory painting, camera movements, or motion transfer and see results unfold in real-time, achieving state-of-the-art results in both motion following and video quality.", "questions": "1. The authors acknowledge that fixed attention sinks limit the model's ability to handle complete scene changes, favoring initial scene preservation; they should provide concrete visual examples demonstrating this failure case to better illustrate when and how the method breaks down under drastic environmental transitions.\n2. In Figure 5, when conducting the guidance ablation study, the authors use different guidance scales for the same conditioning across experiments, such as w_m=1.5 in Hybrid versus w_m=5.0 in Motion Only; could the observed performance differences be primarily attributed to these inconsistent hyperparameter settings rather than the conditioning strategies themselves?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper achieves sub-second latency for motion-controlled video generation at 24 FPS on a single GPU, representing a two-order-magnitude speedup over existing methods through causal distillation of a bidirectional teacher model using Self Forcing with distribution matching loss.\n\n2. This paper enables infinite-length video generation at constant speed by introducing sliding window causal attention with attention sinks and KV cache rolling, which prevents quality degradation and error accumulation while maintaining fixed computational costs regardless of video length.\n\n3. This paper delivers truly interactive motion control experiences by allowing users to paint trajectories, control cameras, or transfer motion with real-time visual feedback, achieving state-of-the-art results in both motion following and video quality.", "weaknesses": "1. The technical contributions are somewhat limited. The proposed MotionStream seems to be a straightforward combination of existing full-attention controllable video generation with the self forcing framework.\n2. While the paper claims to enable long video generation, only a single long-form demonstration is provided; the authors should present more extensive examples to substantiate the robustness and consistency of their approach across diverse scenarios.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)