Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion
Authors: Xun Huang
Zhengqi Li
Guande He
Mingyuan Zhou
Eli Shechtman
Venue: NeurIPS 2025 spotlight ()
Keywords: Video generation
Autoregressive models
Diffusion models
World models
Primary area: deep_learning
TL;DR: We introduce a new method to train autoregressive video diffusion models by performing autoregressive self-rollout with KV caching during training.

Abstract: We introduce Self Forcing, a novel training paradigm for autoregressive video diffusion models. It addresses the longstanding issue of exposure bias, where models trained on ground-truth context must generate sequences conditioned on their own imperfect outputs during inference. Unlike prior methods that denoise future frames based on ground-truth context frames, Self Forcing conditions each frame's generation on previously self-generated outputs by performing autoregressive rollout with key-value (KV) caching during training. This strategy enables supervision through a holistic loss at the video level that directly evaluates the quality of the entire generated sequence, rather than relying solely on traditional frame-wise objectives. To ensure training efficiency, we employ a few-step diffusion model along with a stochastic gradient truncation strategy, effectively balancing computational cost and performance. We further introduce a rolling KV cache mechanism that enables efficient autoregressive video extrapolation. Extensive experiments demonstrate that our approach achieves real-time streaming video generation with sub-second latency on a single GPU, while matching or even surpassing the generation quality of significantly slower and non-causal diffusion models.

Self Forcing: Bridging the Train- Test Gap in Autoregressive Video Diffusion
Xun Huang'
Zhengqi Li
Guande He²
Mingyuan Zhou? Eli Shechtman'
Adobe Research
The University of Texas Austin
https://selfffoott github.thiii 1o/
Abstract
We introduce Self Forcing. a novel training paradigm for autoregressive video dif- fusion models. It addresses the longstanding issue of exposure bias, where models trained on ground-trutt context must generate sequences conditioned on their own imperfect outputs during inference. Unlike prior methods that denoise future frames based on ground-trum trrththhthhhhh context frames, Self Forcing conditions each frame's ' genera- tion on previously self-generated outputs by performing autoregressive rollout with key-value (KV) caching during training. This strategy enables supervision through holistic loss the video level that directly evaluates the quality of the entire generated sequence, rather than relying solely on traditional frame-wise objectives. To ensure training efficiency, we employ a few-step diffusion model along with a stochastic gradient truncation strategy, effectively balancing computational cost and performance. We further introduce rolling K'V cache mechanism that enables efficient autoregressive video extrapolation. Extensive experiments demonstrate that our approach achieves real-time streaming video generation with sub-seee second latency on single GPU, while matching or even surpassing the generation quality of significantly slower and non-causal diffusion models.
Introduction
Recent years have witnessed tremendous progress in video synthesis, with state-of-the-art systems now capable of generating remarkably realistic content with complex temporal dynamics [6]. However, these results are typically achieved with diffusion transformers (DiT) 622 g
that denoise all frames
simultancously using bidirectional attention. This design allows the future to affect the past and requires generating the entire video at once, fundamentally limiting their applicability real-time streaming applications where future information is unknown when generating the current frame.
In contrast, autoregressive (AR) models 17 27
104 generate videos sequentially, a paradigm that naturally aligns with the causal structure of temporal media. This approach not only signif- icantly reduces the viewing latency of generated videos but also unlocks numerous applications,
94
including real-time interactive content creation 99 46), game simulation [I]] 61 78 102), and
robotics learning [42, 96 [OT. However, AR models often struggle to match the visual fidelity achieved by state-of-the-ar video diffusion models due to their reliance on lossy vector quantization
techniques [79].
To combine the best of both worlds, two recent techniques have emerged to equip video diffusion models with AR generation capabilities: Teacher Forcing (TF) [16] 28 33 [06] and Diffusion Forcing (DF)
X 10 20 69 73 100]. Teacher Forcing, a well-established paradigm in sequence modeling, trains the model to predict the next token conditioned ground-truth tokens. When applied video diffusion, TF involves denoising each frame using clean, ground-truth context frames (Figure [ (a)), strategy commonly referred to next-frame prediction. In contrast, Diffusion Forcing trains the model on videos with noise levels independently sampled for each frame, denoising each frame based
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

p(e (e (a "Io(a" (o)) ( paat,a" (7)
(( ( ( of p(at 49,
p( to (a(" (a p(8', 29)
p(a*)
p(a*)
p( |a m
p(a*)
p(29
pl
Causal DiT
Causal DiT
Causal DiT
(a) Teacher Forcing Training
(b) Diffusion Forcing Training
(c) Self Forcing Training (ours)
Figure 1: Training paradigms for AR video diffusion models. (a) In Teacher Foreing, the model
trained to denoise each frame conditioned on the preceding clean, ground-truth context frames. (b) In Diffusion Forcing, the model is trained to denoise each frame conditioned on the preceding context frames with varying noise levels. Both (a) and (b) generate outputs that do not belong to the distribution the model generates during inference. (c) Our Self Forcing approach performs autoregressive self-rollout during training. denoising the next frame based on previous context frames generated by itself. A cetrriuution-mtcchiin loss (e.g., SiD, DMD, GAN) is computed on the final output video to align the distribution of generated videos with that real videos. Our training paradigm closely mirrors the inference process, thereby bridging the train-test distribution gap.
on noisy context frames (Figure (b)). This ensures the autoregressive inference scenario, where context frames are clean and the current frame is noisy, is covered by the training distribution.
accumulation during autoregressive generation, leading to degraded video quality over time [84] 100, [05). This issue is more broadly known exposure bias 660 711, where model is trained exclusively ground-truth context but must rely on its own imperfect predictions at inference time, resulting in a distributional mismatch that compounds errors generation progresses. While some approaches attempt to mitigate this issue in video diffusion models by incorporating noisy context frames during inference [ 11 [05],
However, models trained with TF or DF often suffer from
such design sacrifices temporal consistency, complicates the KV-cache design, increases generation latency, and does not fundamentally resolve the exposure bias problem.
In this work, propose Self Forcing (SF), novel algorithm addressing exposure bias in autoregres- sive video generation. Inspired by carly RNN-era sequence modeling techniques [40, 65 [03], our approach bridges the train-test distribution gap by explicitly unrolling autoregressive generation during training, generating each frame conditioned on previously self-generated frames rather than ground-
truth ones. This enables supervision with holistic distribution-mate matbhiinnn losses [18 98, 99] applied to complete generated video sequences. By forcing the model to encounter and learn from its own prediction errors, Self Forcing effectively mitigates exposure bias and reduces error accumulation.
While Self Forcing may seem computationally prohibitive due to its sequential nature preventing parallel training, we demonstrate that it can be efficiently implemented as algorithm in the post- training stage where the model does not require a large number of gradient updates to converge. By employing few-step diffusion backbone and a carefully designed gradient truncation strategy, Self Forcing is surprisingly more efficient than alternative parallel strategies, achieving superior performance within the same wall-clock training time. Additionally, we introduce rolling KV cache mechanism that enhances the efficiency of video extrapolation.
Extensive experiments demonstrate that our model enables real-time video generation at 17 FPS with sub-second latency on a single H100 GPU, while achieving competitive or superior generation quality compared to recent slow bidirectional and autoregressive video diffusion models. These advances open the door to genuinely interactive video generation use cases-live live streaming, gaming, and world Limulatio-whhe latency budgets measured in milliseconds rather than minutes.

---

2 Related Work
GANs for Video Generation. Early video generation approaches relied primarily generative adversarial networks (GANs) [18], either using convolutional networks to generate entire videos in employing recurrent architectures to produce frames sequentially [44 77. ST] Recently, GANs have also been applied to distill video diffusion models [47] [6
parallel [3 68 82
44 49
108].
Since the generator in GANs follows the same process during training and inference, inherently avoids exposure bias. Our work draws inspiration from this fundamental GAN principle by directly optimizing the alignment between the generator's output distribution and the target distribution.
Autoregressive/Diflus Models for Video Generation. Modern video generation models have largely shifted toward diffusion or autoregressive models due to their stronger scaling abilities. Video diffusion models typically adopt bidirectional attention mechanisms to simultaneously denoise all video frames 3 4 6. [3] 23 (26 (3 64 80 [33 197. Autoregressive models, in contrast, trained with next-tokent prediction objectives and generate spatiotemporal tokens sequentially at inference
time 38 66
86 88 (94.
Hybrid Models. Very recently, hybrid models integrating autoregressive and diffusion frameworks have emerged as a promising direction in generative modeling of videos g
16
20 [2
22
33
45
50
52
89
100 [06 107 as well as other sequence domains M 43 53
110. They typically rely on long, iterative prediction chain (both temporally autoregressive and spatially iterative denoising), which could lead to significant error accumulation. Our work addresses this issue by training the model conditioned on its own predictions and teaching it to correct its own mistakes.
20
trains video diffusion models with progressive noise schedule, where the noise level gradually increases from earlier to later frames. While these methods support sequential long video generation with less accumulated errors and are sometimes also referred to as autoregressive, they do not strictly follow the autoregressive chain rule decomposition. Consequently, they would exhibit significant latency in interactive applications, as future frames are partially pre-generated before the current frame is presented to the user. This premature commitment restricts the impact of real-time user-injected controls, resulting in limited responsiveness in immediately subsequent frames.
Rolling Diffusion and Variants. Another line of work B5
67
69
76
93
105
Caus Our work is most closely related to CiusVid [100), which trains few-step autoregressive diffusion models using the DF scheme and distribution matching distillation (DMD). However, Caus Vid suffers from a critical flaw that its training outputs (generated via DF) do not come from the distribution the model produces at inference time, therefore the DMD loss is matching the wrong distribution. We pinpoint this issue and propose solution that matches the true model distribution.
3
Self Forcing: Briding Train-Test Gap via Holistic pest-Training
We first provide a formal definition of autoregressive video diffusion models and describe standard training approaches in Section 3.1 In Section 3.2, we introduce the main part of our Self Forcing training algorithm and describe how it can be efficiently implemented with few-step diffusion model. In Section 3.3] we describe various choices of holistic, video-level Eistrbution-matthii training objectives. Finally, we introduce rolling key-value cache mechanism that enables efficient generation of arbitrarily long videos in Section 3.4
3.1 Preliminaries: Autoregressive Video Diffusion Models
Autoregressive video diffusion model is hybrid generative model that combines autoregressive chain-rule decomposition with denoising diffusion models for video generation. Specifically, given a
sequence of N video frames 1=: (x .². ..... , ,N\,, it factorizes the joint distribution into product of conditionals using the chain rule > = II ((=" Each conditional distribution p(=*|><<) is then modeled using a diffusion process, where a frame is generated by progressively denoising initial Gaussian noise conditioned on previously generated frames. This formulation combines the strengths of both autoregressive models and diffusion models for capturing sequential dependencies while enabling high-quality generation of continuous-valuee visual signal. In practice,
3

---

K
Low SNR
High SNR
No atteation
Spatial denoising
Clean KV
Noicy KV w/ varging SNR
Temporal AR rollout
KV caching
20
caching
(a) Teacher Forcing
(b) Diffusion Forcing
(c) Self Forcing/AR Inference
Figure 2: Attention mask configurations. Both Teacher Foreing (a) and Diffusion Forcing (b) train the model on the entire video parallel, enforcing causal dependencies with custom attention masks, In contrast, our Self-Forcing Training (c) mirrors the autoregressive (AR) inference process with KV caching and does not rely on special attention masks. For illustration purposes, we show a scenario where the video contains 3 frames, and each frame consists of 2 tokens.
we can also choose to generate one chunk of frames rather than single frame at a time 669 [00]. For simplicity of notation, however, we continue to denote each chunk as a frame in this section.
Most existing autoregressive video diffusion models trained using frame-wise denoising loss within the paradigm of Teacher Forcing (TF) or Diffusion Forcing (DF). Specifically, each frame x² corrupted by the forward process qe ja [= (zg) such that = (xx , t²) au oye', where ar, Tp are pre-defined noise schedule within finite time horizon t' E [o, 1000] and et -V,01 I) is Gaussian noise. In TF, the timesteps ( are usually shared across all frames, whereas in DF, they are sampled independently for each frame. generative model learned through the time-reversal of the forward process, where each denoising step can be achieved by predicting the noise e added to each frame with neural network é == Go (Z),,t tt, conditioned on the context c. The context consists of clean ground-truth frames x in TF or noisy context frames j<í in DF. The model is trained minimize the frame-wise mean squared error (MSE) between the predicted noise and the E ,','
true added noise: CDM
[w>+ 1| dl1 where Wy is weighting function.
We focus the transformer-based architecture [62) of diffusion models with text conditioning (omit- ted from equations for clarity) operating in compressed latent space encoded by a causal 3D variational autoencoder (VAE) 37 The autoregressive chain-rule decomposition implemented via causal attention. Figures 2 (a) and (b) illustrate the attention mask configurations of Teacher Foreing and Diffusion Forcing approaches. For Teacher Foreing, we describe an efficient variant that processes all frames in parallel using block sparse attention masks, rather than denoising one frame each training iteration
33 Such design has been used in MAR-based
4 autoregressive video
generation 111 and concurrently in other autoregressive video diffusion models [106]] 107
3.2 Autoregressive Diffusion Post- Bust-Training via Self-Rollout
The core idea of Self Forcing is generate videos through autoregressive self-rollout during training following the inference-ting time recipe. Specifically, we sample batch of videos {x } po (x NNN-
11 (=' where each frame x' generated by performing iterative denoising conditioned on self-generated outputs, including both clean context frames in the past and noisy frames at the current time step. Unlike most previous autoregressive models that only utilize KV caching during inference. our Self Forcing method innovatively employs KV caching during training, as shown in Figure (c).
Nevertheless, implementing Self Forcing with standard many diffusion models would be compu- tationally prohibitive, as it requires unrolling and backpropagation through long denoising chains. Therefore, we choose to use few-step diffusion model Go to approximate each conditional dis- tribution po (a* xx) in the autoregressive factorization. Consider (to = 0, 0, br 1000) subsequence of timesteps [O...., 1000), each denoising step t, and frame index i, the model denoises an intermediate noisy frame æi, conditioned previous clean frames <i It then injects Gaussian noise with lower noise level into the denoised frame through the forward process V to

---

Algorithm Self Foreing Training
Algorithm Autoregressive Diffusion Inference with Rolling KV Cache
Require: Denoise timesteps (f.y...,fr)
Require: Number of video frames N Require: AR diffusion model Ge (returns KV em- beddings via %
Require: KV cache of size L frames
Require: Denoise timesteps (41..... Cr)
Require: Number of generated frames M
loop
Require: AR diffusion model Gg (returns KV em-
2: Initialize model output Xg f-
beddings via G
3: Initialize KV cache V +
1: Initialize model output X Initialize KV cache KV <
4: Sample Uniform( 2,
for I..... do Initialize E,p ~ (0, for j T 8 do
for i L...... M do Set 20 Ga i i4, KV)
4: Initialize wi, (O.. 7)
7:
5: for j = T ....., do
8:
j = then Enable gradient computation Set 2b - Ga(44,14 KV)
9:
if j = then
8:
Xa. append (6)
11:
(f))
Cache kv <- GKV (#g: 0, KV)
12:
Disable gradient computation
10:
[KV = then
13:
Cache kv + G (Zhi "Z)
KV)
11:
KV (op((0)
Cache eviction
KV. avp.apeen(kkv)
12:
end
15:
else
13:
KV append(kv')
16:
Disable gradient computation
14:
else
17:
Set & Go(zl, ; i,,, KV)
15:
Sample ~ ((,
18:
Sample ~ N)
16:
Set -i t ( (, r,t-11)
19:
Set wi 4- e
17:
end if
20:
end if end for
18: end for
21:
19:
end for
22:
end for
20:
return X,
23:
Update via distribution matching loss
24:
end loop
obtain the noisy frame -i,_, as the input to the next denoising step, following the standard practice in few-step diffusion models 774 98) The model distribution p=<">>>>53 is implicitly defined as fo.t, o o...2.... o...ofo.tt, (xt, where So.,, (mt,) V(Go(r) Lj, x x*t), (y and oy ~ N( 7).
Even with few-step models, however, naively backpropagating through the entire autoregressive diffusion process would still lead to excessive memory consumption. To address this challenge, propose gradient truncation strategy that limits the backpropagation to only the final denoising step of each frame. Moreover, instead of always using T denoising steps (as in inference time), randomly sample a denoising step s from [1, T) for each sample sequence at each training iteration, and use the denoised output of the s-th step the final output. This stochastic sampling approach ensures all intermediate denoising steps receive supervision signals. We additionally detach the gradients of the previous frames from the current frame during training by restricting gradient flow into KV cache embeddings. For a complete description of the training process, see Algorithm []
3.3 Holistic Distribution Matching Loss
Autoregressive self-rollout generates samples directly from the inference-time model distribution, enabling apply holistic, video-level losses that align the distribution of generated videos po(x::N) with that of real videos (x ). To leverage pre-trained diffusion models and enhance training stability (32) we inject noise both distributions and match (z. and P (x!::N,, where each represents the respective distribution after applying the forward diffusion process; (..( (= I:N )p. IxN " Our framework is generally applicable to various divergence measures and distribution matching frameworks, and we consider three approaches this paper:
This approach minimizes the re- verse Kullback- divergence IE, Dxi, (po.s [aaan.)) by leveraging the score difference between distributions to guide gradient updates.
Distribution Matching Distillation (DMD) 98 99

---

Score Identity Distillation (SiD)
112 113 This method performs distribution matching via Fisher divergence Ex.p... V Po.t
Pa. V log Pdata.t
Generative Adversarial Networks (GANs) [8] It approximately minimizes the Jensen-
Shannon divergence through minimax game between the generator (our autoregressive diffusion model) and discriminator that distinguishes between real and generated videos.
Importantly, our training objective matches the holistic distribution of the entire video sequence to the data distribution D ( (x ([)p)(=):>)) )). In contrast, TF/DF can be understood as per- forming frame-wwis distribution matching: (==+)-rmm DKL (Pasu (x' St) [po [ < [ where DF
additionally samples context frames from a noise-corrupted data distribution {a<i) Pdaat- Our formulation fundamentally transforms the training dynamics- context frames {x*t] are sampled from the model's own distribution Pa rather than from the data distribution (clean or noisy). This alignment between training and inference distributions effectively addresses exposure bias and forces the model to learn from its own imperfections, thereby developing robustness to error accumulation.
While all three objectives have been used in the context of timestep distillation of diffusion models, our primary motivation differs fundamentally from distillation: we aim to enhance the quality of autoregressive video generation by addressing exposure bias via distribution matching, rather than merely accelerating sampling. This distinction makes other popular distillation methods (74] inap- plicable to our framework as they only focus on timestep reduction without directly aligning the generator output distribution. Although Caus [100) similarly employs DMD to match the distribu- tion of generated videos, the distribution it optimizes during training (using Diffusion Forcing outputs) deviates from the actual inference-time distribution, significantly undermining its effectiveness.
3.4 Long Video Generation with Rolling KV Cache
key advantage of autoregressive models over standard video diffusion models their extrapolative ability, in principle allowing the generation of infinitely long videos via windowwwwwwww//oo inference. While bidirectional attention models trained with Diffusion Forcing 110 7
can also generate videos
autoregressively, they do not support KV caching, requiring complete recomputation of attention matrices for each new frame. This leads to excessive computational complexity of O(T)2) (where T represents the number of denoising steps and L is thewindow size), as shown in Figure (a).
Models with causal attention, on the other hand, can leverage KV caching to improve efficiency. However, existing implementations [69 100) require recomputing KV cache for overlapping frames between consecutive sliding windows, as illustrated in Figure (b). This leads to O(L + TL) complexity when employing dense sliding windows. As result, prior implementations adopt larger strides with minimal overlap to reduce computational costs, which compromises temporal consistency since frame at the beginning of each window relies on significantly reduced historical context.
Inspired by research in large language models [92], we propose rolling KV cache mechanism for autoregressive diffusion models that allows infinitely long video generation without any need of recomputing the KV cache. As illustrated Figure 3 (c). we maintain fixed-size KV cache that stores the KV embeddings of tokens in the most recent I. frames. When generating a new frame, we first check if the KV cache full. If is, we remove the oldest KV cache entry before adding the new one. This approach enables endless frame generation with a time complexity of OTT,), while still maintaining a sufficient context length when generating each new frame. Algorithm 2 provides a detailed description of our autoregressive long video generation algorithm with rolling KV cache.
However, naive implementation of this mechanism leads severe flickering artifacts due to distribu- tion mismatch. Specifically, the first latent frame has different statistical properties than other frames: it only encodes the first image without performing temporal compression. The model, having always seen the first frame as the image latent during training, fails to generalize when the image latent is no longer visible in the rolling KV cache scenario. Our solution is straightforward but effective: during training, we restrict the attention window so the model cannot attend to the first chunk when denoising the final chunk, thereby simulating the conditions encountered during long video generation.
'With a specific weighting per noise level [36 75 denoising loss approximates the maximum likelihood objective, equivalent to minimizing the KL divergence between per- frame data and model distributions.

---

Noisy tokens
Clean tokems
KV cache (clean tokens)
Discarded tokens
Denoising sleps
Window size
Imput
Recompute KV cache
Exiet aldest KV cache
Bidirectional DiT
Causal DiT
Denoise xF
Causal DiT
Danoise xT
Ouggut
Window
Window -.d
Window ---
(a) Bidirectional sliding window
O(TL?)
Casual sliding window
O(L2+TIL
Casual sliding window with Rolling KY Cache (ours)
O (TL)
with KV recomputation
Figure 3: Efficiency comparisons for video extrapolation. When performing video extrapolation through sliding window inference, (a) bidirectional diffusion models trained with TR/D [T0] 73 do
support KV cache. (b) Prior causal diffusion models [69 [00] require re-computing KV when shifting the window. (c) Our method does not recompute KV and enables more efficient extrapolation.
Experiments
Implementation. We implement Self Forcing with Wan2.1-T2V-1.3B [83. Flow Matching 48
based model that generates 58 videos at 16 FPS with a resolution of 832 x 480. Following Caus Vid's initialization protocol [TOO], we first finetune the base model with causal attention masking on 16k ODE solution pairs sampled from the base model. For both ODE initialization and Self Forcing training, we sample text prompts from a filtered and LLM-extended version of VidProM [85]. We use 4-step diffusion and implement both frame-wist and chunk- autoregressive variants, with the latter generating chunk of 3 latent frames at time. We adopt the R3GAN 29 objective, which consists of relativistic pairing GAN loss 64 with RI + R2 regularization 58 We use the 14B base model to generate 70k videos as the dataset for training GANs 70 and fine-tuning many-step TF/DF AR diffusion baselines. Notably, DMD/SiD implementations of our algorithm remain data-free, capable converting a pre-trained video diffusion model into an autoregressive model without any video training data. Additional implementation details are provided in Appendix A
Evaluation metries. We adopt VBench 31 and user preference study to evaluate both visual quality and semantic alignment. We also rigorously evaluate the efficiency of our method for real-time applications. While some recent works claim "real-time" video generation abilities 24 109] based solely on throughput, we argue that true real-time performance requires both sufficient throughput (exceeding video playback rate) and lower latency than the perceptual threshold which could be application-depert dependent [41]. We therefore evaluate both throughput and first-frane frst-frame latency to provide a comprehensive assessment of real-time capabilities, with all speed tests conducted on a single NVIDIA H100 GPU.
Comparison with existing baselines. We com- pare our model with relevant open-source video generation models of similar scale. Our com- parisons include two diffusion models: Wan2. -
Ours
66.1%
33.9%
CausVid
Ours
62.7%
37.3%
Wan2.1
1.3B
(our initialization weights) and LTX- (known for efficiency). We also com- pare with several autoregressive models includ-
83
Video 24
Ours
57.9%
42. 1%
SkyReds-V2
ing Pyramid Flow 33 NOVA [33, SkyReels- V2 100, MAGI-1 [69]. and Vid [00) (also
Ours
54.2%
45.8%
MAGI-1
initialized from Wan-1.3B).
40
100
Preference Rate (%)
As shown in Table our chunk-wise autoregres- sive model achieves the highest VBench scores across all compared models while simultaneously delivering real-time throughput (17.0 FPS) with sub-second latency, low enough for certain real-time applications such as live video streaming 2 Figure Shouss//sows the user study results comparing our chunk-wise Self Forcing model against several important baselines. Our approach consistently preferred over alternatives, including the many- step diffusion model Wan2.1 that our model is initialized from. Our frame-wise variant maintains strong generation quality while providing the lowest. latency (0.45s), making it particularly suitable for latency-sensinn real-time applications. Results here are obtained using the DMD loss objective;
Figure 4: User preference study. Self Forcing outperforms all baselines in human preference.

---

Table 1: Comparison with relevant baselines. We compare Self Foreing with representative open-source video generation models of similar parameter sizes and resolutions.
Evaluation scores 1
Model
#Params Resolution
Throughput (FPS)
Latency (s) L
Total
Quality Score
Semantic Score
Score
Diffusioo models
XTX-Videv [4]
1.9B
768x512 832x4400
8.98 0.78
13.5
80.00
82,30
70.79
Wan2.1 83
1.3B
103
84.26
85.30
80.09
Chunk-wise antoregressivee models
SkyReels- V2 [0]
1.3B
960x540
0.49
112
82.67
84.70
74.53
MAGI-I 69
4.5B
x82
282
79.18
82.04
67.74
Caus 100
1.3B
832 480
17.0
0.69
81.20
84.05
69.80
Self Forcing (Ours, chunk-wise)
1.3B
832x/100
17.0
0.69
84.31
85.07
81.28
Autoregressive models'
NOVA 13
0.6B
768x480
0.88
4.1
80.12
80,39
79.05
Pyramid Flow
33
2B
640x384
6.7
2.5
81.72
84.74
69.62
Self Forcing (Ours, frame-wise wise)
1.3B
x32
8.9
0,45
84.26
85,25
80.30
We compare with the official implementation of CausVic that uses the same hase model (Wan-1.3B).
The distinction of AR/non-AR applies to the temporal dimension.
0s
2.55
5c
0s
2-555
55
Wan2.1
SkyReels-V2
CausVid
Ours
W
SkyReels-V2
CausVid
Ours
Qualitative comparisons. We visualize videos generated Self Forcing (Ours) against those by Wan2.1 833. SkyReels-Vi V2 [I0], and Caus [100] at three time steps. All models share the same architecture with 1.3B parameters.
Figure 5:
models trained with SiD and GAN objectives achieve comparable performance as detailed in our ablation studies. As shown Figure Caus Vid suffers from the error accumulation problem that causes the saturation to increase over time. Our approach obtains slightly better visual quality than Wan2. ReSky V2, while heing around 150x faster in latency. More example videos are provided in the project website (https: :/set////efff forcing. tttb.iio///

---

Table 2: Ablation study. We conduct controlled ablation studies comparing different training paradigms and distribution matching objectives under our training setup across chunk-wise (left) and frame-wise (right) AR models. Self Forcing works well with all different distribution matching objectives and consistently outperforms alternative training approaches.
Evaluation scores 1
Evaluation scores
Chunk-wise AR
Frame-wise AR
Total Quality Semantic Score Score
Total Quality Semantic Score Score
Score
Score
Many (50x2)-ste models
Many (50x 2) 2-step models
Diffusion Forcing (DF)
82.95
83.66
80.09
Diffusion Forcing (DF)
77.24
79.72
67.33
Teacher Foreing (TF)
83.58
84.34
80.52
Teacher Forcing (TF)
80.34
81.34
76.34
Few (t)- models
Few (4)- models
DF + DMD
82.76
83.49
79.85
DF DMD
80,56
81.02
78.71
TF + DMD
82.32
82.73
80.67
TF + DMD
78.12
79.62
72.11
Sclf Forcing (Ours, DMD)
84.31
85.07
81.28
Self Forcing (Ours, DMD)
84.26
85.25
80.30
Self Forcing (Ours, SiD)
84.07
85.52
78.24
Self Foreing (Ours, SiD)
83.54
84.71
78.86
Self Forcing (Ours, GAN)
83.88
85.06
79.16
Self Forcing (Ours, GAN)
83.27
84.57
78.08
Ablation Studies. We perform controlled comparisons of Self Forcing with alternative autoregressive diffusion training approaches. We evaluate: (1) AR Diffusion models trained with denoising diffusion loss using either Teacher Forcing Diffusion Forcing, and (2) few-step AR Diffusion models trained with TF/DF inputs but optimized with distribution matching objectives. The latter configuration with DF and DMD essentially replicates Caus [100] within our implementation framework, allowing direct comparison under identical training conditions.
Table demonstrates that Self Forcing performs robustly across various distribution matching objectives (DMD, SiD, and GAN), consistently outperforming all baselines. While baseline methods exhibit notable quality degradation when shifting from chunk-wise to frame-wise AR due to error accumulation associated with increased AR unrolling steps, usually manifesting as progressive over- saturation over-sharpening (similar Caus Vid in Appendix B Fig. 5). Self Forcing maintains consistent performance across both setups, highlighting its effectiveness at addressing exposure bias.
Rolling KV cache. We observe that recomputing KV cache when shifting sliding window (Fig. (b)) results in significantly reduced throughput (only 4.6 FPS) when generating 10-second videos. While naive rolling KV cache maintains high throughput, it introduces severe visual artifacts, illustrated the examples in Appendix B By training the model to generate frames without seeing the initial image latent, effectively mitigate these artifacts while maintaining high throughput (16.1 FPS).
Training efficiency. One might expect Self Forcing training to be computationally prohibitive given its sequential nature that contradicts the parallelizable paradigm of transformers. Surprisingly. our experiments reveal that Self Foreing actually outperforms alternative strategies in training efficiency. As shown in Fig. ( (left), Self Forcing achieves comparable per-iteration training time to Teacher Forcing and Diffusion Foreing. Furthermore, Fig. [ (right) demonstrates that Self Foreing achieves superior quality given same wall-clock training budgets compared to both alternative approaches. Each Self Forcing experiment with DMD converges in approximately 1.5 hours on 64 H100 GPUs.
This counter-intuitive result stems from two key factors: First, while Self Forcing performs sequential rollout, still processes all tokens within each individual frame/chunk in parallel, maintaining high GPU utilization during training. Second, TF and DF require specialized attention masking patterns to enforce causal dependencies, introducing additional computational overhead even with specialized implementations like AtexAntnooion [15]. On the other hand, Self Forcing always uses full attention during training and can leverage highly optimized attention kernels such as Itashtttentoon-3 (72).
5 Discussion
In this section, we examine the broader implications of our results, discuss additional perspectives, and outline potential directions for future research.

---

Per-Iteration Training Time (DMD loss)
Quality vs. Training Time
0.840
17.5
Diffusion Forolng
Tescher Foreing
15.0
Saff Forcing (1 (1-step)
VBench Totel Score
12.5
Forcing (2-step)
0.850
Forcing, (S-step)
Time
10.0
Salf Forcing (4-stop)
0.825
0.820
5.0
Diftision Forcing
Teacher Forcing
25
0.810
Forving
Generator Update
Critic Update
Training Wall Clock Time (minutes)
Figure 6: Training efficiency comparison. Left: Per-iteration time across different chunk- wise, ste- autoregressive video diffusion training algorithms (using DMD as the distribution matching objective). Right: Video quality (VBench score) vs. wall clock training time.
Fundamental limitation of the parallelizable training paradigm. Parallelizable training has been pivotal to transformers' success by enabling efficient scaling. However, this parallelism introduces fundamental limitations. Prior research [57] demonstrates that parallel architectures inherently limit expressiveness in sequential Sate-tracking problems. Our work highlights another critical limitation: parallelizable training paradigms creates misalignment between training and inference distributions, leading the accumulation of errors time. We advocate new paradigm of parallel pre-training and sequential gaining momentum in language modeling through reinforcement learning (1], our work represents the first step towards this direction for the video domain. We believe our framework is general and can be applied to other sequence domains, especially where the data is continuous.
that combines the best of both worlds. While this paradigm shift is
Interplay between AR, Diffusion, and GANs. Autoregressive models, diffusion models, and GANs have traditionally been viewed distinct paradigms in generative modeling. Our work highlights their complementary nature and demonstrates how they can be effectively integrated. Specifically, autore- gressive and diffusion models provide complementary ways factorize distributions (chain-rui thhle-rrul vs. variallvaiaable,, which can be composed in nested manner. The core idea behind GANs-matche the distribution of implicit generator to the target distribution by drawing samples from the implicit generator-can be employed to train a generator powered by autoregressive- diffussion factorization.
Limitation and future directions. While our method effectively mitigates error accumulation within training context length, quality degradation remains observable when generating videos substantially longer than those seen during training. Additionally, our gradient truncation strate- gics-while necessary for memory efficiency- may limit the model's 's ability to learn long-range dependencies. Future work could explore both improved extrapolation techniques and inherently recurrent architectures like state-space models 19 63 that better balance memory efficiency with long-context modeling.
Acknowledgments
We thank Tianwei Yin, Beidi Chen, Kaiwen Zheng, Kai Zhang, Gaurav Parmar, Yi Gu, Sai Bi, and Jianming Zhang for valuable discussions. G. He and M. Zhou acknowledge the support of NSF-IIS 2212418 and NIH-R37 CA271186.
References
[1] Marianne Arriola, Aaron Gokaslan, Justin T Chiu, Zhihan Yang. Zhixuan Qi. Jiaqi Han. Subham Sekhar Sahoo, and Volodymyr Kuleshov. Block diffusion: Interpolating between autoregressive and diffusion language models. In ICLR, 2025.
[2] Abdelhak Bentaleb, May Lim, Mehmet N Akeay, Ali C Begen, Sarra Hammoudi, and Roger Zimmermann. Toward one-second latency: Evolution of live media streaming. IEEE Communications Surveys & Tatorials, 2025.
10

---

[3] Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch. Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Sealing latent video diffusion models to large datasets. arXiv preprint arXiv2211.11122: 2023.
[4] Andreas Blattmann, Rohin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent diffusion models. In CVPR, 2023.
[5] Tim Brooks, Janne Hellsten, Miika Aittala, Ting-Chun Wang. Timo Aila, Jaakko Lehtinen, Ming Yu Liu, Alexei Efros, and Tero Karras. Generating long videos of dynamic scenes. NeurIPS, 2022.
[6] Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh. Video generation models world simulators, 2024.
[7] Jake Bruce, Michael D Dennis, Ashley Edwards, Jack Parker-Holder, Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richic Steigerwald, Chris Apps, al. Genie: Generative interactive environments. ICML, 2024.
[8] Boyuan Chen, Diego Martí Monsó, Yilun Du, Max Simchowitz, Russ Tedrake, and Vincent Sitzmann. Diffusion forcing: Next-token prediction meets full-sequence diffusion. NeurlPS, 2024.
[9] Feng Chen, Zhen Yang, Bohan Zhuang, and Qi Wu. Streaming video diffusion: Online video editing with diffusion models. arXiv preprint arXiv:2405. 19726, 2024.
[10] Guibin Chen, Dixuan Lin, Jiangping Yang. Chunze Lin, Juncheng Zhu, Mingyuan Fan, Hao Zhang, Sheng Chen, Zheng Chen, Chengehen Ma, et al. Skyreels-v2: Infinite-enngth film generative model. arXiv preprint arXiv:2504. 1002200444..4..... 2025.
[11] Julian Decart, Quinn Quevedo. Spruce McIntyre, Xinlei Campbell, Robert Chen, and Wachen. Oasis: A universe in transformer, 2024.
[12] Chaorui Deng, Deyao 7hu, Kunchang Li, Shi Guang, and Haoqi Fan. Causal diffusion transformers for generative modeling. arXiv preprint 2tX:x::4/2.//000 2024.
[13) Haoge Deng, Ting Pan. Haiwen Diao, Zhengxiong Luo, Yufeng Cui, Huchuan Lu, Shiguang Shan, Yonggang Qi, and Xinlong Wang. Autoregressive video generation without vector quantization. In ICLR, 2025.
[14] Emily L Denton et al. Unsupervised learning of disentangled representations from video. In NeurlPS, 2017.
[15] Jucchu Dong, Boyuan Feng, Driss Guessous, Yanho Liang, and Horace He. Flex attention: A program- ming model for generating optimized attention kemels. ArXiv, abs/2412.05496, 2024.
[16] Kaifeng Gao, Jiaxin Shi, Hanwang Zhang, Chunping Wang, Jun Xiao, Long Chen. Ca2-vdm: Efficient autoregressive video diffusion model with causal generation and cache sharing. arXiv preprint arXiv:2411.16375. 2024.
[17] Songwei Ge, Thomas Hayes, Harry Yang, Xi Yin, Guan Pang, David Jacobs, Jia-Bin Huang, and Devi Parikh, Long video generation with time-agnostic vggan and time-sensitive transformer. In ECCV, 2022.
[18] Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In NeurlPS, 2014.
[19] Alhert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. In COLM, 2024.
[20] Yuchao Gu, Weijia Mao, and Mike Zheng Shou. Long-contex autoregressive video modeling with next-frame prediction. arXiv preprint arXiv:2503. /9325, 2025.
[21] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song. Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang. Xiao Bi, al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement lcarning, arXiv preprint arXiv:2501. 12948, 2025.
[22] Yuwei Guo, Ceyuan Yang, Ziyan Yang. Zhibei Ma, Zhijie Lin, Zhenheng Yang, Dahua Lin, and Lu Jiang. Long context tuning for video generation. arXiv preprini arXiv:2503. 10589, 2025.
[23] Agrim Gupta, Lijun Yu, Kihyuk Sohn, Xiuye Gu, Meera Hahn, Fei-Fei Li, Irfan Essa, Lu Jiang, and José Lezama. Photorealistic video generation with diffusion models. In ECCV, 2024.
11

---

[24] Yoav HaCohen, Nisan Chiprut, Benny Brazowski, Daniel Shalem, Dudu Moshe, Eitan Richardson, Eran Levin, Guy Shiran, Nir Zabari, Ori Gordon, et al. Lix-video: Realtime video latent diffusion. arXiv preprint arXiv:2501.0011 2024.
[25] Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey A. Gritsenko, Diederik P. Kingma, Ben Poole, Mohammad Norouzi, David Fleet, and Tim Salimans. Imagen video; High definition video generation with diffusion models. ArXiv, abs/2210.02303, 2022.
[26] Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. In NeurlPS, 2022.
[27] Wenyi Hong. Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. Cogvideo: Large-scale pretraining for text-to-video generation via transformers. In ICLR, 2023.
[28] Jinyi Hu, Shengding Hu, Yuxuan Song, Yufei Huang, Mingxuan Wang. Hao Zhou, Zhiyuan Liu, Wei- Ying Ma, and Maosong Sun. Acdit: Interpolating autoregressive conditional modeling and diffusion transformer. arxiv preprint arXiv:2412.0772 2024.
[29] Nick Huang, Aaron Gokaslan, Volodymyr Kuleshov, and James Tompkin. The gan is dead; long live the gan! a modern gan baseline. In NeurIPS, 2024.
[30] Zemin Huang, Zhengyang Geng, Weijian Luo, and Guo-jun Qi. Flow generator matching. arXiv preprint 2rXiv:2410.1953 2024.
[31] Ziqi Huang, Yinan He. Jiashuo Yu. Fan Zhang. Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang. Dahua Lin, Yu Qiao, and Ziwei Liu. VBench: Comprehensive benchmark suite for video generative models. In CVPR, 2024.
[32] Simon Jenni and Paolo Favaro. On stabilizing generative adversarial training with noise. In CVPR, 2019.
[33] Yang Jin, Zhicheng Sun, Ningyuan Li, Kun Xu, Hao Jiang, Nan Zhuang, Quzhe Huang, Yang Song, Yadong Mu, and Zhouchen Lin. Pyramidal flow matching for efficient video generative modeling, In ICLR, 2025.
[34] Alexia Jolicoeur-Marticaau. The relativistic discriminator: a key element missing from standard gan. In ICLR, 2019.
[35] Jihwan Kim, Junoh Kang, Jinyoung Choi, and Bohyung Han. Fifo-diffusion: Generating infinite vidcos from text without training. In NeurlPS, 2024.
[36] Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. In NeurlPS, 2021.
[37] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. In ICLR, 2014.
[38] Dan Kondratyuk, Lijun Yu, Xiuye Gu, Jose Lezama, Jonathan Huang, Grant Schindler, Rachel Hornung. Vighnesh Birodkar, Jimmy Yan, Ming-Chang Chiu, et al. Videopoet: large language model for zero-shot video generation. In /CML, 2024.
[39] Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvidco: A systematic framework for large vidco generative models. arXiv preprint
2024.
[40] Alex M Lamb, Anirudh Goyal ALIAS PARTH GOYAL, Ying Zhang, Saizheng Zhang, Aaron C Courville, and Yoshua Bengio. Professor forcing: A new algorithm for training recurrent networks. In NeurIPS. 2016.
[41] Qing Li, Xun Tang, Junkun Peng, Yuanzheng Tan, and Yong Jiang. Lateney reducing in real-time internet video transport: A survey, SSRN 4654242, 2023.
[42] Shuang Li, Yihuai Gao, Dorsa Sadigh, and Shuran Song. Unified video action model. arXiv preprint arXiv:2503.00200 2025.
[43] Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. Autoregressive image generation without vector quantization. In NeurIPS, 2024.
[44] Zhengqi Li. Qianqian Wang. Noah Snavely, and Angjoo Kanazawa, Infinitenature-zero:: Learning perpetual view generation of natural scenes from single images. In ECCV, 2022.
12

---

[45] Zongyi Li, Shujie Hu, Shujie Liu, Long Zhou, Jeongsoo Choi, Lingwei Meng. Xun Guo, Jinyu Li. Hefei Ling, and Furu Wei. Arlon: Boosting diffusion transformers with autoregressive models for long video generation. ICLR, 2025.
[46] Feng Liang, Akio Kodaira, Chenfeng Xu, Masayoshi Tomizuka, Kurt Keutzer, and Diana Marculescu. Looking backward: Streaming video-to-video translation with feature banks. In ICLR, 2025.
[47] Shanchuan Lin, Xin Xia, Yuxi Ren, Ceyuan Yang. Xuefeng Xiao, and Lu Jiang. Diffusion adversarial post-training for one-step video generation. arXiv preprint arXiv:2501.08311 2025.
[48] Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. In ICLR, 2023.
[49] Andrew Liu, Richard Tucker, Varun Jampani, Ameesh Makadia, Noah Snavely, and Angjoo Kanazawa. Infinite nature: Perpetual view generation of natural seenes from single image. In ICCV, 2021.
[50] Haozhe Liu, Shikun Liu, Zijian Zhou, Mengmeng Xu, Yanping Xie, Xiao Han, Juan C Pérez, Ding Liu, Kumara Kahatapitiya, Menglin Jia, et al. Mardini: Masked autoregressive diffusion for video generation at scale. arXiv preprint arXiv:2410.20280, 2024.
[51] Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In ICLR, 2023.
[52] Yaofang Liu, Yumeng Ren, Xiaodong Cun, Aitor Artola, Yang Liu, Tieyong Zeng. Raymond H Chan, and Jean-michel Morel. Redefining temporal modeling in video diffusion: The vectorized timestep approach. arXiv preprint
2024.
[53] Zhijun Liu, Shuai Wang, Sho Inoue, Qibing Bai, and Haizhou Li. Autoregressive diffusion transformer for text-to-speech syathesis. arXiv preprint arXiv:2406.05551, 2024.
[54] Weijian Luo, Tianyang Hu, Shifeng Zhang. Jiacheng Sun, Zhenguo Li, and Zhihua Zhang. Diff-instruct: A universal approach for transferring knowledge from pre-trained diffusion models. In NewrIPS, 2023.
[55] Weijian Luo, Zemin Huang, Zhengyang Geng, J Zico Kolter, and Guo-jun Qi. One-step diffussin distillation through score implicit matching. NeurlPS, 2024.
[56] Xiaofeng Mao, Zhengkai Jiang, Fu- Yun Wang, Jiangning Zhang, Hao Chen, Mingmin Chi, Yabiao Wang. and Wenhan Luo. Osv: One step is enough for high-quality image to video generation. In CVPR, 2025.
[57] William Merrill and Ashish Sabharwal. The parallelism tradeoff: Limitations of llog-precision transformers. TACL, 2023.
[58] Lars Mescheder, Andreas Geiger, and Sebastian Nowozin. Which training methods for gans do actually converge? In ICML, 2018
[59] Sicheng Mo, Thao Nguyen, Xun Huang, Siddharth Srinivasan Iyer, Yijun Li, Yuchen Liu, Abhishck Tandon, Bli Shechtman, Krishna Kumar Singh, Yong Jac Lee, et al. X-fusion: Introducing new modality frozen large language models. arXiv preprint arXiv:2504.2089 20996, 2025.
[60] Mang Ning. Mingxiao Li, Jianlin Su, Albert Ali Salah, and Itir Onal Ertugrul. Elucidating the exposure bias in diffusion models. In ICLR, 2024.
[61] Jack Parker-Holder, Philip Ball, Jake Bruce Vibhavari Dasagi, Kristian Holsheimer, Christos Kaplanis, Alexandre Moufarek, Guy Scully, Jeremy Shar, Jimmy Shi, Stephen Spencer, Jessica Yung, Michael Dennis, Sultan Kenjeyev, Shanghang Long, Vlad Mnih, Harris Chan, Maxime Gazeau, Bonnie Li, Fabio Pardo, Luyu Wang. Lei Zhang, Frederic Besse, Tim Harley, Anna Mitenkova, Jane Wang, Jeff Clune, Demis Hassabis, Raia Hadsell, Adrian Bolton, Satinder Singh, and Tim Rocktäschel. Genie large-scale foundation world model, 2024.
[62] William S Peebles and Saining Xie. Scalable diffusion models with transformers. In ICCV, 2023.
[63] Ryan Po, Yotam Nitzan, Richard Zhang, Berlin Chen, Tri Dao, Eli Shechtman, Gordon Wetzstein, and Xun Huang. Long-context state-space video world models. arXiv preprint 2rXiv:2505.2017), 2025. 2025.
[64] Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apporv Vyas, Bowen Shi, Chih-Yan Ma, Ching-Yan Yao Chuang, et al. Movie gen: A cast of media foundation models. arXiv preprint arXiv:2410.13720, 2024.
[65] Mare Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba. Sequence level training with recurrent neural networks. In ICLR, 2016.
13

---

[66] Shuhuai Ren, Shuming Ma, Xu Sun, and Furu Wei. Next block prediction: Video generation via semt-auutoregrressvve modeling. arXiv preprint arXiv:2502.07337, 2025.
[67] David Ruhe, Jonathan Heek, Tim Salimans, and Emiel Hoogeboom. Rolling diffusion models. In ICML, 2024.
[68] Masaki Saito, Eiichi Matsumoto, and Shumta Saito. Temporal generative adversarial nets with singular value clipping. In /CCV, 2017.
[69] Sand-AL Magi-I: I: Autoregressive video generation at scale, 2025.
[70] Axel Sauer, Frederic Bocsel, Tim Dockhorn, Andreas Blattmann, Patrick Esser, and Robin Rombach. Fast high-resolution image synthesis with latent adversarial diffusion distillation. In SIGGRAPH Asia 2024 Conference Pupers, pages 1-11, 2024.
[71] Florian Schmidt. Generalization in generation: A closer look at exposure bias. EMNLP-IJCNIP 2019,
page 157, 2019.
[72] Jay Shah, Ganesh Bikshandi. Ying Zhang, Vijay Thakkar, Pradeep Ramani, and Tri Dao. Flashattention-:: Fast and accurate attention with asynchrony and low-precision. NeurIPS. 2024.
[73] Kiwhan Song, Boyuan Chen, Max Simchowitz, Yilun Du, Russ Tedrake, and Vincent Sitzmann. History- guided video diffusion. arXiv preprint arXiv:2502.06760 2025.
[74] Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. In ICML, 2023.
[75] Yang Song, Conor Durkan, lain Murray, and Stefano Ermon. Maximum likelihood training of score-based diffusion models. In NewrlPS, 2021.
[76] Mingzhen Sun, Weining Wang, Gen Jiawei Liu, Jiahui Sun, Wanquan Feng, Shanshan Lao, Yu Zhou, Qian He, and Jing Liu. Ar-diffusion: Asynchronous video generation with ruto-regressive diffusion. In CVPR, 2025.
[77] Sergey Tulyakov, Ming-Yu Yu Liu, Xiaodong Yang, and Jan Kautz, Mocogan: Decomposing motion and content for video generation. In CVPR, 2018
[78] Dani Valevski, Yaniv Leviathan, Moab Arar, and Shlomi Fruchter. Diffusion models are real-time game engines. In ICLR, 2025.
[79] Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. In NeurlPS, 2017.
[80] Villegas, H Moraldo, Castro, Babacizadch, H Zhang, Kunze, PJ Kindermans, MT Saffar, and D Erhan. Phenaki: Variable length video generation from open domain textual descriptions. In ICLR, 2023.
[81] Ruben Villegas, Jimei Yang, Seunghoon Hong, Xunyu Lin, and Honglak Lee. Decomposing motion and content for natural video sequence prediction. In ICLR, 2017.
[82] Carl Vondrick. Hamed Pirsiavash, and Antonio Torralba. Generating videos with scene dynamics. NeurlPS, 2016.
[83] Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen Wei Xie, Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, et al. Wan: Open and advanced seree-scale video generative models. arXiv preprint arXiv:2503.20314, 2025.
[84] Jing Wang, Fengzhuo Zhang, Xiaoli Li, Vincent YF Tan, Tianyu Pang, Chao Du, Aixin Sun, and Zhuoran Yang. Error analyses of auto-regressive vidco diffusion models: A unified framework. arXiv preprint arXiv:2503. 2025.
[85] Wenhao Wang and Yi Yang. Vidprom: million-scale real prompt-gallery dataset for text-to-video diffusion models. In NewrlPS, 2024.
[86] Yuqing Wang, Tianwei Xiong, Daquan Zhou. Zhijie Lin, Yang Zhao, Bingyi Kang. Jiashi Feng, and Xihui Liu. Loong: Generating minute-level long videos with autoregressive language models. arXiv preprint arXiv:2410.0275: 2024.
[87] Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Prolificdreamer: fideliidelity and diverse text-to-3d generation with variational score distillation. NeurIPS, 2023.
14

---

[88] Dirk Weissenbom, Oscar Täckström, and Jakob Uszkoreit. Scaling auloregressive video models. In ICLR, 2020.
[89] Wenming Weng, Ruoyu Feng. Yanhui Wang. Qi Dai, Chunyu Wang. Dacheng Yin, Zhiyuan Zhao, Kai Qiu, Jianmin Bao, Yuhui Yuan, et al. Art-v: Auto-regressive text-to-video generation with diffusion models. In CVPR, 2024.
[90] Tong Wu, Zhihao Fan, Xiao Liu, Hai-Tao Zheng, Yeyun Gong, Jian Jiao, Juntao Li, Jian Guo, Nan Duan, Weizhu Chen, et al. Ar-diflusion: Auto-regressive diffusion model for text generation. In NeurlPS, 2023.
[91] Yushu Wu, Zhixing Zhang, Yanyu Li, Yanwu Xu, Anil Kag, Yang Sui, Huseyin Coskun, Ke Ma, Alcksei Lebedev, Ju Hu, et al. Snapgen-v: Generating five-second vidco within five seconds on mobile device. CVPR, 2025.
[92] Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming language models with attention sinks. In ICLR, 2024.
[93] Desai Xie, Zhan Xu, Yicong Hong. Hao Tan, Difan Liu, Feng Liu, Arie Kaufman, and Yang Zhou. Progressive autoregressive video diffusion models. arXiv preprint arX1v:2410008151, 2024.
[94] Wilson Yan, Yunzhi Zhang, Pieter Abbeel, and Aravind Srinivas. Videogpt: Video generation using vq-vae and transformers. arXiv preprint arXiv:2 20::::2101111 10157, 2021.
[95] An Yang, Baosong Yang, Beichen Zhang. Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprini 11Xiv:2412.15115, 2024.
[96] Mengjiao Yang, Yilun Du, Kamyar Ghasemipour, Jonathan Tompson, Dale Schuurmans, and Pieter Abheel. Learning interactive real-world simulators. In /CLR, 2024.
[97] Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. In ICLR, 2025.
[98] Tianwei Yin, Michaël Gharbi, Tacsung Park, Richard Zhang, Eli Shechtman, Fredo Durand, and Bill Freeman. Improved distribution matching distillation for fast image synthesis. NeurlPS, 2024.
[99] Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Fredo Durand, William T Freeman, and Taesung Park. One-step diffusion with distribution matching distillation. In CVPR, 2024.
[100] Tianwei Yin, Qiang Zhang, Richard Zhang, William T Freeman, Fredo Durand, Eli Shechtman, and Xun Huang. From slow bidirectional to fast autoregressive video diffusion models. In CVPR, 2025.
[101] Alan Yu, Ge Yang, Ran Choi, Yajvan Ravan, John Lconard, and Phillip Isola. Learning visual parkour from generated images. In CoRL. 2024.
[102] Jiwen Yu, Yiran Qin. Xintao Wang. Pengfei Wan, Di Zhang, and Xihui Liu. Gamefactory: Creating new games with generative interactive videos. arXiv preprint arXiv:2501.088 083255, 2025.
[103] Lantao Yu, Weinan Zhang, Jun Wang, and Yong Yu. Seqgan: Sequence generative adversarial nets with policy gradient. In AAAI, 2017.
[104] Lijun Yu, José Lezama, Nitesh B Gundavarapu, Luca Versari, Kihyuk Sohn, David Minnen, Yong Cheng, Vighnesh Birodkar, Agrim Gupta, Xiuye Gu, al. Language model beats diffusion-tokeniizr key visual generation. In ICLR, 2024.
[105] Lvmin Zhang and Maneesh Agrawala. Packing input frame context in next-frame prediction models for video generation. arXiv preprint arXiv:2504. 12626, 2025.
[106] Tianyuan Zhang, Sai Bi, Yicong Hong. Kai Zhang, Fujun Luan, Songlin Yang, Kalyan Sunkavalli, William T Freeman, and Hao Tan. Test-time training done right. arXiv preprint arXiv:2505.2 2025.
[107] Yuan Zhang, Jiacheng Jiang, Guoqing Ma, Zhiying Lu, Haoyang Huang, Jianlong Yuan, and Nan Duan. Generative pre-trained autoregressive diffusion transformer. arXiv preprint arXiv:2505.07334 2025.
[108] Zhixing Zhang, Yanyu Li, Yushu Wu, Anil Kag, Ivan Skorokhodov, Willi Menapace, Aliaksandr Siarohin, Junli Cao, Dimitris Metaxas. Sergey Tulyakov, et al. Si-v: Single forward video generation model. In NewrIPS, 2024.
[109] Xuanlei Zhao, Xiaolong Jin, Kai Wang, and Yang You. Real-time video generation with pyramid attention broadcast. In ICLR, 2025.
15

---

[110] Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, and Omer Levy. Transfusion: Predict the next token and diffuse images with one multi-modal model. In ICLR, 2025.
[111] Deyu Zhou, Quan Sun, Yuang Peng, Kun Yan, Runpei Dong, Duomin Wang. Zheng Ge, Nan Duan, Xiangyu Zhang, Lionel M Ni, et al. Taming teacher forcing for masked autoregressive video generation. In CVPR, 2025.
[112] Mingyuan Zhou, Huangjie Zheng, Yi Gu, Zhendong Wang, and Hai Huang. Adversarial score identity distillation: Rapidly surpassing the teacher in one step. In ICLR, 2025.
[113] Mingyuan Zhou, Huangjie Zheng. Zhendong Wang. Mingzhang Yin, and Hai Huang. Score identity distillation: Exponentially fast distillation pretrained diffusion models for one-step generation. In ICML, 2024.
A
Implementation Details
Our implementation is largely based on the open-source code of Wan2.1 1 The attention implementation of Diffusion Forcing and Teacher Forcing baselines is based on
83 and Caus Vid [100.
FlexAttention [[5]
while the attention in Self Forcing is based FlashAttention-3 72)
Noise schedule and model parameterization. Following the Wan2.1 series, we adopt the flow matching framework [48 [I], with time step shifting f(c,, 8) = (kt/ (tt/1000)// (1 + (k l) ((tt)))0))-
1000 and a shift factor le = 5. The forward process is specified as Ie 1000 x + 1- with E [0, 1000].
1000 e, ~ 1).
The data prediction model is given by:
Gø(x, c) Cskip - Cout Ug (Cin It, Croise (t),
We keep the preconditioning coefficients the same as the base models' configuration, i.c., Cskip Cin = Cout = and Cnoise (t) = t. Our few-step diffusion process employs uniform 4-step schedule [tq, f3, t2, t1] = [1000, 750, 500, 250].
Prompt preprocessing. We use the VidProS subset from VidProM 85 which contains around 1M semantically unique user-written text-to-video prompts. We filter out prompts that are too short (less than 20 characters), contain command line arguments (e.g.. -ar 16:9), or have a NSFW probability greater than 0.01 for any annotated category (toxicity, obscenity, identity attack, insult, threat, and sexual explicitness). This results in total of around 250k prompts. We then expand those prompts with Owen/Qwen2 open-source implementation of 1 [83] For VBench evaluation, we similarly rewrite the test prompts using Quen/ /Qwnn2. 5- Instruct... We note that the VBench results of the Wan2. 1 base model are also obtained with prompt rewriting, and report baseline results with prompt rewriting, provided that the model supports such enhancements.
InI-tutt [95), using the system prompt (English version) provided the
Training details. Most of our training runs use 64 NVIDIA GPUs (80GB memory each) with Per-GPU batch size of 1. We implement gradient accumulation for configurations requiring larger effective batch size than 64. Our DMD training runs take only approximately 1.5 hours converge, while SiD/GAN training takes 2-3 hours on 64 H100 GPUs. We initialize the real score network and critic network using the pretrained weights of the base model. We list all other training configurations. as well as the choice of real score network and critic network for different distribution matching objectives, Table We describe detailed training configurations for each distribution matching objective below.
For DMD, the gradient of the reverse Kullback-Leiblee divergence is given by [54
87 99
Oi
V&Le (pe.ell (ppoae))Pdaaa.)) (2 (s (sical (
Stake (2e, t))
(2)
00
where Sreal (:, 1) the score function for ta approximated by a pretrained diffusion model fol. 1).
also referred to the real score network, and Stake (-, t) is the score function for Pa, and is learned
16

---

Table 3: Specification of training hyperparameters
Hyperperumeters
DMD
SiD
GAN
Real score network
Wan2.1-T2V-144
N/A
Real score CEG weight
3.0
3.0
N/A
Critic network initialization
1-22-1-22V-1.3B
1an2.1-T2V-1.33
Wan2. 1122.1-TB 1-22----
Batch size
64
64
768
AdamW, A1 U, 82 0.999.
Adam, 8 0, 0.999,
AdamW, 8 82 0.999,
Optimizer (Ge)
10-8, weight decay= 0.01
1c-8, weight decay=
1c-8, weight decay= 0.01
AdamW. 01 0, 82 0.999,
Adam, 01 0. 02 0.999.
AdamW. 3, = 82 0.999.
Optimizer (fp)
1e-8, weight decay= 0.01
1e-8, weight_decay=
1e-8, weight_decay== 0.01
Leaming rate (Ga)
2c-6
2--6
2--6
Learning rate (fu)
4--7
2e-6
2e-6
Generstor/eritic update ratio
5
EMA decay
0.99
0.99
0.99
through a critic network (e(:;)) () via the standard diffusion loss. The gradient in Eqn. to the following loss function:
2 equivalent
1
LDMD
2 || sg l (fo( fo(
(3)
where sgl-] denotes the stop gradient operator.
Similar to the pipeline of DMD, the SiD loss is given by 113
(0) Er,a,, [(o( () fo
5) (1- (o)laalll( (o 1)
which can be shown that the case of a = 0.5 corresponds the gradient of the Fisher divergence
Et.po.. [I] log po.: V log Paaa.ell [l [30 (5). Empirically, it is observed that the second term often
leads to unstable training and thus a is typically adopted for better performance [12 113)
which is also followed in this work.
For GAN training, we add additional cross-attention layers and classification heads to the initialized critic network. We employ relativistic loss [34] and approximate the regularization terms (R1 and R2) using finite difference following Seaweed-APP APP [47]. Specifically, we perturb the noisy real/fake data with additional small Gaussian noise and encourage the discriminator output to be similar to the original one. The final training objective is defined as:
Lug 2 Ee, (xr) - fulzz + a e) + (If - fol a -
(5)
(D(i) - -E & [log (sigmoid (Ix( fo + ACreg
(6)
Co( E.... [log (sigmoid (ft( fu (z))))
(7)
where Pdata.t. &, ~ P are the noisy real and fake data, respectively, e and é are Gaussian
noise sampled from /'0 1), and f is the critic network (discriminator) of GAN. We use 30,
= 0.05 for all experiments. For video generated from the output of the s-th step (see Algorithm I for details), we find that only sampling from (t to] helps stabilize the training. We also adopt large batch size of 768 for training stability.
B Importance of local attention training in rolling KV cache
We qualitatively ablate two training settings for video extrapolation using the rolling KV cache technique. In the naive baseline, the model is trained such that every chunk always attends to the first chunk during denoising. In contrast, our proposed method restricts the attention window to prevent the model from attending to the first chunk when denoising the last chunk. As shown in Fig. 7 the naive baseline exhibits visual artifacts when extrapolating videos beyond the training context length, whereas our proposed solution mitigates this issue.
17

---

Starting frames
Extrapolatod Frames
Starting frames
Extrapolated Frames
Rolling KV Cache (naive)
Rolling V Cache (ours)
Rolling KV Cache (naive)
Rolling KV Cache (ours)
Figure 7: Qualitative comparisons on video extrapolation. We present a visual comparison between the naive baseline and our proposed technique for rolling KV cache- based video extrapolation. Compared to our method using local attention window training, extrapolated video frames from the naive baseline exhibit severe visual artifacts.
C VBench Scores Across All Dimensions
In Fig. we evaluate Self Fore- ing (both chunk-wise and frame-wise AR versions) using all 16 VBench metrics against representative mod- els. Self Forcing generally outper- forms other models in terms of se- mantic alignment, evidenced by the high scores scene, object class, mul-
baceground consistancy
overal consistency
tempory Tlikk-iigg ting
temporal Syle
mation
ROPRRTMTEE
tiple objects, and human action di- mensions. Our methods also achieve good Trame-wise quality, indicated by the high scores in aesthetic qual- ity and imaging quality. Our frame- wise AR variant exhibits more dy- namic motion (high dynamic degree score) but worse temporal consis- tency (worse background consistency, motion smoothness, and larger tem- poral flickering) than the chunk-wise AR variant.
degree
weathetic
antonshpp
imaging quality
cdlor
class
human action
Selt Forcing (chunk-wise) Self Forting (frame-wisel
SkyReeis-V2
UTX
D Broader Societal Impact
MAGI-1
Wan-1.38
Figure 8: VBench scores visualization. We compare Self Forcing with SkyReels-V2 (10), Wan2. [83] MAGI- 69 and Caus Vid [100] using all 16 VBench metrics.
Generative modeling particulally for videos-care carries significant poten- tial for misuse. It can lead to serious societal consequences, most notably
the spread of disinformation through deepfakes that become increasingly difficult to distinguish from authentic content. Additionally, these models can reinforce harmful stereotypes and amplify existing socictal biases without careful governance and responsible deployment.
18

---

Our research on real-time video generation creates additional complexities, as it removes one of the practical barriers (computational cost) that currently limits widespread misuse. While our methods enable positive applications like creative content production and accessibility tools, we acknowledge the dual-use nature of this technology and encourage continued research into detection methods, watermarking techniques, and policy frameworks that can help mitigate potential harms.
E
User Study Details
In the user preference study, we show users two videos side by side using the same text prompt. We ask the users to select the that is overall better, considering both quality and prompt alignment. Detailed instructions are shown in Fig. | We use all 1003 prompts from MovieGenBench [64] and each prompt is evaluated by single user.
better video.
Instruction:
You will ba shown two Al generated videos side by side. Each video was created based on the same text prompt. Your task compare both videos and select the one that Is of higher quality and more acourately reflects the content of the prompt
evaluating
consider
oritaria:
Visual Quality: Which video has more relieric, aestheticall pleasing visuass without color or sturctural artitcct??
Prompt Alignment Which video better reproenns the detail,, themes, and intent of the original ledt prompt?
Choose
balances visual tainful
Text Prompt
Several giant wooly mammoths approach treacing through snowy meadow, their long wooly fur lightly blows the wind as they walk, snow covered dramaric capped
light vispy clouds high low camera view stunning capturing the furry mammaa with beautiful photography, depth field.
glow,
ACCEPT
Figure 9: User study instruction screenshots.
19

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our main contribution is a new training algorithm for autoregressive diffusion models that addresses the exposure bias problem.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of work in Section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
20

---

Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
We provide the information needed to reproduce the main experimental results in Section 4 and in Appendix A
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
21

---

Answer: [Yes]
Justification:
We include the code and data in the supplemental material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ http://cccc./ccl..../coo////aaaa Coapae///ooSSSebbiieeeee sei.aiisssooooiioyyo. for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide all the information needed to reproduce the main experimental results in Section 4 and in Appendix A
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Error bars are not reported because it would be too computationally expen- sive (each run requires hundreds of H100 hours of compute).
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
22

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The information is provided the Appendix A
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
We reviewed the NeurIPS Code of Ethics and our research conforms to it.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We include discussion in Appendix D
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
23

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [Yes]
Justification: Our base model weights (Wan 2.1) are safeguarded and we do not use additional video data during our training process that could be unsafe. We also filter out NSFW prompts from our training data, as described in Appendix A
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Section 4 we mention that our model is initialized from Wan2.1 1.3B and we also generate synthetic data using their pretrained models. In Appendix A we acknowledge that our implementation is largely based on Wan2.1 and Caus Vid open-source implementations. We obey the Apache License of Wan2.1 and the CC-NC license of Caus Vid.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

---

If assets are released, the license, copyright information, and terms of use in the thcode has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssit
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
See supplemental material for the documentation of our code.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [Yes]
Justification:
We include this information in Appendix E
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification:
IRB review is not required.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
25

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM for what should or should not be described.
26


Reviews:
[{"review_id": "AV1N6HHdFk", "reviewer": "Reviewer_Vwja", "summary": "The paper addresses a critical issue in autoregressive video generation setups: error accumulation in the network's predictions, leading to train/test domain gaps. The authors employ an Infinite Nature approach, where instead of teacher forcing (training on ground truth sequences), they use student-forcing or self-forcing, in which the model also observes its own outputs during training. This is challenging because the model must backpropagate through all diffusion time steps and all previous generation steps. To address this, the authors propose several techniques: 1. Use a few step distribution matching backbone and training scheme instead of a full ~50-step SDE/ODE model; 2. Stochastic gradient truncation, where the gradients flow only through the last denoising step of each frame; earlier steps and prior frames are detached. The model works well and achieves 17FPS (on par with its backbone CausVid), and have certain quality improvement.", "questions": "N/A", "limitations": "N/A", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength:\n\n-- The problem of error accumulation is a critical and long-standing problem of AR models, and the paper is well-motivated and proposed a reasonable solution.\n\n-- The training is very efficient and could converge within hours.\n\n-- The authors provide thorough empirical section: VBench, human study, latency/throughput, ablations on loss functions and training paradigms.\n\nWeakness:\n\n-- The idea of student forcing/self forcing is not new. Prior autoregressive work such as Infinite Nature did some thorough experiments on such methods.\n\n-- I personally work on this topic and tried the code as soon as it was released. However, I am not certain how much improvement is actually made by the proposed self-forcing training scheme -- efficiency-wise, the FPS and workload is exactly the same as CausVid, so efficiency-based improvements cannot be claimed by this paper; quality-wise, I am also not certain how much improvement is made on top of CausVid, the short 5-sec videos look on par with CausVid, and longer extrapolation still has the drifting issue, and I personally think it is arguably more severe than CausVid, potentially because self-forcing is trying to accommodate and ignore the error, but at the same time it also forgets some portions of the context.\n\nOverall, this is a good paper that should be accepted, as it provides some interesting insights. However, I am not sure how \"functional\" the method actually is — the error accumulation is still evident, and it is hard to argue how much more robust the model gets compared with the backbone. Therefore, I vote for acceptance, but not a strong acceptance.", "was_revised": true, "final_justification": "I do not have many concerns initially and would keep my positive score.", "rebuttal": "We thank the reviewer for their valuable feedback. We are encouraged by the overall positive assessment, with reviewers agreeing that our work addresses a \"critical and long-standing problem\" [Vwja] and is \"very well motivated\" [2kqq]. The method is \"clearly presented\" [9RYz], \"intuitive\" [N5mF], and the paper is “well-written and easy to follow” [2kqq]. Our \"quantitative results are solid\" [9RYz] and were praised as a \"thorough empirical section\" [Vwja], demonstrating \"broad success across VBench, human preference studies, and efficiency benchmarks\" [N5mF].\n\nWe will now address the concerns and questions raised to further clarify our contributions.\n\n- **Novelty**: We agree with the reviewer and acknowledge in our paper (L49, L70-74) that training with self-rollout was explored in early RNN+GAN-based models, including InfiniteNature. However, as Reviewer N5mF kindly noted, our work is the first to successfully adapt this principle to **modern Autoregressive Diffusion Transformers**. This adaptation is highly non-trivial, requiring a careful design for combining a few-step diffusion, KV caching, and gradient truncation. Successfully bridging this established concept to the AR-DiT architecture is a core technical contribution of our work, leading to significant gains in generation quality over older paradigms.\n\n- **Improvement over CausVid**: We want to clarify that Self-Forcing has **significantly better quality than CausVid-Wan-1.3B** implemented by the original author under the same-speed setting. This advantage is quantitatively supported by VBench scores (Total Score: 84.31 vs. 81.20), user preference study (66.1% preference), the qualitative examples in the appendix (Fig. 7), and the html website in the supplementary material. We respectfully hypothesize that the reviewer's observation that our results “look on par with CausVid” may refer to a different implementation of CausVid built upon a larger closed-source model that runs significantly slower (9.4 FPS for 360P, compared with ours 17FPS for 480P). Our paper provides an apples-to-apples comparison against the public CausVid implementation to fairly evaluate the specific impact of our training paradigm on the same 1.3B architecture."}, {"review_id": "b27o8Q7EJl", "reviewer": "Reviewer_9RYz", "summary": "The paper proposes Self-Forcing, a fine-tuning technique to enable autoregressive generation in video diffusion models. The method starts with fine-tuning the base model with causal masking to initialize it with some autoregressive capability. Then it follows with training on autoregressively self-generated data using distribution matching losses to make sure the generated chunks form a plausible temporally consistent video as a whole. KV caching and few-step distillation are used to speed up the training and the inference with the model. Self-forcing achieves better results than the prior work on VBench and human preference.", "questions": "1. I recommend moving the details about the different training objectives to the main paper, as they are essential for understanding key aspects of the method, such as how the pseudo ground truth scores are obtained and how the critic is trained.\n2. Could you clarify how many rollouts are used during training? What is the total number of frames? Does this total align with the native number of frames of the base model or the critic?\n\nI think in the current state the presentation weaknesses overweigh the method's strengths, therefore I am leaning towards rejection. However, I am open to revising my score if the rebuttal adequately addresses my concerns, as I do believe the contributions (the achieved efficiency through KV caching and autoregressive distillation) are noteworthy.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n\n1. The motivation and the method itself are clearly presented. \n2. Applying KV caching to video models is a meaningful and valuable contribution.\n3. The quantitaive results are solid.\n\n## Weaknesses\n\n1.The proposed Self-Forcing method is essentially a distillation or fine-tuning technique, whereas Teacher Forcing (TF) and Data Forcing (DF) were originally designed for training from scratch. Presenting these approaches as direct counterparts can be confusing and makes the overall story harder to follow.\n2. The method is presented as addressing exposure bias and enabling long video generation, yet no quantitative long generation results are provided (only some qualitative examples in the supplementary material’s limitations section). Moreover, the authors acknowledge that the model struggles with sequences longer than those seen during training. This seems like an inherrent limitation of the approach, as the distributional losses do not seems to go beyond the native generation window of the base model.\n3. The visual results, aside from the local attention ablation, do not clearly demonstrate the benefits of the proposed method.", "was_revised": true, "final_justification": "The authors did a good job during the rebuttal and addressed most of my concerns in both the rebuttal and the follow-up discussion phase. Enabling AR generation in modern VDMs offers valuable benefits, and the proposed method provides a reasonable solution to this problem. The application of KV-caching for videos is a meaningful contribution.", "rebuttal": "We thank the reviewer for their valuable feedback. We are encouraged by the overall positive assessment, with reviewers agreeing that our work addresses a \"critical and long-standing problem\" [Vwja] and is \"very well motivated\" [2kqq]. The method is \"clearly presented\" [9RYz], \"intuitive\" [N5mF], and the paper is “well-written and easy to follow” [2kqq]. Our \"quantitative results are solid\" [9RYz] and were praised as a \"thorough empirical section\" [Vwja], demonstrating \"broad success across VBench, human preference studies, and efficiency benchmarks\" [N5mF].\n\nWe will now address the concerns and questions raised to further clarify our contributions.\n\n- **Presentation versus TF/DF**: We believe comparing Self-Forcing directly to Teacher-Forcing (TF) and Diffusion-Forcing (DF) is useful for understanding our core contribution, as the **fundamental axis of comparison is the source of the conditioning context during training**. As illustrated in Figure 1, these methods represent three distinct choices for this context:\n  - Teacher Forcing: Conditions on clean, ground-truth frames.\n  - Diffusion Forcing: Conditions on noisy, ground-truth frames.\n  - Self Forcing (Ours): Conditions on the model's own previously generated frames.\n\n  Although TF/DF were originally introduced for pre-training and Self-Forcing is described as a post-training algorithm for practical considerations, we argue that algorithms used at different stages can still form valid comparisons. As an analogy, while today GANs are often used only for post-training/distillation and diffusion models for pre-training, they are still routinely compared as alternative generative modeling paradigms.\n\n  Moreover, the application of TF/DF is not limited to pre-training. Our key baseline, CausVid, applies DF in a post-training phase with a DMD objective. Our ablation studies (Table 2) apply all three paradigms in the same post-training setting. This ensures an apples-to-apples comparison to isolate the impact of the conditioning strategy—the central variable our paper investigates.\n\n- **Long video generation**: The goal of this paper is **not to improve long video generation quality** in the extrapolation setting. Our work aims to address exposure bias and to reduce error accumulation for videos generated *within the training context length*. This is a distinct problem from extrapolation (i.e., generating videos substantially longer than the training context), where we do not claim strong performance, as discussed in our limitations (L318-320). Extrapolation introduces a second distributional mismatch (of video length), in addition to the distribution mismatch that we address (ground-truth vs. generated history). While resolving exposure bias is a necessary step for long video generation, it is not sufficient by itself. To truly enable high-quality long video generation, future work needs to either scale up the video length *during training*, or investigate orthogonal techniques that enable *better extrapolation*.\n\n- **Visual results not convincing enough**: The visual results in Fig. 7 show **much better quality than CausVid** (less oversaturation). While videos obtained with Wan/SkyReels/MAGI-1 indeed sometimes have visually comparable quality, those methods are **150-400x slower** than us in terms of latency. So **even same quality is still a big win** for us.\n\n- **Moving training details**: We thank the reviewer’s suggestion and will move the training objective descriptions to the main paper in the final version.\n\n- **Training rollout**: We generate 21 latent frames (corresponding to 81 video frames) during training, which is aligned with the setting of the base model. The number of rollouts is 21 divided by the chunk size (which equals to 3 or 1 in our experiments). We will add these details in the final version."}, {"review_id": "xYdczLCWIo", "reviewer": "Reviewer_2kqq", "summary": "This paper presents self-forcing, a diffusion framework for long video generation. Different from prior approaches such as diffusion forcing or teacher forcing, self-forcing aims to remove the gap between training and inference by explicitly doing rollout during training. To reduce the computation and memory burden for this training strategy, the paper introduces several tricks to mitigate this, e.g., using few-step diffusion models or only doing backpropagation for a single random timestep per frame. The paper also adapts the KV caching trick (which has been popular in LLMs) by for long video generation. By incorporating all of these compoments, the paper shows self-forcing can generate long videos in real-time with high quality, outperforming prior long video generation methods.", "questions": "Please see my weaknesses", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n- The paper is generally well-written and easy to follow.\n- The paper is very well motivated, as the length limit of existing video generation models is one of the biggest bottlenecks for them to be extended to real-world videos.\n- Removing the discrepancy between training and inference is important and a reasonable approach. \n- Some components seem technically novel (e.g., how to handle gradients).\n\n**Weaknesses**\n- There are very limited qualitative comparisons, making it difficult to know how the proposed method generates better long video compared with other baselines. In particular, I wonder how self-forcing can be qualitatively better than diffusion-forcing or teacher-forcing but I cannot find qualitative comparisons. Could the authors provide more comparisons in general (like comparison with other baselines, eg, CausVid) as well as comparison with DF or TF? \n- I think real-time generation seems great, but the provided examples do not seem to be long, as they are just 10 seconds videos. \n- Minor: [1] also deals with chunk-wise autoregressive long video generation and exposure bias but the paper misses discussion. \n- Minor: Could the authors provide memory consumption in training and inference? \n\n[1] Yu et al., MALT Diffusion: Memory-Augmented Latent Transformers for Any-Length Video Generation, 2025", "was_revised": true, "final_justification": "I think this paper tackles and an important problem with a quite elegant solution. I recommend acceptance for this paper", "rebuttal": "We thank the reviewer for their valuable feedback. We are encouraged by the overall positive assessment, with reviewers agreeing that our work addresses a \"critical and long-standing problem\" [Vwja] and is \"very well motivated\" [2kqq]. The method is \"clearly presented\" [9RYz], \"intuitive\" [N5mF], and the paper is “well-written and easy to follow” [2kqq]. Our \"quantitative results are solid\" [9RYz] and were praised as a \"thorough empirical section\" [Vwja], demonstrating \"broad success across VBench, human preference studies, and efficiency benchmarks\" [N5mF].\n\nWe will now address the concerns and questions raised to further clarify our contributions.\n\n- **Long video generation**: The goal of this paper is **not to improve long video generation quality** in the extrapolation setting. Our work aims to address exposure bias and to reduce error accumulation for videos generated *within the training context length*. This is a distinct problem from extrapolation (i.e., generating videos substantially longer than the training context), where we do not claim strong performance, as discussed in our limitations (L318-320). Extrapolation introduces a second distributional mismatch (of video length), in addition to the distribution mismatch that we address (ground-truth vs. generated history). While resolving exposure bias is a necessary step for long video generation, it is not sufficient by itself. To truly enable high-quality long video generation, future work needs to either scale up the video length *during training*, or investigate orthogonal techniques that enable *better extrapolation*.\n\n- **Visual comparisons with DF/TF**: We will add additional qualitative comparisons with DF/TF in the final version of the paper (we are not allowed to add them in rebuttal due to NeurIPS policy restrictions). Visually, videos generated with DF/TF tend to exhibit **significantly more error accumulation** and often suffer from over-saturation over time.\n\n- **MALT Diffusion** reduces error accumulation by adding noise to context frames during training. This was also used in earlier works including Diffusion Forcing and GameNGen, and we have some discussions in L44-47. We will add citations to MALT Diffusion in the final version.\n\n- **Memory usage**: We use mostly bfloat16 precision for both training and inference. The peak allocated memory during training (batch size 1) is 53.2GB. During inference, it is 23.9GB Inference without text encoder offloading and 13.0GB with offloading. We will add these details in the final version."}, {"review_id": "FfLOViEflU", "reviewer": "Reviewer_N5mF", "summary": "This paper introduces Self Forcing, a post-training approach for autoregressive video diffusion models. The key idea is to simulate inference-time behavior during training by letting the model condition on its own generated frames rather than ground truth. This helps close the gap between training and inference, which has long been a challenge due to exposure bias. It combines few-step diffusion and stochastic gradient truncation to efficiently expose models to their own prediction errors and optimize a holistic video-level distribution-matching objective.", "questions": "1. Did the authors experiment with applying varying noise intensities to individual frames before denoising and then enforcing a holistic distribution-matching constraint?  How do different noise levels impact the mitigation of cumulative errors?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "**Strengths**\n1. This paper is the first to introduce autoregressive self-rollout in video diffusion post-training, fully aligning training and inference distributions to eliminate exposure bias.\n2. It uses holistic video-level distribution-matching losses (DMD/SiD/GAN)， so the model learns to correct its own residual errors, improving both visual fidelity and temporal consistency.\n3. It is intuitive why this works: by matching the model’s output to the true distribution at the video (holistic) level, the method enforces a global constraint on cumulative errors.\n4. This paper also demonstrates broad success across VBench, human preference studies, and efficiency benchmarks, outperforming both autoregressive and bidirectional diffusion baselines.\n\n**Weaknesses**\n1. Gradient truncation (only backpropagating the final denoising step) and detaching historical KV embeddings save memory but may weaken the model’s ability to capture long-range, cross-frame dependencies.\n2 .Comparison of DMD, SiD and GAN objectives is limited; the paper does not clearly identify which loss is best for which scenario.\n3 .Key hyperparameters (diffusion steps, truncation point, KV-cache size) are highly sensitive, yet the paper offers little guidance on their tuning.", "was_revised": true, "final_justification": "The authors have addressed my concerns. I keep my initial rating.", "rebuttal": "We thank the reviewer for their valuable feedback. We are encouraged by the overall positive assessment, with reviewers agreeing that our work addresses a \"critical and long-standing problem\" [Vwja] and is \"very well motivated\" [2kqq]. The method is \"clearly presented\" [9RYz], \"intuitive\" [N5mF], and the paper is “well-written and easy to follow” [2kqq]. Our \"quantitative results are solid\" [9RYz] and were praised as a \"thorough empirical section\" [Vwja], demonstrating \"broad success across VBench, human preference studies, and efficiency benchmarks\" [N5mF].\n\nWe will now address the concerns and questions raised to further clarify our contributions.\n\n- **Gradient truncation**: We implemented more heavily-optimized activation checkpointing and additional training parallelism to support Self Forcing training without gradient truncation. However, we find that **models trained without gradient truncation do not produce better results** than models with truncation. We hypothesize that models already learn representations that capture long-range dependencies during the pretraining stage, so that enabling long-range gradient flow during post-training becomes unnecessary. We will add those discussions in the final version of the paper.\n\n- **Comparison of DMD/SiD/GAN**: We find GAN training is generally less stable and requires more hyperparameter tuning efforts than DMD/SiD. DMD is preferred over SiD when using a larger teacher model. Unlike SiD, DMD does not require backpropagating through the teacher and is therefore much more efficient. We will add a discussion in the final version.\n\n- **Hyperparameter sensitivity**: Our method is **not sensitive** to the mentioned hyperparameters. The maximum KV-cache size is determined by the base model, and we always apply truncation to the last denoising step to minimize memory consumption. We do not observe better results with full backpropagation (without truncation), despite larger memory consumption. The number of diffusion steps is a hyperparameter that controls the quality/speed trade-off. Our method does not fail catastrophically when using less denoising steps. For example, we trained a 3-step model and it obtains a VBench total score of 84.26, only slightly lower than the 4-step version (84.31).\n\n- **Varying noise intensities**: We interpret the question as follows: instead of using clean self-generated context frames as conditioning (as in the current Self-Forcing algorithm), can we add noise to self-generated context frames during training and inference (maybe with progressive intensity levels similar to rolling diffusion), and does it further reduce error accumulation?\n\n  We do not find adding noise to context frames helpful in our case. Intuitively, when there is a train-test distribution mismatch (e.g., in teacher forcing), adding Gaussian noise to both distributions makes them  have larger overlap and thereby mitigates cumulative errors. However, since our method does not have a train-test distribution mismatch, adding noise to the context is no longer necessary and only introduces drawbacks as discussed in L45-47."}]

Author rebuttal:

Meta-review: The submission proposes a training approach termed Self Forcing, to solve the problem of error accumulation for autoregressive video diffusion models.  Specifically, Self Forcing performs autoregressive self-rollout during training, denoising the next frame based on previously generated frames by itself.

The reviewers acknowledged the author's rebuttal and gave all positive final ratings: three “accept” and one "borderline accept".
After reviewing all the materials, the AC agrees with the ratings of the reviewers and suggests a solid accept.
AC also suggests a Spotlight recommendation considering the significance of autoregressive video generation and the effectiveness of the proposed Self Forcing approach.
Decision: Accept (spotlight)