Self-Forcing++: Towards Minute-Scale High-Quality Video Generation
Authors: Justin Cui
Jie Wu
Ming Li
Tao Yang
Xiaojie Li
Rui Wang
Andrew Bai
Yuanhao Ban
Cho-Jui Hsieh
Venue: ICLR 2026 Poster (2026)
Keywords: long video generation
diffusion model
autoregressive video generation
Primary area: generative models
TL;DR: autoregressive long video generation

Abstract: Diffusion models have revolutionized image and video generation, achieving unprecedented visual quality. However, their reliance on transformer architectures incurs prohibitively high computational costs, particularly when extending generation to long videos. Recent work has explored autoregressive formulations for long video generation, typically by distilling from short-horizon bidirectional teachers. Nevertheless, given that teacher models cannot synthesize long videos, the extrapolation of student models beyond their training horizon often leads to pronounced quality degradation, arising from the compounding of errors within the continuous latent space. In this paper, we propose a simple yet effective approach to mitigate quality degradation in long-horizon video generation without requiring supervision from long-video teachers or retraining on long video datasets. Our approach centers on exploiting the rich knowledge of teacher models to provide guidance for the student model through sampled segments drawn from self-generated long videos. Our method maintains temporal consistency while scaling video length by up to 20$\times$ beyond teacher's capability, avoiding common issues such as over-exposure and error-accumuation without recomputing overlapping frames like previous methods. When scaling up the computation, our method shows the capability of generating videos up to 4 minutes and 15 seconds, 
equivalent to 99.9\% of the maximum span supported by our base model’s position embedding and more than 50x longer than that of our baseline model.  Experiments on standard benchmarks and our proposed improved benchmark demonstrate that our approach substantially outperforms  baseline methods in both fidelity and consistency. Our long-horizon videos demo can be found at http://self-forcing-plus-plus.github.io/

Published as a conference paper at ICLR 2026
SELF-FORCING++; TOWARDS MINUTE-SCALE HIGH- QUALITY VIDEO GENERATION
Justin Cui Jie Wur Ming Li Tao Yang', Xiaojie Li", Rui Wang', Andrew Bai', Yuanhao Ban', Cho-Jui Hsieh 1.3
UCLA 2 University of Central Florida 3 ByteDance Seed f Project Leader, Corresponding Author
Vidro Scoling
s0- 1205 Bacand Minuel
120x-180x (Thie Minute)
Gypoamic Deiyree
19kb-24th Bouthh Minute)
CANCOa
(wer)
Figure I: Self-forcing++ generates videos up to four minutes long. The radar chart highlights our model's superiority, while the line plot shows its sustained motion dynamics over long durations.
ABSTRACT
Diffusion models have revolutionized image and video generation, achieving un- precedented visual quality. However, their reliance on transformer architectures incurs prohibitively high computational costs, particularly when extending gen- eration long videos. Recent work has explored autoregressive formulations for long video generation, typically by distilling from short-horizon bidirectional teachers. Nevertheless, given that teacher models cannot synthesize long videos, the extrapolation of student models beyond their training horizon often leads pronounced quality degradation, arising from the compounding of errors within the continuous latent space. In this paper, we propose simple yet effective approach to mitigate quality degradation in long-horizon video generation with- out requiring supervision from long-video teachers or retraining datasets. Our approach centers on exploiting the rich knowledge of teacher mod- els to provide guidance for the student model through sampled segments drawn from self-generater long videos. Our method maintains temporal consistency while scaling video length by up 20x beyond teacher's capability, avoiding common issues such as over-exposure and error-accumulation without recomput- ing overlapping frames like previous methods. When scaling up the computa- tion, our method shows the capability of generating videos up to 4 minutes and 15 seconds, equivalent to 99.9% of the maximum span supported by our base model's position embedding and more than 50x longer than that of our baseline model. Experiments on standard benchmarks and our proposed improved bench- mark demonstrate that approach substantially outperforms baseline methods
long video

---

Published as a conference paper at ICLR 2026
in both fidelity and consistency. Our long-horizon videos demo can be found at self-forcing-plee pwt.-plusu-uppappe pplus.iithub.ii
INTRODUCTION
The field of video generation is advancing at a remarkable pace, catalyzed by the advent of diffusion models. Seminal works such Sora (OpenAT 2024), Wan (Wan et al. 2025), Hunyuan-DiT (Kong) et al. 2024), and Veo (Google DeepMind| 2025) are progressively closing the gap between gen- erated content and reality. Despite this progress, formidable challenge remains: the majority of state-of-the-al models are confined to generating short-form videos, typically capped at 5-10 sec- onds. This constraint is inherent the architectural design of the underlying Diffusion Transformers (DiT) (Peebles & Xie 2023). the inherently non-streaming and non-causal nature of the vanilla DiT architecture poses significant challenge to achieving temporal scalability.
A promising avenue for transcending this limitation lies in shifting from bidirectional diffusion ar- chitectures autoregressive, streaming- models. One such approach, Diffusion Forcing (Chen) eet al. 2024 Kim et al. 2024), applies heterogeneous noise schedules across frames to enable se- quential generation. However, the combinatorial complexity of noise scheduling often leads train- ing instability and has proven difficult to scale (Chen et al.) 2025 Sand-Al] [2025 He et al. 20225) more tractable strategy involves predicting the next frame or chunk from clean context, with KV caching emerging as a key mechanism for enabling performant, real-time streaming. For in- stance, Caus Vid (Yin et al.] 2025) proposes a method to distill a bidirectional teacher model into a streaming student model using heterogeneous distillation. However, its reliance overlapping frames for temporal consistency and a pronounced train-inference mismatch often results in over- exposure artifacts. The Self Self-Forcing (Huang et al., 2025) method mitigates the over-exposure issue by aligning the training and inference distributions. While this sets a new benchmark for short-form video quality, its capacity remains bottlenecked by the fixed-duration teacher model. Consequently, when tasked with generating content beyond this intrinsic temporal window (e.g.. > 10 seconds), the model's ss visual quality degrades precipitously.
A primary challenge limiting the quality of autoregressive long-video generation models is signif- icant -inferencee misalignment. This misalignment manifests in two principal ways. First, a temporal mismatch occurs: during training, models generate short clips of up to 5 seconds the maximum horizon of the teacher model-whereat inference, they must generate videos of sig- nificantly greater length. Second, error accumulation caused by supervision misalignment during long-horizon generation. In training, the teacher model provides abundant supervision for every frame within the short clip. This intensive guidance, however, means the student model is rarely exposed to the compounding errors that naturally arise in long rollouts, leaving it ill-equipped to handle them. As result, generation quality rapidly deteriorates beyond the 5-second training hori- zon, often collapsing into static or stalled content.
In this paper, introduce Self-Forcingt+. which directly targets the above two issues. Building upon the observation from previous works (Bruce et al.] [2224] Alonso et al./ 2024 Li ct al. 2025a) that a teacher model, despite its own 5-second generation limit, possesses rich knowledge for cor- recting errors in quality-degraded videos due to its training on a vast video corpus. We leverage this insight by extending the student's generation horizon far beyond 5 seconds (up 100 seconds in our experiments). This process intentionally produces candidate long videos that contain accumulated errors. To enable the student model to handle these errors, we then re-inject noise into these de- graded rollouts and apply distribution setributioo-matchign distillation with the strong teacher model, process combined with a long- student to recover from degraded states and sustain high-quality coherent video generation over ex- tended durations. Experimental results demonstrate that our method can scale video generation up to 100 seconds, 20x increase over the baseline, while maintaining high visual quality. By scaling up computation through extended training, our method is capable of generating videos up to minutes and 15 seconds"] utilizing 99.9% of the base model's positional embedding capacity and represent- ing a 50x improvement over the baseline. Furthermore, our investigation revealed that the widely used VBench (Huang et al.) 2024) benchmark exhibits bias that favors over-exposed and degraded
rolling KV cache and windowed sampling. This strategy teaches the
'The maximum number of latent frames Wan2.1 1-T2V-1.3B supports is 1024, since we generate videos in trunk size of the maximum length can reach is 1023 which is 99.9% of maximally supported length 1024.
2

---

Published as a conference paper at ICLR 2026
frames when evaluating long videos, undermining the reliability of its results. To remedy this, propose a new metric, Visual Stability, designed to systematically capture both quality degradation and over-exposure in long video generation. Our work paves the way for building more robust and reliable long video generation models. Our contributions are summarized as follows:
Identifying Horizon Scaling Bottlenecks: We reveal the primary obstacle extending the gener- ation horizon of autoregressive models: a dual mismatch in temporality and supervision during training versus inference. This insight provides clear target for overeoming previous limitations on the generation length.
A Simple Solution: We propose a simple training framework, named Self-Forcing++. By gen- crating beyond the teacher's ss horizon and correcting the student model on its own long. error- accumulated rollout trajectories, Self-Forcing++ extends high-quality video generation to 100 seconds, far surpassing previous state-of-the-art methods without reusing overlapping frames.
SOTA Performance and Horizon Scalability: Self-Forcing++ achieves state-of-the-art (SOTA) performance in long-video generation across a range of durations (e.g., 10s, 50s, 100s). Further- more, we discover significant scaling property: by scaling the training computation, model's generation capability extends to multiple minutes, a feat previously considered out of reach.
2 RELATED WORKS
Long Video Generation
Despite their success in short video generation, DiT-based models are costly to train and infer, and most remain limited to 5-10 outputs. Several methods aim to extend
generation length Jiang et al. 2025 Kodaira et al. 2025 Fang et al. 2025 Lin et al. 2025). RI- FL.Ex (Zhao et al. 2025) offers training-free solution by revisiting positional encodings, doubling horizon length and outperforming prior approaches (Chen et al. 2023 Peng et al. 2023 Zhuo et al.
2024). Autoregressive strategies tackles it from a different perspective: Pyramid-Flow (Jin et al.
2025 links flows across pyramidal resolutions for end-to-end autoregressive generation: SkyReels-
V2 (Chen et al. 2024) applies diffusion forcing for potentially infinite rollouts; MAGI-1 Teng
et al. 2025) predicts consecutive segments via progressive chunk denoising; CausVid (Yin et al.
2025 leverages block causal attention and KV caching; and Self-Forcing (Huang et al.) (025) further aligns training with inference through self-rollout during training, yielding higher quality. APT2 (Lin et al. 2025) adopts an adversarial training paradigm that first transforms a bidirectional model into one-step autoregressive generator, then enhances generation quality through joint train- ing with student forcing and a discriminator trained on real and generated video segments. In con- trast, our method relies pretrained diffusion models and does require a real training dataset.
Reinforcement Learning Reinforcement learning is central to post-training large language mod- els (Ouyang et al. 2022: Kumar et al. 2024 Hou al., 2025 Xie al.) 2025) and has proven similarly effective for generative models. Early efforts focused on image-specific rewards such as ImageReward (Xu al. 2023), Pick-a-Pic (Kirstain al. 2023), , and HPS V2 (Wu et al. 2023),
later extended to video with VedeoReward (Liu et al. (025b) and VisionReward (Xu et al. 2024)
for temporal and motion assessment. Optimization techniques from LLMs including DPO (Rafailov
et al. 2023) Wallace et al.] (2024) and GRPO (Shao et al. 2024 Liu et al. 2025a
Xue et al. 2025)
have since been adapted diffusion model.
METHODOLOGY
This section details our methodology for long video generation. We begin by revisiting the conver- sion of bidirectional models into streaming autoregressive generators (Yin et al., 2025 Huang et al.) 2025). Building upon this, we introduce our novel strategies tailored for long-form video synthesis. The complete generative process is formalized in Algorithm
3.1 BACKGROUND
Video diffusion models, while powerful, typically require denoising along a multi-step noise sched- ule, which renders the generation process computationally intensive. A prevalent strategy to mitigate this computational burden is to distill the foundational model into few-step generator. Prominent
approaches in this domain include Distribution Matching (DM) (Yin al. 2024h
Luo et al.
3

---

Published as a conference paper at ICLR 2026
2025b) and Consistency Models (CM) (Song et al., 2023 Wang al. 2024). Building upon the methodologies of Caus Vid and Self-Forcing, we distill the original bidirectional teacher model into few-step generator, then convert it into an autoregressive model. This conversion is accomplished by training a student model to replicate the Ordinary Differential Equation (ODE) trajectories sam- pled from the teacher. We refer this procedure for implementation details). The Self-Forcing method extends this approach by training the dis- tilled model on self-generated rollouts of up five seconds using techniques such Distribution Matching Distillation (DMD) loss (Yin et al. 2024a). While this technique effectively mitigates the over-exposure artifacts present in Caus Vid, exhibits a critical limitation: significant degradation in generative quality when producing sequences that exceed its constrained training horizon.
initialization stage (see appendix [A.5]
3.2 EXTEND TRAINING BEYOND TEACHER'S 'S LIMIT
Motivation As discussed earlier, the teacher model is trained exclusively on five-second video segments. Consequently, distillation-based methods such as CausVid (Yin et al. (025) and Self- Forcing (Huang et al. 2025) only enforce temporal window. This constrained training objective leads to precipitous decline in quality when generation extends beyond this five-second horizon. Despite this performance collapse, we make critical observation: videos rolled out beyond the training horizon often retain structural coherence, even if this coherence manifests as undesirable artifacts such as motion stagnation (a common failure mode in Self-Foreing). This suggests that the core problem not fundamental breakdown of the autoregressive mechanism, which correctly leverages the history KV cache to maintain context. Rather, the primary issue is the compounding of autoregressive errors during extended rollouts. These errors accumulate and eventually manifest as motion loss, scene freezing, and catastrophic degradation of visual fidelity. This insight motivates us to introduce a simple yet effective method to mitigate error accumulation, which is described in the following sections.
distribution alignment within this limited
Backwards Noise Initialization A central challenge in extending student-teacher distillation to long-horizon video generation resides in the noise initialization strategy. In the short-horizon set- ting (i.e., for videos with a length up to M frames), the student model can be directly supervised on complete trajectories sampled from the teacher, each originating from random noise. However, for long-horizon generation, a trajectory initialized from pure random noise is decoupled from the preceding video content, leading to a fundamental context misalignment since the sampled noise does not preserve the temporal dependencies of previously generated frames. Based on the obser- vation mentioned above, we add noise back to the denoised latent vectors and use as the starting noise which is also shown to boost the performance of distillation (Yin et al. (024a). While similar techniques of re-injecting noise have been employed in prior work (Yin et al. 2024a 2025 Huang et al.) 2025), our motivation and application are distinct. Whereas they used this for viirt distillation, primarily to enhance single-shot quality or circumvent the need for real training data. We leverage it as mechanism to enforce temporal consistency across long videos. Specifically, the student model is first rolled out to a sequence of N clean frames, with N >> M, where M denotes the maximum horizon the teacher can reliably generate such as 5 seconds. We then re-inject noise into the student roll-out according to the same diffusion noise schedule {o,)T=] Formally, given the clean trajectory (a: N generated by the student, the generation is perturbed as:
Fhitt = o,) + oeC, where T4,0 Fi,-- 0 ( 1),
and e (0, denotes Gaussian noise, and éo is the noise prediction network parameterized by 0 which serves as the initial state for computing the teacher and student distributions. This approach ensures that the distribution divergence between student and teacher model evaluated on trajectories that retain temporal consistency and correctly structured.
Extended Distribution Matching Distillation Our strategy for extending training to long videos is grounded in the observation that although the bidirectional teacher model is trained exclusively on short, five-second clips, it implicitly captures the underlying data distribution of the "world" from its training data. From this perspective, any short, contiguous video segment can be viewed sample from the marginal distribution of a valid, longer video sequence (Bruce et al., 2024 Alonso
et al. 2024 Li et al. 2025a). This intuition motivates our core methodological extension. Since baseline method Forcing (Huang et al. (025) restricts the training duration to the first M frames (typically ~ seconds), we instruct the student model to roll out to N frames where N > M. We
4

---

Published as a conference paper at ICLR 2026
CaueVil
Real Data
Sell Foriag
Self-tallu Rulling Cucha
Traske
Pood Traking Dambia
0V38 Sumoah lng 1080
TacherOndeer Block Crand
Befioedeoal Twccter Made gland Drodertkhneegeeeie Stedee Made DamuleelDedddn
Teacher Outjats
Teowe Corpett
Student Ougans
blesial Maden (Mtsts
fraring blage
inferonce Stage
"AQUAVV""
Treiinn
Twisngloker Lakeemo Vor
Figure 2: Workflow between baselines and Self-Forcing++. Our method employ backward noise initialization, extended DMD and rolling KV Cache to effectively mitigates train-test discrepancies.
then uniformly sample a contiguous window of length from the generated sequence, and compute the distributional discrepancy between the student and teacher models within this window, This sliding-window distillation process formalized as eq. (2):
VgL. DMD extended
= E,Ex
V KL (KKLLPp,... (I
- B, Battittn N-K -
dCu(zs)
dz
Here, Gz(z) denotes the student generator rollout given a latent 2, and ((-.t) is the transformation process timestep PỘ., and Pt represent the student and teacher distributions at time with corresponding scores 3 and s² We uniformly sample a starting index i Unif(O,..., NN K} from the student rollout of length N, and extract window of length K. The student is then trained to minimize the average KL divergence between its distribution and the teacher's distribution across this window. The window size K is typically chosen to match the horizon which the teacher model was originally trained to generate.
Remark Bi-directional diffusion can be seen as a process to gradually restore a degraded target in different denoising time-steps. Our method adapts the idea to autoregressive video generation regime by having short-horizon teacher gradually restore student's 's degraded rollouts different temporal time-frames and then distills these correction knowledge back into the student model.
Training with rolling KV Cache Despite using KV cache at inference time, Caus (Yin et al, 2025) still relies recomputing overlapping frames and suffers from severe over-exposure prob- lem. Self-Forcing (Huang et al., (025) attempts to address this but introduces train-inference mismatch by using fixed cache during training and rolling cache inference. Although this is partially mitigated by masking the first latent frame, the mismatch still leads to substantial error accumulation and temporal flickering in long videos (see fig. In contrast, our method naturally eliminates this mismatch by employing rolling KV cache during both training and inference. At training time, this cache is used to roll sequences far beyond the teacher's ss supervisory horizon to compute the extended DMD as detailed above. Consequently, our approach greatly simplifies the entire process, requiring neither the recomputation of overlapping frames nor latent frame masking.
3.3 IMPROVING LONG-TERM SMOOTHNESS VIA GRPO
A common drawback of generative models (iao et 2023 Luo et al. 2025a) employing sliding- window or sparse attention mechanisms for long sequences generation is the gradual loss long- term memory. This degradation often manifests temporal inconsistencies such objects abruptly
5

---

Published as a conference paper at ICLR 2026
emerging or vanishing unnaturally rapid scene transitions. Although the method proposed above has achieved strong results, we show that Group Relative Policy Optimization (GRPO), a reinforcement learning technique (Liu et al. 2025a Xue al. 2025). be utilized in autoregres- sive video generation framework when such phenomenon presents. The per step importance weight Tolas,i ss.)) where To ( (s.,)) denotes the policy function for output o; time step can Pt.j Teata (Rt. st.z)
be computed according to eq. ① and the overall generation probability can be computed as the sum of all the log probabilities in current autoregressive rollouts which show in appendix [A.6 To guide the optimization process towards temporally smooth outputs, we follow prior work (Chefer al. 2025 Nam al., 2025) and the relative magnitude of optical flow between consecutive frames as a proxy for motion continuity.
Algorithm Self-Forcing++ with Backward Noise Initialization (ours)
Require: Student Cg, teacher Tg. cache size L; rollout length >> 58; slice length (Ss); denoise steps (f1,...,fr]
1: loop
2:
V Rollout(Ge,
Pick ~ P (I,..., N- set W + VV:: Vi:itKK1]]
uniform slice
4:
Sample ~ (U,...,,t)})
5:
Backward noise initialization: (W( < BackwardNoisseni((W:
6: LDMD + DMD(GG( (W)
end loop
9:
< Oplicall[ow(Rewar((GG.)) +- CRPO.updatee00. It)
3.4 NEW METRICS FOR LONG VIDEOS EVALUATION
Most prior works rely on VBench (Huang et al. 2024 to assess image and aesthetic quality in long video generation. We find, however, that outdated evaluation models make the benchmark favor over-exposed videos (e.g., Caus Vid) and degraded long videos (e.g.. Self- Seff-Forcing). leading to inaccurate scores. To address this, we adopt Gemini-2.5-Pa Comanici al.
2025), a state-of-the
2025c Our protocol art video MLLM with strong reasoning ability (Chiang et al. 2024 Liu et al.
defines key long-video issues such over-exposure and error accumulation, prompts Gemini-2.5- Pro to rate videos along these axes, and aggregates the results onto a 0-100 scale termed visual stability for consistent comparison. More details are provided in fig. 3 and appendix A.8
Image Quality
Little Degradation
Normal exposure
77.13 Severe Degradation
69.03
59.05
64.01
67.29
53.64
43.03
50.88
Over Exposure
Figure 3: The left figure shows the issue of image score issue and the right figure shows the issue of aesthetic score regular and degraded images from earlier and later frames of the same video. VBench tends to overrate degraded and over-exposed frames rendering these two metrics unreliable.
4 EXPERIMENTAL RESULTS
4.1 SETTINGS
Baseline methods We include the following baseline methods such as NOVA (Deng et al. 2025)
Pyramid Flow (Jin et al. (025). SkyReels-V2-1.3B (Chen et al. 2025). MAGI-I-A 14.5B
(Sand-)
AI 2025) distilled to 16 steps for long video generation, Caus YYn et al. 2025) and Self-
Foreing (Huang al. 2025), both 1.3B distilled few-step generators similar to Additional two of bidirectional models --dee (HaCohen et al. 2024) and Wan2.1 (Wan et al. 2025) are included for references.
Evaluation metrics We conduct evaluations under two primary settings. The first setting follows the general VBench protocol (Huang et al. 2024), which measures generation quality on short videos

---

Published as a conference paper at ICLR 2026
of 5 seconds using 946 prompts across 16 dimensions. The second setting examines the model's capacity to extend generation up to 50/75/100 seconds with the same prompt set used in Caus Vid, consisting of 128 prompts from MovieGen (Polyak et al. 2024). Performance in this setting is assessed with both VBench Long and our proposed improved evaluation metric.
4.2 EMPIRICAL RESULTS LONG VIDEO GENERATION
Quantitative and qualitative results are presented in tables] [nd and figs. [and respectively. Our method achieves competitive performance in short-horizon generation and demonstrates substantial advantages as the generation horizon extends.
Short-Horizon (5s): Although not specifically trained for the initial seconds, our model performs comparably Self-Foreing on short clips, achieving strong overall results with semantic score of 80.37 and total score of 83.11 11, both surpassing the remaining baselines.
Table I:
Performance comparisons on 5s short videos and 50s long videos. Bascline methods achieve high temporal quality scores primarily due to stagnation reflected by their dynamic degree.
Model
Params
Thrughhut (FPS) f
Results on f
Results on 50s
Total Score
Quality Score
Semantic Score
Text Alignment
Temporal Quality
Dynamic Degree
Visual Stability
Framewise Quality
Ridirectional models
LTX. Video
1.9B
8.98
80.00
82.30
70.79
1.3B
Antoregressive models
NOVA
0.6B
0.88
80.12
80.39
79.05
24.58
86.33
31,96
45.94
34.45
Pyramid Flow
2B
6.7
81.72
84.74
69.62
MAGI-I
4.5B
0.19
79.18
82.04
67.74
26.04
88.34
28.49
51.25
54.20
Sky Reyleee--V2 V2
1,3B
0.49
82.67
84.70
74.53
23.73
88.78
39.15
60.41
54.13
Caus Vid
1.3B
17.0
82.46
83.61
77.84
25.25
89.34
37.35
40.47
61.56
Self Forcing
1.3B
17.0
83.00
83.71
80.14
24.77
88.17
34.35
40.12
61.06
Ours
1.3B
17.0
83.11
83.79
80.37
26.37
91.03
55.36
90.94
60.82
indicatio the cither generate videos specified lengih oulput collapses into (rumewise quality unrelialle for long videos, we include it here reference.
disccused scction
3.4
Long-Horizon (50s/75</100s): The superiority of method becomes more pronounced in long- horizon generation. We observe consistent improvements across key metrics. E.g. our model achieves aligmment score of 26.04 and dynamic degree of 54. 12 with 100-second video, outper- forming Casu Vid by 6.67% and 56.4% respectively which relies on recomputing overlapping frames and our baseline method Self-Foreing by 18.36% and 104.9% respectively as shown in fig. 4 This suggests that our approach effectively mitigates error accumulation during long rollouts.
Table 2: Performance comparisons on 75s and 100s long videos. Baseline methods achieve high temporal quality scores primarily due to stagnation or degrade pure noise.
Results on 100x 1
Model
Text Alignment
Temporal Quality
Dynamic Dogree
Visual Stability
Pramewise Quality
Text Aligument
Temporal Quality
Dynamic Degree
Visual Stability
Framewise Quality
Autoregressive models
NOVA
23.37
86.32
31.24
34.06
31.53
22.89
86.24
31.09
32.97
31.03
MAGI-I
24.95
87.89
24.82
43.28
$2.04
23.75
87.62
22.21
39.38
50.90
Reylsees-V2
22.70
88.99
39.89
55.47
51.55
22.05
88.80
38.75
56.72
50.48
CausVid
24.76
89.14
35.82
39.84
60.96
24.41
89.06
34.60
39.21
61.01
Self Forcing
23.39
87.79
29.15
35.00
60.02
22.00
87.39
26.41
32.03
58.25
91.00
55.62
In contrast, baseline methods exhibit significant degradation when generating long videos. Their pri- mary failure modes are: i) Motion Collapse: While maintaining short-term temporal structure, their videos frequently collapse into nearly static sequences, reflected by their low dynamic degree scores. Our method, however, sustains coherent motion throughout the entire sequence. ii) Fidelity Degradation: Baselines often suffer from exposure instability. For instance, Caus Vid trends towards ever-exposure, while Self-Forcing videos progressively darken. Our model maintains stable bright- ness and visual quality. This degradation in Self-Forcing is direct consequence of accumulated
7

---

Published as a conference paper at ICLR 2026
errors without explicit long-horizel training. While diffusion foreing methods show sporadic recovery from noise collapse such as SkyReels, the resulting content of low fidelity.
t Os
t=258
5
t=758
t=
100s
MAGI-I SkyReels
Ours Self Forcing Caus CausVid Vid
Figure 4: 100-second video generated for prompt "A vibrant tropical fish glides gracefully through colorful ocean reefs, surrounded by swaying coral...". Baseline methods usually suffer from error accumulation and over-exposure, causing severe quality degradation when generating long videos.
4.3
ABLATION STUDY
4.3.1
LENGTH OF ATTENTION WINDOW
straightforward way mitigate Self- Forcing's training-inference mismatch is to shorten the attention span during training, ex- posing the model more diverse cache states within a limited horizon. For instance, 5- second clip corresponds 21 latent frames; by reducing the attention window, the model is forced to slide attention multiple times. As shown in table 3 and visualized in Appendix fig. 7 smaller windows bring modest gains. For example, visual stability improves from 40.12 to 52.50 with window of latent frames. However, this comes the cost of increased inconsistency, since the model now relies on much less context compared to the original 21-frame history.
Table 3: Ablation study on various methods to re- duce error accumulation measured by visual sta- bility on 50s videos.
Causvid
Self-Forcing
Attn-15
Attn-12
Attn-9
Ours
40.47
40.12
44.69
42.19
52.50
90.94
4.3.2 THE IMPACT OF GRADIENT STEPS
Our method builds upon the same underlying backbone Self-Forcing (Huang al. 2025) which is a four-step distilled generation model. In the main experiments, we enable gradi- ents exclusively at the final denoising step. To further understand the impact of enabling gra- dient at different steps, we compare this setting with an alternative strategy that uniformly sam- ples the steps at which gradients are enabled similar to that of Self-Foreing (Huang al. 2025). We evaluate the models on 50-second generation and the results are shown in table We observe that enabling gradients over uni- formly sampled steps improves visual stability and yields higher framewise quality. However, this advantage comes with reduction in dy- namism, as the resulting videos exhibit notice- ably slower motion. This is likely due to that
Table Ablation study gradient step sampling.
Teat Alignment
Temporal Quality
Dynamic Degree
Visual Stability
Framewise Quality
62.48
Ours
26.37
91.03
$5.36
90.94
61.2
8

---

Published as a conference paper at ICLR 2026
restricting gradient computation to only the final step provides the student model with more flexibil- ity, allowing greater freedom. In contrast, enabling gradients at earlier steps may impose stronger alignment with the teacher model, inadvertently constraining the motion dynamics.
4.3.3 THE IMPACT OF WINDOW SAMPLING DISTRIBUTION
For our main method, we uniformly sample from the roll-out sequences. Here we also evalu- ate an alternative sampling strategy based on a Beta distribution. Specifically, we draw a value Be 2) and map the target range using the transformation where & and h denote the lower and upper bounds of the roll-out interval. This sampling scheme places more weight toward earlier steps in the sequence. Results comparing uni- form sampling with 2)) sampling on 50-second generation are provided in table can be seen that when focusing on shorter horizons, the model still achieves substantial improvement over baseline methods. However, it produces videos with noticeably reduced mo- tion, e.g, a dynamic degree of 45.66 compared 55.36 under uniform sampling. Although this setting yields a higher alignment score, it's more prone to stagnation, which in turn leads to
round + (h- f)),
Table 5: Ablation study on window sampling.
Text Alignment
Temperal Quality
Dynamic Degree
Visual Subiliit
Framewise Quality
Self Foecing
24.77
88.
34.35
40.12
61.06
Beta Sampling
26.65
90.14
45.06
86.25
60.83
Curs
26.37
91.03
55.36
90.94
60.82
greater eтror accumulation over long sequences.
4.3.4 THE IMPACT OF K/N RATIO
central limitation of Self-Foreing (Huang al.] 2025) is that its training horizon is restricted to 5 seconds, matching the teacher's supervision window. To better understand how extending this horizon affects downstream performance, we conduct an experiment that doubles the training dura- tion to 10 seconds which exceeds the teacher's horizon and forces rolling KV while avoiding pro- hibitive training cost. We evaluate the model on 50-second generation and show results in table 6
The darkening and temporal--fickering artifacts
Table 6: Ablation study on training horizon.
observed in fig. 4 are substantially reduced in
the early portions of the sequences, consis- tent with the improved text-alignment score. However, because the training horizon remains short, the model is still not exposed to suffi- ciently diverse failure modes for the teacher to correct. As result, the generations gradually stagnate toward near-static frames, leading to strong error accumulation over longer roll-outs.
Text Alignment
Temporal Quality
Dymmic Degres
Visual Stabiliiy
Framewisc Quality
Forcing 10s Horizon
25.36 26.37
88.78
35.91
50.78
59.50
Ours
91.03
55.36
90.91
60.82
4.3.5 THE EFFECT OF GRPO WITH OPTICAL- REWARD
Here we show its effectiveness for enhancing temporal consistency by examining the opti- cal flow magnitude, proxy for temporal sta- bility. As visualized in fig. 5 videos gen-
20.0
17.5
6.89
15.0
12.5
erated without GRPO may suffer from abrupt scene transitions. These transitions manifest as sharp spikes in the optical flow magni- tude, an artifact that is exacerbated by the rolling window mechanism used during infer- ence. By promoting smoother temporal transi- tions, our GRPO method effectively suppresses these spikes. This results in marked improve- ment in oongrange consistency and overall per- ceptual quality of the generated videos.
10.0
Optical
7.5
25
50
75
100
125
150
175
200
Frame Index
Figure 5: Comparison of video generation out- comes with and without GRPO. Variance is com- puted with window size of 8 frames.
4.4 TRAINING BUDGET SCALING
Finally, we investigate the effect of scaling the training budget on the model's long- video generation capabilities. We observe that increasing either the roll-out duration the total training

---

Published as a conference paper at ICLR 2026
iterations improves long-tern generation quality. Here, mainly scale up the number of training iterations with a 50-second training horizon, using a batch size of 8 on 8 H100 GPUs and the same hyperparameters described in appendix [A.2] As illustrated in fig. 6 our model, following ODE initialization, exhibits only a nascent ability to generate short, low-fidelity clips. We establish a baseline (1 budget) the training required to produce coherent 5-second video which is around 500 iterations. At this scale, extending generation leads to significant temporal flickering and error accumulation, failure mode similar that of Self-Forcing (Huang al. 2025). Increasing the budget to 4x enables the model to maintain semantic coherence over longer horizons, successfully rendering a consistent subject like the specified elephant. At 8x, the model begins to generate detailed backgrounds and more semantically accurate subjects, although motion dynamics remain limited and temporal quality degradation persists. further scaling 20x yields substantial improvement, producing high-fidelity videos that remain stable for over 50 seconds. Remarkably, at a 25x budget with around 12500 iteration, the model successfully generates a 255-second video with negligible quality loss. These findings indicate that scaling the training budget is viable path toward high-quality, long-duration video synthesis, circumventing the reliance on large-scale real video datasets, which are notoriously difficult to acquire.
Training Budget Staling
0-5s
Vidto Horton Scaling
2505
Forcing
DDE
Self Forcing++
Figure 6: Scaling phenomenon observed in 255-second generation for prompt: "A massive elephant walks slowly across a sunlit savannah, dust rising around its feet, the warm glow of sunset...""
5 CONCLUSION
We introduce Self-Forcing++, method that mitigates enror accumulation in autoregressive long- video generation. By leveraging a short-video teacher to guide the student on own self-generated long rollouts, our approach learns to correct errors without requiring long-video supervision. Ex- periments demonstrate that our method significantly extends video length to even over 4 minutes (a 50x improvement over the baseline) while maintaining high fidelity. We also propose new met- ric, Visual Stability, to address critical biases in existing long-video evaluation benchmarks. Our contributions pave the way for more robust and scalable long-via long-video synthesis.
LIMITATIONS AND FUTURE WORK
Our method, while effective, inherits certain limitations from its Self-Forcing foundation and the capacity of the underlying Wan2.1-T2V. 1.3B model. Key drawbacks include slower training speed compared to teacher-forcing and lack of long-term memory, which cause content divergence in regions occluded for extended periods. To address these challenges, we identify several promising future directions. First, tackle the high training cost of self-rollout, we will explore parallelizing the training process. Second, to further mitigate quality degradation over long sequences, we plan investigate techniques for controlling the fidelity of latent vectors. This includes quantizing latent representations stored in the KV cache, suggested by prior works (Zhang & Agrawala 2025), or normalizing the KV cache to prevent distributional shift. Finally, we aim to incorporate long-term memory mechanisms (Li et al. 2025b Liu et al. 2025d) into our autoregressive framework, which we believe is crucial for achieving true long-range temporal coherence.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
We sincerely thank the conference reviewers for their constructive feedback and helpful suggestions, and we also thank the authors of the works on which our paper builds.
ETHICS STATEMENT
Our work builds on the open-source Wan2.1-T2V-I. model, which publicly available and does not involve new dataset collection or release. No human subjects or private data are used, and we emphasize the importance of responsible use of generative models.
REPRODUCIBILITY STATEMENT
Due the simple design, our proposed method be reproduced by following the described steps in algorithm 1, Our training hyperparameter are also released in the appendix for reproducibility.
REFERENCES
Eloi Alonso, Adam Jelley, Vincent Micheli, Anssi Kanervisto, Amos Storkey, Tim Pearce, and François Fleuret. Diffusion for world modeling: Visual details matter atari. Advances in Neural Information Processing Systems, 37:58757-58777 2024.
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127 2023.
Jake Bruce, Michael D Dennis, Ashley Edwards, Jack Parker-Holder, Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, et al. Genie: Generative inter- active environments. In Forty-first International Conference on Machine Learning, 2024.
Hila Chefer, Uriel Singer. Amit Zohar, Yuval Kirstain, Adam Polyak, Yaniv Taigman, Lior Wolf, and Shelly Sheynin. Videojam: Joint appearance-mottonn representations for enhanced motion generation in video models. arXiv preprint arXiv:2502.02499, 2025.
Boyuan Chen, Diego Martí Monsó, Yilun Du, Max Simchowitz, Russ Tedrake, and Vincent Sitz- mann. Diffusion forcing: Next-token prediction meets full-sequence diffusion. Advances in Neural Information Processing Systems, 37:24081-241155, 2024.
Guibin Chen, Dixuan Lin, Jiangping Yang, Chunze Lin, Junchen Zhu, Mingyuan Fan, Hao Zhang, Sheng Chen, Zheng Chen, Chengcheng Ma. et al. Skyreels-v2: Infinite-length film generative model. arXiv preprint arXiv:2504.1 13074, 2025.
Shouyuan Chen, Sherman Wong, Liangjian Chen, and Yuandong Tian. Extending context window of large language models via positional interpolation. arXiv preprint arXiv:2306. 15595, 2023.
Wei-Lin Chiang, Anastasios Angelopoulos, L Zheng, Y Sheng, L Dunlap. Chou, T Li, E Frick, N Jain, D Li, et al. Chatbot arena. LMArena, https://hmarena. ai, 2024.
Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capa- bilities. arXiv preprint arXiv:2507.0626) 2025.
Haoge Deng, Ting Pan, Haiwen Diao, Zhengxiong Luo, Yufeng Cui, Huchuan Lu, Shiguang Shan, Yonggang Qi, and Xinlong Wang. Autoregressive video generation without vector quantization. arXiv preprint arXiv:2412.1416 2024.
Haoge Deng, Ting Pan, Haiwen Diao, Zhengxiong Luo, Yufeng Cui, Huchuan Lu, Shiguang Shan, Yonggang Qi, and Xinlong Wang. Autoregressive video generation without vector quantization. In ICLR, 2025.
11

---

Published as a conference paper at ICLR 2026
Xueji Fang, Liyuan Ma, Zhiyang Chen, Mingyuan Zhou, and Guo-jun Qi. Infivg: Reinforce inference-time consistent long video generation with grpo arXiv preprint arXiv:2505.17574, 2025.
Google Deep.indd Veo.
2025. Accessed:
2025-09-09.
Yoav HaCohen, Nisan Chiprut, Benny Brazowski, Daniel Shalem, Dudu Moshe, Eitan Richardson, Eran Levin, Guy Shiran, Nir Zabari, Ori Gordon, et al. Lxx-video: Realtime video latent diffusion. arXiv preprint arXiv:2501.00100 2024.
Xianglong He, Chunli Peng. Zexiang Liu, Boyang Wang, Yifan Zhang, Qi Cui, Fei Kang, Biao Jiang, Mengyin An, Yangyang Ren, et al. Matrix-game 2.0: An open-source, real-time, and streaming interactive world model. arXiv preprint arXiv:2508. /3009, 2025.
Yingqing He, Tianyu Yang, Yong Zhang, Ying Shan, and Qifeng Chen. Latent video diffusion models for high-fidelity long video generation. arXiv preprint arXiv:2211.1322. 2022.
Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al. Imagen video: High definition video generation with diffusion models. arXiv preprint arXiv:2210.023003 2022a.
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David Fleet. Video diffusion models. Advances in neural information processing systems, 35:8633- 8646, 2022b.
Wenyi Hong Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. Cogvideo: Large-scale pre- training for text-to-video generation via transformers. arXiv preprint 1xXi:22205.18860 2022.
Zhenyu Hou, Xin Lv, Rui Lu, Jiajie Zhang, Yujiang Li, Zijun Yao, Juanzi Li, Jie Tang, and Yux- iao Dong. Advancing language model reasoning through reinforcement learning and inference scaling, arXiv preprint S..V::2501.116511 2025.
Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 12):3, 2022.
Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman. Self foreing: Bridging the train-test gap in autoregressive video diffusion. arXiv preprint arXiv:25066.080 2025.
Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Yuming Jiang, Yuanhan Zhang, Tianx- ing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive benchmark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 21807-21818, 2024.
Jiaxiu Jiang, Wenbo Li, Jingjing Ren, Yuping Qiu, Yong Guo, Xiaogang Xu, Han Wu, and Wang- meng Zuo. Lovic: Efficient long video generation with context compression. arXiv preprint
1rX.v:2507.12952, 2025.
Yang Jin, Zhicheng Sun, Ningyuan Li, Kun Xu, Hao Jiang, Nan Zhuang, Quzhe Huang, Yang Song, Yadong Mu, and Zhouchen Lin. Pyramidal flow matching for efficient video generative modeling. In ICLR, 2025.
Jihwan Kim, Junoh Kang, Jinyoung Choi, and Bohyung Han. Fifo-diffusion: Generating infinite videos from text without training. Advances in Neural Information Processing Systems, 37: 19834-89868, 2024.
Diederik P Kingma and Max Welling, arXiv:1312.6114 2013.
Auto-encoding variational bayes.
arXiv preprint
Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. Pick- a-pic: An open dataset of user preferences for text-to-image generation. Advances neural information processing systems, 36:36652-36663, 2023.
12

---

Published as a conference paper at ICLR 2026
Akio Kodaira, Tingbo Hou, Ji Hou, Masayoshi Tomizuka, and Yue Zhao. Streamdit: Real-time streaming text-to-video generation. arXiv preprint arXiv:2507.03745, 2025.
Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603, 2024.
Aviral Kumar, Vincent Zhuang, Rishabh Agarwal, Su, John Co-Reyes, Avi Singh, Kate Baumli, Shariq lqbal, Colton Bishop, Rebecca Roelofs, al. Training language models to self-correct via reinforcement learning. arXiv preprint arXiv:2409.12911 12917, 2024.
Jiaqi Li, Junshu Tang, Zhiyong Xu, Longhuang Wu, Yuan Zhou, Shuai Shao, Tianbao Yu, Zhiguo High-dynamic interactive game video generation with
Cao, and Qinglin Lu. hybrid history condition. arXiv preprint arXiv:2506. 17201, 2025a.
Jiaqi Li, Junshu Tang, Zhiyong Xu, Longhuang Wu, Yuan Zhou, Shuai Shao, Tianbao Yu, Zhiguo Cao, and Qinglin Lu. Mlunuaan-aamecree High-dynamic interactive game video generation with hybrid history condition, 2025b. URL //t.///tpss///aarrrr.rr////////
/tc///axxiie arxiv. wttg//r////b.......
17201
Bin Lin, Yunyang Ge, Xinhua Cheng, Zongjian Li, Bin Zhu. Shaodong Wang, Xianyi He, Yang Yc, Shenghai Yuan, Liuhan Chen, et al. Open-sora plan: Open-source large video generation model. arXiv preprint 11 2024.
Shanchuan Lin, Ceyuan Yang, Hao He, Jianwen Jiang, Yuxi Ren, Xin Xia, Yang Zhao, Xuefeng Xiao, and Lu Jiang. Autoregressive adversarial post-training for real-time interactive video gen- eration. arXiv preprint ar 0rxiv:2506.09335 2025.
Jie Liu, Gongye Liu, Jiajun Liang, Yangguang Li, Jiaheng Liu, Xintao Wang, Pengfei Wan, Di Zhang, and Wanli Ouyang. Flow-grpo: Training flow matching models via online rl. arXiv preprint arXiv:2505.054: 2025a.
Jie Liu, Gongye Liu, Jiajun Liang, Ziyang Yuan, Xiaokun Liu, Mingwu Zheng, Xiele Wu, Qiulin Wang, Wenyu Qin, Menghan Xia, et al. Improving video generation with human feedback. arXiv preprint arXiv:2501.139/1 2025b.
Yuanxin Liu, Kun Ouyang, Haoning Wu, Yi Liu, Lin Sui, Xinhao Li, Yan Zhong, Y Charles, Xinyu Zhou, and Xu Sun. Videoreasonbeechh: Can milms perform vision-centric complex video reason- ing? arXiv preprint arXiv:2505.23335 2025c.
Zhiheng Liu, Xueqing Deng. Shoufa Chen, Angtian Wang. Qiushan Guo, Mingfei Han, Zeyue Xue, Mengzhao Chen, Ping Luo, and Linjie Yang. Worldweaver: Generating long-horizon video worlds via rich perception. arXiv preprint arXiv:2508. 15720, 2025d.
Yang Luo, Xuanlei Zhao, Mengzhao Chen, Kaipeng Zhang, Wenqi Shao, Kai Wang, Zhangyang Wang, and Yang You. Enhance-a-video: Better generated video for free. arXiv preprint arXiv:2502.07508. 2025a.
Yihong Luo, Tianyang Hu, Jiacheng Sun, Yujun Cai, and Jing Tang. Learning few-step diffusion models by trajectory distribution matching. arXiv preprint arXiv:2503.06671 06674, 2025b.
Hyelin Nam, Jaemin Kim, Dohun Lee, and Jong Chul Ye. Optical-flow guided prompt optimization for coherent video generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 7837-7846, 2025.
OpenAl. port, OpenAl, February 2024.
Video generation models
world
simulators.
Technical
URL sttts:://ppeaat www.cooiiiinioe /nne//
Ludde--generation--oo www.-s--mommdeaaaa-------------- wtw.llddddimmaarssssoooooo Technical repport..rrpprr.....o.
Long Ouyang, Jeffrey Wu, Xu Jiang. Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to fol- low instructions with human feedback. Advances in neural information processing systems, 35: 27730-27744, 2022.
13

---

Published as a conference paper at ICLR 2026
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the JEEE/CVF international conference on computer vision, pp.
2023.
Bowen Peng, Jeffrey Quesnelle, Honglu Fan, and Enrico Shippole. Yarn: Efficient context window extension of large language models. arXiv preprint arXiv:2309.0007 0000,, 2023.
Xiangyu Peng, Zangwei Zheng, Chenhui Shen, Tom Young, Xinying Guo, Binluo Wang. Hang Xu, Hongxin Liu, Mingyan Jiang, Wenjun Li, et al. Open-sora 2.0: Training a commercial-level video 2025.
generation model in 200k. arXiv preprint arXiv:2503.0964
Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apporv Vyas, Bowen Shi, Chih- Yao Ma. Ching- Yao Chuang, et al. Movie gen: A cast of media foundation models. arXiv preprint arXiv:2410. 13720, 2024.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 36:53728-53741, 2023.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedi- cal image segmentation. In International Conference on Medical image computing and computer- assisted intervention. pp. 231-241. Springer, 2015.
Sand-AL. Magi-l: Autoregressivve video generation scale, 2025. URL (RLLittps://staiic.mmg worlld/statii//iiees//AA pdf
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300 2024.
Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. 2023.
Hansi Teng, Hongyu Jia, Lei Sun, Lingzhi Li, Maolin Li, Mingqiu Tang, Shuai Han, Tianning Zhang, WQ Zhang, Weifeng Luo, et al. Magi-1: Autoregressive video generation at scale. arXiv preprint arXiv:2505.13211, 2025.
Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model alignment using direct preference optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8228-8238, 2024.
Team Wan, Ang Wang, Baole Ai. Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang. Jianyuan Zeng, Jiayu Wang, Jingfeng Zhang, Jingren Zhou, Jinkai Wang, Jixuan Chen, Kai Zhu, Kang Zhao, Keyu Yan, Lianghua Huang. Mengyang Feng, Ningyi Zhang, Pandeng Li, Pingyu Wu, Ruihang Chu, Ruili Feng. Shiwei Zhang, Siyang Sun, Tao Fang, Tianxing Wang, Tianyi Gui, Tingyu Weng, Tong Shen, Wei Lin, Wei Wang, Wei Wang. Wenmeng Zhou, Wente Wang, Wenting Shen, Wenyuan Yu, Xianzhong Shi, Xiaoming Huang, Xin Xu, Yan Kou, Yangyu Lv, Yifei Li, Yijing Liu, Yiming Wang, Yingya Zhang, Yitong Huang. Yong Li, You Wu, Yu Liu, Yulin Pan, Yun Zheng, Yuntao Hong, Yupeng Shi, Yutong Feng, Zeyinzi Jiang, Zhen Han, Zhi-Fan Wu, and Ziyu Liu. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.22 203 14, 2025.
Fu-Yu Yun Wang, Zhaoyang Huang, Alexander Bergman, Dazhong Shen, Peng Gao, Michael Lingelbach, Keqiang Sun, Weikang Bian, Guanglu Song, Yu Liu, et al. Phased consistency models. Advances in neural information processing systems, 37:83951-84009 2024.
Wenhao Wang and Yi Yang. Vidprom: A million-scale real prompt-gallery dataset for text-to-vides diffusion models. Advances in Neural Information Processing Systems, 11665618-65644 2024.
Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis. arXiv preprint Xrxiv:2306. 00341, 2023.
Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming lan- guage models with attention sinks. arXiv preprint arXiv:2309. 17453, 2023.
14

---

Published as a conference paper at ICLR 2026
Zhihui Xie, Liyu Chen, Weichao Mao, Jingjing Xu, Lingpeng Kong, et al. Teaching language models to critique via reinforcement learning. arXiv preprint arXiv:2502.03492, 2025.
Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: learning and evaluating human preferences for text-to-image generation. In Proceed- ings of the 37th International Conference on Neural Information Processing Systems, pp. 15903- 15935. 2023.
Jaazheng Xu, Yu Huang, Jiale Cheng. Yuanming Yang, Jiajun Xu, Yuan Wang. Wenbo Duan, Shen Yang. Qunlin Jin, Shurun Li, Jiayan Teng, Zhuoyi Yang. Wendi Zheng, Xiao Liu, Ming Ding, Xi- aohan Zhang, Xiaotao Gu, Shiyu Huang, Minlie Huang, Jie Tang, and Yuxiao Dong. Visionreward: Fine-grained muulti-dimensional human preference learning for image and video generation, 2024.
URL http:::tp//...... //arxii///xxvvv arxiv. org/ haa//abs//aa 2412. 21059
Zeyue Xue, Jie Wu, Yu Gao, Fangyuan Kong, Lingting Zhu, Mengzhao Chen, Zhiheng Liu, Wei Liu, Qiushan Guo, Weilin Huang, et. Dancegrpo: Unleashing grpo on visual generation. arXiv preprint arXiv:2505.07816 2025.
Zhuoyi Yang. Jiayan Teng. Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408. arXiv:2408.06007, 06072, 2024.
Tianwei Yin, Michaël Gharbi, Taesung Park, Richard Zhang, Eli Shechtman, Fredo Durand, and Bill Freeman. Improved distribution matching distillation for fast image synthesis. Advances in neural information processing systems, 37:47455-47487, 2024a.
Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Fredo Durand, William T Freeman, and Taesung Park. One-step diffusion with distribution matching distillation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 6613-66222 2024b.
Tianwei Yin, Qiang Zhang, Richard Zhang, William T Freeman, Fredo Durand, Eli Shechtman, and Xun Huang. From slow bidirectional to fast autoregressive video diffusion models. In CVPR, 2025.
Lymin Zhang and Maneesh Agrawala. Packing input frame context in next-frame prediction models for video generation. arXiv preprint arXiv:2504. /2626, 2025.
Min Zhao, Guande He, Yixiao Chen, Hongzhou Zhu, Chongxuan Li, and Jun Zhu. Riflex: A free lunch for length extrapolation in video diffusion transformers. arXiv preprint arXiv:2502.15880 2025.
Le Zhuo, Ruoyi Du, Han Xiao, Yangguang Li, Dongyang Liu, Rongjie Huang, Wenze Liu, Xiangyang Zhu, Fu-Yu Yun Wang, Zhanyu Ma, et al. Lumina-next: Making lumina-t2x stronger and faster with ditt-dit, Advances in Neural Information Processing Systems, 37:131278-1311 2024.
15

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 THE USE OF LARGE LANGUAGE MODELS (LLMS)
Large Language Models (LLMs) such as GPT were used solely for language polishing and clarity improvements in the writing of this paper. All technical content, dataset design, experimental results, and analyses were created by the authors. The models were not used to generate ideas, data, or experimental outcomes.
A.2 DETAILED EVALUATION RESULTS FOR ALL DIMENSIONS
Due to limited space, only report VBench aggregated metrics in tables [and the main text. Here we show the full evaluated data for each dimension in table
Table Comparison of models across multiple quality metrics on 50s, 75s, and 100s videos for our main results. The gray metrics are aggregated metrics by VBench Long.
lest alignment
overall consistency
clip score
temporal quiiity
subject consistency
background consistency
motion smoochness
dynamic degree
Irame-wise quality
aesthetic quality
imaging quality
50) seroonds
MAGE-1
26.04
21.55
30.53
88.34
98. 08
97.50
99.36
28.40
54.20
52.15
56.26
SkyReels V2
23.73
18.94
28.53
88.78
96.06
96.43
98.67
39.15
54.13
50.44
57.82
Caus
25.25
20.71
24.80
89.34
98.
97.27
98.45
37.35
61.56
57,91
65.21
26.37
22.03
30.71
91.03
17
98.
67.87
75 Anconnss
NOWA
23.37
19.19
21.54
86.32
91. 78
95.55
99.
31.24
31.53
37.34
25.73
MAGE-1
24.95
20.36
29.53
87.89
98.
93.70
99.29
24.82
52.04
54.71
SkyRoels
Cau Vid
21.76
19.99
29.53
89.11
98.
97.
35.82
60.96
56.87
65.06
Sell- FalrFForing
23.39
18.85
933
87.79
97 47
96. 50
96.77
29.15
60.02
$3.28
66.77
Ours
26.31
22.09
30.53
91.00
96
95.45
98.29
55.62
53.38
67.97
700
18.63
99.13
MAGI-1
23.75
18:94
28.57
87.62
98.
97. 99
99.20
22.21
50.90
47.25
$4.55
Sky Rodls-V V2
22.45
16.84
27.25
88
96
96. 52
98% 86
38.75
50:48
46.33
54.62
CineVid
24.41
19.61
97.46
57.22
64.79
Self.Forzing
17.16
Ours
26.04
21.75
30.34
90.87
97.09
95.53
98.35
$4.12
60.66
53.00
68.31
IMPLEMENTATION DETAILS
We adopt the same base model Wan2.1-T2V-1133 (Wan et al., 2025) as Causvid and Self-Forcing, which later converted into an autoregressive model as describe above. The model initialized with sampled 16K ODE training trajectories by optimizing the loss in eq. (4). We use the same filtered and LLM-extended version of VidProM (Wang Yang 2024) as Sell-Forcing for training. In the training phase, since utilize backward noise initialization, thus don't need real data for training. We utilize the same 1-n2.1-T2V-1- as the teacher model.
TRAINING DETAILS
We use 8 H100 GPUs each with 80GB GPU memory during training with a training batch size of 8 and training length 100 seconds for total of 48 H100 GPU days for full training. The cost could be further reduced when training with LoRA (Hu et al. 2022). For inference, it takes around 50 seconds to generate a 50-second video which occupies around 22GB of memory on one H100 GPU with 80GB memory. The hyperparameters are mostly adopted from Self-Forcing such as the denoising steps of 1000,750,500 and 250 with a generator learning rate of 2e -6 and critic learning rate of -7 The generator and critic update ratio 5. AdamW optimizer is used for both generator and critic both with 3 = 0 and 32 0.999. Our rolling KV cache window size is 21 latent frames in all cases except ablation study. The model is updated with EMA starting at 200 epochs. We have also inspected the version without EMA which can also generate long high quality videos but the EMA version performs better. Our method videos such as videos up to 4minute 15 seconds before GPRO, the ablation study, we show that it's possible to further boost the model's performance with properly designed rewards.
already generate consistent high quality long
16

---

Published as a conference paper at ICLR 2026
A.3 ADDING NOISE TO CONTEXT WINDOW
As demonstrated in table T methods such MAGI-1 (Sand-Al 2025) and SkyReels-V2 (Chen itt al. 2025). which rely on variable noisy context injection following a predefined schedule Chen al. 2024p. are insufficient to mitigate error accumulation when rolling long videos. To further investigate its effect on methods, we conduct an experiment where noise is manually injected into the KV cache explicitly simulate the effect of accumulated errors over extended sequences. Specifi- cally, before adding any query or key to the KV cache, we inject random Gaussian noise into them. While this strategy yields a slight improvement in both image quality and visual stability compare to the original Self-Forcing, it nonetheless fails to prevent substantial degradation in long-horizon video generation which can be seen in fig. 7
t=ss
100
1=20s
t=30s
1=50s
Atin-9
Attn- 12
Noisy-KV Attn-
Figure 7: Ablation study for various methods of mitigating error accumulation. Here is one visual- ization of generated 50-second video for prompt: "Drone view of waves crashing against the rugged cliffs along Big Sur's garay point beach..."
A.4 MORE RELATED WORKS
Video Diffusion Models Video diffusion models have advanced rapidly, beginning with UNet (Ronneberger et al., 2015) based approaches that extended image diffusion backbones into the temporal domain (Ho et al.) 2022a He al. 2022 Blattmann et al. (202).. These early mod- els enabled short-form video generation but faced limitations in scalability. The introduction of the Diffusion Transformer (DiT) & 2023 represented a turning point by replacing convolu- tional hierarchies with transformer blocks, which allowed models to capture global ppatio-temporal dependencies more effectively and to scale with larger datasets and computational resources. This shift led to
wave of architectures, such as Spra(OpenAl] 2024). which produces realistic, coherent videos with strong temporal consistency and diverse motion, and Hunyuan Video (Kong al. 2024), which employs causal 3D VAE (Kingma & Welling 2013) for spatio-temporal ken compression in latent space combined with a large language model for text conditioning. Wan 2.1 (Wan al.] 2025) further demonstrates the benefits of massive pretraining for righlresolution video generation. Cog VideoX (Hong et al. 2022; Yang et al.) 2024) introduces an expert trans- former with adaptive LayerNorm to enhance cross-modal fusion, supported by 3D VAE, pro- gressive training, and rulti-resolution frame packing, thereby achieving strong text alignment and motion coherence. Open-Sora (Lin et al. 2024 Peng et al. (025) and Open-Sora-Plan (Lin et al.) 2024) extend these advancements in the open-source community, delivering high-quality video gen- eration and significantly accelerating progress efficiency and realism. Collectively, these works illustrate how scaling strategies and architectural innovations have transformed video diffusion from modest UNet adaptations into transformer-drivee models capable of generating controllable and high-quality videos.
17

---

Published as a conference paper at ICLR 2026
architec- art models remain limited to generating videos of 5-10 seconds. To over- come this constraint, number of techniques have been introduced extend generation to longer durations (Jiang et al. |025] Kodaira et al. 2025 Fang et al. 2025: Lin et al. 2025). RIFLEx (Zhao et al.) 2025) is training-frees approach that revisits positional encoding. effectively doubling the gen- eration length by avoiding encodings that induce repetitive motion, and surpassing prior methods by large margin (Chen al., 2023, Peng et al. 2023 Zhuo al. (024). Another promising direc- tion is autoregressive video generation. Nova (Deng et al. (024) reformulates video synthesis as non-quantized autoregressive problem, jointly modeling temporal frame-by-frae b--mmameeemmm prediction and spatial set-by-set prediction, which enables flexible in-context learning. Pyramid-Flow (Jin et al.) 2025) interprets denoising as hierarchical process across multi-stage pyramids, linking flows across resolutions and time to support end-to-end autoregressive video generation with single diffusion transformer. SkyReels- V2 adopts diffusion forcing (Chen et al. 2024) to support potentially infinite rollouts, while MAGI-1 (Teng al.) 2025) trains model to progressively denoise per-chunk noise that increases over time, autoregressively predicting fixed-length segments of consecutive frames. Caus Vid (Yin et al. 2025) employs block causal attention and KV cache autoregressively extend sequences, and Self-Forcing (Huang et al. 2025) further aligns training with inference by incorpo- rating the KV cache directly during training, producing high-quality short videos.
Long Video Generation Due to the substantial training and inference cost of
tures, most
Reinforcement Learning Reinforcement learning has become central component in the post- training of large language models (Ouyang et al., 2022 Kumar et al.] 2024 Hou et al.) 2025 Xie al. 2025). With the rise of image generation, has also proven effective for improving generative models, with early efforts introducing reward models tailored to images, such as ImageReward (Xu
al. 2023). Pick-a-Pic (Kirstain al., 2023). and HPS V2 (Wu et al.. 2023). These concepts have since been extended to video through reward functions like VideoReward (Liu et al.) (025b) and VisionReward (Xu et al., 2024), which assess temporal coherence and motion quality. Building on these reward signals, optimization techniques first developed for language models, including Di- rect Preference Optimization (DPO))Rafailovv et al.] 2023) and Group Relative Policy Optimization (GRPO) Shao et al., 2024). have been adapted to diffusion-basee generation. Notably, Diffusion- training directly to diffusion models, while Flow-GRPO (Liu et al. 2025a Xue al., 2025) leverages GRPO to fine-tune video diffusion mod- els, resulting in improvements in both visual fidelity and motion consistency.
DPO (Wallace et al. 2024) applies preference-base
A.5 MORE BACKGROUND
Video diffusion models are typically trained to denoise along fixed noise schedule, which makes video generation computationally expensive. A common strategy to reduce this cost is to distill the model into few-step generator, using approaches such Distribution Matching (Yin et al. 2024b a Luo et al. 2025b) and Consistency Model (Song et al. 2023 Wang el al. (024). In line with Caus Vid and Self-Forcing. we also adopt DMD to distill the original bidirectional model into few-step model, which can be viewed as minimizing the reverse KL divergence between the student and teacher models, as formulated eq.
VoLDMD = F. Vo (T Preat.c)
= = Ee S (((o)((((G)()))) - Stake (oa))
dGo(2) d0
(3)
dz
After the model is distilled into few-step form, it converted into an autoregressive model by introducing causal attention. The conversion is carried out by sampling ODE trajectories from the teacher and training the autoregressive model on these trajectories. This stage functions as warm- up phase, distinct from the main training procedure described below. The ODE training process is formally expressed in eq. 4).
Lode = Ex,
| (1x MEn (42))
(+)
teacher [i=l
1.
(4)
18

---

Published as a conference paper at ICLR 2026
A.6 IMPROVING LONG-TERN SMOOTHNESS VIA GRPO
Following the discussion in the main text, here we show the general form of GRPO which can be written as:
G
T
J(0) (o:) m r (11))
Bo (1p))))
Σ
min mninppr.A, clip(pr. 1 (S)
G
where Pt, Aeaaa (9c.s #t.s) is the importance weight, To (a(,),)5)).)) denotes the policy function for
xo(or. i st.s)
output o; at time step whose value can be computed according to eq. ( e is clipping hyper- parameter, and A, the advantage computed across generation group. The advantage is computed across a group of G outputs as:
Ti r2, sid((r), 72, r2....ra))
A
(6)
In order to generate adopt our model for GRPO, we inject Gaussian noise at each non-terminal step according to the noise scheduler. The probability of the final generated video can be formulated as below. log logp(E11.)
log p(In Ien)
T Σ
[ - (1 σr) )()))) t.
(n)
ΣΣ
logo, 12g(
20
where (m) is computed following eq. conditioned the previously generated samples Z<n- D denotes the latent dimension size, T is the number non-terminal sampling steps, and N is the total number of autoregressive steps.
A.7 TEMPORAL REPETITION
As highlighted in RIFLEx Zhao et al. 2025), one of the primary challenges in long video genera- tion temporal repetition, where videos begin to cycle with fixed, recurring patterns. In contrast, autoregressive approaches such as Self-Forcing (Huang et al.) (225)) and our method, which rely exclusively on the KV cache to produce new frames, are less prone to this failure mode. To quantify this, we adopt the NoRepeat Score introduced in RIFLEx and report the results in table
From table g we observe that NOVA, MAGI- I and Cius are more susceptible re- peated temporal patterns when extended to long videos. In contrast, methods that generate solely from the KV cache such as Self-Forcing and ours achieve stronger resistance to tempo- ral repetition without requiring recomputation overlapping frames.
Table 8: NoRepeat scores (f) across different methods, computed following RIFLEx. The RI- FLEx score reported here corresponds its best published result and serves only as reference.
RIFLEX
Nova
MAGE-1
SkyReels V2
Caus Vid
Self Forcing
Our
39.0
67.19
75:44
95.31
92.97
1000
98.44
A.8 EVALUATION WITH GEMINI- 2.5-- AND MANUALLY VERIFICATION
We present several representative results with Gemini-2.55PPr Pro on 50-second videos generated by our method and by baseline methods, as shown figs. and None of the baselines sustain high quality at this length, and each displays distinct failure modes. CiusVid consistently shows pronounced over-exposure even within its trained 5-second horizon, which worsens as the video progresses until motion collapses entirely. Self-Forcing suffers from severe error accumulation. leading to global darkening and stagnation. MAGI-I initially avoids over-exposure, likely due to its reliance diffusion forcing, but rapidly deteriorates into heavy over-exposure and structural collapse. SkyReels-V2 V2, as seen in fig. 9 generally preserves structure but exhibits moderate to severe over-exposure, resembling Caus Vid's failure pattern.
19

---

Published as a conference paper at ICLR 2026
CausVid
Self Forcing
Ours
Roting: Noticoable Exposure Problems
Rating: 2 Naticoable Exposure Problems
Rating: 5/5 Well- Well-Exposed
Reason: video consistently displays primarily with high contrast that toads to a loss of detail.
The large celestial body in the background significantly overoxpased, with large arees blown out to pure white, ceusing complete loss surface detail. astronaut's suit also very bright and appear clipped.
Reason:
The video well exposed, demonstrating excalient handling of high- contrast scone. The deop blacks of the space background and the harsh shadows on the luner surf face are appropriste for the setting and do not represent crushed blacks or detail. throughout the tonal range
Reason:
significant exposure
While the scame remains perfectly understandable,
highlights
"clear persistant clippingin highlights or shadows" affect mejor portions of the frams (the main subject and its shadow], which aligns perfectly with the definition for a rating 2f
These exposure issues are not fletiing: they constant throughout the entire video.
preeereeervedd
Figure 8: Example evaluation using Gemini-2.5-pii the results generated by method, Caus Vid and Self-Forcing for prompt "An astronaut runs on the surface of the moon, the low angle shot shows the vast background of the moon, the movement is smooth and appears lightweight.". Gemini-2.5-Prr is tasked to rate the whole video with thinking and output reasoning first before outputting the final rating.
SkyReelsV2
MAGI-1
Ours
Rating:1 - Severe Exposure Failure
Rating: 1 Severe Exposure F Failure
Rating: 5/5 Well-Exposed
While the begins reesonably well-en exl-sexpoeed image (0:00-0:09), it undergoes artetic transition where the exposure repidly deliberately approwim mately the 15 secennd mark onvard, aree portions foreground coral anemones become completaly blown out to white. This expossuuure dominates majority of the video's is runtime, severely distorting visibility losing detailin arees.
Reason: The video begine well-exposed for first 8-10 seconds, showing clear and balanced underwater scene. However, after approxim attely ly the 15 I--second mark, the image quality dagradas catestrophically, The latter two thirds of the video is domineted by waar-exposure
video maintains balanced and censietent exposure throughout. There noticesble areas where highighhs to pure white
shedows are crushed to pure black. All the details colorful coral, backgg are cllerdy sisibie, resulting in clean and well. presented image.
the frame become a flat, blown out cyan.
lower the
Figure Example evaluation using Gemini-2.5-pro on the results generated by our method, Caus Vid and Self-Forcing for prompt "A gorgeously rendered papereraft world of coral reef, rife with colorful fish and sea creatures.". "". Gemini-2.5-Prrs is tasked to rate the whole video with thinking and output reasoning first before outputting the final rating.
As further illustrated in and and on our project page peef--forrin--p.iihuuse all baselines demonstrate systematic breakdown long-vider generation that state-of-the-an MLLMs readily detect. To ensure alignment with human judgment, we conducted manual verification: 20 ran- domly sampled MovieGen videos were independently annotated by two authors, and the averaged For 50-second sequences, Spearman's rank correlation
scores were compared with Gemini-2.5-PP5 reached 100% for the top three methods and 94.2% across all six baselines. Similar results are ob- served for the 75-second and 100-second videos, where the generation quality of bascline methods further declines.
Overall, our method achieves sustained long-term visual stability. Methods trained with diffusion forcing, such as SkyReels- V2 and MAGI-1, rank next, followed by Caus Vid, which maintains struc- ture only under severe exposure. Both Self-Forcing and NOVA degrade to comparable low levels.
The prompt use for evaluating is as following:
20

---

Published as a conference paper at ICLR 2026
You tasked with rating the exposure stability of video. Assign according the following scale:
Catastrophic Exposure. Nearly the entire frame is either blown out (pure white) or crushed (pure black), rendering the scene unreadable.
Severe Exposure Failure. Large portions the frame are dominated by over-exposure or under-exposure, substantially impairing visibility.
2: Noticeable Exposure Problems. Persistent clipping is present in highlights or shadows. Significant areas lose detail, though the frame remains viewable.
Moderate Exposure Issues. Over-exposed highlights or under-exposed shadows occur but are limited in extent duration.
Minor Exposure Flaws. Small regions are occasionally too bright or dark, but these do not meaningfully disrupt overall visibility.
5: Well-Exposed. Balanced lighting across the frame. No distracting over-exposure or darkening; both highlights and shadows retain detail.
Do not claim that the observations in any video are of specific artistic style or scene transitions unless the prompt explicitly states The prompt for generating the video is as follows:
First, provide a brief explanation of your reasoning, describing the observed exposure char- acteristics. Then, state your final score according the scale.
A.9 DISCUSSION OF DIFFUSION FORCING VS AUTOREGRESSIVE
Both diffusion forcing such as SkyReels and MAGI and autoregressive models with clean context such Self-Forcing and ours generate long videos. Diffusion forcing works by keeping large number of frames in the current stage and apply different noise level for different frames. Thus, naturally comes with better long cost of training instability as the number of different noise level combinations can be extremely huge due the nature of diffusion models which needs multiple step denoising. Thus methods such as SireamDiT (Kodaira et al. 2025) has opted to distill the model first to limit the number of combina- tions which reduces the tranining instability. However, and 7 that a context with variable noises it not absolutely required to achieve long horizon genera- tion with little quality degradation. As shown in our ablation study, the model gradually learns to generate long videos with increased training budget even without using long video training set. We hope our work can help the community with generating better and more consistent long videos.
memory. However, such long term memory comes the
the resutls shown in our work tables T |
A.10 MORE VISUALIZATIONS
Please checkout our demo page for more videos at https::/sel--foriing-luu--- github.io/
21


Reviews:
[{"review_id": "HLT5Wnc7mA", "reviewer": "Reviewer_avME", "summary": "The paper proposes an algorithm to improve the length extrapolation capability of Self-Forcing-style autoregressive video diffusion models. In Self Forcing, due to the limitation that the teacher model is only trained on short duration, the student perform short-duration rollout (e.g. 5s) and receives supervision from the teacher trained on the same length. Here, the student model performs extended-duration rollout", "questions": "- What is the exact thing that is scaled in the \"Training Budget Scaling\" section (4.4)? Is it the batch size, number of iterations, or per-iteration rollout-duration. Could you provide detailed experiment config for each (1x, 4x, 8x, 20x, 25x) training setup?\n- Have you compared the proposed strategy (rollout an entire, long video then uniformly sample a single segment, e.g., rollout 1 min then sample 5s) versus an alternative strategy that rollout one segment at a time, compute loss for it and then rollout another segment (e.g., rollout 5s, compute loss and backprop, then rollout 5s).", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The paper demonstrates strong empirical results with multi-minute-long generation with no significant error accumulation. This is achieved without supervision from long videos by leveraging teacher feedback from short individual video segments generating by the student via self-rollout. \n- The experiments are very comprehensive. The paper not only demonstrates the advantage over prior work (e.g., Self Forcing) but also compares with alternative strategies (noisy kv or local attention) that alleviate error accumulation. The paper also identifies an important issue in existing benchmark (VBench) and proposes a better benchmarking.", "weaknesses": "- The description of \"Backwards Noise Initialization\" in Section 3.2 is very unclear and confusing. \n  - Is the \"Backwards Noise Initialization\" term comes from \"backward simulation\" in DMD2? It seems to be the case when I read the sentence \"they used this for... circumvent the need for real training data\". But in Fig. 2, it seems to refer to injecting noise to student output before computing fake/real scores. The noise injection is employed in the original DMD and all follow-up works, is unrelated to \"circumvent real training data\", and should not be an original contribution.\n  - Does t represent noise level or frame index? Sometime it seems to refer to noise level (x_t = (1 − σ_t)x_0 + σ_tϵ) but sometime it seems to refer to the frame index (N clean frames and clean trajectory {x^S_t}_{t=1}^N)\n  - Is there still a fake score network with separate parameters, like in DMD? If so, in eq. 2 the fake score s^S_\\theta should not be denoted with parameter theta - theta is the parameter of the student.\n  - What is S in x_t^S (L198)?\n- The way Figure 2 compares CausVid/SF/SF++ is confusing. The second row of frames of CausVid (frames with independent noise levels) are the input to the **student** model. But the second row of frames of the SF/SF++ appears to be the input to the **teacher** model and the fake score network. The input of the student model for SF/SF++ is not shown and the output of the student model is illustrated in the first row. CausVid should also have the corresponding \"Backward Noise Initialization\" step but it's subsumed in the \"DMD Alignment\" process. In short, the frames depicted in the same row, which are supposed to be side-to-side comparisons, do not correspond to things at the same stage.\n- Would be great if the authors can discuss the relationship with APT2 (Lin et al. 2025), which also uses a similar long video training method to improve long video generation ability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nimghCnXfv", "reviewer": "Reviewer_PrTo", "summary": "This paper introduces Self-Forcing++, a method that achieves a landmark breakthrough in the stability of long-horizon autoregressive video generation. It directly confronts the critical issue of quality degradation that occurs when models distilled from short-video teachers are tasked with generating minute-scale sequences. The core contribution is an elegant and highly effective training strategy that requires no long-video data. The method involves prompting the student model to generate long, error-accumulated rollouts, and then using the short-horizon teacher to perform \"segment-wise correction\" on randomly sampled clips from these rollouts. This process effectively teaches the model to recover from its own extrapolation errors.\nEmpirically, the results are groundbreaking. The method extends high-fidelity video generation from a baseline of ~10 seconds to over four minutes, a feat previously considered out of reach. It demonstrates an unprecedented ability to maintain local visual quality, motion dynamics, and temporal coherence over these extended durations.", "questions": "1. On the Inherent Trade-offs of the Global-to-Local Distillation: Your approach involves distilling knowledge from a bidirectional, full-attention teacher into a causal, autoregressive student. This global-to-local translation necessarily involves trade-offs. Could you comment on what capabilities are inherently lost in this process? For example, a bidirectional model can generate \"perfectly looping\" videos or effects where the beginning is influenced by the end. Are such temporally non-causal phenomena fundamentally beyond the reach of your framework, and do you see this as a necessary price for achieving scalability and streaming capabilities?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper's most significant and undeniable contribution is its success in mechanically extending the stable generation horizon of autoregressive models by an order of magnitude. The ability to produce minute-scale videos without collapsing is a remarkable engineering feat that directly addresses a major bottleneck in the field. This establishes a new, albeit signal-level, state-of-the-art for generation duration.\n2. The conceptual reframing of the long-video problem is the paper's most elegant intellectual contribution. The core insight—that a short-horizon teacher can be repurposed as a \"local error corrector\" for a student's long, degraded rollouts—is a highly non-obvious and clever idea. This formulation provides a novel, data-efficient paradigm for tackling extrapolation challenges.\n3. A key practical finding of this work is the demonstrated correlation between the training budget and the achievable horizon of stable generation (Figure 6). This provides the community with a clear, albeit potentially brute-force, recipe for progress: longer, stable videos can be achieved by investing more computation into the self-correction training loop. This is a valuable and actionable result.\n4. The authors should be credited for their critical examination of existing evaluation protocols. Identifying and attempting to remedy the biases of VBench when applied to long-form video demonstrates a level of scientific rigor and thoughtfulness that is often missing. While their proposed metric is deeply flawed, the motivation to improve evaluation standards is in itself a positive contribution to the community's discourse.", "weaknesses": "1. Architectural Constraint: A Bounded Information Horizon via the KV Cache.\nThe model's autoregressive generation relies on a fixed-size rolling KV cache of length L. This mechanism architecturally imposes a strict Markov assumption on the generative process. The conditional probability of generating the current latent state x_t is not conditioned on the entire history x_{<t}, but is instead approximated by p(x_t | x_{t-L:t-1}). Consequently, the model's information horizon is strictly bounded by L. Any semantic dependency or causal relationship requiring the integration of information over a temporal span greater than L (e.g., long-term entity consistency, narrative causality) cannot be modeled, as the requisite information has been discarded from the context window. This is a fundamental architectural limitation, not a matter of training scale.\n2. Objective Function Constraint: Optimization of Local, not Global, Properties.\nThe proposed Self-Forcing++ training objective is formulated as an expectation over uniformly sampled, short, contiguous segments of length K (where K is the teacher's horizon). The loss function is a sum of local distribution matching objectives, each minimizing the KL divergence between the student's and teacher's distributions within that local window: KL(p_θ(x_i:i+K) || p_T(x_i:i+K)). This objective function provides a powerful supervisory signal for enforcing local properties—such as texture fidelity, motion dynamics, and exposure consistency—that are observable within a segment of length K. However, it provides no direct supervisory signal for dependencies between non-overlapping segments, W_i and W_j, where j > i+K. The model is therefore trained to be an expert local generator, but it is not explicitly optimized for global semantic coherence. The observed long-term stability is an emergent property of chaining together high-fidelity local segments, not a result of learning a true global model of the video distribution.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hXPOjF24L6", "reviewer": "Reviewer_hYZq", "summary": "The paper introduces Self-Forcing++, a training framework for minute-scale autoregressive video generation that mitigates long-horizon degradation without long-video teachers or datasets. It identifies a dual train–test mismatch: models train with dense teacher supervision on short clips but must sustain much longer rollouts at inference, causing compounding errors (motion stalling, exposure drift).  \nSelf-Forcing++ rolls the student out far beyond the teacher’s horizon, re-injects diffusion noise (Backward Noise Initialization) to preserve temporal context, and performs Extended Distribution Matching Distillation by uniformly sampling K-length windows (teacher horizon) from the long rollout for student–teacher alignment. Training consistently uses a rolling KV cache, eliminating cache mismatches and overlapping-frame recomputation. An optional GRPO stage with an optical-flow smoothness reward further enhances temporal continuity.  \nExperiments show substantial gains over CausVid, Self-Forcing, SkyReels-V2, and MAGI-1 on 50/75/100s tasks, especially in motion dynamics and a proposed Visual Stability metric that addresses VBench’s long-video bias. With increased compute, the model scales to 4m15s while maintaining quality.", "questions": "-", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1.\tThe paper clearly diagnoses the core barrier to long-horizon video generation as a dual train–inference mismatch in temporal horizon and supervision density. The overall pipeline is communicated cleanly, with a step-by-step flow (long self-rollout, backward noise re-injection, sliding-window distillation, rolling KV cache). The failure modes of prior work, including motion collapse, exposure drift, and cache-state mismatches, are explained convincingly and tied to design choices.\n2.\tThe method achieves strong, consistent gains over baselines (CausVid, Self-Forcing, SkyReels-V2, MAGI-1) on comparable 50–100s tasks, especially in motion dynamics and exposure/visual stability. With increased training budget, and without long-video data or teachers, it scales to 4m15s (near positional embedding limits) while maintaining coherent motion and stable exposure. \n3.\tThe authors identify and analyze a bias in VBench that can favor over-exposed or degraded frames, which are common failure modes in prior systems. They propose Visual Stability, an evaluation protocol leveraging a strong video MLLM and human verification to better capture long-horizon degradation and exposure issues, thereby improving the reliability of benchmarking for long video generation.", "weaknesses": "1.\tThe paper should provide a clearer accounting of computational resources for both training and inference, such as FLOPs, GPU hours, and memory, to enable fair cost–quality comparisons with baselines.\n2.\tWhile the paper includes ablations (attention window, noisy-KV, GRPO), it would be even stronger with finer analyses of key design choices, e.g., alternative window sampling distributions, different noise schedules/intensities for backward initialization, and systematic sweeps over K/N ratios.\n3.\tThere are minor punctuation and typography issues. For example, in the Introduction’s first paragraph the last sentence ends with a comma instead of a period, and around line 202 the phrase \"by $\\Theta$\" is followed by an extra period. A careful proofreading pass to fix such punctuation/formatting inconsistencies would improve presentation quality.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)