Alicezrzhao/Mask-Forcing

Mask Forcing: Improving Autoregressive Video Diffusion Distillation via Dual-Noise Masking Rollout

Zhuoran Zhao^1,2 · Shengju Qian³ · Tongtong Liang⁴ · Xianghao Kong² · Songchun Zhang² · Junchao Huang⁵ · Guian Fang⁶ · Xin Wang³ · Pan Hui^1,2 · Anyi Rao²

¹HKUST(GZ)  ·  ²HKUST  ·  ³LIGHTSPEED  ·  ⁴UCSD  ·  ⁵CUHK(SZ)  ·  ⁶NUS

📖 Overview

Self-rollout DMD enables few-step autoregressive video generation, but its mode-seeking reverse-KL objective may concentrate the student rollout distribution on a narrow set of teacher modes. Moreover, the DMD objective is evaluated only on the completed rollout output and does not directly regularize each intermediate transition, allowing errors to accumulate through subsequent denoising steps and later chunks. Together, these limitations reduce visual quality and realism.

Mask Forcing introduces a Dual-Noise Masking Rollout strategy that injects cleaner tokens into noisy rollout inputs through random masks. The resulting perturbations diversify student rollout trajectories to cover more teacher modes, while cleaner tokens guide the denoising of noisier tokens to improve intermediate predictions and reduce error accumulation.

Mask Forcing requires no real-video supervision or additional post-training stages, introduces no additional network forward passes, and leaves inference unchanged.

🎥 Demo

Sample 1	Sample 2
Sample 3	Sample 4
Sample 5	Sample 6

For more qualitative and quantitative comparisons, please visit our Project Page.

---

📢 News

• [2026-09] Paper released on arXiv and Project Page is online.
• [2026-09] Inference code is released.
• Training code will be released in this repository. Stay tuned.

📋 TODO

• Release paper and project page
• Release inference code and checkpoints
• Release training code

🚀 Quick Start

Installation

conda create -n mask_forcing python=3.10 -y
conda activate mask_forcing
pip install -r requirements.txt
pip install flash-attn --no-build-isolation
python setup.py develop

Download Checkpoints

hf download Wan-AI/Wan2.1-T2V-1.3B  --local-dir wan_models/Wan2.1-T2V-1.3B
# Mask Forcing checkpoints -> checkpoints/chunkwise/
hf download Alicezrzhao/Mask-Forcing --local-dir checkpoints

Inference

bash inference_chunkwise.sh

🎓 Citation

If you find Mask Forcing useful in your research, please consider citing:

@misc{zhao2026maskforcingimprovingautoregressive,
      title={Mask Forcing: Improving Autoregressive Video Diffusion Distillation via Dual-Noise Masking Rollout}, 
      author={Zhuoran Zhao and Shengju Qian and Tongtong Liang and Xianghao Kong and Songchun Zhang and Junchao Huang and Guian Fang and Xin Wang and Pan Hui and Anyi Rao},
      year={2026},
      eprint={2609.09123},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2609.09123}, 
}

🤝 Acknowledgements

This project builds upon the following outstanding open-source works:

• Self-Forcing — Self-forcing training for autoregressive video diffusion
• Causal Forcing — AR teacher distillation with causal attention
• LongLive — Long-form video generation via generative extrapolation
• Wan2.1 — Base video diffusion transformer