Long Context Tuning for Video Generation
Authors: name: Yuwei Guo, name: Ceyuan Yang, name: Ziyan Yang, name: Zhibei Ma, name: Zhijie Lin, name: Zhenheng Yang, name: Dahua Lin, name: Lu Jiang

Recent advances in video generation can produce realistic, minute-long
single-shot videos with scalable diffusion transformers. However, real-world
narrative videos require multi-shot scenes with visual and dynamic consistency
across shots. In this work, we introduce Long Context Tuning (LCT), a training
paradigm that expands the context window of pre-trained single-shot video
diffusion models to learn scene-level consistency directly from data. Our
method expands full attention mechanisms from individual shots to encompass all
shots within a scene, incorporating interleaved 3D position embedding and an
asynchronous noise strategy, enabling both joint and auto-regressive shot
generation without additional parameters. Models with bidirectional attention
after LCT can further be fine-tuned with context-causal attention, facilitating
auto-regressive generation with efficient KV-cache. Experiments demonstrate
single-shot models after LCT can produce coherent multi-shot scenes and exhibit
emerging capabilities, including compositional generation and interactive shot
extension, paving the way for more practical visual content creation. See
https://guoyww.github.io/projects/long-context-video/ for more details.