LongVie: Multimodal-Guided Controllable Ultra-Long Video Generation
Authors: name: Jianxiong Gao, name: Zhaoxi Chen, name: Xian Liu, name: Jianfeng Feng, name: Chenyang Si, name: Yanwei Fu, name: Yu Qiao, name: Ziwei Liu

Controllable ultra-long video generation is a fundamental yet challenging
task. Although existing methods are effective for short clips, they struggle to
scale due to issues such as temporal inconsistency and visual degradation. In
this paper, we initially investigate and identify three key factors: separate
noise initialization, independent control signal normalization, and the
limitations of single-modality guidance. To address these issues, we propose
LongVie, an end-to-end autoregressive framework for controllable long video
generation. LongVie introduces two core designs to ensure temporal consistency:
1) a unified noise initialization strategy that maintains consistent generation
across clips, and 2) global control signal normalization that enforces
alignment in the control space throughout the entire video. To mitigate visual
degradation, LongVie employs 3) a multi-modal control framework that integrates
both dense (e.g., depth maps) and sparse (e.g., keypoints) control signals,
complemented by 4) a degradation-aware training strategy that adaptively
balances modality contributions over time to preserve visual quality. We also
introduce LongVGenBench, a comprehensive benchmark consisting of 100
high-resolution videos spanning diverse real-world and synthetic environments,
each lasting over one minute. Extensive experiments show that LongVie achieves
state-of-the-art performance in long-range controllability, consistency, and
quality.