FullDiT: Multi-Task Video Generative Foundation Model with Full
  Attention
Authors: name: Xuan Ju; name: Weicai Ye; name: Quande Liu; name: Qiulin Wang; name: Xintao Wang; name: Pengfei Wan; name: Di Zhang; name: Kun Gai; name: Qiang Xu
Published: 2025-03-25 17:59:06

Abstract: Current video generative foundation models primarily focus on text-to-video
tasks, providing limited control for fine-grained video content creation.
Although adapter-based approaches (e.g., ControlNet) enable additional controls
with minimal fine-tuning, they encounter challenges when integrating multiple
conditions, including: branch conflicts between independently trained adapters,
parameter redundancy leading to increased computational cost, and suboptimal
performance compared to full fine-tuning. To address these challenges, we
introduce FullDiT, a unified foundation model for video generation that
seamlessly integrates multiple conditions via unified full-attention
mechanisms. By fusing multi-task conditions into a unified sequence
representation and leveraging the long-context learning ability of full
self-attention to capture condition dynamics, FullDiT reduces parameter
overhead, avoids conditions conflict, and shows scalability and emergent
ability. We further introduce FullBench for multi-task video generation
evaluation. Experiments demonstrate that FullDiT achieves state-of-the-art
results, highlighting the efficacy of full-attention in complex multi-task
video generation.