Phantom: Subject-consistent video generation via cross-modal alignment
Authors: name: Lijie Liu; name: Tianxiang Ma; name: Bingchuan Li; name: Zhuowei Chen; name: Jiawei Liu; name: Qian He; name: Xinglong Wu
Published: 2025-02-16 11:02:50

Abstract: The continuous development of foundational models for video generation is
evolving into various applications, with subject-consistent video generation
still in the exploratory stage. We refer to this as Subject-to-Video, which
extracts subject elements from reference images and generates
subject-consistent video through textual instructions. We believe that the
essence of subject-to-video lies in balancing the dual-modal prompts of text
and image, thereby deeply and simultaneously aligning both text and visual
content. To this end, we propose Phantom, a unified video generation framework
for both single and multi-subject references. Building on existing
text-to-video and image-to-video architectures, we redesign the joint
text-image injection model and drive it to learn cross-modal alignment via
text-image-video triplet data. In particular, we emphasize subject consistency
in human generation, covering existing ID-preserving video generation while
offering enhanced advantages. The project homepage is here
https://phantom-video.github.io/Phantom/.