Subject-driven Video Generation via Disentangled Identity and Motion
Authors: name: Daneul Kim; name: Jingxu Zhang; name: Wonjoon Jin; name: Sunghyun Cho; name: Qi Dai; name: Jaesik Park; name: Chong Luo
Published: 2025-04-23 06:48:31

Abstract: We propose to train a subject-driven customized video generation model
through decoupling the subject-specific learning from temporal dynamics in
zero-shot without additional tuning. A traditional method for video
customization that is tuning-free often relies on large, annotated video
datasets, which are computationally expensive and require extensive annotation.
In contrast to the previous approach, we introduce the use of an image
customization dataset directly on training video customization models,
factorizing the video customization into two folds: (1) identity injection
through image customization dataset and (2) temporal modeling preservation with
a small set of unannotated videos through the image-to-video training method.
Additionally, we employ random image token dropping with randomized image
initialization during image-to-video fine-tuning to mitigate the copy-and-paste
issue. To further enhance learning, we introduce stochastic switching during
joint optimization of subject-specific and temporal features, mitigating
catastrophic forgetting. Our method achieves strong subject consistency and
scalability, outperforming existing video customization models in zero-shot
settings, demonstrating the effectiveness of our framework.