DreamRelation: Relation-Centric Video Customization
Authors: name: Yujie Wei, name: Shiwei Zhang, name: Hangjie Yuan, name: Biao Gong, name: Longxiang Tang, name: Xiang Wang, name: Haonan Qiu, name: Hengjia Li, name: Shuai Tan, name: Yingya Zhang, name: Hongming Shan

Relational video customization refers to the creation of personalized videos
that depict user-specified relations between two subjects, a crucial task for
comprehending real-world visual content. While existing methods can personalize
subject appearances and motions, they still struggle with complex relational
video customization, where precise relational modeling and high generalization
across subject categories are essential. The primary challenge arises from the
intricate spatial arrangements, layout variations, and nuanced temporal
dynamics inherent in relations; consequently, current models tend to
overemphasize irrelevant visual details rather than capturing meaningful
interactions. To address these challenges, we propose DreamRelation, a novel
approach that personalizes relations through a small set of exemplar videos,
leveraging two key components: Relational Decoupling Learning and Relational
Dynamics Enhancement. First, in Relational Decoupling Learning, we disentangle
relations from subject appearances using relation LoRA triplet and hybrid mask
training strategy, ensuring better generalization across diverse relationships.
Furthermore, we determine the optimal design of relation LoRA triplet by
analyzing the distinct roles of the query, key, and value features within
MM-DiT's attention mechanism, making DreamRelation the first relational video
generation framework with explainable components. Second, in Relational
Dynamics Enhancement, we introduce space-time relational contrastive loss,
which prioritizes relational dynamics while minimizing the reliance on detailed
subject appearances. Extensive experiments demonstrate that DreamRelation
outperforms state-of-the-art methods in relational video customization. Code
and models will be made publicly available.