EchoVideo: Identity-Preserving Human Video Generation by Multimodal
  Feature Fusion
Authors: name: Jiangchuan Wei; name: Shiyue Yan; name: Wenfeng Lin; name: Boyuan Liu; name: Renjie Chen; name: Mingyu Guo
Published: 2025-01-23 08:06:11

Abstract: Recent advancements in video generation have significantly impacted various
downstream applications, particularly in identity-preserving video generation
(IPT2V). However, existing methods struggle with "copy-paste" artifacts and low
similarity issues, primarily due to their reliance on low-level facial image
information. This dependence can result in rigid facial appearances and
artifacts reflecting irrelevant details. To address these challenges, we
propose EchoVideo, which employs two key strategies: (1) an Identity Image-Text
Fusion Module (IITF) that integrates high-level semantic features from text,
capturing clean facial identity representations while discarding occlusions,
poses, and lighting variations to avoid the introduction of artifacts; (2) a
two-stage training strategy, incorporating a stochastic method in the second
phase to randomly utilize shallow facial information. The objective is to
balance the enhancements in fidelity provided by shallow features while
mitigating excessive reliance on them. This strategy encourages the model to
utilize high-level features during training, ultimately fostering a more robust
representation of facial identities. EchoVideo effectively preserves facial
identities and maintains full-body integrity. Extensive experiments demonstrate
that it achieves excellent results in generating high-quality, controllability
and fidelity videos.