AnchorCrafter: Animate CyberAnchors Saling Your Products via
  Human-Object Interacting Video Generation
Authors: name: Ziyi Xu; name: Ziyao Huang; name: Juan Cao; name: Yong Zhang; name: Xiaodong Cun; name: Qing Shuai; name: Yuchen Wang; name: Linchao Bao; name: Jintao Li; name: Fan Tang
Published: 2024-11-26 12:42:13

Abstract: The automatic generation of anchor-style product promotion videos presents
promising opportunities in online commerce, advertising, and consumer
engagement. However, this remains a challenging task despite significant
advancements in pose-guided human video generation. In addressing this
challenge, we identify the integration of human-object interactions (HOI) into
pose-guided human video generation as a core issue. To this end, we introduce
AnchorCrafter, a novel diffusion-based system designed to generate 2D videos
featuring a target human and a customized object, achieving high visual
fidelity and controllable interactions. Specifically, we propose two key
innovations: the HOI-appearance perception, which enhances object appearance
recognition from arbitrary multi-view perspectives and disentangles object and
human appearance, and the HOI-motion injection, which enables complex
human-object interactions by overcoming challenges in object trajectory
conditioning and inter-occlusion management. Additionally, we introduce the
HOI-region reweighting loss, a training objective that enhances the learning of
object details. Extensive experiments demonstrate that our proposed system
outperforms existing methods in preserving object appearance and shape
awareness, while simultaneously maintaining consistency in human appearance and
motion. Project page: https://cangcz.github.io/Anchor-Crafter/