DragAnything: Motion Control for Anything using Entity Representation
Authors: name: Wejia Wu; name: Zhuang Li; name: Yuchao Gu; name: Rui Zhao; name: Yefei He; name: David Junhao Zhang; name: Mike Zheng Shou; name: Yan Li; name: Tingting Gao; name: Di Zhang
Published: 2024-03-12 08:57:29

Abstract: We introduce DragAnything, which utilizes a entity representation to achieve
motion control for any object in controllable video generation. Comparison to
existing motion control methods, DragAnything offers several advantages.
Firstly, trajectory-based is more userfriendly for interaction, when acquiring
other guidance signals (e.g., masks, depth maps) is labor-intensive. Users only
need to draw a line (trajectory) during interaction. Secondly, our entity
representation serves as an open-domain embedding capable of representing any
object, enabling the control of motion for diverse entities, including
background. Lastly, our entity representation allows simultaneous and distinct
motion control for multiple objects. Extensive experiments demonstrate that our
DragAnything achieves state-of-the-art performance for FVD, FID, and User
Study, particularly in terms of object motion control, where our method
surpasses the previous methods (e.g., DragNUWA) by 26% in human voting.