MagicMotion: Controllable Video Generation with Dense-to-Sparse
  Trajectory Guidance
Authors: name: Quanhao Li, name: Zhen Xing, name: Rui Wang, name: Hui Zhang, name: Qi Dai, name: Zuxuan Wu

Recent advances in video generation have led to remarkable improvements in
visual quality and temporal coherence. Upon this, trajectory-controllable video
generation has emerged to enable precise object motion control through
explicitly defined spatial paths. However, existing methods struggle with
complex object movements and multi-object motion control, resulting in
imprecise trajectory adherence, poor object consistency, and compromised visual
quality. Furthermore, these methods only support trajectory control in a single
format, limiting their applicability in diverse scenarios. Additionally, there
is no publicly available dataset or benchmark specifically tailored for
trajectory-controllable video generation, hindering robust training and
systematic evaluation. To address these challenges, we introduce MagicMotion, a
novel image-to-video generation framework that enables trajectory control
through three levels of conditions from dense to sparse: masks, bounding boxes,
and sparse boxes. Given an input image and trajectories, MagicMotion seamlessly
animates objects along defined trajectories while maintaining object
consistency and visual quality. Furthermore, we present MagicData, a
large-scale trajectory-controlled video dataset, along with an automated
pipeline for annotation and filtering. We also introduce MagicBench, a
comprehensive benchmark that assesses both video quality and trajectory control
accuracy across different numbers of objects. Extensive experiments demonstrate
that MagicMotion outperforms previous methods across various metrics. Our
project page are publicly available at
https://quanhaol.github.io/magicmotion-site.