Motion Anything: Any to Motion Generation
Authors: name: Zeyu Zhang; name: Yiran Wang; name: Wei Mao; name: Danning Li; name: Rui Zhao; name: Biao Wu; name: Zirui Song; name: Bohan Zhuang; name: Ian Reid; name: Richard Hartley
Published: 2025-03-10 06:04:31

Abstract: Conditional motion generation has been extensively studied in computer
vision, yet two critical challenges remain. First, while masked autoregressive
methods have recently outperformed diffusion-based approaches, existing masking
models lack a mechanism to prioritize dynamic frames and body parts based on
given conditions. Second, existing methods for different conditioning
modalities often fail to integrate multiple modalities effectively, limiting
control and coherence in generated motion. To address these challenges, we
propose Motion Anything, a multimodal motion generation framework that
introduces an Attention-based Mask Modeling approach, enabling fine-grained
spatial and temporal control over key frames and actions. Our model adaptively
encodes multimodal conditions, including text and music, improving
controllability. Additionally, we introduce Text-Music-Dance (TMD), a new
motion dataset consisting of 2,153 pairs of text, music, and dance, making it
twice the size of AIST++, thereby filling a critical gap in the community.
Extensive experiments demonstrate that Motion Anything surpasses
state-of-the-art methods across multiple benchmarks, achieving a 15%
improvement in FID on HumanML3D and showing consistent performance gains on
AIST++ and TMD. See our project website
https://steve-zeyu-zhang.github.io/MotionAnything