How Animals Dance (When You're Not Looking)
Authors: name: Xiaojuan Wang; name: Aleksander Holynski; name: Brian Curless; name: Ira Kemelmacher; name: Steve Seitz
Published: 2025-05-29 17:58:02

Abstract: We present a keyframe-based framework for generating music-synchronized,
choreography aware animal dance videos. Starting from a few keyframes
representing distinct animal poses -- generated via text-to-image prompting or
GPT-4o -- we formulate dance synthesis as a graph optimization problem: find
the optimal keyframe structure that satisfies a specified choreography pattern
of beats, which can be automatically estimated from a reference dance video. We
also introduce an approach for mirrored pose image generation, essential for
capturing symmetry in dance. In-between frames are synthesized using an video
diffusion model. With as few as six input keyframes, our method can produce up
to 30 second dance videos across a wide range of animals and music tracks.