Segment Any Motion in Videos
Authors: name: Nan Huang; name: Wenzhao Zheng; name: Chenfeng Xu; name: Kurt Keutzer; name: Shanghang Zhang; name: Angjoo Kanazawa; name: Qianqian Wang
Published: 2025-03-28 09:34:11

Abstract: Moving object segmentation is a crucial task for achieving a high-level
understanding of visual scenes and has numerous downstream applications. Humans
can effortlessly segment moving objects in videos. Previous work has largely
relied on optical flow to provide motion cues; however, this approach often
results in imperfect predictions due to challenges such as partial motion,
complex deformations, motion blur and background distractions. We propose a
novel approach for moving object segmentation that combines long-range
trajectory motion cues with DINO-based semantic features and leverages SAM2 for
pixel-level mask densification through an iterative prompting strategy. Our
model employs Spatio-Temporal Trajectory Attention and Motion-Semantic
Decoupled Embedding to prioritize motion while integrating semantic support.
Extensive testing on diverse datasets demonstrates state-of-the-art
performance, excelling in challenging scenarios and fine-grained segmentation
of multiple objects. Our code is available at https://motion-seg.github.io/.