Segment Any Motion in Videos
Authors: name: Nan Huang, name: Wenzhao Zheng, name: Chenfeng Xu, name: Kurt Keutzer, name: Shanghang Zhang, name: Angjoo Kanazawa, name: Qianqian Wang

Moving object segmentation is a crucial task for achieving a high-level
understanding of visual scenes and has numerous downstream applications. Humans
can effortlessly segment moving objects in videos. Previous work has largely
relied on optical flow to provide motion cues; however, this approach often
results in imperfect predictions due to challenges such as partial motion,
complex deformations, motion blur and background distractions. We propose a
novel approach for moving object segmentation that combines long-range
trajectory motion cues with DINO-based semantic features and leverages SAM2 for
pixel-level mask densification through an iterative prompting strategy. Our
model employs Spatio-Temporal Trajectory Attention and Motion-Semantic
Decoupled Embedding to prioritize motion while integrating semantic support.
Extensive testing on diverse datasets demonstrates state-of-the-art
performance, excelling in challenging scenarios and fine-grained segmentation
of multiple objects. Our code is available at https://motion-seg.github.io/.