MotionBench: Benchmarking and Improving Fine-grained Video Motion
  Understanding for Vision Language Models
Authors: name: Wenyi Hong; name: Yean Cheng; name: Zhuoyi Yang; name: Weihan Wang; name: Lefan Wang; name: Xiaotao Gu; name: Shiyu Huang; name: Yuxiao Dong; name: Jie Tang
Published: 2025-01-06 11:57:38

Abstract: In recent years, vision language models (VLMs) have made significant
advancements in video understanding. However, a crucial capability -
fine-grained motion comprehension - remains under-explored in current
benchmarks. To address this gap, we propose MotionBench, a comprehensive
evaluation benchmark designed to assess the fine-grained motion comprehension
of video understanding models. MotionBench evaluates models' motion-level
perception through six primary categories of motion-oriented question types and
includes data collected from diverse sources, ensuring a broad representation
of real-world video content. Experimental results reveal that existing VLMs
perform poorly in understanding fine-grained motions. To enhance VLM's ability
to perceive fine-grained motion within a limited sequence length of LLM, we
conduct extensive experiments reviewing VLM architectures optimized for video
feature compression and propose a novel and efficient Through-Encoder (TE)
Fusion method. Experiments show that higher frame rate inputs and TE Fusion
yield improvements in motion understanding, yet there is still substantial room
for enhancement. Our benchmark aims to guide and motivate the development of
more capable video understanding models, emphasizing the importance of
fine-grained motion comprehension. Project page: https://motion-bench.github.io .