VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement
  Fine-Tuning
Authors: name: Xinhao Li, name: Ziang Yan, name: Desen Meng, name: Lu Dong, name: Xiangyu Zeng, name: Yinan He, name: Yali Wang, name: Yu Qiao, name: Yi Wang, name: Limin Wang

Recent advancements in reinforcement learning have significantly advanced the
reasoning capabilities of multimodal large language models (MLLMs). While
approaches such as Group Relative Policy Optimization (GRPO) and rule-based
reward mechanisms demonstrate promise in text and image domains, their
application to video understanding remains limited. This paper presents a
systematic exploration of Reinforcement Fine-Tuning (RFT) with GRPO for video
MLLMs, aiming to enhance spatio-temporal perception while maintaining general
capabilities. Our experiments reveal that RFT is highly data-efficient for
task-specific improvements. Through multi-task RFT on spatio-temporal
perception objectives with limited samples, we develop VideoChat-R1, a powerful
video MLLM that achieves state-of-the-art performance on spatio-temporal
perception tasks without sacrificing chat ability, while exhibiting emerging
spatio-temporal reasoning abilities. Compared to Qwen2.5-VL-7B, VideoChat-R1
boosts performance several-fold in tasks like temporal grounding (+31.8) and
object tracking (+31.2). Additionally, it significantly improves on general QA
benchmarks such as VideoMME (+0.9), MVBench (+1.0), and Perception Test (+0.9).
Our findings underscore the potential of RFT for specialized task enhancement
of Video MLLMs. We hope our work offers valuable insights for future RL
research in video MLLMs.