Video-RTS: Rethinking Reinforcement Learning and Test-Time Scaling for
  Efficient and Enhanced Video Reasoning
Authors: name: Ziyang Wang, name: Jaehong Yoon, name: Shoubin Yu, name: Md Mohaiminul Islam, name: Gedas Bertasius, name: Mohit Bansal

Despite advances in reinforcement learning (RL)-based video reasoning with
large language models (LLMs), data collection and finetuning remain significant
challenges. These methods often rely on large-scale supervised fine-tuning
(SFT) with extensive video data and long Chain-of-Thought (CoT) annotations,
making them costly and hard to scale. To address this, we present Video-RTS, a
new approach to improve video reasoning capability with drastically improved
data efficiency by combining data-efficient RL with a video-adaptive test-time
scaling (TTS) strategy. Based on observations about the data scaling of RL
samples, we skip the resource-intensive SFT step and employ efficient pure-RL
training with output-based rewards, requiring no additional annotations or
extensive fine-tuning. Furthermore, to utilize computational resources more
efficiently, we introduce a sparse-to-dense video TTS strategy that improves
inference by iteratively adding frames based on output consistency. We validate
our approach on multiple video reasoning benchmarks, showing that Video-RTS
surpasses existing video reasoning models by an average of 2.4% in accuracy
using only 3.6% training samples. For example, Video-RTS achieves a 4.2%
improvement on Video-Holmes, a recent and challenging video reasoning
benchmark, and a 2.6% improvement on MMVU. Notably, our pure RL training and
adaptive video TTS offer complementary strengths, enabling Video-RTS's strong
reasoning performance.