Title: Video-R1: Reinforcing Video Reasoning in MLLMs
Venue: NeurIPS 2025
Accepted: yes
Keywords: Multimodal Large Language Models

Video Reasoning
Abstract: Inspired by DeepSeek-R1's success in eliciting reasoning abilities through rule-based reinforcement learning (RL), we introduce Video-R1 as the first attempt to systematically explore the R1 paradigm for incentivizing video reasoning within multimodal large language models (MLLMs). However, directly applying RL training with the GRPO algorithm to video reasoning presents two primary challenges: (i) a lack of temporal modeling for video reasoning, and (ii) the scarcity of high-quality video-reasoning data. To address these issues, we first propose the T-GRPO algorithm, which encourages models to utilize temporal information in videos for reasoning. Additionally, instead of relying solely on video data, we incorporate high-quality image-reasoning data into the training process. We have constructed two datasets: Video-R1-CoT-165k for SFT cold start and Video-R1-260k for RL training, both comprising image and video data. Experimental results demonstrate that Video-R1 achieves significant improvements on video reasoning benchmarks such as VideoMMMU and VSI-Bench, as well as on general video benchmarks including MVBench and TempCompass, etc. Notably, Video-R1-7B attains a 37.1\% accuracy on video spatial reasoning benchmark VSI-bench, surpassing the commercial proprietary model GPT-4o. All code, models, and data will be released.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces an R1-style framework for video reasoning task.\nThe main contributions, as mentioned in the paper, are as follows:\n1. Introduce T-GRPO, an adaptation of GRPO for temporal-based video reasoning.\n2. Video-R1, a Qwen-based model trained using T-GRPO for video reasoning.\n3.  2 datasets for cold starting and RL training on video reasoning tasks.\n\nThe paper provides detailed information about the T-GRPO algorithm. It uses the GRPO algorithm allong with an additional reward based on reasoning using the right order of frames. This forces the model to use temporal information during reasoning allowing for accurate outputs.\n\nThe contribution of the datasets allows for exapnding this methodology with outher foundation models as the backbone as well as addding to the pool of present data for future works. \n\nThe authors of the work undertake extensive experimentation on multiple video reasoning datasets.\nThe work shows a significant performance increase on various video reasoning datasets using their methodology, showing the viability of their method with a strong ablation study measuring the upliftment due to each part of their framework.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1.  The paper introduces a new framework and potential direction for video reasoning.\n2. The paper shows significant performance in various datasets.\n3. The work presents extensive experimentation and ablation study along with a detailed appendix.\n\nWeakness:\n1. Some variables in the equations are not clearly described or assume prior knowledge, making it difficult to follow. For example,\n    a. In Equation 2, it is unclear whether $R_i$ is calculated as the reward for each output or the overall set of outputs. \\\n    b. In Equation 3, $\\pi_{ref}$ is not defined when describing the GRPO algorithm.\n    \nWhile the definition of variables in other places can be inferred from context, it leaves room for ambiguity, leading to difficulty in reproducing the results.\n\nQUESTIONS:\nThe same as described in the weaknesses.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-06-21"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this work, the authors propose a Video-R1 paradigm for reinforcing video reasoning with MLLMs, inspired by the learning style of DeepSeek-R1. First, they build two reasoning datasets, i.e., Video-R1-CoT-165k and Video-R1-260k, respectively for SFT and RL training. Second, they introduce a T-GRPO learning method that leverages temporal modeling in RL to enhance video reasoning.\n\nSTRENGTHS AND WEAKNESSES:\n*Strengths\n\n1 The main contribution of this work is an early attempt on video reasoning. Different from the previous works, this paper systematically explores the R1 learning paradigm on video MLLMs,  based on SFT (with Video-R1-CoT-165k) and RL training (with Video-R1-260k). It would benefit video understanding community from basic perception towards deep thinking on complex video content.\n\n2 The claims of this work are basically supported by their experiments on six video benchmarks: VSI-Bench, VideoMMMU, MMVU, MVBench, TempCompass, and VideoMME. Especially on video reasoning benchmarks like VSI-Bench, VideoMMMU and MMVU, the proposed Video-R1 achieves a preferable performance than recent open-source models and baselines, showing its effectiveness.\n\n3 The structure of this paper is basically well organized. \n\n\n\n*Weaknesses\n\n1 The temporal reward is mainly based on frame order. The order-relevant supervision has been widely investigated in the video undestanding community such as [1-3].\n\n2 The evalution is mainly based on video question answering. There are more temporal-relavent tasks such as temporal grounding and object tracking, which should be further investigated like VideoChat-R1 [4].\n\n3 Small Concern: What is Qwen2.5-VL-7B (CoT)? Moreover, Qwen2.5-VL-7B (CoT) is generally worse than Qwen2.5-VL-7B-SFT on video reasoning benchmarks, while it is often better than Qwen2.5-VL-7B-SFT  on video general benchmarks such as TempCompass and VideoMME. Why this happens?\n\n*Reference\n\n[1] Shuffle and Learn: Unsupervised Learning using Temporal Order Verification, ECCV2016\n\n[2] Unsupervised Representation Learning by Sorting Sequences, ICCV2017\n\n[3] Self-Supervised Video Representation Learning With Odd-One-Out Networks, CVPR2017\n\n[4] VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuning, arxiv 2025\n\nQUESTIONS:\nPlease see weakness section.\n\nLIMITATIONS:\nPlease see weakness section.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a variant of GRPO for video understanding, enhancing temporal reasoning. The authors claim that existing multi-modal large language models do not fully utilize the temporal information. To address this, the authors introduced Video-R1-260K and Video-R1-CoT-165K curated datasets. Following DeepSeek-R1, the authors used rule-based rewards depending on data types. As presented in Figure 3, the variant of GRPO, T-GRPO, utilized randomly shuffled videos and their outputs. Based on the relation between responses from original and shuffled videos, rewards are differently augmented. Also, if the length of the response is reasonable, a higher reward is given. This simple variant provides considerable performance gain.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n1. **Simplicity.** The proposed method is a simple modification of the GRPO using data augmentation and modified rule-based reward functions with additional rewards when the rate of correct responses of the original data is higher than shuffled videos. Also, the responses have reasonably lengths, the authors give a higher reward. This is extremely simple and applicable to various criteria. \n2. **Motivation.** After the advance of DeepSeekMath trained with GRPO, numerous papers have studied variants of GRPO. In MLLMs, or LLM-based agents, post-training became important to achieve competitive performance in downstream applications. Especially, the \n\n**Weaknesses**\n\n1. **Limited technical contribution.** The simplicity of solutions is desirable but no theoretical analysis or derivation is provided. Also, the relation between the proposed method and other alternatives are not discussed. GRPO is relatively new but many variants of GRPO have been studied. Without comparing the proposed method with existing GRPO variants and discussing the difference, it is difficult to the novelty of this work.\n2. **Hyperparameters.** The manually designed reward functions and additional rewards entail hyperparameters. Presumably, this work is sensitive to those hyperparameters. No in-depth analysis is provided and large dependency on hyperparameters raises questions on generalizability of this work to a new data or model.\n3. **Curated data.** This work requires another non-trivial manual effort to construct the dataset. The authors provided Video-R1-260K and Video-R1-CoT-165K datasets. These datasets are curated. To this extent, this work improves the performance with more annotations. As Table 1 shows mixed results. Qwen2.5-V(CoT) is better than SFT in two Video General Benchmarks: TempoCompass and  VideoMME (wo sub). However for other benchmarks, it is opposite.\n\nQUESTIONS:\n1. Are the responses to original and shuffled videos paired? \n2. How did you curate newly introduced datasets?\n3. How sensitive is the proposed method to the additional reward values for temporal reasoning, and length in (2) and (5)?\n4. Regarding temporally augmented reward defined in (2) why r_i  is given when the response is not correct?\n5. Is the proposed method applicable to other temporal data? applications?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Video-R1, a framework that systematically explores the R1 paradigm to enhance video reasoning in large multimodal models. The authors identify two main challenges when applying the GRPO algorithm to video reasoning: (1) insufficient reward for temporal reasoning, which can lead to shortcut solutions, and (2) a lack of high-quality video reasoning data. To address these, they propose the T-GRPO algorithm, which incentivizes the use of temporal information by rewarding the model only when it performs better on temporally ordered frames compared to shuffled ones. Additionally, they construct two new datasets: Video-R1-CoT-165k for SFT and Video-R1-260k for RL. Experimental results show that Video-R1 significantly improves performance on both video reasoning and general video benchmarks.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n\nVideo-R1 is a representative early exploration of video reasoning with large multimodal models (open-source), distinguished by its strong performance. In addition to the benchmark results reported in the paper, Video-R1 also achieves top performance on recent released benchmark [a]. The proposed T-GRPO algorithm is both simple and effective, encouraging the model to reason more deeply about temporal relationships in video data.\n\n[a] Video-Holmes: Can MLLM Think like Holmes for Complex Video Reasoning? arXiv: 2505.21374.\n\n### Weaknesses\n\n- 1. It seems that the paper does not report the original Qwen2.5-VL-Instruct performance. Please include this baseline using the same inference parameters as your model. Qwen2.5-VL-7B-Instruct should also be competitive. This comparison is essential for readers to clearly understand the benefits of your approach.\n\n- 2. Given access to 8x80G GPUs, why not train on longer sequences? With Qwen-style smart resize, the model should support at least 24k sequence length (approximately 240 frames) during training. To convincingly demonstrate improvements in temporal reasoning, please train and evaluate the model on more frames and provide comparative results.\n\n- 3. Concerns with T-GRPO Algorithm Setting:\n    - a. Since training is limited to 16 frames, longer videos (e.g., >2 minutes. BTW, please specify the video durations in Video-R1-CoT-165k and Video-R1-260k) will lose significant temporal correlation. This may make the distinction between temporally ordered and randomly shuffled frames ambiguous, potentially weakening the proposed temporal reward.\n    - b. The rationale for why temporally ordered frames should yield higher rewards is unclear. If the model can reason correctly from randomly shuffled frames, does this not indicate even stronger temporal reasoning, as it can \"recover\" the original order?\n    - c. While most video LLMs use causal visual token order, some employ full-attention encoders, making it difficult to guarantee that token order reflects true temporal order. Your approach is based on Qwen2.5VL uses a single-frame encoder, ensuring visual order matches temporal causality. Please clarify that in your methods section.\n\nQUESTIONS:\nPlease refer to weakness part.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-08"}