Title: Scaling RL to Long Videos
Venue: NeurIPS 2025
Accepted: yes
Keywords: Vision language models

Long video reasoning

Reinforcement Learning
Abstract: We introduce a full-stack framework that scales up reasoning in vision-language models (VLMs) to long videos, leveraging reinforcement learning. We address the unique challenges of long video reasoning by integrating three critical components: (1) a large-scale dataset, LongVideo-Reason, comprising 104K long video QA pairs with high-quality reasoning annotations across diverse domains such as sports, games, and vlogs; (2) a two-stage training pipeline that extends VLMs with chain-of-thought supervised fine-tuning (CoT-SFT) and reinforcement learning (RL); and (3) a training infrastructure for long video RL, named Multi-modal Reinforcement Sequence Parallelism (MR-SP), which incorporates sequence parallelism and a vLLM-based engine tailored for long video, using cached video embeddings for efficient rollout and prefilling. In our experiments, LongVILA-R1-7B achieves strong performance on video benchmarks, reaching 65.1% and 71.1% accuracy on VideoMME without and with subtitles, respectively, and consistently outperforming LongVILA-7B across multiple benchmarks. Moreover, LongVILA-R1-7B supports processing up to 8,192 video frames per video, and configurable FPS settings. Notably, our MR-SP system achieves up to 2.1x speedup on long video RL training. In addition, we release our training system for public availability that supports RL training on various modalities (video, text, and audio), various models (VILA and Qwen series), and even image and video generation models. On a single A100 node (8 GPUs), it supports RL training on hour-long videos (e.g., 3,600 frames). Code and models are available at https://github.com/NVlabs/Long-RL


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper describes a dataset, benchmark and RL-based training methodology for reasoning over long videos. The dataset is created by automatically generating annotations (using existing foundation models) to videos from an existing video dataset. A model trained using the introduced data and training methodology is evaluated on the VideoMME benchmark as well as on the newly introduced benchmark. It shows that the resulting model outperforms several existing models.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n- The paper addresses a difficult (and increasingly popular) task: reasoning in long videos. \n- It addresses the task by introducing a new (automatically generated) dataset and a multi-GPU training strategy supporting RL training on long videos. \n- The paper also introduces a new hand-curated benchmark, which can be useful for revaluating future models (assuming it will be made publicly available, see below).  \n\nWeaknesses: \n- The performance of the newly introduced model is very similar to (and only marginally higher than) LongVILA [3], of which this is in a sense a derivative work. For example, results on MME reported in [3] are 60.1 (vs 60.3 in this work) in the \"w/o subtitle\"-setting and 65.1 (vs 65.9) in the \"w subtitle\"-setting. \n- The performance of the model is actually lower than that of Video-R1-7B reported in [12], which is 61.4 in the 64-frames-setting (Table 1 in [12]). Instead, only the performance of the lower performing 16-frames setting of that work is shown (which is 59.3). Why is that? Am I missing something here? \n- By comparison, the only strong performance over existing methods is on the own (1000 example-)benchmark. \n- The paper is fairly hard to read due to typos, inconsistencies, and overly touting language. (To name a few examples: Line 44: \"long-video understandings\", Line 44: \"we strategically construct a high-quality dataset\", Figure 3 caption: \"..., from which 52K QAs with reasoning annotations in LongVideo-Reason.\", Line 111 \"We then apply a high-quality automated CoT annotation pipeline\", ...\n\nQUESTIONS:\n- The results of LongVILA [3] are missing from Table 2. \n- Why is the 64-frames result from Video-R1-7B [12] not shown or mentioned? \n- Will the code for data generation and the MR-SP training strategy be made available? \n- \"question-answer pairs are generated via a leading open-source reasoning LLM\" - which one? \n- Line 52: \"We also manually curate a balanced set of 1K long-video samples\" - what does \"balanced\" here mean? \n- (small comment: the soccer example in Figure 1 is strange in that the prediction to be made is very stochastic and model correctness will be almost random)\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-24"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents LongVILA-R1: a framework for long video reasoning in VLMs and introduces a new dataset (LongVideo-Reason). The proposed LongVILA-R1 framework consists of a two-stage CoT-SFT and RL training approach. The paper also introduces MR-SP, for efficient for scalable long video RL. The evaluation is performed on the proposed LongVideo-Reason dataset and Video-MME.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n•\tThe LongVideo-Reason dataset is diverse and contains complex video reasoning questions, e.g., tracking balls in a shell game (Figure 12).\n\n•\tThe paper introduces a novel and interesting training paradigm (Multi-modal Reinforcement Sequence Parallelism) that shows promising improvements in training complexity.\n\n•\tThe paper shows promising performance on the proposed LongVideo-Reason-eval benchmark and other datasets such as Video-MME.\n\n**Weaknesses:**\n\n•\tThe proposed LongVideo-Reason dataset is based only on the Shot2Story dataset. As the video score is a single dataset, this might limit the diversity of the LongVideo-Reason dataset. The paper should discuss the diversity of the LongVideo-Reason dataset in more detail.\n\n•\tNovelty: The dataset generation and training scheme – supervised fine-tuning followed by GRPO based RL training is largely based on prior work, e.g., the Deepseek family of models. The novelty of the training scheme should be discussed in more detail.\n\n•\tEvaluation on Video-MME: The paper does not compare the state of the art approaches such as VideoLLaMA3 which achieves better results -- 54.9% accuracy on long videos without subtitles – without the need of RL based training.\n\n•\tEvaluation on LongVideo-Reason-eval: While the proposed model outperforms the Gemini-1.5-Pro and GPT-4o models on this benchmark, the proposed model was already fine-tuned on this dataset. This raises the question whether the comparison with zero-shot methods such as Gemini-1.5-Pro and GPT-4o is fair.\n\n•\tLong videos: The proposed method seems to be limited to 512 frames, while state of the art approaches, e.g., Qwen-2.5-VL can process >=1200 frames.\n\n•\tThe model seems to rely on a stable and low frame rate, it is unclear if the model can deal with variable or high frame rates.\n\nQUESTIONS:\n•\tThe paper should clarify the novelty of the training scheme in more detail.\n\n•\tThe paper should provide more detailed analysis of the diversity of the proposed LongVideo-Reason dataset.\n\n•\tThe paper should clarify the choice of evaluation datasets and baseline methods.\n\nLIMITATIONS:\nYes, but the limitations of the evaluation scheme should also be discussed.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe key idea of this paper is applying GRPO-style reinforcement learning to improve long video understanding. The authors identify key bottlenecks that limit this and solve them, the three main contributions being (1) a LongVideo-Reason training set (2) Multi-modal Reinforcement Sequence Parallelism (MR-SP) training strategy that allows extremely long videos to be used with many parallel inferences, which is required for GRPO and finally (3) LongVideo-Reason-eval benchmark, a handpicked benchmark with 1000 samples. The final result is LongVILA-R1-7B LLM, which can ingest up to 512 video frames while still being able to carry out spatio-temporal reasoning.\n\nThe LongVideo-Reason training corpus supplies 52 K multiple-choice question–answer pairs (with chain-of-thought) drawn from 18 077 long videos, evenly covering Temporal, Goal & Purpose, Spatial, and Plot & Narrative reasoning. Heuristics are used to split the training data into easy, medium and hard subsets.\n\nLongVILA-R1 is trained in two stages: (i) chain-of-thought supervised fine-tuning on \"easy\" and \"hard\" examples, followed by (ii) Group-Relative Policy Optimisation (GRPO) reinforcement learning on \"medium\" cases . Stage 1 is a SFT-CoT stage designed to \"warmup\" the model's reasoning capabilities. In Stage 2, the authors also add data from Video-R1, a prior work, to scale up RL.  To keep RL feasible, the paper introduces Multi-modal Reinforcement Sequence Parallelism (MR-SP), which caches video embeddings and shards giant token contexts so policy and reference models prefill in parallel. \n\nFor evaluation, the authors curated LongVideo-Reason-eval, 1,000 hand-checked problems that mirror the four reasoning categories in the training dataset. LongVILA-R1 averages 67.9% accuracy, exceeding GPT-4o performance. On prior benchmarks like VideoMME, it outperforms open source VideoLLMs by a large margin.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n+ The dataset contributions are valuable to the Video LLM community\n+ The overall execution is thorough and polished, resulting in strong performance across benchmarks\n\nWeaknesses\n\n- Evaluations on more Video understanding benchmarks which are common in the VideoLLMs literature would be appreciated\n- The primary technical contributions are in the Sequence Parallel training, the architecture and training methods (GRPO) are all just directly applied.\n\nQUESTIONS:\nResult on standard video benchmarks like MVBench, ApolloBench etc would be appreciated.\n\nAs dataset is a big part of the contribution, the authors should give more details and visualizations of it, so we can understand the diversity of the dataset.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces LongVILA-R1, a framework that enhances vision-language models (VLMs) for long video reasoning through reinforcement learning. The authors address challenges in long video understanding by developing three core components: (1) LongVideo-Reason dataset with 52K long video QA pairs with reasoning annotations, (2) a two-stage training pipeline combining chain-of-thought supervised fine-tuning (CoT-SFT) with reinforcement learning (RL), and (3) Multi-modal Reinforcement Sequence Parallelism (MR-SP) system for efficient long video RL training. The method scales from 16 to 512 frames and demonstrates competitive performance on VideoMME and their proposed LongVideo-Reason-eval benchmark.\n\nSTRENGTHS AND WEAKNESSES:\n***Strengths***\n\n-\tThe paper is generally well-written with clear explanations of the technical contributions. \n-\tThe paper presents a well-structured approach to long video reasoning with clear methodology. The MR-SP system design is technically sound, achieving 9.2× speedup for long video RL training.\n-\tThe dataset represents a valuable contribution as a resource for training reasoning models on video content.\n\n***Weaknesses***\n\n-\tMissing Key Baseline: The evaluation lacks comparison with LongVILA, an important baseline for this work.\n-\tLimited Performance Gains: The performance improvements over existing baselines are modest and may not justify the substantial computational investment. Specifically, compared to the baseline LongVILA paper [1], LongVILA-7B achieved 60.1 (without subtitles) and 65.1 (with subtitles) on VideoMME with 256 frames, while LongVILA-R1-7B achieves only marginally better results at 60.3 (without subtitles) and 65.9 (with subtitles). Given the extensive additional training (CoT-SFT + RL) and computational resources required, these improvements appear insufficient.\n-\tLimited Evaluation Scope: The evaluation relies primarily on VideoMME and the authors' proprietary LongVideo-Reason-eval benchmark. More comprehensive evaluation across established long video benchmarks (such as EgoSchema, ActivityNet-QA, and LongVideoBench) would better validate the claimed reasoning improvements.\n\n[1] Chen, Yukang, et al. \"Longvila: Scaling long-context visual language models for long videos.\", ICLR2025.\n\nQUESTIONS:\n-\tComprehensive Benchmarking: Could the authors provide evaluation results on additional established long video benchmarks (such as EgoSchema, ActivityNet-QA, and LongVideoBench) to better demonstrate the generalizability and robustness of the reasoning improvements?\n-\tCost-Benefit Analysis: Given the substantial computational investment required for the RL training pipeline, could the authors provide a more detailed analysis of the performance improvements achieved by their method compared to baseline models? Specifically, how do the gains justify the additional computational overhead and training complexity?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-04"}