Scaling RL to Long Videos
Authors: Yukang Chen
Wei Huang
Baifeng Shi
Qinghao Hu
Hanrong Ye
Ligeng Zhu
Zhijian Liu
Pavlo Molchanov
Jan Kautz
XIAOJUAN QI
Sifei Liu
Hongxu Yin
Yao Lu
Song Han
Venue: NeurIPS 2025 poster ()
Keywords: Vision language models
Long video reasoning
Reinforcement Learning
Primary area: applications
TL;DR: Long-RL enables RL on hour-long videos on a single A100; LongVILA-R1-7B supports 8,192 frames and scores 65.1%/71.1% on VideoMME.

Abstract: We introduce a full-stack framework that scales up reasoning in vision-language models (VLMs) to long videos, leveraging reinforcement learning. We address the unique challenges of long video reasoning by integrating three critical components: (1) a large-scale dataset, LongVideo-Reason, comprising 104K long video QA pairs with high-quality reasoning annotations across diverse domains such as sports, games, and vlogs; (2) a two-stage training pipeline that extends VLMs with chain-of-thought supervised fine-tuning (CoT-SFT) and reinforcement learning (RL); and (3) a training infrastructure for long video RL, named Multi-modal Reinforcement Sequence Parallelism (MR-SP), which incorporates sequence parallelism and a vLLM-based engine tailored for long video, using cached video embeddings for efficient rollout and prefilling. In our experiments, LongVILA-R1-7B achieves strong performance on video benchmarks, reaching 65.1% and 71.1% accuracy on VideoMME without and with subtitles, respectively, and consistently outperforming LongVILA-7B across multiple benchmarks. Moreover, LongVILA-R1-7B supports processing up to 8,192 video frames per video, and configurable FPS settings. Notably, our MR-SP system achieves up to 2.1x speedup on long video RL training. In addition, we release our training system for public availability that supports RL training on various modalities (video, text, and audio), various models (VILA and Qwen series), and even image and video generation models. On a single A100 node (8 GPUs), it supports RL training on hour-long videos (e.g., 3,600 frames). Code and models are available at https://github.com/NVlabs/Long-RL

Scaling RL to Long Videos
Yukang Chen I*
Wei Huang! 1.34
Baifeng Shi!.4t
Qinghao Hu2t
Hanrong Ye Xiaojuan Qi
Ligeng Zhu'
Zhijian Liu'
Pavlo Molchanov?
Jan Kautz*
Sifei Liu'
Hongxu Yin'
Yao Lu
Song Han
NVIDIA
MITT "KKU UCC Berkeley
Abstract
We introduce a full-stack framework that scales up reasoning in llaguae models (VLMs) to long videos, leveraging reinforcement learning. We address the unique challenges of long video reasoning by integrating three critical components: (1) large-scale dataset, Long
comprising 104K long video QA pairs with high-quality reasoning annotations across diverse domains such as sports, games, and vlogs; (2) two-stage training pipeline that extends VLMs with chain- of-thought supervised fine-tuning (CoT-SFT) and reinforcement learning (RL); and (3) training infrastructure for long video RL, named Multi-modal Reinforcement Sequence Parallelism (MR-SP), which incorporates sequence parallelism and a vLLM-based engine tailored for long video, using cached video embeddings for efficient rollout and prefilling. In our experiments, Long VILA-RI-7E achieves strong performance on video benchmarks, reaching 65.1% % and 71.1% accuracy on VideoMME without and with subtitles, respectively, and consistently outperforming LongVII.A-BB across multiple benchmarks. Moreover, LongVILA-RI--7 supports processing up to 8,192 video frames per video, and configurable FPS settings. Notably, our MR-SP system achieves up to 2.1 x speedup on long video RL training. In addition, we release our training system for public availability that supports RL training on various modalities (video, text, and audio), various models (VILA and Qwen series), and even image and video generation models. On single A100 node (8 GPUs), it supports RL training on hour-long videos (e.g., 3,600 frames). Code and models are available at httppp//ggiuub..o///alabbb////// and
RL
Introduction
Understanding long videos requires more than simple recognition- demands reasoning from temporal, spatial, goal-oriented, and narrative perspectives [51]. As illustrated in Figure [] answering high-level questions often hinges on a model's ability integrate clues distributed across time, infer hidden goals or strategies, track entities spatially, and comprehend the evolving plot. For instance, predicting the winner of foothall penalty shootout involves assessing emotional cues and tactical behavior (temporal and goal reasoning), while determining the final location of a hidden ball requires precise spatial tracking. Likewise, evaluating a poker player's 's decision demands interpreting implicit strategies beyond surface actions (goal reasoning) and understanding character's 's development or match trajectory reflects the need for plot reasoning. These examples underscore that reasoning is indispensable for long video understanding that goes beyond recognition alone. Despite the clear importance of reasoning in long video understanding. enabling such capabilities in long video VLMs
poses significant challenges 88 43 47 [12 33]. First, the collection of high quality long video reasoning datasets inherently difficult. Unlike domains such as math or code reasoning, where
"qqual contribution
Core contribution
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Football Game
Texas Hold'em
Spatial Track
2022 FIFA Argentina Netherlands
2025 Triton Super High Roller Series
Moving Cup and Guess where the ball
Question: shown 30-minute vidco segment from foothall gume. The score remains 2-2 throughout period, no goals are scored. Based players' physical coodition, lactical behavior, cmotional state. and overall match performance abserved during extra time, which team is more likely the upcoming penalty shootout?
Question: This is a competirive video of a
Question: There are three boxes, a purple ball is initially placed in the middle box. Then positions the boxes are swapped. Please analyze stop by stop the movemaal trajectory through the
Texas Hold'em poker game featuring two
players: Pascal
Leftancols(wcaring sunglasses and green on one side) and Eelis Parssinen (wearing white buseball cap). Based on their betting patterns
of the box containing video and deternine the final position of the ball.
video. deternine Felis Parssinen potential hand. Assume you Pascal Lefrancols after Relis Parssinen goes all-in, decide whether you should call.
The Netherlands the 4-2 negulation time Aegestiaa win the monniiiinn conpouure Nether the shootorr Dobegentica score withor going to peraltis
Arpertin
Fold
Ae Middle
Right.
D:None the above
Rewsoning: <think-Thh score remains 2-2
Reusoning: <think-Pascall Lefrancols the King of Clubs and the Jack of Diamonds, which is a strong hand but not the strongest possible hand. A Koyal Flush (A, K, Q, 10 of the same suit) is the strongest hand, but only the Ace of Spades and King of Hearts are part of Pascal 's hand ...... The Turn card (Ten of Diamonds) and River card (Four of Diamonds) do not complete ...... PASCAL's hand (K. J) is not strong enough outrank Full House (thrce of one tank and two of another).
Reasoning: <shink--The middle box initially contains the purple ball, When the boxes are swapped, the left box becomes the new middle box, the middle box moves to the right, und the right box Since the left box now in the middlo position, the ball, which was previously the middle box, is now in the left box. The swapping, continues with boxes rotating left. but the ball's position does not chango because always the left box. Therefore. the final position of the hall is in the left hox. <thinke-
thronghout time, indicaring high- inteusity match with both icams committed to winning... Argentinn's 's goalkeaper,
unitially struggled, regains confidence and makes crucial save, demonstrating his improved focus and determination. Key players from teams, such Messi and de Ligt. are seen holding their heads in disuppoinrment or celehnarion, reflecting the
the left.
onotionna highs and lows that often docide
penally shoctouts... Given the evidence.
best PASCAL can hope Therefore, PASCAL should fold, as ERLIS's hand is likely superior, </hiiks>
Argentina
likely win the penalty
shootout 4 3, with their goalkeepper playing a
decisive role and key players maintaining composure. <thiuk>>
<Answer> A: Left </Answer>
<Answer> B: Argentina win the penalty shootout 4-3, with their goalkesper playing decisive role and key players maintaining composure. </Answor>
Video Frames
SAnwer> B: Fold </Arywer>
32
64
128
256
B
Figure I: Examples of LongVILA-RI. The illustration demonstrates sample tasks and their reasoning. From left to right, the examples include predicting the results of a football match, decision-making reasoning in Texas Hold' em Poker, and trajectory for spatial dynamics of objects. Notably, the spatial tracking video involves a relatively complex dynamic moving, for which the model fails to achieve accurate reasoning until the number of input video frames increases to 128.
structured supervision and benchmarks are readily available 35 24], long video reasoning requires annotating complex temporal dynamics, goals. spatial relations, and narrative elements- across minutes or hours of footage [14]. This process is labor-intensive and subjective, making large- scale dataset construction slow and costly. Second, the RL training framework for long videos is challenging. Reinforcement learning, a common strategy for aligning models with complex reasoning objectives, is computationally expensive and sample-inete inefffciiet [38. When applied to long videos, RL becomes even more burdensome due to the extended video frames, requiring more memory and longer rollout runtime. These challenges jointly hinder the development of effective long video VLMs with strong reasoning capabilities.
In this work, we introduce LongVIIL--R a comprehensive framework exploring the reasoning capabilities for long video understandings. Firstly, we strategically construct high quality dataset with CoT annotations for long video reasoning, named Long Video-Reason. Leveraging a powerful VLM (NVILA-8B) (27) and a leading open-source reasoning LLM, we develop a dataset comprising 104K high quality PhrestionRReasoning samples for Long-CoT-SFFT to initialize the model's 's reasoning and instruction-followee
pairs for long videos. We use 36K high quality abilities, for reinforcement learning. This two-stage training combines high quality reasoning annotations with reinforcement learning, enabling Long VILA-RI to achieve superior and generalized video reasoning. We also manually curate a balanced set of 1K long video samples to build new benchmark,
and 68K samples with an additional 102K video data (53] 46 31 [8]
44
that evaluates performance from four perspectives: Temporal, Goal and Purpose, Spatial, and Plot and Narrative, for comprehensive assessment.
2

---

500
Qwea2.5-VV VLL7B
101
LongVILLA-RL -1-8
WWO MR-SP
O0M
200
Time (ole/nn
(Nage only)
500
only)
400
200
WW MR-SP (Stage
vol MIK SP (Bogge &
300
1.
151
164
50
200
161
360
154
100
77
42
26
100
25
Kunkerof Franaze
Nunher Pramcx
Figure 2: Training efficiency comparison with MR-SP (SP degree=4) Qwen2.5-VVL VL-BB and LongVILA-RI-e and a single node 8x A100 GPUs. It achieves 2.1 x speed-up and avoids GPU OOM issue long frames.
Long Videos
TAK
reanons)
164K
Ruul 2068
Events
11%
10%t
Sports
Edacation
Anirais
24%
30%
Goal and Purpose Recsoning
Roconia Rosoning
VLVVAVVide (T3K)
News Politcck
Music
Techoology
Spatid Baacoring
35%
50%
PesseptionTls
Next
13%
Entestatmennt
Counedy
Rossoning
a (LLIVER
Finn
Figure Data Distribution of Long Reasonn and total data in the LongVILA-RI training framework. Long Video-Reason comprises a total of 18K videos and 104K QAs with reasoning annotations. Additionally, we include 102K QAs from existing works
18 44
Secondly, propose training framework for VLMs advance long video reasoning. As illustrated in F this framework incorporates two stages, ie., Stage-1: Long CoT-SFT, and Stage-2: RL for long video reasoning. To address the unique challenges of long video RL, including massive visual embeddings, heavy rollouts, and long-con long-context LLM prefilling, we develop an efficient and scalable solution, referred as Multi-modal Reinforcement Sequence Parallelism (MR-SP). incorporates vLLM engine [19] tailored for LongVILA and a caching scheme for video embeddings. The MR-SP system alleviates the problem of intensive memory and facilitates RL training of long video VLMs. As shown in Figure 2 our MR-SP system achieves up to 2.1 x speedup on 512-frame video RL training on 7B models and enables longer training frames without out-of- -omory (OOM).
In our experiments, LongVILA-H 1-7B demonstrates strong performance on video benchmarks, achieving 65.1 % and 71.1% % accuracy VideoMME without and with subtitles, respectively. It consistently outperforms Long VILA--BB across range of benchmarks, including ActivityNer-QA, LongVideo VideoBench, Perception NExT-QA, VNBench, and VideoMME. Additionally, Long RI supports processing up to 8,192 video frames per video, and configurable FPS settings. On our Long -easo--asssoo benchmark, Long LongVILA-RI-71 achieves an average accuracy of 72.0%, surpassing Video-RI -1-eo-R1-7BB3 as well as proprietary models such as Gemini- Geminii-15-PPrg 1.5-
2 Related Work
Multi-modal reasoning models. The field of multi-modal reasoning has advanced significantly, particularly in Vision-Language Models (VLMs). GPT-4o [29] improves visual understanding
through enhanced reasoning, while Gemini-1.5-Pro 34 extends context length to 1 million tokens, Following the substantial progress of
achieving state-of-the-arr performance in VideoMME 9
architecture and training algorithms in open-source VLMs 56
27
48 23 39], multi-modaa reasoning
has been further explored works including LMM-RI
[32 which employs a two-stage training
strategy, Vision-RI (15] that addresses post-cold-start overthinking, and Video-RI [8] which enhances
RL for video via T-GRPO 16 frames. However, these approaches primarily focus on single images or short videos, and long video reasoning still poses great challenges.
Sequence parallelism. Training with long contexts often exceeds the memory capacity single device [4] necessitating efficient distribution strategies. Sequence parallelism (SP) has become
3

---

For example, ring-based systems like LightSeq 20 and Ring Attention 25 use point-to-poin (P2P) communication, while DeepSpeed-Ulysee [ employs all-to-all (A2A) primitives to optimize attention computations. Additionally, USP and Loong Toain Train 11 were introduced to integrate Ring-style SP and Ulysses SP. Long VILAA 47 further proposed multi-modal SP (MM-SP), enabling vision-language models to handle long-contexi inputs. However, multi-modal reinforcement learning introduces additional challenges, as it requires extensive sampling from long, mixed-token sequences [38], particularly in complex group optimization tasks [35]
widely adopted solution 22 16
Reinforcement Learning (RL) has become a key strategy for enhancing Large Language Models (LLMs), particularly through Reinforcement Learning with Human Feedback (RLHF) [0) or Direct Preference Optimization 33] which aligns model outputs with human preferences. Recent advancements demonstrate that RL significantly improves LLM reasoning abilitics. For example. DeepSeck-R. R1 [6] utilizes the Group Relative Policy Optimization (GRPO) algorithm (36,, integrating group-based based sampling and rule-based rewards. On the other hand, RL [1 poses a unique challenge of heavy computational cost, especially in multi-modal settings. To address this problem, HybridFlow (8) is introduced, leveraging Ray (88] for efficient data flow and vL..M [19] for faster sampling. Nevertheless, this remains a bottleneck when processing long video sequences, with group-based sampling constrained by the high computational cost of long-context sampling and visual encoding. In this work, we propose MR-SP that provides up to 2. I x speedup.
RL frameworks for LLMs/VLMs.
3 Long Video-Reason Data Construction
3.1 Overview of Data Curation
We first curate 18K long videos from the Shot2Story dataset 13 (Figure 3 left). We further
incorporated 2k additional 4K-resolution videos spanning scenarios such as autonomous driving, video games, household robotics, and wildlife. We then apply high quality automated annotation pipeline for CoT as detailed in Section 3.2 and end up with a total of 104K Quustion-Reasonee Answer pairs where each sample, based on the type of question it is reasoning about, can be categorized into Temporal Reasoning, Goal and Purpose Reasoning, Spatial Reasoning, or Plot and Narrative Reasoning (Figure 3 middle). This dataset is designed to support various types of long-video reasoning tasks comprehensively.
Given the sensitivity of GRPO to batch sampling 40 32 a data filtering approach is adopted. Specifically, we employ a test-scaling method where LongVILA 47 performs inference 10 times on the original datasets. Questions consistently answered correctly or incorrectly are labeled as easy or hard while those inducing diverse predictions are labeled as medium. We filter out too easy and too hard questions, and use remaining questions for training. The reason is that GRPO expects different rollouts of each sample to be diverse in order to have meaningful advantages, and the gradient vanishes if all the rollouts predict correct or incorrect answers. The COT-SFT subset (36K) features high quality CoT reasoning processes formatted in standard <think>><hhink> <aciiiik><aasww>>>////mees
structure, providing abundant resources for warm-up training during Stage 1 of the model's reasoning capabilities. Meanwhile, the RL subset contains 68K challenging long-video Q&A, which is leveraged in Stage 2 for scaling reasoning through reinforcement learning. To further enhance RL scaling, we incorporate an additional 102K high quality open-source videos (Figure 3 right) from other
datasets 53 46 18 44. This combination improves the model's generalization.
3.2 Long-Vides Reasoning Generation
We introduce an automated annotation pipeline (Figure 4 that generates high quality Question- Reasoning-Answw pairs from long videos. This process begins by segmenting videos into short clips (-10 seconds each), each of which is annotated using the NVILA-80 8B (2 model to provide descriptive captions. For the spatial reasoning category, We employed VILA-HD object bounding boxes in video frames and constructed spatial reasoning QAs based on these boxes and corresponding captions. Leveraging the breakthrough text-based reasoning, we then deploy a leading open-source reasoning LLM, DeepSeek-R R R1-671B ( provide the captions of all the clips in Ansser pairs that involve
39 to generate
each video, and prompt it to generate diverse types of reasoning over the content across the whole video.

---

Long Video Collection
Group-clips Caption
Long- Generation
Captaan
Rasssoming LLM
Temporal Spatia
Lang- Cat Reasming sleps
Goat & Purpose
VILA
Captare
Flot Narrative
Capesn
NQ: Quaetion
(arp The frui involer
Saep &n Drreenss of drsctinal
Long Videos
Short Video Clips
Captun
Question
Reasoning Generation
Figure 4: Data generation process for the Long Video-Reason dataset. This process begins with segmenting videos into 10-second clips and generating captions for each clip using NVILA-SB Then based on the captions all clips in a video, we generate question-anse -nnswer pairs that involve reasoning across the content of the whole video. along with the reasoning annotations using leading open-source reasoning LLM. Reasoning questions are categorized into Temporal, Goal and Purpose, Spatial, and Plot and Narrative. Finally, the reasoning annotations reformatted for conciseness and alignment with video details. We present a more detailed figure of data generation process in Figure
Specifically, design four types of prompts to encourage the LLM to generate Question-Reasoning- Answer pair that focuses on one of the four types of reasoning: Temporal Reasoning, Goal and Purpose Reasoning, Spatial Reasoning, or Plot and Narrative Reasoning. To ensure VLMs focus visual details, we also craft the prompts with phrases such as "checking the video" and "analyzing the scene", which guide iterative examination of visual content. Finally, L.L. is then used to refine and streamline the reasoning steps. We also manually curated 1,000 high quality complex reasoning questions across four reasoning categories serve for evaluating VLMs in reasoning abilities. Note that we also include open-ended questions. Among the total 104K QA pairs, approximately half are multiple-choice and hald are open-ended. This entire data procedure consumes about 80,000 H100 GPU hours.
benchmark (Long Video-Reason--ee
LongVILA Training Pipeline
As shown in Figure 5 there two extended training stages in LongVILA-RI, i.e., (I) warm-up for long video reasoning, utilizing 36K data with high quality CoT for SFT on the MM-SP system [47]; (2) reinforcement learning with dense frames from long videos.
4.1 Long Video CoT Supervised Fine- Sine-Tuning
pairs, we apply the data filtering method described in Section 3.1 to select 36K examples for long CoT-SFT, serving as warm-up phase for subsequent RL. This stage equips the model with fundamental reasoning abilities and instruction- following skills for long video scenarios. To efficiently perform SFT on hundreds of frames, we adopt the MM-SP [47] training system from VInA As demonstrated in Section 66.2] SFT solely on our LongVideo- Siidoo-Resoo dataset also effectively improves the model with reasoning capabilities.
Utilizing 104K high quality question-reassnnnng-an
4.2 GRPO for Long Video
Building the advancements of the GRPO (35] algorithm and prior explorations of multi-moda reasoning training 8 32), we adhere to the standard GRPO framework to train our model. For each given question g. the policy model generates group of candidate responses {01, o2, OG) from the old policy To,.. accompanied by their corresponding rewards {r1, T2, ... rG which are computed based on rule-based reward functions (format/accuracy). The model my subsequently optimized by maximizing the following objective function:
G To Ta(0)14)
O(0) = Eyy,(o.)
Σ
(min(
Toae (o:1g) A,
G
1=1
clip( Te (o q)
- (1)
Naae (o:1g)
where and /3 are hyper-parameters, set as 8 in our experiments, and the sampled rewards above normalized to get the advantages (A,) for updating the model:
5

---

Mid-training Contect Extension
Reasoning Warmup
Reasoning Scaling
Long Video SFT
VILA
w/ MM-SP
LongVILA
Stage=1
Long Video RL LoAgVILA-R1
Stage-2
Long CoT SFT
MM-SP
MR-SP
Figure 5: The Long training pipeline. LongVILA-RE VILA-RI builds upon the base training pipeline for Long MM-SP is further employed for SFT on long video understanding tasks with long CoT. Then, reinforcement scaling learning is conducted through Multi-modal Reinforcement Sequential Parallelism (MR-SP).
Videos
Embeddinggs colling out pre reelfiliing
Policy Model
Pollcy Model
W/SP
Policy Model
w'vilm engine 52
Reforence Model
Update policy
For rolling
Responses
For pro-filing
Logis
Figure 6: LongVILA-RI RL training framework. The framework integrates multi-modal rein- forcement sequence parallelism (MR-SP) for scalable video frame encoding and LLM prefilling. RL utilizes bLsee engine with cached video embeddings, tailored for Long rollout. Rewards for accuracy and format guide policy optimization.
A; -0e-mman(ee (11, rG)...
strd(f1.))) P2.,, ral)
However, RL for long videos presents significant challenges due to the high computational demands of processing hundreds to thousands of frames. Existing RL frameworks struggle with such long- context training in rollout and LLM prefilling. To address this, we develop the MR-SP framework (Section which efficiently scales reinforcement learning for long-context video reasoning.
Considering the sensitivity GRPO to sampling during training [40), we use the 68K filtered data for reinforcement learning described in Section 3.1 Additionally, extra 102K samples from other datasets [53] 46 31 18, 44 are incorporated to scale up the RL. This approach aims to guide
the model in freely exploring and developing
effective and generalized reasoning strategies.
5 Multi-modal Reinforcement SP
Existing RL frameworks for VLMs, such as RI-V I and EasyRI [55], are not designed for long videos which present unique challenges due to their high token volume. To address this, we introduce Multi-modal Reinforcement Sequence Parallelism (MR-SP), a framework for efficient RL. training on long videos. MR-SP leverages sequence parallelism in both rollout and pre-filling stages, enabling long videos RL, with reduced overhead. We show the training curve with MR-SP Figure S
5.1 Stage Rollout with Paralleled Encoding
To support long-video reinforcement learning efficiently, we adopt sequence parallelism (SP) for the video encoding stage. As shown in Figure 7 the input video frames are first evenly divided across multiple GPUs (c.g., GPU GPU 3), each equipped with its vision tower. Each GPU independently processes a slice of the vidco, encoding only subset of the frames. The resulting video embeddings then aggregated with text embeddings via an all-gather operation indicated by the "All-Gather" arrow in the figure. This strategy distributes the encoding workload, allowing the system handle significantly longer videos by leveraging more GPUs, while avoiding the risk of GPU memory overflow. The parallel encoding scheme ensures balanced utilization the vision towers and enables scalable long-video processing that would otherwise be infeasible.
After the video embeddings are globally gathered, they are reused for downstream usage throughout the RL pipeline. As illustrated in Figure 7 the gathered embeddings reused during multiple

---

GPU
Visaa
Towes
LLM
(2) Podding Sequenze
GPU
A A A A A
Vistian
Torwer
A
GPU
(1) Sharing Local GPU CPU
? vIL.M engine (LIMM)
GPUS
GPU
GPU1
Vision
Tower
Rallout
D
O
Rallout
O
O
Text Pronpt
Rallout
O
P2P Communication
Enable
Stage Rollout with Paraileeed Fncoding
Stage 2 Prefilling with Sequence Parallelism
Figure 7: The workflow of multi multi-modal reinforcement sequential parallel (MR-SP). accommodate multi-modal input in reinforcement learning, we develop custom sharding strategy to ensure balanced workload distribution and compatibility with SP communication. Efficient video embedding reuse and vLLM rollout acceleration strategies implemented, while meeting the demands of policy model prefilling for dense video frames.
rowariToernat
suggel eo
long
Wy
0.5
3.5
300
0
Figure 8: Reward curve of our LongVILA-RI-1 with MR-SP. During training, LongVILA-RII7D exhibits stable reward improvements across overall, format, and accuracy dimensions.
rollouts without recomputation. For instance, in each training step. we typically perform 8 16 rollouts. Without recycling. the same video would need to be re-encoded dozens of times per step, severely impacting training speed. By caching and reusing the gathered embeddings. MR-SP eliminates this redundancy and significantly accelerates training.
5.2 Stage 2 Prefilling with Sequence Parallelism
For each rollout, both the reference and policy models require prefilling compute-intensively in RL for long videos. With the gathered embedding from Stage reused, we parallelize the inference stage across devices using sequence parallelism. As illustrated in Figure 7 we globally gathered input embeddings are first padded to uniform length (Padding Sequence) and then evenly partitioned across GPUs (Sharding to Local GPU). This allows each GPU to handle only a portion of the input sequence during prefilling. This parallelism is applied to both policy and reference model prefilling. Then, each GPU locally computes logits for its token slice, prefilling in parallel.
Experimental Results
6.1 Main Results
Table
shows the performance comparison on 6 video benchmarks 49 31
46 54
9
Long
R1-7B consistently outperforms LongVILA, across all benchmarks, with performance gaps that vary according to the complexity of the reasoning tasks. Table 3 presents the general performance of
VILA-RL comparing to existing advanced models [22 [2 21
42 51
g 57
27 50
under comparable model sizes on the Video-MME 9 benchmark. The LonnVVLA--R1-TI is
tested using 512 video frames as inputs. Long VILA-R1-7B achieves leading scores across different
video lengths, obtaining scores of 65.1% % and 7 1. % in the settings without subtitles and with subtitles,

---

Table 1: Performance ActivityNet-QA
49
LongVideoBench 45
Perception 31, NExT-
QA [46] VNBench [54], and VideoMME 19. LongVILA-RR-77B outperforms LongVILA across all these benchmarks, with varying margins.
Net.Net-QA
L.Bench val
PerTest val
NEXT-QA
VNBench val
VideoMME
Model
w/o sub.
w/ sub.
GPT-4o mini
56.5
64.8
68.9
GPT-4o
61.9
66.7
64.4 66.7
71.9
77.2
Gemini-1.5-Pro
57.5
64.0
75.0
81.3
Video-LLaVA-731
45.3
37.6
12.4
39.9
41.6
Dash--SSteeam-71
51.9
61.6
ShareGPT4Video- 8B
50.8
41.8
39,9
43.6
Vide2LLaMA2-7B
50.2
51.4
4.5
47.9
50.3
Videol Lideo1.1aMA2.1-78
53.0
54.9
54.9
56.4
Kangaroo-8B
54.8
56.0
57.6
PLLaVA-7B
56.3
39.2
LLaVA-OV-7B
56.7
56.4
57.1
79.4
51.8
58.2
61.5
LongVILA-7B
59.5
57.1
58.1
80.7
63.0
60.1
65.1
LongVIL.A-R1-7B
64.8
58.0
68.9
81.5
75.5
65.1
71.1
Table 2: Performance on Long LongVideo-Reasonee
LongV VILA------ achieves a strong overall score.
Model
Temporal
Goal
Plot
Spatial
Overall
Video-R1-7B 8
61.4
85.0
62.0
58.5
68.1 I
Gemini-1.5- 34
65.4
81.9
67.8
53.3
69.3
LongVIILA-7B
58.0
80.2
57.1
46.7
62.7
LongVILA LongVILA-
68.1
85.7
70.6
53.3
72.0
respectively. Table[ccoppares the results of our LongVideo- Lodde--Reason-evar benchmark. LongVILA-RI- 7B model achieves strong performance with an average score 72.0%, surpassing Video-R1-7B and slightly outperforms Gemini-1.5.4 334 Qualitative results are in the appendix.
6.2 Ablation Study
Scaling video frames. The reasoning capability of LongVILA-RI scales consistently with the number of input video frames. Specifically, Table 5 illustrates the performance of LongVIL VILA- 1.5B (grey line) and Long VILL--1.5--E (red line) on the long-video reasoning benchmark under varying frame inputs. With only 16 input frames, LongVILA-PR -I-L--R---.SBB
close Long VILA-1..... However, as the number of frames increases to 512, Long VILA-RI-A R1-1.5B consistently outperforms and eventually achieves score of 64.3%. Notably, LongVILA-R---L.5 demonstrates steady performance improvements throughout the scaling process. In contrast, LongVILA VILA-115B hits a performance bottleneck with 256 frames, and got degradation on 512 frames. The enhanced reasoning capabilities of Long VILA----1.55 allow it to effectively integrate and infer information from long videos.
Ablation on pipeline and datasets. As shown in Table we ablate the effectiveness of training stages and datasets, using LongVILA-1.5B Seeggviieo--oo----ee
starting point. The accuracies are evaluated on X means skipping this stage, means training this stage with our datasets, and O means training this stage with other datasets [53] 46 18 44. Our CoT-SFT dataset results a better performance than that of other datasets. addition, incorporating RL on top of the warm-up phase (CoT-SFT) yields additional improvements compared to using only SFT. We show that we skip CoT-SFT and train our models directly with RL, the accuracy drops. we apply Video-RI datasets in both CoT-SFT and RL stages, the performance is inferior to using ours.
Training efficiency on MR-SP. We conduct the training efficiency comparison for our MR-SP system, one A100 node, ie.. 8xA100 (80GB) GPUs. We measure the forward time for each training

---

Table 3: Performance comparison VideoMME 9 benchmark in details.
w/o subtitle
subtitle
Model
Overall
Short
Medium
Long
Overall
Short
Medium
Long
Video-RI-7B
61.4
Apollo-7B [5
61.1
63.3
ALaVA-Vidco-22 153
63.3
69.7
NVILA-8B-Vider 27
64.2
70.0
Vidco-LLaVA-77B
39,9
45.3
38.0
36.2
41.6
46.1
40.7
38,1
SliME-8B [52]
45.3
53.3
55.4
39.8
47.2
55.4
44.4
41.7
ShareGPT4Video-SB
39.9
48.3
36.3
35.0
43.6
53.6
39.3
37.9
VidcoChat2-7B [21
39.5
48.3
37.0
33.2
43.8
52.8
39.4
39.2
Chat-Univi-v1.5-7B
40.6
45.7
40.3
35.8
45.9
51.2
44.6
41.8
Kangaroo-SB [26
56.0
66.1
55.3
46.7
57.6
68.0
55.4
49.3
ShareGemini-7B
43.2
41.3
47.9
49.1
47.3
43.4
LongVA-7B [5T]
52.6
61.1
50.4
46.2
54.3
61.1
53.6
47.6
VIIA-1.5-7B [10
56.1
67.0
54.2
47.1
58.7
69.9
55.7
50.4
LongVII.A-7BB [47]
60.1
69.0
58.3
53.0
65.1
72.9
64.9
57.4
LongVII.ARRI-7B
65.1
76.8
63.2
55.2
71.1
79.2
69.7
64.3
Table 4:
Ablations on frames and reasoning (R.).
Table Ablations on CoT-SFT and RL.
Frames
16
32
64
128
256
512
CoT-SFT RL
X
0
0
x
X
0
w/o R.
55.7
56.4
58.1
60.5
60.7
60.2
w/
55.9
56.7
61.9
62.6
64.1
64.3
Accuracy
58.1
60.2
52.4
59.1
59.4
61.9
step. The results obtained after 10 warming up iterations and averaged over 5 iterations. We LongVILA-77-RI model with training batch size as per GPU and rollout number as 5.
Figure 2 presents the training efficiency comparison across different numbers of frames. The figure plots the runtime per step (in seconds) for three settings: the plain RL system without MR-SP, MR-SP Stage only, and the full MR-SP system (Stage & 2). The baseline runtime increases steeply as the frame count grows. Using only Stage of MR-SP significantly improves efficiency up to 512 frames but encounters GPU out-of memory (OOM) issues beyond that point. In contrast, the full MR-SP system consistently reduces runtime, achieving up to a 2. 2.1x speedup at 512 frames and scaling efficiently to 1024 frames without OOM, highlighting the benefit of combining sequence reuse and sequence parallelism for long video RL. training.
Conclusion
We present comprehensive framework designed to fully scale VLMMs for reasoning over long videos. LongVILA-RI encompasses a meticulously constructed large-scale dataset, Long Wideo-Reasoo and parallelized training Framework, MR-SP. Leveraging our curated dataset of 104K long video ceattion-reasonin--anee
pairs, combined with other open-source video datasets, we adopt two- stage training process that integrates CoT-SFT and RL. LongVILA-RII-P7 demonstrates outstanding performance on mainstream video benchmarks, achieving 65. % and 71.1% on VideoMME without and with subtitles. LongVILA-RP supports processing up to 8,192 video frames per video, and configurable FPS settings. Notably, our MR-SP leads to a speed up of 2.1x for long video RL. training, supporting hour-level (3,600 frames) RL training on single node addition, we release our training system to the public, which supports RI. training across multiple modalities (video, text, and audio), various models (ineluding VILA and Qwen series), and even image and video generation models.
A100 GPUs. In
While our RL training system capable of handling long video frames efficiently (3,600 frames single node of A100 GPUs), scaling significantly longer sequences, more modalities (like include audio for omni VL.Ss) or large batch sizes would require distributed training across multiple GPUs. This requires more GPUs, making it less feasible to run limited resources.
Limitations

---

Lang
Group-elips Caption
Duta Reformatted
Thoupicnss dipkia
VIL
dare The Bergueri avelisn
Te sere vmuape on cnuer viM
poseid and dul aneepta..
The (hrooppotio
13 ael siandie
Thoss IoRo oepaaes seaeane (ey legs
The emgg saee
High Quality Reasoning
Thehhikprood la sooe
kach
keniiig lhsugh the groo in raedk
Large Frames Long Video
10-4ccond Short Video Clips
Captions Each Clip
You
U&&AA Long-C Cut Geseration
"day cellew
Reasoning
Eill tarn
cemuuty vallo and denem mad procies
Teuppral
jgg What is sptiray
Sasall tha aecottpp lictered
valoe"
poin
(M Par.Pronpe
dee
Kourminng Arouge
You art33
WR Step 1:Th Eaily
2017 Furract
Langelat
planing
Siendin ppectinn Jasloce
ExEei wien The fealoops
Langle
plensin criec
cuplann
beguage and
Spotil
Narrativ
capp undooraad
Spatial Longg Col
Narmitive Lang
relitrcte theoe
how an ac dadepeeiinnd
AAcA Bi.. Ch.cl...
Shor in inaiherr ae vode
AN de... Co.ilee.........................
with D.
vakp cerace...
Pe Al 14 the hods
aht Step Fooe 200-30
Cheilaphe Noton
Pirt
Question and Answer Pairs
Generation Prompt
Long-Cor Cut Reasoning Generation
Duta Subclass
Figure 9: Detailed data generation process for the LongVideo Reason dataset, supplementing Figure
8 Border Impacts
The development of LongVILA-R represents a significant advancement in long video reasoning, enabling real world systems to perform sophisticated temporal and compositional understanding across diverse and extended visual contexts. This progress holds transformative potential across multiple domains, including embodied AI, robotics, autonomous systems, and AR/VR applications. By equipping vision-language models (VLMs) with the ability to process and reason over long- duration video data, LongVILA lays the foundation for AI systems capable of understanding event sequences, and inferring causal and physical relationships over extended frames.
Long video reasoning technologies powered by LongVILA-RE VILA-RI can significantly enhance embodied Al and robotics by enabling agents to sustain coherent, long-term understanding of their environ- ment. Robots equipped with such capabilities would excel in performing complex, multi-stage tasks, adapting to dynamic contexts, and building richer world models for planning and decision- These advancements also promise to unlock new opportunities in education, healthcare, and enter- tainment. For instance, long video understanding could enable AI tutors to analyze and summarize extended instructional videos, or assist healthcare professionals in reviewing lengthy procedural recordings. Furthermore, such systems could enhance sports analytics, and other areas requiring nuanced temporal reasoning.
Figure 9 shows the the data-generation where our supervision is built from grouped short clips that form long videos, per-clip captions, and L.Le-egerrtted Q&A with Long-Col that is formatted as multiple-choice reasoning rather than identity labels or form attribution. This design choice intentionally steers the supervision signal toward temporal and causal evidence (what happens, when, and why) and away from sensitive attributes (who person is), thereby reducing privacy and profiling risks at the data layer. Concretely, we derive questions from captions and temporally grounded observations instead of collecting personal metadata; we avoid tasks that require demographic inference, face recognition, or persistent identity tracking; and we release structured Q&A rather than user-originating personal content. In this way, the dataset construction supports the intended societal benefits of long-video reasoning while mitigating risks such as privacy leakage, surveillance-Iie use, and stereotype amplification.
In conclusion, Long VOLA-ILA- demonstrates the potential of long video reasoning to drive progress across wide range of applications, from robotics to immersive virtual environments. However, unlocking the full promise of this technology requires a steadfast commitment to ethical principles, privacy protection, and the broader goal of benefiting humanity. By addressing these challenges, the Al community can ensure that advancements in long video reasoning contribute positively to society while mitigating associated risks.
10

---

References
[1] Liang Chen, Lei Li, Haozhe Zhao, Yifan Song, and Vinci. R1-v: Reinforcing super generaliza- tion ability vision-language models with less than $3, 2025. Accessed: 2025-02-02.
[2] Lin Chen, Xilin Wei, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Zhenyu Tang, Yuan, et al. Sharegpt4video: Improving video understanding and generation with better captions. NeurIPS, 177119472-199995 2024.
[3] Qiaoling Chen, Diandian Gu, Guoteng Wang. Xun Chen, YingTong Xiong, Ting Huang, Qinghao Hu, Xin Jin, Yonggang Wen, Tianwei Zhang, and Peng Sun. Internevo: Efficient long-sequence large language model training via hybrid parallelism and redundant sharding. CoRR. abs/2401.091499 2024.
[4] Yukang Chen, Shengju Qian, Haotian Tang, Xin Lai, Zhijian Liu, Song Han, and Jiaya Jia. Longlora: Efficient fine-tuning of long-context large language models. In ICLR, 2024.
[5] Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui. Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, Lixin Gu, Xuehui Wang, Qingyun Li, Yimin Ren, Zixuan Chen, Jiapeng Luo, Jiahao Wang. Tan Jiang, Bo Wang. Conghui He, Botian Shi, Xingcheng Zhang, Han Lv, Yi Wang, Wenqi Shao, Pei Chu, Zhongying Tu, Tong He, Zhiyong Wu, Huipeng Deng, Jiaye Ge, Kai Chen, Min Dou, Lewei Lu, Xizhou Zhu, Tong Lu, Dahua Lin, Yu Qiao, Jifeng Dai, and Wenhai Wang. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. CoRR, abs/2412.00277/ 2024.
[6] DeepSeek-AI. Deepseek-r1: Incentivizing reasoning capability in Ilms via reinforcement learning. CoRR, abs/2501.12944 2025.
[7] Jiarui Fang and Shangchun Zhao. USP: unified sequence parallelism approach for long context generative AI. CoRR, abs/2405.07719, 2024.
[8] Kaituo Feng, Kaixiong Gong, Bohao Li, Zonghao Guo, Yibing Wang, Tianshuo Peng. Benyou Wang, and Xiangyu Yue. Video-rl: Reinforcing video reasoning in milms. CoRR, abs/2503.21776, 2025.
[9] Chaoyou Fu, Yuhan Dai, Yondong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, Peixian Chen, Yanwei Li, Shaohui Lin, Sirui Zhao, Ke Li, Tong Xu, Xiawu Zheng, Enhong Chen, Rongrong Ji, and Xing Sun. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal lims in video analysis. CoRR, abs/2405.210055, 2024.
[10] Chaoyou Fu, Haojia Lin, Xiong Wang, Yifan Zhang, Yunhang Shen, Xiaoyu Liu, Haoyu Cao, Zuwei Long, Heting Gao, Ke Li, Long Ma, Xiawu Zheng, Rongrong Ji, Xing Sun, Caifeng Shan, and Ran He. VITA-1.5: towards gpt-4o level real-time vision and speech interaction. CoRR,
[11] Diandian Gu, Peng Sun, Qinghao Hu, Ting Huang, Xun Chen, Yingtong Xiong, Guoteng Wang, Qiaoling Chen, Shangchun Zhao, Jiarui Fang, Yonggang Wen, Tianwei Zhang, Xin Jin, and Xuanzhe Liu. Loongtrain: Efficient training of long-sequence llms with head-context parallelism. CoRR, pdf/2406. 18485, 2024.
[12] Yanan Guo, Wenhui Dong, Jun Song, Shiding Zhu, Xuan Zhang, Hanqing Yang, Yingbo Wang, Yang Du, Xianing Chen, and Bo Zheng. Fila-video: Spatio-temporal compression for fine-grained long video understanding. CoRR, abs/2504.20384, 2025.
[13] Mingfei Han, Linjie Yang, Xiaojun Chang, and Heng Wang. Shot2story20k: A new benchmark for comprehensive understanding of multi-shot videos. CoRR, abs/2312. 10300. 2023.
[14] Songhao Han, Wei Huang, Hairong Shi, Le Zhuo, Xiu Su, Shifeng Zhang, Xu Zhou, Xiaojuan Qi. Yue Liao, and Si Liu. Videoespresso: A large-scale sfain-of-thoughh dataset for fine-grained video reasoning via core frame selection. CoRR, abs/2411.14794, 2024.
11

---

[15] Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. Vision-rl: Incentivizing reasoning capability in multimodal large language models. CoRR, 0hns/200.00674999 2025.
[16] Sam Ade Jacobs, Masahiro Tanaka, Chengming Zhang, Minjia Zhang, Shuaiwen Leon Song, Samyam Rajbhandari, and Yuxiong He. Deepspeed ulysses: System optimizations for enabling training of extreme long sequence transformer models. CoRR, abs/2309. 14509, 2023.
[17] Peng Jin, Ryuichi Takanobu, Wancai Zhang, Xiaochun Cao, and Li Yuan. Chat-univi: Unified visual representation empowers large language models with image and video understanding. In CVPR, pages 13700-13710, 2024.
[18) Justin Johnson, Bharath Hariharan, Laurens van der Maaten, Li Fei-Fei, C. Lawrence Zitnick, and Ross B. Girshick. CLEVR: A diagnostic dataset for compositional language and elementary visual reasoning. In CVPR, pages 1988-1997, 2017.
[19] Woosuk Kwon, Zhuohan Li, Siyuan Zhuang. Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large lan- guage model serving with pagedattention. Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023.
[20] Dacheng Li, Rulin Shao, Anze Xie, Eric Xing, Joseph E Gonzalez, Ion Stoica, Xuezhe Ma, and Hao Zhang. Lightseq: Sequence level parallelism for distributed training of long context transformers. 2023.
[21] Kunchang Li, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang. Limin Wang, and Yu Qiao. Videochat: Chat-centric video understanding. CORR, fos/2305.06355 2023.
[22] Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, and Li Yuan. Video-llava -lava: Learning united visual representation by alignment before projection. EMNLP, pages 5971-5984. ACL, 2024.
[23] Ji Lin, Hongxu Yin, Wei Ping, Pavlo Molchanov, Mohammad Shoeybi, and Song Han. Vila: On pre-training for visual language models. In CVPR, pages 26689-26699, 2024.
[24] Changshu Liu, Shizhuo Dylan Zhang, and Reyhaneh Jabbarvand. Codemind: A framework to challenge large language models for code reasoning. CoRR, abs/2402.09664, 2024.
[25] Hao Liu, Matei Zaharia, and Pieter Abbeel. Ring attention with blockwise transformers for near-infinite context. CoRR, abs/2310.01826 2023.
[26] Jiajun Liu, Yibing Wang. Hanghang Ma, Xiaoping Wu, Xiaoqi Ma, Xiaoming Wei, Jianbin Jiao, Enhua Wu, and Jie Hu. Kangaroo: A powerful ladeu-language model supporting long-context video input. CoRR, abs/2408. 15542, 2024.
(27] Zhijian I. Liu, Ligeng Zhu, Baifeng Shi, Zhuoyang Zhang, Yuming Lou, Shang Yang, Haocheng Xi, Shiyi Cao, Yuxian Gu, Dacheng Li, et al. NVILA: efficient frontier visual language models. CoRR, abs/2412.044
2024.
[28] Philipp Moritz, Robert Nishihara, Stephanie Wang, Alexey Tumanov, Richard Liaw, Eric Liang, Melih Elibol, Zongheng Yang, William Paul, Michael I Jordan, al. Ray: A distributed framework for emerging (AI) applications. In OSDI, pages 561-577, 2018.
[29] OpenAl. Gpt-4o. 2025. 2025.
[30] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. NeurIPS, 35:27730-77744, 2022.
[31] Viorica Patraucean, Lucas Smaira, Ankush Gupta, Adrià Recasens, Larisa Markeeva, Dylan Banarse, Skanda Koppula, Joseph Heyward, Mateusz Malinowski. Yi Yang, Carl Doersch, Tatiana Matejovicova, Yury Sulsky, Antoine Miech, Alexandre Fréchette, Hanna Klimezak, Raphael Koster, Junlin Zhang, Stephanie Winkler, Yusuf Aytar, Simon Osindero, Dima Damen, Andrew Zisserman, and João Carreira. Perception test: A diagnostic benchmark for multimodal video models. In NewrIPS, 2023.
12

---

[32] Yingzhe Peng, Gongrui Zhang, Miaosen Zhang, Zhiyuan You, Jie Liu, Qipeng Zhu, Kai Yang. Xingzhong Xu, Xin Geng, and Xu Yang. LMM-RI: empowering 3b Imms with strong reasoning abilities through two-stage rule-based RI.. CoRR, abs/2503.075.3 2025.
[33] Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. In NeurlPS, 2023.
[34] Machel Reid, Nikolay Savinov, Denis Teplyashin, Dmitry Lepikhin, Timothy Lillicrap, Jean- Baptiste Alayrac, Radu Soricut, Angeliki Lazaridou, Orhan Firat, Julian Schritwieser, loannis Antonoglou, Rohan Anil, Sebastian Borgeaud, Andrew M. Dai, Katie Millican, Ethan Dyer, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. CoRR, abs/2403.05530, 2024.
[35] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. CoRR, abs/2402.033000, 2024.
[36] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y. K. Li, Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. CoRR, abs/2402.03300) 2024.
[37] Yunhang Shen, Chaoyou Fu, Shaoqi Dong.
Xiong Wang, Peixian Chen, Mengdan Zhang, Haoyu Cao, Ke Li, Xiawu Zheng. Yan Zhang, et al. Long-vita; Scaling large modai models to 1 million tokens with leading short-context accuray. CoRR, abs/2502.05177, 2025.
[38] Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng. Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient RLHF framework. In EuroSys, pages 279-1297. ACM, 2025.
[39] Baifeng Shi, Boyi Li, Han Cai, Yao Lu, Sifei Liu, Marco Pavone, Jan Kautz, Song Han, Trevor Darrell, Pavlo Molchanov, and Hongxu Yin. Scaling vision pre-training to 4k resolution. CoRR, 199025 2025.
[40] Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1.5: Scaling reinforcement learning with llms. CoRR, abs/2501..15999 2025.
[41] Peng Wang. Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. Qwen2-vl. vl: Enhancing vision- language model's perception of the world at any resolution. CoRR, abs/2409.122 12191, 2024.
[42] Xiao Wang, Qingyi Si, Jianlong Wu, Shiyu Zhu, Li Cao, and Liqiang Nie. Retake: Redueing temporal and knowledge redundancy for long video understanding. CoRR, abs/2412.20504,, 2024.
[43] Yuetian Weng, Mingfei Han, Haoyu He, Xiaojun Chang, and Bohan Zhuang. Longvlm: Efficient long video understanding via large language models. In ECCV, pages 453-470. Springer, 2024.
[44] Bo Wu, Shoubin Yu, Zhenfang Chen, Josh Tenenbaum, and Chuang Gan. STAR: A benchmark for situated reasoning in real-world videos. In NeurlPS Datasets and Benchmarks, 2021.
[45] Haoning Wu, Dongxu Li, Bei Chen, and Junnan Li. Longvideobench: A benchmark for long-context interleaved video-language understanding. CoRR, abs/2400.15575. 2024.
[46] Junbin Xiao, Xindi Shang, Angela Yao, and Tat-Seng Chua. Next-qa: Next phase of question- answering explaining temporal actions. In CVPR, pages 9777-9786, 2021.
[47] Fuzhao Xue, Yukang Chen, Dacheng Li, Qinghao Hu, Ligeng Zhu, Xiuyu Li, Yunhao Fang, Haotian Tang, Shang Yang, Zhijian Liu, Ethan He, Hongxu Yin, Pavio Molchanov, Jan Kautz, Linxi Fan, Yuke Zhu, Yao Lu, and Song Han. Longvila: Scaling long-context visual language models for long videos. In ICLR, 2025.
13

---

[48] An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2.5 technical report. CoRR, abs/2412.15115. 2024.
[49] Zhou Yu, Dejing Xu, Jun Yu, Ting Yu, Zhou Zhao, Yueting Zhuang, and Dacheng Tao. Activitynet-qa: A dataset for understanding complex web videos via question answering. In AAAI, pages 9127-91344 2019.
[50] Boqiang Zhang, Kehan Li, Zesen Cheng, Zhiqiang Hu, Yuqian Yuan, Guanzheng Chen, Sicong Leng, Yuming Jiang. Hang Zhang, Xin Li, Peng Jin, Wenqi Zhang, Fan Wang, Lidong Bing, and Deli Zhao. Videollama 3: Frontier multimodal foundation models for image and video understanding. CoRR, abs/2501. 10000..1106,... 2025.
[51] Peiyuan Zhang. Kaichen Zhang. Bo Li, Guangtao Zeng, Jingkang Yang, Yuanhan Zhang, Ziyue Wang, Haoran Tan, Chunyuan Li, and Ziwei Liu. Long context transfer from language vision. CoRR, abs/2406. 16852, 2024.
[52] Yifan Zhang, Qingsong Wen, Chaoyou Fu, Xue Wang, Zhang Zhang, Liang Wang, and Rong Jin. Beyond Ilava-hd: Diving into high-resolution large multimodal models. CoRR, abs/2406.08487, 2024.
[53] Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Ziwei Liu, and Chunyuan Li. Video instruction tuning with synthetic data. CoRR, abs/2410.02713, 2024.
[54] Zijia Zhao, Haoyu Lu, Yuqi Huo, Yifan Du, Tongtian Yue, Longteng Guo, Bingning Wang, Weipeng Chen, and Jing Liu. Needle in A video haystack: scalable synthetic framework for benchmarking video milms. CoRR, abs/2406.09366 2024.
[55] Yaowei Zheng, Junting Lu, Shenzhi Wang. Zhangchi Feng. Dongdong Kuang, and Yuwen Xiong. Easyrl: An efficient, scalable, multi-modality rl training framework, 2025.
[56] Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, Zhangwei Gao, Erfei Cui, Xuehui Wang, al. Intern113: Exploring advanced training and test-time recipes for open-source multimodal models. CoRR, abs/2504.10473 2025.
[57] Orr Zohar, Xiaohan Wang, Yann Dubois, Nikhil Mehta, Tong Xiao, Philippe Hansen-Estrude Licheng Yu, Xiaofang Wang. Felix Juefei-Xu, Ning Zhang, Serena Yeung-Levy, and Xide Xia. Apollo: An exploration of video understanding in large multimodal models. In CVPR, pages 18891-18901, 2025.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope, as they align with the theoretical and experimental results presented in the paper and provide a clear understanding of the paper's goals.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The paper acknowledges the limitations of the work performed by the authors.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiitiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
15

---

Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
The paper fully discloses all the information needed to reproduce the main experimental results, ensuring that the main claims and conclusions can be independently ver- ified. This includes providing relevant details, methodologies, and any necessary parameters or configurations for conducting the experiments.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
16

---

reproducible, they 5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
The paper will provide open access the data and code necessary to reproduce the main experimental results. It also includes sufficient instructions in the supplemental material on how to faithfully replicate the experiments conducted in the paper.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please the NeurlPS code and data submission guidelines (ht.ps://m/ /sscc//
tttp:///wueeeeeeeeeeeeeee public/ des CCodeSumm ss cessoonollcc
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https?
//nips.cc/pul wepublicc/ie htt//giieeee Subm Submi oasionPoll iicyy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The paper specifies all the training and test details necessary to understand the results. This includes information on data splits, hyperparameters, the methodology for selecting hyperparameters. the type of optimizer used, and any other relevant details that crucial for replicating and comprchending the reported results.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: The paper provides suitable information about the statistical significance of the experiments. This indicates that the authors have appropriately addressed the need for statistical analysis and have reported the relevant measures to support the reliability and significance of their experimental findings.
Guidelines:
17

---

The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of exccution) needed to reproduce the experiments?
Question:
Answer: [Yes]
The paper provides sufficient information on the computer resources required to reproduce each experiment. This includes details such as the type compute workers used, the amount of memory required, and the time taken for the execution of the experiments. This information allows for accurate replication of the experiments and provides transparency regarding the computational requirements of the study.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
ceu.ee/eeeeeeecccceeeeeeeeeeeeee ines?
Answer: [Yes]
Justification: The research conducted in the paper conforms, every respect, with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
18

---

Question:
Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The paper discusses both potential positive and negative societal impacts of the work performed, especially in the resource limited application.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (c.g., dep.lymeln of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification:
The paper poses no such risks
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
The paper only use the opensource datasets with legal license.
Guidelines:
19

---

The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms of use the package should be provided. For popular datasets, paper www.wwwwwwwowooooooooooooocctece datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The new assets introduced in the paper are well documented and the documentation provided alongside the assets.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
20

---

Justification:
The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy htttss//ccoiiiccccooofoeeeeeeee/
LLM) for what should should not be described.
21

---

Appendix
In this section, we selected examples of four types of reasoning tasks from the benchmark to evaluate and compare the reasoning processes and answers provided by Demmini-1.5-Pre Video-R1-7B, and LongVILA-RR--7B On Figure [2 a 20-minute StarCraft match is depicted, where the models analyze the players' unit compositions, strategies, and play styles to predict the potential developments on produced a correct prediction of the outcome, its reasoning
the battlefield. While Gemini-1.5- process contained factual inaccuracies. In contrast, Video-RI-7B, influenced by the characteristics of training data, tended to summarize answers based on options, neglecting critical video details and resulting in incorrect reasoning. LongVILA-H players' operational styles and specific moments marked in the video. Icading to a comprehensive and accurate prediction of the match's 's trajectory. On Figure 1 another example demonstrates the models' abilities in narrative reasoning and visual information analysis. Gemini- 1.5-- failed to correctly infer why the appearing for the second time in the video is not the husband. In contrast, successfully reasoned that the man's 's habit of wearing a
however, is able to accurately analyze the
both Video-R1- R1-7B and VongVIL--R ring on his left hand is key indicator, providing accurate answers.
Figure 14 illustrates the models' spatial perception and reasoning abilities as the camera moves through room. Gemini-1.5-PP effectively identified the key information within the video and provided the correct answer through straightforward reasoning. In contrast, Video-R1-70 experienced significant localization errors during the reasoning process, leading to critical issue for reasoning models: a mismatch between the reasoning analysis and the final answer. LongVILA-RI-78 demonstrated superior performance by leveraging dense frame analysis to accurately infer the spatial relationships between rooms and furniture across different levels, ultimately delivering a coherent reasoning process and the correct answer. On Figure [5] the focus shifts to temporal analysis in Lego video featuring diverse events and interactions. All three models successfully reasoned through the sequence of events and provided correct answers, showcasing their proficiency in temporal reasoning tasks. As a supplement to Figure D Figure 10 provides a more comprehensive comparison of two examples: "2022 FIFA Argentina
Netherlands" and "Moving the Cup and Guessing Where the Ball Is." In the football match example, while Gemini-1.5-Pro produced the correct answer, its output contained hallucinatory content influenced by biases in its pre-Icarned knowledge. Video-RI not only failed to provide accurate video analysis reasoning but also made incorrect predictions. In contrast, LongVILA-RA successfully analyzed the players' performance and emotions during the match, integrating these factors through its robust reasoning capabilities to make accurate predictions about the outcome. For the more challenging task of tracking the ball, Gemini-1.5-Proi 's reasoning is inconsistent with the spatial content throughout, while Video-RI R1 failed to deduce the ball's 's final position accurately. Remarkably, Long VILA-RI precisely analyzed the spatial transformations following the movement of the box, demonstrating superior interpretative and reasoning abilities.
22

---

Temporal 2022 FIFA Argentina v.s. Netherlands
3Y
You are shown a 30-minute video segment from a football game. The score remains 2 2 throughout this period, and no goals are scored. Based on players' physical condition...whit .which team is more likely to win the upcoming penalty shootout?
Question:
A: The Netherlands win the 4-2.. regulation time.
B: Argentina win the 1-3...maintaining composure.
C: The Netherlands win the shootout 5-4...by Argentina.
D: Argentina
..332 without going to penalties
Gemini-L.5-Pro: The video shows the extra time of the Netherlands vs Argentina match. Both teams score no goal, leading to penalty shootout....Basee on the video, 00:11, the narrator points toumament eight years ago and went into a penalty shootout, in which the Argentine goalkeeper emerged as This suggests that the players were exhausted and playing aggressively. It is pointed out that Argentina has played 9 periods of extra time in the world cup history.
that the two teams played draw in the previous
<answer-B</nnswer
<think~Let's break down the situation step by diep:..Scoreline The score tied at 2-2, meaning neither team has won the match yet. This....3. Players' Physical Condition:- Both teams appear be physically drained after 120 minutes of play. However, the Netherlands might have slight edge as they were the first to score in the match, potentially giving them bit more stamina.... They have shown more consistency and have better chance of converting their penalties, </think->/
Video-RI:
<answer>A</answer>
Long VInAVILL-R1: <think> The remains 2 throughout extra time, indicating high-intensity match with both teams committed to winning... Argentina's goalkeeper, who initially struggled, regains confidence and makes a crucial save, demonstrating his improved focus and determination. Key players from both teams, such Messi and de Ligt, are seen holding their heads in disappointment or celebration, reflecting the emotional highs and lows that often decide penalty shootouts... Given the evidence, Argentina is more likely to win the penalty shootout 4-3, with their goalkeeper playing a decisive role and key players maintaining composure. </think Sanswer-B</anstee
Figure 10: Detailed comparisons in the football game example. The video is available Link
23

---

Spatial Track
Where the ball is
Levaf1
How Many Can Righe?
There are three boxes, and purple ball is initially placed in the middle box. Then the positions of the boxes are swapped. Please analyze step by step the movement trajectory of the box containing the ball through the video and determine the final position the ball.
Question:
A: Left
B: Middle
C: Right
D: None of the above
Gemini-1.5-Pro: Here is the step-by-step analysis of the movement of the boxes: At the beginning of the video, the purple ball located in the middle box.2. The middle box moves the right.3. The right box moves the middle.4. The middle box moves the right, then to the left.5. The left box moves the middle.6. The middle box moves the left.7. The right box moves to the middle, then to the left. The middle box moves the right, then to the left. Based the box movements, ball is in the left box. <answer> A</anseer
track that the
<think-Let's break down the problem step by .tep, let's consider the movement of the box containing the ball:- The box that originally had the ball the ball will
Video-RI:
(middle box) will now be in the right position after the swap....Therffor be in the right box...o corggiall right box moves to the left. The ball is in the box that was originally the middle, which is now in the right position. Final position of the ball: right box. </think>
<answer <annwwr-C</answer
LongVILA-RI:
<think The middle box initially contains the purple ball. When the boxes are swapped, the left box becomes the new middle box, the middle box moves to the right, and the right box moves to the left. Since the left box is now in the middle position, the ball, which swapping continues with the boxes rotating left, but the ball's position does not change because it is always in the left box. Therefore, the final position of the ball is in the left box. </think> <answer>As/answer
previously in the middle box, is now in the left box. The
Figure 11: Detailed comparisons in the moving cup example. The video is available at Link
24

---

Goal and Purpose- Strategic Intent Prediction in StarCraft 2
SCULLL
Question:
You are presented with the first minutes a StarCraft II game. What are the likely strategic intentions of both players, their next possible actions, and who is more likely to win the match?
A: Zest executes successful Stargate-baser air harassment strategy...
Reynor adapts Zest's early aggression, transitions into Roach-Ravager... C: Zest's early proxy strategy catches Reynor off-guard, leading to quick win... Both players engage in prolonged macro game, Zest's superior...
overlord to scout Zest's base. Reynor starts building Queens in response to Zest showing a void ray. At 0:30, Zest sends an oracle into Reynor's 's base. Zest's zealots clear out Reynor's early expansion. 2:21, Zest's void ray damages Reynor's 's base. At 4:05, Reynor's roach-ravager is quite big. At 6:48, Reynor's ground armor is completed. At 6:57, Reynor defeats Zest.... options C and D can be climinated. Reynor adapts to Zest's early aggression and secures victory by securing his economy. Option false. <answer-B</answer
Gemini-I.5-Pro: At 0:04,
Reynor
Video-RI: <think-Let's break down the situation step by step: 1. Initial Observations:- The game appears to be at early stage, with both players having established some form of...Option suggests Zest's success with the Stargate, which seems plausible given his early advantage... Option C suggests a quick win for Zest, which is less likely due to the early stage and Reynor's potential adaptability. Option D suggests prolonged samee..<thinkk
<answer>A<answwer
<think> The initial frames show Zest using Stargate-based air harassment strategy, focusing on early aggression with air units like Stalkers..HHoeeverr However, by the 15-minute mark, Reynor's economy is stabilizing, and they are building a mix of ground and air units, including Marauders, Overlords, and Zealots. This suggests transition to balanced strategy... The final frames highlight Reynor's dominance in the late game, supported by their balanced unit composition, including Roaches, Zealots, and Marauders, which allowed them to adapt to Zest's ss early strategy and counterattack effectively. Therefore, the correct answer B: Reynor adapts to Zest's early aggression, transitions into Roach-Ravager composition, and secures victory through sustained ground assaults. </think> <answer-B</answer>>>
LongVILA-RI:
Figure 12: LongVILA-RI reasoning example in the "Goal and Purpose" category. The video is
available at Link
25

---

Plot and Narrative Detector Game
Question:
Dylan and Susan had been happily married for 10 years, One day, Dylan went on business trip. When he returned, Susan immediately understood that the man not her husband. How did she figure it out?
man doesn't look like Dylan
B: The man's sleeve is open
The man is not wearing a wedding ring on left hand
D: Susan is hallucinating
Gemini-1.5-PPe Here are the steps to think about this riddle/'n'nl. The video shows a man greeting Susan. This man seems to look like Dylan first glance. Option can be eliminated..44 .4. Then Susan keeps looking at the man's left hand. She is supposed to check if the ring is on the right finger. Option C can be eliminated. Vnin5. The man opens another sleeve at 0:22. Hc wears his wedding ring on his right hand. He is not Dylan. The answer is B.
Video-RI: <think~Let's break down the situation step by step: 1. Dylan and Susan have been married for years, which suggests a strong bond and familiarity...B. "The man's sleeve is open." " This detail is not specific enough determine if the man is the husband. It could be coincidence or result of casual behavior...conte contexx and the details provided, the most logical and specific indicator that Susan figured out the man was not her hushand is the absence of a wedding ring on his left hand. </think>> Canswer-C</nnwwe
Long VILAVIL::: <think> Susan recognized the man as her husband because Dylan wears a wedding ring, which symbolizes marital status. the video, the man seen wearing ring his left hand while standing in front of window. This detail indicates his marital status...he also seen wearing a gray suit and a white shirt, but there is no mention of a wedding ring..The key detail is that Dylan wears a wedding ring, which Susan would recognize as her hushand's symbol marriage. The man's appearance, attire, and lack of wedding ring are distractions. The correct answer is based on the symbolic importance of the ring, which confirms Dylan's identity as her husband. </think
Sanswer-C</answer
Figure 13: LongVILA-RD reasoning example in the "Plot and Narrative" category. The video is available at Link
26

---

Spatial House Tour
Question: Based the video, which room is most likely located the upper floor adjacent to the staircase with the wedding photo, requiring understanding of spatial and temporal progression across multiple scenes?
A: Kitchen with gray cabinets and pink backsplash Nursery with white crib and light blue walls
C: Bathroom with white bathtub and floral wallpaper
Dining room with a rectangular wooden table
Gemini-1.5-Per Here are the thinking steps:
1. Locate the staircase with the wedding photo. The staircase appears the screen at [00:00:48]. The wedding photo is shown at [00:00:50].
2. Identify the rooms adjacent to the staircase with the wedding photo. The adjacent room appears at [00:00:52]. contains a crib and blue wallpaper with small white stars. Emily refers to this room as the "baby's room".
Video-RI: <hink-Lett me think about this carefully... Thiis requires us consider the spatial layout and the sequence of rooms shown in the video... This room is also shown in the video, but it's not clear if it's on the upper floor or not... This room is shown in the video, but it's not clear if it's the upper floor or not. Oh, I see. The kitchen is the most logical choice based the spatial layout and the fact that it's common the upper floor...Final answer: A Let's break it down one last time to ensure. </think> <answer-B<answeer>>>>>>
LongVILA-RI: <think> The staircase has a black railing on the left and white wall on the right, with a wedding photo visible on the right wall...adjacem to the living room. The bathroom with white bathtub and floral wallpaper described as having window with a white curtain, and it is never mentioned to be on the first floor. The nursery with a white crib and light blue walls described as having a staircase with black railing on the left, matching the staircase in the first scene, and it is explicitly stated to be on the second floor. Therefore, the bathroom and kitchen are on the first floor, while the nursery is on the second floor adjacent to the staircase with the wedding photo, </think> <answer>\<<answer>
COT
Figure 14: LongVILA-RI reasoning example in the "Spatial" category. The video is available at Link
27

---

Temporal LEGO PINATA
Question: Which cake's preparation process involved both the use of a blue hammer for smashing and a red silicone mold for shaping decorative elements, based on the temporal progression of tools and ingredients observed in the video?
Greentyellow/red Lego block cakes
B: Red velvet cake with batter mixed in a blue bow!
C: Chocolate cake with parchment paper and red cake pops
D: Lego-themed cake smashed at 0:00:30-0.00:40 and decorated with bricks
Gemini-1.5-PP The question is about the cake's preparation process shown in the video.. Thh yellow Lego-themed cake is smashed with an orange hammer. The green Lego-themed cake is opened with knife. 3. At 0:30-0:40, red silicone mold is used for shaping decorative brick by pouring melted chocolate into its cases. 4. So, both tools are present only at the preparation of the red Lego-themed cake in the video.
<answer-D</answee
Video-RI: <think<Let me think about this carefully.. let's consider the options: A: Green/yellowired Lego block cakes This option mentions Lego blocks, but there's s no mention of blue hammer or red silicon...afte going through all the options, it's clear that the correct choice is D.... Let's verify last time make sure. Yes, the video indeed shows the use a blue hammer and a red silicone mold in the preparation of the Lego-themed cake. So, the final answer is D</think>> <answer=D</answer>
LongVILA-RI: think The preparation of the Lego-themed cake involved the use of a blue hammer to smash colorful Lego blocks, which were then used as decorations. The red silicone mold was used shape the red decorative elements for the cake. The process started with the woman using blue hammer smash green, yellow, and red Lego blocks, which confirmed the use of the blue hammer. The red silicone mold was explicitly mentioned when shaping the red decorative elements. The cake was then decorated with bricks, indicating the use of the red silicone mold for the red elements.. The red velvet cake used blue bowl for mixing batter but did use the blue hammer or red silicone mold. Therefore, the correct answer is D. D/tthiik>> <answer-D</answer>
Figure 15: LongVIL.A-R reasoning example the "Temporal"" category. The video is available Link
28

---

Taboo Game
This a video of two people playing the game Taboo on the topic of 'machine learning.' One person describes a word, and the other person guesses it. Your task is to analyze the video and choose the most likely word being described from the following options:
Question:
Logistic Regression Pruning Attention Mechanism
B: Support Vector Machine Quantization Transformer
C: Neural Network Pruning Gradient Boosting
D: Neural Network Quantization Transformer
The description mentions similarity to the human brain, layers of nodes, weighted connections, and origin the 1950s. This strongly suggests "Neural Network." All other options in the sets don't 't have these defining features...3. Third Clue: The clue references science fiction film with characters from Cybertron, Optimus Prime, and Megatron. This leads to "Transformer." powerful neural network architecture, and its naming convention evokes the Trranfformers franchise.
Gemini-L.5-Pro: Here are the thinking steps:1. First Clue:
Canswer-D</nnswer
<think> The question involves analyzing the context provided by the video and matching it with the given options. The participants are discussing machine learning concepts, which narrows down the focus. Firstly, let's consider the options... Neural Network Quantization Transformer... After analyzing the options, B seems the most balanced and relevant
Video-RI;
context the video. The participants are likely discussing mix of fundamental and advanced concepts in machine Larming..s//hink. Sanswer>B<<answer
<think> The word "Logistic Regression" is not actually mentioned; instead, the description "similar to the human brain, with lot of layers of nodes and weighted connections" clearly refers to Neural Network, a fundamental machine learning model originating in the 1950s. The second clue, about "cutting the digits of an algorithm and localizing to an edge device," " matches Quantization, which reduces bit-width make models more efficient. The mention of "Optimus Prime and Megatron" directly points to Transformer, named after the Transformers franchise and used widely in modern NLP. Together, Neural Network, Quantization, and Transformer form a coherent chain of concepts matching the video's clues. Options A and C omit Transformer include unrelated terms. Therefore, the best answer D,</think>> <answer=D</answer>>
LongVILA-RI:
Figure 16: Long VILA-RI reasoning example taboo game. The video is available at Link
29


Reviews:
[{"review_id": "gsULw2csy3", "reviewer": "Reviewer_HSHu", "summary": "This paper introduces LongVILA-R1, a framework that enhances vision-language models (VLMs) for long video reasoning through reinforcement learning. The authors address challenges in long video understanding by developing three core components: (1) LongVideo-Reason dataset with 52K long video QA pairs with reasoning annotations, (2) a two-stage training pipeline combining chain-of-thought supervised fine-tuning (CoT-SFT) with reinforcement learning (RL), and (3) Multi-modal Reinforcement Sequence Parallelism (MR-SP) system for efficient long video RL training. The method scales from 16 to 512 frames and demonstrates competitive performance on VideoMME and their proposed LongVideo-Reason-eval benchmark.", "questions": "-\tComprehensive Benchmarking: Could the authors provide evaluation results on additional established long video benchmarks (such as EgoSchema, ActivityNet-QA, and LongVideoBench) to better demonstrate the generalizability and robustness of the reasoning improvements?\n-\tCost-Benefit Analysis: Given the substantial computational investment required for the RL training pipeline, could the authors provide a more detailed analysis of the performance improvements achieved by their method compared to baseline models? Specifically, how do the gains justify the additional computational overhead and training complexity?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "***Strengths***\n\n-\tThe paper is generally well-written with clear explanations of the technical contributions. \n-\tThe paper presents a well-structured approach to long video reasoning with clear methodology. The MR-SP system design is technically sound, achieving 9.2× speedup for long video RL training.\n-\tThe dataset represents a valuable contribution as a resource for training reasoning models on video content.\n\n***Weaknesses***\n\n-\tMissing Key Baseline: The evaluation lacks comparison with LongVILA, an important baseline for this work.\n-\tLimited Performance Gains: The performance improvements over existing baselines are modest and may not justify the substantial computational investment. Specifically, compared to the baseline LongVILA paper [1], LongVILA-7B achieved 60.1 (without subtitles) and 65.1 (with subtitles) on VideoMME with 256 frames, while LongVILA-R1-7B achieves only marginally better results at 60.3 (without subtitles) and 65.9 (with subtitles). Given the extensive additional training (CoT-SFT + RL) and computational resources required, these improvements appear insufficient.\n-\tLimited Evaluation Scope: The evaluation relies primarily on VideoMME and the authors' proprietary LongVideo-Reason-eval benchmark. More comprehensive evaluation across established long video benchmarks (such as EgoSchema, ActivityNet-QA, and LongVideoBench) would better validate the claimed reasoning improvements.\n\n[1] Chen, Yukang, et al. \"Longvila: Scaling long-context visual language models for long videos.\", ICLR2025.", "was_revised": true, "final_justification": "The authors have provided a comprehensive rebuttal addressing key concerns in my review. Hence, I decided to raise my rating accordingly.", "rebuttal": "## **General Response**\n\nWe sincerely appreciate your detailed and constructive feedback. Before addressing each concern individually, we first to highlight the major updates as follows.\n\n1. **Larger Dataset:** We double the size of our dataset, LongVideo-Reason, from 52K to 104K reasoning data, with carefully filtering and data cleaning. Moreover, we include both multi-choice and open-ended QAs, making it more general.\n2. **VideoMME Update**: Thanks to the updated dataset, our model achieved strong performance, e.g., **65.1% / 71.1%** on VideoMME [1] without and with subtitles, outperforming the LongVILA [2] baseline, which achieved 60.1% / 65.1% on VideoMME [1].\n3. **Expanded Evaluation**: We have included **7 additional video benchmarks** for evaluation: ActivityNet-QA [3], PerceptionTest [4], VNBench [5],  LongVideoBench [6], EgoSchema [7],  NExT-QA [8], and MVBench [9]. Alongside VideoMME [1], we now compare LongVILA-R1 against LongVILA [2] baseline, and other methods [10, 11, 12, 13 ,14, 15] on a total of **8 benchmarks**, demonstrating consistently strong performance. Results are show in the table below.\n4. **Extended Inputs**: LongVILA-R1-7B is capable of processing **2,048** video frames per video, with customizable FPS settings.\n\n| Model | VideoMME (w/o sub) | VideoMME (w/ sub) | ActivityNet-QA | PerceptionTest | VNBench | LongVideoBench | EgoSchema | NExT-QA | MVBench |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |\n| VIdeo-LLaVA-7B | 39.9 | 41.6 | 45.3 | - | 12.4 | 37.6 | 38.4 | - | 43.5 |\n| ShareGPT4Video-8B | 39.9 | 43.6 | 50.8 | - | - | 41.8 | - | - | 51.2 |\n| VideoLLaMA2.1-7B | 54.9 | 56.4 | 53.0 | 54.9 | - | - | 53.1 | - | 57.3 |\n| Kangaroo-8B | 56.0 | 57.6 | - | - | - | 54.8 | 62.7 | - | 61.1 |\n| LLaVA-OV-7B | 58.2 | 61.5 | 56.7 | 57.1 | 51.8 | 56.4 | 60.1 | 79.4 | 56.7 |\n| Video-R1-7B | 61.4 | - | - | - | - | - | - | - | 64.8 |\n| LongVILA-7B | 60.1 | 65.1 | 59.5 | 58.1 | 63.0 | 57.1 | 67.7 | 80.7 | 67.1 |\n| LongVILA-R1-7B | **65.1** | **71.1** | **64.8** | **68.9** | **75.5** | **58.0** | **68.9** | **81.5** | **68.2** |\n\n## **Detailed Response for Each Question**\n\nWe provide detailed responses to each of the concerns below.\n\n**Q1: “Missing Key Baseline, Limited Performance Gains, Limited Evaluation Scope”**\n\n**A:** As mentioned in the **General Response**, we have evaluated LongVILA-R1-7B on eight benchmarks, including the three you suggested: ActivityNet-QA[3], LongVideoBench [6], and EgoSchema[7], and compared to the LongVILA-7B baseline. Across all benchmarks, LongVILA-R1-7B consistently outperforms LongVILA-7B  [9], achieving 65.1% and 71.1% on VideoMME [1], with **+5.0% and +6.0%** improvements over LongVILA-7B [2].\n\n**Q2: “Cost-Benefit Analysis”**\n\n**A**: Thank you for your constructive suggestion. We justify the performance gains from LongVILA-7B to LongVILA-R1-7B on VideoMME [1], along with the associated training overhead on H100.\nBoth CoT SFT and RL fine-tuning contribute to meaningful improvements, from 65.1 → 66.9 → 71.1 (with subtitles). While CoT SFT and RL require 82 GPU-days and 224 GPU-days, the performance gains are non-trivial. \n\nIn addition, as shown in the table in **General Response**, LongVILA-R1-7B achieves notable improvements over LongVILA-7B on several other benchmarks—for example, from **59.5% to 64.8% on ActivityNet-QA** [3], **58.1% to 68.9% on PerceptionTest** [4], and **63.0% to 75.5% on VNBench** [5].\n\n| Model | VideoMME (w/o sub) | VideoMME (w/ sub) | Overhead |\n| --- | --- | --- | --- |\n| Baseline | 60.1 | 65.1 | - |\n| +CoT SFT | 61.4 | 66.9 | 82 GPU days |\n| +RL | 65.1 | 71.1 | 224 GPU days |\n\n### **References**\n\n[1] Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis. CVPR 2025\n\n[2] LongVILA: Scaling Long-Context Visual Language Models for Long Videos. ICLR 2025\n\n[3] ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answering. AAAI 2019\n\n[4] Perception Test: A Diagnostic Benchmark for Multimodal Video Models. NeurIPS 2023\n\n[5] Needle In A Video Haystack: A Scalable Synthetic Framework for Benchmarking Video MLLMs. ICLR 2025\n\n[6] LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding. NeurIPS 2024\n\n[7] EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understanding. NeurIPS 2023\n\n[8] NExT-QA: Next Phase of Question-Answering to Explaining Temporal Actions. CVPR 2021\n\n[9] MVBench: A Comprehensive Multi-modal Video Understanding Benchmark. CVPR 2024\n\n[10] Video-LLaVA: Learning United Visual Representation by Alignment Before Projection. EMNLP 2024\n\n[11] ShareGPT4Video: Improving Video Understanding and Generation with Better Captions. NeurIPS 2024\n\n[12] VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs. ArXiv 2024\n\n[13] Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Input. ArXiv 2024\n\n[14] LLaVA-OneVision: Easy Visual Task Transfer. Trans. Mach. Learn. Res. 2025\n\n[15] Video-R1: Reinforcing Video Reasoning in MLLMs. ArXiv 2025"}, {"review_id": "4HrVOVFWLb", "reviewer": "Reviewer_QtJd", "summary": "The key idea of this paper is applying GRPO-style reinforcement learning to improve long video understanding. The authors identify key bottlenecks that limit this and solve them, the three main contributions being (1) a LongVideo-Reason training set (2) Multi-modal Reinforcement Sequence Parallelism (MR-SP) training strategy that allows extremely long videos to be used with many parallel inferences, which is required for GRPO and finally (3) LongVideo-Reason-eval benchmark, a handpicked benchmark with 1000 samples. The final result is LongVILA-R1-7B LLM, which can ingest up to 512 video frames while still being able to carry out spatio-temporal reasoning.\n\nThe LongVideo-Reason training corpus supplies 52 K multiple-choice question–answer pairs (with chain-of-thought) drawn from 18 077 long videos, evenly covering Temporal, Goal & Purpose, Spatial, and Plot & Narrative reasoning. Heuristics are used to split the training data into easy, medium and hard subsets.\n\nLongVILA-R1 is trained in two stages: (i) chain-of-thought supervised fine-tuning on \"easy\" and \"hard\" examples, followed by (ii) Group-Relative Policy Optimisation (GRPO) reinforcement learning on \"medium\" cases . Stage 1 is a SFT-CoT stage designed to \"warmup\" the model's reasoning capabilities. In Stage 2, the authors also add data from Video-R1, a prior work, to scale up RL.  To keep RL feasible, the paper introduces Multi-modal Reinforcement Sequence Parallelism (MR-SP), which caches video embeddings and shards giant token contexts so policy and reference models prefill in parallel. \n\nFor evaluation, the authors curated LongVideo-Reason-eval, 1,000 hand-checked problems that mirror the four reasoning categories in the training dataset. LongVILA-R1 averages 67.9% accuracy, exceeding GPT-4o performance. On prior benchmarks like VideoMME, it outperforms open source VideoLLMs by a large margin.", "questions": "Result on standard video benchmarks like MVBench, ApolloBench etc would be appreciated.\n\nAs dataset is a big part of the contribution, the authors should give more details and visualizations of it, so we can understand the diversity of the dataset.", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n+ The dataset contributions are valuable to the Video LLM community\n+ The overall execution is thorough and polished, resulting in strong performance across benchmarks\n\nWeaknesses\n\n- Evaluations on more Video understanding benchmarks which are common in the VideoLLMs literature would be appreciated\n- The primary technical contributions are in the Sequence Parallel training, the architecture and training methods (GRPO) are all just directly applied.", "was_revised": true, "final_justification": "The expanded results on video benchmarks made me more confident of the results, the consistent improvement across datasets shows the value of the method better.", "rebuttal": "## **General Response**\n\nWe sincerely appreciate your detailed and constructive feedback. Before addressing each concern individually, we first to highlight the major updates as follows.\n\n1. **Larger Dataset:** We double the size of our dataset, LongVideo-Reason, from 52K to 104K reasoning data, with carefully filtering and data cleaning. Moreover, we include both multi-choice and open-ended QAs, making it more general.\n2. **VideoMME Update**: Thanks to the updated dataset, our model achieved strong performance, e.g., **65.1% / 71.1%** on VideoMME [1] without and with subtitles, outperforming the LongVILA [2] baseline, which achieved 60.1% / 65.1% on VideoMME [1].\n3. **Expanded Evaluation**: We have included **7 additional video benchmarks** for evaluation: ActivityNet-QA [3], PerceptionTest [4], VNBench [5],  LongVideoBench [6], EgoSchema [7],  NExT-QA [8], and MVBench [9]. Alongside VideoMME [1], we now compare LongVILA-R1 against LongVILA [2] baseline, and other methods [10, 11, 12, 13 ,14, 15] on a total of **8 benchmarks**, demonstrating consistently strong performance. Results are show in the table below.\n4. **Extended Inputs**: LongVILA-R1-7B is capable of processing **2,048** video frames per video, with customizable FPS settings.\n\n| Model | VideoMME (w/o sub) | VideoMME (w/ sub) | ActivityNet-QA | PerceptionTest | VNBench | LongVideoBench | EgoSchema | NExT-QA | MVBench |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |\n| VIdeo-LLaVA-7B | 39.9 | 41.6 | 45.3 | - | 12.4 | 37.6 | 38.4 | - | 43.5 |\n| ShareGPT4Video-8B | 39.9 | 43.6 | 50.8 | - | - | 41.8 | - | - | 51.2 |\n| VideoLLaMA2.1-7B | 54.9 | 56.4 | 53.0 | 54.9 | - | - | 53.1 | - | 57.3 |\n| Kangaroo-8B | 56.0 | 57.6 | - | - | - | 54.8 | 62.7 | - | 61.1 |\n| LLaVA-OV-7B | 58.2 | 61.5 | 56.7 | 57.1 | 51.8 | 56.4 | 60.1 | 79.4 | 56.7 |\n| Video-R1-7B | 61.4 | - | - | - | - | - | - | - | 64.8 |\n| LongVILA-7B | 60.1 | 65.1 | 59.5 | 58.1 | 63.0 | 57.1 | 67.7 | 80.7 | 67.1 |\n| LongVILA-R1-7B | **65.1** | **71.1** | **64.8** | **68.9** | **75.5** | **58.0** | **68.9** | **81.5** | **68.2** |\n\n## **Detailed Response for Each Question**\n\nWe provide detailed responses to each of the concerns below.\n\n**Q1: “Evaluations on more benchmarks”**\n\n**A:** Thank you for your suggestion. We fully agree on the importance of evaluating the performance of our reasoning model on a more comprehensive set of benchmarks. As mentioned in **General Response**, we evaluated LongVILA-R1 on eight benchmarks, including VideoMME (w/o sub), VideoMME (w/ sub)  [1], ActivityNet-QA [3], PerceptionTest [4], VNBench [5], LongVideoBench [6], EgoSchema [7], NExT-QA [8], and MVBench [9]. Under these benchmarks, LongVILA-R1 consistently outperforms LongVILA [2] and demonstrates more comprehensive imporvements compared to other video VLMs with similar parameter scales.\n\n**Q2: “The primary technical contributions are in the Sequence Parallel training, while the architecture and training methods (GRPO) are directly applied.”**\n\n**A:** We really appreciate the reviewer's acknowledgement of one of our core contributions, MR-SP system, which enable training on long videos with up to 3,600 frames using a single node with 8 A100 GPUs.\n\nIn addition to the technical contributions of MR-SP, our work also provides a dataset, a benchmark, and, more importantly, **a comprehensive pipeline for generating large-scale datasets tailored for long-video RL training**. We believe this contribution has a big value in this area.\n\nThe novelty of our training scheme lies in **the co-design of the algorithm and infrastructure**, even though the SFT+RL algorithm itself is not new. The infrastructure features several key design elements:\n\n1. We first implemented **video encoding with balanced distribution** across multiple GPUs. (Traditional RL frameworks often run into **GPU OOM** issues when processing more than 512 frames.)\n2. We introduce a **chunked distributed gathering** strategy for multimodal inputs and embeddings. (Traditional RL frameworks often suffer from **CPU OOM** issues due to the combination of long video frames and a large number of rollouts.)\n3. We further **cache and reuse video embeddings** during both the rollout and pre-filling phases, effectively reducing redundant computation. (Traditional RL frameworks often suffer from **high latency** due to repetitive processing.)\n\nWithout this co-design, the training algorithm alone is insufficient to handle long-video training effectively. In contrast, our training framework is capable of training on 3,600 video frames using a single node with 8 A100 GPUs. \n\nWe are fully committed to releasing the dataset, the complete code of the dataset generation pipeline, and the training framework, including the MR-SP system, to support future research in the community.\n\n**Q3: “Result on standard video benchmarks like MVBench, ApolloBench”**\n\n**A:** Thank you for the reminder. As mentioned above, we have reported results on seven additional benchmarks, including MVBench [9] as you suggested. As for ApolloBench [16], the benchmark is not publicly available at this time. We will evaluate our model on it when it becomes open-sourced.\n\n**Q4: “More details and visualizations of the dataset”**\n\n**A:** Thank you for the reminder. We will include additional details and visualizations of the dataset in the revised version. \n\nWe illustrate the data distribution in Figure 3 and the dataset build-up pipeline in Figure 4. The videos span a wide range of categories, including travel, sports, education, animals, blogs, news, music, technology, comedy, entertainment, film, and gaming. The QA pairs are categorized into four reasoning types: **Temporal**, **Goal and Purpose**, **Spatial**, and **Plot and Narrative**. After thorough filtering and cleaning, the LongVideo-Reason dataset contains **104K** QA pairs with explicit reasoning annotations. Representative visual examples are provided in Figures 10 and 11.\n\nIn addition, we further collected 2,000 additional 4K-resolution videos spanning scenarios such as **autonomous driving, video games, household robotics,** and **wildlife**. We employed VILA-HD [17] to generate object bounding boxes in video frames and constructed spatial reasoning Q&A pairs based on these boxes and corresponding captions. \n\nAdditionally, the dataset contains a balanced mix of **multiple-choice** and **open-ended questions**, enhancing its versatility. To facilitate future research in video reasoning, we will release the dataset and the complete dataset construction pipeline to the community. We will also add more visualizations of the dataset in the future version of the paper.\n\n### **References**\n\n[1] Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis. CVPR 2025\n\n[2] LongVILA: Scaling Long-Context Visual Language Models for Long Videos. ICLR 2025\n\n[3] ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answering. AAAI 2019\n\n[4] Perception Test: A Diagnostic Benchmark for Multimodal Video Models. NeurIPS 2023\n\n[5] Needle In A Video Haystack: A Scalable Synthetic Framework for Benchmarking Video MLLMs. ICLR 2025\n\n[6] LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding. NeurIPS 2024\n\n[7] EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understanding. NeurIPS 2023\n\n[8] NExT-QA: Next Phase of Question-Answering to Explaining Temporal Actions. CVPR 2021\n\n[9] MVBench: A Comprehensive Multi-modal Video Understanding Benchmark. CVPR 2024\n\n[10] Video-LLaVA: Learning United Visual Representation by Alignment Before Projection. EMNLP 2024\n\n[11] ShareGPT4Video: Improving Video Understanding and Generation with Better Captions. NeurIPS 2024\n\n[12] VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs. ArXiv 2024\n\n[13] Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Input. ArXiv 2024\n\n[14] LLaVA-OneVision: Easy Visual Task Transfer. Trans. Mach. Learn. Res. 2025\n\n[15] Video-R1: Reinforcing Video Reasoning in MLLMs. ArXiv 2025\n\n[16] Apollo: An Exploration of Video Understanding in Large Multimodal Models. CVPR 2025\n\n[17] Scaling Vision Pre-Training to 4K Resolution. CVPR 2025"}, {"review_id": "tpiCbV3rRF", "reviewer": "Reviewer_oP4r", "summary": "This paper presents LongVILA-R1: a framework for long video reasoning in VLMs and introduces a new dataset (LongVideo-Reason). The proposed LongVILA-R1 framework consists of a two-stage CoT-SFT and RL training approach. The paper also introduces MR-SP, for efficient for scalable long video RL. The evaluation is performed on the proposed LongVideo-Reason dataset and Video-MME.", "questions": "•\tThe paper should clarify the novelty of the training scheme in more detail.\n\n•\tThe paper should provide more detailed analysis of the diversity of the proposed LongVideo-Reason dataset.\n\n•\tThe paper should clarify the choice of evaluation datasets and baseline methods.", "limitations": "Yes, but the limitations of the evaluation scheme should also be discussed.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths:**\n\n•\tThe LongVideo-Reason dataset is diverse and contains complex video reasoning questions, e.g., tracking balls in a shell game (Figure 12).\n\n•\tThe paper introduces a novel and interesting training paradigm (Multi-modal Reinforcement Sequence Parallelism) that shows promising improvements in training complexity.\n\n•\tThe paper shows promising performance on the proposed LongVideo-Reason-eval benchmark and other datasets such as Video-MME.\n\n**Weaknesses:**\n\n•\tThe proposed LongVideo-Reason dataset is based only on the Shot2Story dataset. As the video score is a single dataset, this might limit the diversity of the LongVideo-Reason dataset. The paper should discuss the diversity of the LongVideo-Reason dataset in more detail.\n\n•\tNovelty: The dataset generation and training scheme – supervised fine-tuning followed by GRPO based RL training is largely based on prior work, e.g., the Deepseek family of models. The novelty of the training scheme should be discussed in more detail.\n\n•\tEvaluation on Video-MME: The paper does not compare the state of the art approaches such as VideoLLaMA3 which achieves better results -- 54.9% accuracy on long videos without subtitles – without the need of RL based training.\n\n•\tEvaluation on LongVideo-Reason-eval: While the proposed model outperforms the Gemini-1.5-Pro and GPT-4o models on this benchmark, the proposed model was already fine-tuned on this dataset. This raises the question whether the comparison with zero-shot methods such as Gemini-1.5-Pro and GPT-4o is fair.\n\n•\tLong videos: The proposed method seems to be limited to 512 frames, while state of the art approaches, e.g., Qwen-2.5-VL can process >=1200 frames.\n\n•\tThe model seems to rely on a stable and low frame rate, it is unclear if the model can deal with variable or high frame rates.", "was_revised": true, "final_justification": "The rebuttal addressed by concerns.", "rebuttal": "## **General Response**\n\nWe sincerely appreciate your detailed and constructive feedback. Before addressing each concern individually, we first to highlight the major updates as follows.\n\n1. **Larger Dataset:** We double the size of our dataset, LongVideo-Reason, from 52K to 104K reasoning data, with carefully filtering and data cleaning. Moreover, we include both multi-choice and open-ended QAs, making it more general.\n2. **VideoMME Update**: Thanks to the updated dataset, our model achieved strong performance, e.g., **65.1% / 71.1%** on VideoMME [1] without and with subtitles, outperforming the LongVILA [2] baseline, which achieved 60.1% / 65.1% on VideoMME [1].\n3. **Expanded Evaluation**: We have included **7 additional video benchmarks** for evaluation: ActivityNet-QA [3], PerceptionTest [4], VNBench [5],  LongVideoBench [6], EgoSchema [7],  NExT-QA [8], and MVBench [9]. Alongside VideoMME [1], we now compare LongVILA-R1 against LongVILA [2] baseline, and other methods [10, 11, 12, 13 ,14, 15] on a total of **8 benchmarks**, demonstrating consistently strong performance. Results are show in the table below.\n4. **Extended Inputs**: LongVILA-R1-7B is capable of processing **2,048** video frames per video, with customizable FPS settings.\n\n| Model|VideoMME (w/o sub)|VideoMME (w/ sub)|ActivityNet-QA|PerceptionTest|VNBench|LongVideoBench|EgoSchema|NExT-QA|MVBench|\n|---|---|---|---|---|---|---|---|---|---|\n|VIdeo-LLaVA-7B|39.9|41.6|45.3|-|12.4|37.6|38.4|-|43.5|\n|ShareGPT4Video-8B|39.9|43.6|50.8|-|-|41.8|-|-|51.2|\n|VideoLLaMA2.1-7B|54.9|56.4|53.0|54.9|-|-|53.1|-|57.3|\n|Kangaroo-8B|56.0|57.6|-|-|-|54.8|62.7|-|61.1|\n|LLaVA-OV-7B|58.2|61.5|56.7|57.1|51.8|56.4|60.1|79.4|56.7|\n|Video-R1-7B|61.4|-|-|-|-|-|-|-|64.8|\n|LongVILA-7B|60.1|65.1|59.5|58.1|63.0|57.1|67.7|80.7|67.1|\n|LongVILA-R1-7B|**65.1**|**71.1**|**64.8**|**68.9**|**75.5**|**58.0**|**68.9**|**81.5**|**68.2**|\n\n## **Detailed Response for Each Question**\n\nWe provide detailed responses to each of the concerns below.\n\n**Q1: “The diversity of the LongVideo-Reason dataset”**\n\n**A:** Apologies for the confusion. The videos in LongVideo-Reason are not solely sourced from the Shot2Story [16] dataset. We also incorporated an additional **2,000 4K-resolution** videos covering diverse scenarios, including **autonomous driving, video games, household robotics, and wildlife**. We used VILA-HD [17] to generate object bounding boxes within video frames and constructed spatial reasoning Q&A pairs based on these boxes and their corresponding captions. We will add detailed description of this in the revision.\n\nThe diversity of the LongVideo-Reason dataset also stems from its reasoning-annotated QA pairs, which are categorized into four reasoning types: **Temporal**, **Goal and Purpose**, **Spatial**, and **Plot and Narrative**. After thorough filtering and cleaning, the LongVideo-Reason dataset contains **104K** QA pairs with explicit reasoning annotations. Representative visual examples are provided in Figures 10 and 11.\n\nAdditionally, the dataset contains a balanced mix of **multiple-choice** and **open-ended questions**, enhancing its versatility. As a result, as shown in the **General Response**, this dataset significantly contributes to the model’s strong performance across multiple benchmarks, which cover a diversity of video tasks.\n\n**Q2: “The novelty of the dataset generation and the training scheme”**\n\n**A:** We answer this from two perspectives:\n\n**(1) Dataset Generation**\n\nThe novelty of our dataset generation lies in the **pipeline design**: we first use VLMs to generate text captions for video frames, and then leverage powerful reasoning-oriented LLMs to generate multi-step reasons. While advanced LLMs like DeepSeek-R1-671B [18] possess strong reasoning capabilities, they cannot process video inputs. Our pipeline bridges the gap between long-video VLMs and the reasoning-oriented LLMs.\n\nDifferent from existing video datasets [19] which only provide QAs, our dataset includes detailed reasons for QA pairs. While some datasets do include reasoning annotations, they are designed for LLMs, such as mathematics [20], not videos.\n\nMoreover, this dataset helps LongVILA-R1-7B to achieve strong performance (e.g., 65.1/71.1 on VideoMME) and serves as a solid baseline for future research, making it a meaningful contribution to the field.\n\n**(2) Training Scheme**\n\nThe novelty of our training scheme lies in **the co-design of algorithm and infrastructure**, even though the SFT+RL algorithm itself is not new. A central component of this scheme is the **MR-SP** system, which is carefully tailored for efficient long-video RL. It features several key designs:\n\n1. We first implemented **video encoding with balanced distribution** across multiple GPUs. (Traditional RL frameworks often run into **GPU OOM** issues when processing more than 512 frames)\n2. We introduce a **chunked distributed gathering** strategy for multimodal inputs and embeddings. (Traditional RL frameworks often suffer from **CPU OOM** issues due to the combination of long video frames and a large number of rollouts)\n3. We further **cache and reuse video embeddings** during both the rollout and pre-filling phases, effectively reducing redundant computation. (Traditional RL frameworks often suffer from **high latency** due to repetitive processing)\n\nWithout this co-design, the training algorithm alone is insufficient to handle long-video training effectively. In contrast, our training framework is capable of training on 3,600 video frames using a single node with 8 A100 GPUs. \n\nWe are fully committed to releasing the dataset, the complete code of dataset generation pipeline, and the training framework, including the MR-SP system, to support future research in the community.\n\n**Q3: “Evaluation on Video-MME”**\n\n**A:** Thanks for your reminder, we include more VLMs comparison on VideoMME, including NVILA-8B-Video [21] (CVPR 2025), Apollo-7B [22] (CVPR 2025), and VideoLLaMA3-7B [23] (ArXiv 2025).\n\nLongVILA-R1 achieves performance comparable to VideoLLaMA3. Although VideoLLaMA3 does not incorporate RL training, it uses **5.72 million samples** during video fine-tuning, Table 4 in VideoLLaMA3 [23]. In contrast, **we use only 0.21 million samples** in total across both SFT and RL, resulting in higher data efficiency and much lower training cost than VideoLLaMA3.\n\n|Model|VideoMME (w/o sub)|VideoMME (w/ sub)|\n|---|---|---|\n|Apollo-7B (CVPR 2025)|61.1|63.3|\n|NVILA-8B-Video (CVPR 2025)|64.2|70.0|\n|VideoLLaMA3-7B (ArXiv 2025)|66.2|70.3|\n|LongVILA-R1-7B|65.1|71.1|\n\n**Q4: “The comparison with Gemini-1.5-Pro and GPT-4o”**\n\n**A:** To address concerns of zero-shot evaluation, we further compare LongVILA-R1-7B with Gemini-1.5-Pro, GPT-4o, and GPT-4o-mini on ActivityNet-QA, VNBench, and VideoMME. LongVILA-R1-7B outperforms Gemini-1.5-Pro and GPT-4o on ActivityNet-QA and VNBench, and inferior to them on VideoMME, where LongVILA-R1-7B surpasses GPT-4o-mini with only 7B parameters.\n\n|Model|ActivityNet-QA|VNBench|VideoMME (w/o sub)|VideoMME (w/ sub)|\n|---|---|---|---|---|\n|GPT-4o-mini|-|-|64.8|68.9|\n|GPT-4o|61.9|64.4|71.9|77.2|\n|Gemini-1.5-Pro|57.5|66.7|75.0|81.3|\n|LongVILA-R1-7B|64.8|75.5|65.0|70.7|\n\n**Q5: “Long video frames”**\n\n**A:** As mentioned in the **General Response**, LongVILA-R1-7B is capable of processing up to **2,048** video frames per video. We progressively increase the input frames when evaluating on VNBench, and observe that performance improves as the frames increases.\n\n|Frames|128|256|512|1024|2048|\n|---|---|---|---|---|---|\n|Accuracy|60.5|69.5|73.8|74.2|75.5|\n\n**Q6: “Variable or high frame rates”**\n\n**A:** LongVILA-R1-7B supports both variable and high FPS settings. We evaluate it on VideoMME using FPS values of 8, 16, and 32, all of which result in strong performance.\n\n|FPS|8|16|32|\n|---|---|---|---|\n|VideoMME (w/o sub)|65.3|64.8|64.9|\n|VideoMME (w/ sub)|70.5|71.1|70.7|\n\n### **References**\n[1] Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis. CVPR 2025\n\n[2] LongVILA: Scaling Long-Context Visual Language Models for Long Videos. ICLR 2025\n\n[3] ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answering. AAAI 2019\n\n[4] Perception Test: A Diagnostic Benchmark for Multimodal Video Models. NeurIPS 2023\n\n[5] Needle In A Video Haystack: A Scalable Synthetic Framework for Benchmarking Video MLLMs. ICLR 2025\n\n[6] LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding. NeurIPS 2024\n\n[7] EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understanding. NeurIPS 2023\n\n[8] NExT-QA: Next Phase of Question-Answering to Explaining Temporal Actions. CVPR 2021\n\n[9] MVBench: A Comprehensive Multi-modal Video Understanding Benchmark. CVPR 2024\n\n[10] Video-LLaVA: Learning United Visual Representation by Alignment Before Projection. EMNLP 2024\n\n[11] ShareGPT4Video: Improving Video Understanding and Generation with Better Captions. NeurIPS 2024\n\n[12] VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs. ArXiv 2024\n\n[13] Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Input. ArXiv 2024\n\n[14] LLaVA-OneVision: Easy Visual Task Transfer. Trans. Mach. Learn. Res. 2025\n\n[15] Video-R1: Reinforcing Video Reasoning in MLLMs. ArXiv 2025\n\n[16] Shot2Story: A New Benchmark for Comprehensive Understanding of Multi-shot Videos. ICLR 2025\n\n[17] Scaling Vision Pre-Training to 4K Resolution. CVPR 2025\n\n[18] DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. ACL 2025\n\n[19] Video Instruction Tuning With Synthetic Data. ArXiv 2024\n\n[20] Training Verifiers to Solve Math Word Problems. ArXiv 2021\n\n[21] NVILA: Efficient Frontier Visual Language Models. CVPR 2025\n\n[22] Apollo: An Exploration of Video Understanding in Large Multimodal Models. CVPR 2025\n\n[23] VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understanding. ArXiv 2025"}, {"review_id": "NEr3znCKI2", "reviewer": "Reviewer_XSh3", "summary": "The paper describes a dataset, benchmark and RL-based training methodology for reasoning over long videos. The dataset is created by automatically generating annotations (using existing foundation models) to videos from an existing video dataset. A model trained using the introduced data and training methodology is evaluated on the VideoMME benchmark as well as on the newly introduced benchmark. It shows that the resulting model outperforms several existing models.", "questions": "- The results of LongVILA [3] are missing from Table 2. \n- Why is the 64-frames result from Video-R1-7B [12] not shown or mentioned? \n- Will the code for data generation and the MR-SP training strategy be made available? \n- \"question-answer pairs are generated via a leading open-source reasoning LLM\" - which one? \n- Line 52: \"We also manually curate a balanced set of 1K long-video samples\" - what does \"balanced\" here mean? \n- (small comment: the soccer example in Figure 1 is strange in that the prediction to be made is very stochastic and model correctness will be almost random)", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths: \n- The paper addresses a difficult (and increasingly popular) task: reasoning in long videos. \n- It addresses the task by introducing a new (automatically generated) dataset and a multi-GPU training strategy supporting RL training on long videos. \n- The paper also introduces a new hand-curated benchmark, which can be useful for revaluating future models (assuming it will be made publicly available, see below).  \n\nWeaknesses: \n- The performance of the newly introduced model is very similar to (and only marginally higher than) LongVILA [3], of which this is in a sense a derivative work. For example, results on MME reported in [3] are 60.1 (vs 60.3 in this work) in the \"w/o subtitle\"-setting and 65.1 (vs 65.9) in the \"w subtitle\"-setting. \n- The performance of the model is actually lower than that of Video-R1-7B reported in [12], which is 61.4 in the 64-frames-setting (Table 1 in [12]). Instead, only the performance of the lower performing 16-frames setting of that work is shown (which is 59.3). Why is that? Am I missing something here? \n- By comparison, the only strong performance over existing methods is on the own (1000 example-)benchmark. \n- The paper is fairly hard to read due to typos, inconsistencies, and overly touting language. (To name a few examples: Line 44: \"long-video understandings\", Line 44: \"we strategically construct a high-quality dataset\", Figure 3 caption: \"..., from which 52K QAs with reasoning annotations in LongVideo-Reason.\", Line 111 \"We then apply a high-quality automated CoT annotation pipeline\", ...", "was_revised": true, "final_justification": "Authors made significant improvements during the rebuttal period, addressing my concerns, and resulting in a much stronger paper than the original submission. In the current form, the method and dataset should be useful contribution to the emerging area of reasoning in long videos.", "rebuttal": "## **General Response**\n\nWe sincerely appreciate your detailed and constructive feedback. Before addressing each concern individually, we first to highlight the major updates as follows.\n\n1. **Larger Dataset:** We double the size of our dataset, LongVideo-Reason, from 52K to 104K reasoning data, with carefully filtering and data cleaning. Moreover, we include both multi-choice and open-ended QAs, making it more general.\n2. **VideoMME Update**: Thanks to the updated dataset, our model achieved strong performance, e.g., **65.1% / 71.1%** on VideoMME [1] without and with subtitles, outperforming the LongVILA [2] baseline, which achieved 60.1% / 65.1% on VideoMME [1].\n3. **Expanded Evaluation**: We have included **7 additional video benchmarks** for evaluation: ActivityNet-QA [3], PerceptionTest [4], VNBench [5],  LongVideoBench [6], EgoSchema [7],  NExT-QA [8], and MVBench [9]. Alongside VideoMME [1], we now compare LongVILA-R1 against LongVILA [2] baseline, and other methods [10, 11, 12, 13 ,14, 15] on a total of **8 benchmarks**, demonstrating consistently strong performance. Results are show in the table below.\n4. **Extended Inputs**: LongVILA-R1-7B is capable of processing **2,048** video frames per video, with customizable FPS settings.\n\n| Model | VideoMME (w/o sub) | VideoMME (w/ sub) | ActivityNet-QA | PerceptionTest | VNBench | LongVideoBench | EgoSchema | NExT-QA | MVBench |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |\n| VIdeo-LLaVA-7B | 39.9 | 41.6 | 45.3 | - | 12.4 | 37.6 | 38.4 | - | 43.5 |\n| ShareGPT4Video-8B | 39.9 | 43.6 | 50.8 | - | - | 41.8 | - | - | 51.2 |\n| VideoLLaMA2.1-7B | 54.9 | 56.4 | 53.0 | 54.9 | - | - | 53.1 | - | 57.3 |\n| Kangaroo-8B | 56.0 | 57.6 | - | - | - | 54.8 | 62.7 | - | 61.1 |\n| LLaVA-OV-7B | 58.2 | 61.5 | 56.7 | 57.1 | 51.8 | 56.4 | 60.1 | 79.4 | 56.7 |\n| Video-R1-7B | 61.4 | - | - | - | - | - | - | - | 64.8 |\n| LongVILA-7B | 60.1 | 65.1 | 59.5 | 58.1 | 63.0 | 57.1 | 67.7 | 80.7 | 67.1 |\n| LongVILA-R1-7B | **65.1** | **71.1** | **64.8** | **68.9** | **75.5** | **58.0** | **68.9** | **81.5** | **68.2** |\n\n## **Detailed Response for Each Question**\n\nWe provide detailed responses to each of the concerns below.\n\n**Q1: “The performance is similar compared to LongVILA”**\n\n**A:** As shown in the table in **General Response**, LongVILA-R1-7B benefits from the updated large-scale dataset, achieving **65.1% / 71.1% on VideoMME** [1] without and with subtitles, representing **+5.0% and +6.0% improvements over LongVILA-7B**, respectively.\n\n**Q2: “Missing Video-R1-7B in the 64-frames-setting”**\n\n**A:** Apologies for the oversight. The 64-frame result of Video-R1-7B [15] (**61.4%** on VideoMME without subtitles) was updated in its v2 version on arXiv on May 14, shortly before the NeurIPS submission deadline, and we missed it. LongVILA-R1-7B achieves **65.1%** on VideoMME without subtitles, surpassing Video-R1-7B. **We have included the comparison in the table in the General Response and will correct this value in the revision.**\n\n**Q3: “The only strong performance on the own benchmark”**\n\n**A:** Thank you for the reminder. As shown in the table in **General Response**, we have evaluated LongVILA-R1 on seven additional benchmarks. Alongside VideoMME [1], it demonstrates strong performance across eight benchmarks in total, for example, **64.8% on ActivityNet-QA** [3], **68.9% on PerceptionTest** [4], and **75.5% on VNBench** [5], showing significant improvements over existing models.\n\n**Q4: “Typos, inconsistencies, and overly touting”**\n\n**A:** Thank you for pointing this out. We will correct the typos and carefully refine the language in our revision to address inconsistencies and overly promotional phrasing.\n\n**Q5: “The results of LongVILA are missing from Table 2”**\n\n**A:** Thank you for the reminder. The results of the LongVILA [2] baseline are provided in the table below, and we will include them in the revision.\n\n| Model | Temporal | Goal | Plot | Spatial | Overall |\n| --- | --- | --- | --- | --- | --- |\n| LongVILA-7B | 58.0 | 80.2 |  57.1 | 46.7 | 62.7 |\n| LongVILA-R1-7B | 68.1 | 85.7 | 70.6 | 53.3 | 72.0 |\n\n**Q6: “Will the code for data generation and the MR-SP training strategy be made available?”**\n\n**A:** Yes, we are fully committed to release the training data, benchmark, data generation pipeline, and the whole training pipeline with the MR-SP training system.\n\n**Q7: “The leading open-source reasoning LLM”**\n\n**A:** It is the DeepSeek-R1-671B [16] model, and we will clarify this in our revision.\n\n**Q8: “What does ‘balanced’ here mean?”**\n\n**A:** By \"balanced,\" we mean that the number of questions is balanced across the four categories: temporal, goal and purpose, plot and narrative, and spatial.\n\n**Q9: “Small comment on the soccer example“**\n\n**A:** Thank you for the suggestion. We will include additional examples in the revised version to better demonstrate the advantages of our model.\n\n### **References**\n\n[1] Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis. CVPR 2025\n\n[2] LongVILA: Scaling Long-Context Visual Language Models for Long Videos. ICLR 2025\n\n[3] ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answering. AAAI 2019\n\n[4] Perception Test: A Diagnostic Benchmark for Multimodal Video Models. NeurIPS 2023\n\n[5] Needle In A Video Haystack: A Scalable Synthetic Framework for Benchmarking Video MLLMs. ICLR 2025\n\n[6] LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding. NeurIPS 2024\n\n[7] EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understanding. NeurIPS 2023\n\n[8] NExT-QA: Next Phase of Question-Answering to Explaining Temporal Actions. CVPR 2021\n\n[9] MVBench: A Comprehensive Multi-modal Video Understanding Benchmark. CVPR 2024\n\n[10] Video-LLaVA: Learning United Visual Representation by Alignment Before Projection. EMNLP 2024\n\n[11] ShareGPT4Video: Improving Video Understanding and Generation with Better Captions. NeurIPS 2024\n\n[12] VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs. ArXiv 2024\n\n[13] Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Input. ArXiv 2024\n\n[14] LLaVA-OneVision: Easy Visual Task Transfer. Trans. Mach. Learn. Res. 2025\n\n[15] Video-R1: Reinforcing Video Reasoning in MLLMs. ArXiv 2025\n\n[16] DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. ACL 2025"}]

Author rebuttal:

Meta-review: Final rating:  4: Borderline Accept/ 5. Accept/ 4: Borderline Accept/ 4: Borderline Accept. The paper introduces LongVILA-R1, a long-video reasoning framework combining a 52K LongVideo-Reason dataset, a two-stage CoT-SFT→RL pipeline, and MR-SP training (sequence parallelism + vLLM with cached embeddings). LongVILA-R1-7B shows strong VideoMME results, surpasses Video-R1-7B by 5.2%, matches Gemini-1.5-Pro on the authors’ eval, achieves up to 9.2× RL speedups, and scales from 16 to 512 frames.

Reviewers commend the valuable LongVideo-Reason dataset and a polished training pipeline with strong results, but initially flagged: no LongVILA baseline and only marginal VideoMME gains despite costly CoT-SFT+RL; narrow evaluation (VideoMME + in-house), limited diversity (Shot2Story-based) and modest novelty (SFT→GRPO); incomplete SOTA/fairness comparisons (fine-tuned on own eval vs zero-shot rivals); and scalability/robustness limits (≤512 frames; unclear at high/variable frame rates).

After rebuttal, reviewers said concerns were addressed and leaned toward acceptance. The ACs concur—citing the dataset’s value (104K reasoning QA pairs), efficient MR-SP training, and competitive VideoMME scores (65.1/71.1 w/o/with subtitles)—and recommend acceptance.
Decision: Accept (poster)