ReWatch-R1: Boosting Complex Video Reasoning in Large Vision-Language Models through Agentic Data Synthesis
Authors: Congzhi Zhang
Zhibin Wang
Yinchao Ma
Jiawei Peng
Yihan Wang
Qiang Zhou
Jun Song
Bo Zheng
Venue: ICLR 2026 Poster (2026)
Keywords: Video Reasoning
Large Vision-Language Models (LVLMs)
Agentic Data Synthesis
Multi-Agent ReAct
Reinforcement Learning with Verifiable Reward (RLVR)
Chain-of-Thought (CoT)
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We introduce an agent-based pipeline to synthesize a high-quality video reasoning dataset (ReWatch) and a novel reinforcement learning reward (O&R) to train LVLMs, achieving state-of-the-art performance.

Abstract: While Reinforcement Learning with Verifiable Reward (RLVR) significantly advances image reasoning in Large Vision-Language Models (LVLMs), its application to complex video reasoning remains underdeveloped. This gap stems primarily from a critical data bottleneck: existing datasets lack the challenging, multi-hop questions and high-quality, video-grounded Chain-of-Thought (CoT) data necessary to effectively bootstrap RLVR. To address this, we introduce ReWatch, a large-scale dataset built to foster advanced video reasoning. We propose a novel multi-stage synthesis pipeline to synthesize its three components: ReWatch-Caption, ReWatch-QA, and ReWatch-CoT. A core innovation is our Multi-Agent ReAct framework for CoT synthesis, which simulates a human-like "re-watching" process to generate video-grounded reasoning traces by explicitly modeling information retrieval and verification. Building on this dataset, we develop ReWatch-R1 by post-training a strong baseline LVLM with Supervised Fine-Tuning (SFT) and our RLVR framework. This framework incorporates a novel Observation \& Reasoning (O\&R) reward mechanism that evaluates both the final answer's correctness and the reasoning's alignment with video content, directly penalizing hallucination. Our experiments show that ReWatch-R1 achieves state-of-the-art average performance on five challenging video reasoning benchmarks.

Published as a conference paper at ICLR 2026
REWATCH-RI: BOOSTING COMPLEX VIDEO REASON- ING IN LARGE VISION-LANGUAGE MODELS THROUGH AGENTIC DATA SYNTHESIS
Congzhi Zhang" 1,2.3 Zhibin Wang Yinchao Ma*1
Jiawei Peng*
Yihan Wang" Qiang Zhou 1.2
Jun Song
Bo Zheng
Alibaba Group Holding Limited Future Living Lab of Alibaba
SEU
4USC
DDUT
changcongchil@ gmail. com
jsong.sj @aliiaabaiinnccom
ABSTRACT
While Reinforcement Learning with Verifiable Reward (RLVR) significantly ad- vances image reasoning in Large Vision-Language Models (LVLMss, its application complex video reasoning remains underdeveloped. This gap stems primarily from a critical data bottleneck: existing datasets lack the challenging, multi-hop questions and high-quality, aedeo-grounded Chain-of-Thought (CoT) data nec- essary to effectively bootstrap RLVR. To address this, we introduce Re Watch, large-scale dataset built to foster advanced video reasoning. We propose a novel multi-stage synthesis pipeline to synthesize its three components: ReWatch- Caption, Re BatWhhh-hOLl QA, and Re ReWatch-CoT. core innovation our Multi-Agent ReAct framework for CoT synthesis, which simulates human-like "re-watching" process to generate video- grounded reasoning traces by explicitly modeling infor- mation retrieval and verification. Building on this dataset, develop ReWatch-R1 by post-training a strong baseline LVLM with Supervised Fine- Fine-Tuning (SFT) and RLVR framework. This framework incorporates novel Observation & Reasoning (O&R) reward mechanism that evaluates both the final answer's correctness and the reasoning's 's alignment with video content, directly penalizing hallucination. Our experiments show that ReWatch-RI achieves saaeeoof-the-ate performance on five challenging video reasoning benchmarks. Project Page.
ReWattch
Video Chat R1 Micea-R1
IntemvL3.5 1335 88 GUM-4 IV-98
Qwen2.5 VL-78
ViceoRFT
30
40
Accuracy (%6
20
10
VCR-Bench
Video
VideaMathQa
CG.AV AV-CCnngg
Figure 1: Performance comparison of our ReWatch-R1 with previous state-of-the-art LVLMs on five video reasoning benchmarks. Except for Qwen2.5-VL-7B, all other models use thinking mode. All models were evaluated 192 frames.
"Equal contribution.
1 "oorrsponniig Author.

---

Published as a conference paper at ICLR 2026
INTRODUCTION
While the training paradigm of Supervised Fine-Tuning (SFT) combined with Reinforcement Learning with Verifiable Reward (RLVR)
42 significantly advances image reasoning in Large Vision- [8
Language Models (LVLMs) 12
22
5 1 its application complex video reasoning remains nascent.
10: 39] trained with SFT+RLVR still underperform on high-difficulty benchmarks, especially for multi-step temporal tasks such as causality, state tracking.
Recent open-source video models 16 30; 45
and counting events across long videos 40 38
36..
Recent efforts to apply the SFT+RLVR paradigm video [66 45
10 39]
typically bootstrap
the SFT phase with CoT data synthesized from existing simple video QA datasets, before applying RLVR. However, this approach is fundamentally undermined by the quality of the underlying data. As illustrated in Figure lleft), prevailing open-source data [16] suffers from three flaws: (1) holistic, untimestamped captions that erase temporal structure; (2) simple, perception-based QA that can be answered from short clips or textual priors; and (3) visually unfaithful CoT that relies on commonsense knowledge and process of elimination. This data bottleneck prevents SFT from teaching true video- grounded reasoning, and the subsequent RL phase, lacking a reliable reward video-grounded
signal for process correctness, struggles to penalize hallucination and improve logical fidelity [2]
To address these limitations, we introduce Re Watch, a large-scale dataset explicitly designed to foster advanced video reasoning. Retch is constructed through a multi-stage synthesis pipeline and comprises three tightly coupled components: Re RaWchh Capiion, Re ReWatch-QA QA, and Re ReWatch-Coll First, Re WatchhCaption provides temporally dense video descriptions. We employ a hierarchical captioning method to generate detailed, timestamped narratives that form high-fidelity foundation for complex reasoning. Second, Re Watch-QA features high-difficulty guestion-answer pairs. We use contrastive generation strategy, creating questions from detailed captions that cannot be answered by concise summaries, and apply a three-tier filter to guarantee video dependency. Finally, Re Watch- CoT promotes video-grounded reasoning. We employ novel Multi-Agent ReAct framework to synthesize CoT that simulates human-like "re-watchi reewwachhing" process. This generates reasoning traces that explicitly document information retrieval and verification against the video content. As shown in Figure 2 right), our Re Watch data delivers high-fidelity captions, high-difficulty QAs, and drdeudgr CoTs.
Building Watatch, post-train strong LVLM in two stages to obtain ReWatch-R1. After an initial SFT phase that teaches step-by-step reasoning, we employ RLVR augmented with a novel
(01:9-00::22
[01:30-01:3]]
(01:57 01:591
Video-R1 Dataset
Our ReWatch Dataset
Caption
Non-temparat, Coorse
Caption
Temporal, Fine-groined
The noxt part video begins with towering rock...
The diwer jwith the blande, curly naij holcing sandwilly, which paiss the man behind him (wearing black wetsutt) who then tvbeca bite.
jo2:00 01:12] Two man
the ware builds, a la mileow appears in the mat
The siito then thowle person in brbek wetsil curting down
The stiteo conch with a peroo in bluck wetruit surting down the face masiled ware.....
[01:30-01311] The blande, cur murlyshatred man, now in wetuut [01:50 52) The two men on the jot ski, one driving and.
Simple, Percepptionbauee
Camplixe Timmsstmpp-dhinee
Qavention
phenomenon appears person pariies
Oivaston: After the man with trands, curly hair
rg jet at (01:30 (2.53]
and ater approching werhiiinggg the comera at (01:50-01 12), what specific food hem does he pans to passengar an our ier whale on the jot
AcA curset
Ir nembow Austarry
C-A. lightiing bolt
Energy gei Sandatt
Bi Fruit but
Acswer. 8
Vonier bottle
Annwert
Col
Single hop, Percoptual reasoning
Multi hop, Evidence bosed, Video gr grounded reasoning
athinloust me think a momenn. The question is solling about ratural phorometion that appears in the mis while samsone pard ing en surtharrs. Imm, treak down. The intting Htrly near water, plen surbbard and the presence lighning sunset
Ineed to dennity saeciti: food ham passedity by the mon the jet ski passenger at point carler than the given emmostaps. This means I'll start by looking for the carlisst instance the man on the jet interacting with a possenget specifica by seardhing for food redtiee euchorpes So. cacttonoeer regmeens ssing
mis suggests
typkaily regularly
besuefil yould condition. what ahout nenbow? Ruinbows often minty miny conditions when sunlight refracttdd through water droplets. This itts the sconaria fustly Ol, sec, the most Sitting natural phanomenon in is indeed i rarbbow </thiik Bansver-B-/arswrr
blnds, curly hair on found sogricnt
man recraciing with passer sast/atioo... subservationon' bi-eeeeeeetttonn objects paised
relexant
contree
segment,
segment
looking
cobservationo Buring time, the man being candwith the passcrggos who then taless bite- tttee//bborraiiie donto. This directly arowers the qusition. The frod ttem prosed won cendarr et. cammeen:: San Saw.wwchh//aasssee
bettoeen the and passengee and
cinnsmedee/eee mede//attions.
Figure 2: A comparative of Re Watch dataset and Video-RI dataset on the same source video.
2

---

Published as a conference paper at ICLR 2026
Observation & Reasoning (O&R) reward. Unlike rewards that score only the final answer, O&R also evaluates whether intermediate observations are factually supported by the video and whether the reasoning is sufficient to recover the correct answer from those observations. This dual emphasis on process and outcome explicitly incentivizes verifiable, evidence-linked reasoning. reducing hallucinations and improving logical consistency. As summarized in Figure T ReWatch-R1 sets new state of the art on five challenging video reasoning benchmarks, substantially outperforming models trained on alternative open-source data.
In summary, our contributions are:
A novel, multi-stage stage agentic pipeline for synthesizing a large-scale, high-quality video reasoning dataset (ReWatch).
A new Observation & Reasoning (O&R) reward for RLVR that improves reasoning by rewarding both final-answer correctness and the factual grounding of intermediate steps in video content.
ReWatch-R1, post-trained LVL.M that achieves state-of-the-ard results on five complex video reasoning benchmarks.
DATA CONSTRUCTION: THE REWATCH DATASET
To address the above data bottlenecks, we introduce Re Watch, large, high-fidelityy, bigh-difficulty, and video-grounded dataset for advanced video reasoning. As shown in Figure 3 it is constructed in three stages: Hierarchical Video Captioning, High-Difficulyy QA Generation, and Multi-Agent CoT Synthesis. The dataset contains 10k captions, 170k QA pairs, and 135k CoTs. More details and statistics are in Appendix B
2.1 STAGE 1: HIERARCHICAL VIDEO CAPTIONING
To address the hallucination issue in LVLMs when processing long videos and to generate high- fidelity video descriptions, we propose Hierarchical Dynamic Frame-Rate Generation pipeline for our W-aachh-appiines /0k dataset, The process is applied to our video corpus V, sourced from five public datasets (2
59
Semantic Segmentation. For each video E V. we first partition V into semantically coherent segments S using LVLM Masg. at a low-frame-rate. To strictly preserve long-term contextual integrity, we apply this segmentation only to videos exceeding 10 minutes in duration. Unlike fixed-interval splitting,
approach leverages the LVLM to perform semantic-based partitioning, ensuring that each segment 8; retains a complete narrative structure with an approximate duration of 10 minutes. Each segment 8; corresponds to temporal interval [1start (edd) preserving event integrity.
S {si ................................ 94
MMsgg()
(1)
Detailed Description Generation. We use powerful LVLM Meep to process each segment s; at high frame rate and generate detailed deseription D which includes m, distinct events {cy} along with their relative timestamps {Ti)
D.s {Gij T6)) = (s)
Timestamp Realignment. Finally, function P converts relative timestamps Tıy to absolute ones tij by adding the segment's ' start time.
li = P(Tij, (tart)
1start +
(3)
The final video caption Caetan (V) is the union of all timestamped descriptions.
Caeail (V) U(c,
This hierarchical approach generates temporally precise and semantically rich descriptions while avoiding the hallucination issues associated with LVLMS processing long videos.
3

---

Published as a conference paper at ICLR 2026
Gesovitee
Vidar Captke
Valce Capticn
Captier Qeockes
Loee frmeeratc
Realignnets
BeWarkk
Cxetraatic QA
QA
Segroneation
Ditasied Deverption
Generation
Vidao Summnar
Sureeury Quecitas
Behazis- QU-TEL
Stage I: Hleerarchical Video Captioning
Stage 2: Hlih-difffinlty QA Pair Generation
Thinght
Artion Sigwet Raiiev!
Strattiedd Eancetiee Trane
Naturl Langage Call Sorieg
Quetien
Captina Quers
Tinestanp
-aadaar "/akkkr sabernatoon 2//bsrnatoos
Reasan
Charrer
Junier Seyver Qvery
Capticn Timesiamp
Detail
ReWaide Cut
Simplinng
(c) Stage 3: Multi-Agent Chain-of-Thoeeee Synthesis
Figure 3: The data construction pipeline. (a) Caption Construction. Long videos are semantically segmented to produce detailed, temporally-awand captions. (b) QA Pair Generation. contrastive method using detailed and summary captions generates complex questions, which are then purified by a three-layer filtering mechanism. (c) CoT Synthesis. ReAct framework with a Reasoner Agent and an Observer Agent simulates "re-watee re-watching" process by performing targeted queries on the video caption to generate groeud-groddedd reasoning traces.
2.2 STAGE HIGH-DIFFICULTY QA PAIR GENERATION
To create our ReWatch- -aatcchh-Q--7Ok dataset, we design a pipeline generate challenging QA pairs requiring fine-grained video analysis. It combines Contrastive Prompting with Three-Layer Filtering.
Contrastive QA Generation. Given a detailed caption Csewit we first generate a concise summary Csum = Maum (Cdewa) using lightweight LLM. Then, inspired by previous work [73] 5] our QA generator Maa processes both Cdetall and Csum to create QA pairs (Q, A) that are explicitly answerable from the detailed caption but not from the summary alone. This ensures questions probe fine-grained details while excluding trivial ones.
Araw = MMu Csum)
(5)
To guide generation and ensure diversity, we pre-define 10 question types.
Raw pairs undergo three-layer filtering cascade to ensure quality and Three-Layer Filtering.
video-dependency:
Filter Answer Verification, Fi: A verifier Mvvrify confirms the factual correctness of answer based on Cdetail-
(Q,A) passes Fi .Mventy (Q, A, Cdeaai) = True
(6)
Filter 2: Text Bias Elimination, F2: Ensures the question is unanswerable from general knowledge by probing set of LLMs MIprohe-
(Q, A) passes F2 MPoobel MEM
Σ
1(M(Q) A) 0wst
Filter 3: Summary Bias Elimination, F3: Similarly ensures the question is unanswerable using the summary Caun-
(Q, A) passes F
Σ
1(M(Q, Csum) =s A) < 6umm
(8)
MMprobe
MEM
Where fwe and 6mmm are threshold for consensus. The 85k pairs passing all filters are then rewritten by L.M Mrewrite into multiple-choice questions, yielding total of 170k QA pairs.
4

---

Published as a conference paper at ICLR 2026
2.3 STAGE 3: MULTI-AGENT CHAIN-OF-THOUGH SYNTHESIS
To generate our ReWatch-Co-1138 dataset, we introduce a multi-agent ReAct-based framework that explicitly construct the video-grounded COT. This method extemalizes the observation process for active information retrieval.
We define two agents: Reasoner AR that produces thoughts T and actions Act, and an Observer Ao that executes actions on the video caption Cactail to return observations Obs.
For given question Q, the agents interact in a loop. At each step the Reasoner uses the history H₁-1 - (Q, Tı, Act1, Obs1, T2-1, Act,-1, Obs;-1) to decide the next step:
(Tr Act,) AR(H(-1
(9)
The Observer executes the action to retrieve information from the video context:
Obs: - Ao(Acl,, Cdetail)
(10)
It is important clarify that Observer Agent Ao retrieves observations from the detailed textual captions (Cdetait) rather than processing raw video frames during synthesis. Through manual inspec- tion, confirmed that hierarchical captions from Stage 1 are sufficiently fine-grained to serve as a high-fidelity proxy for visual content. This text-based simulation drastically improves the efficiency and scalability of data synthesis compared to pixel-based methods. While current pipeline is text- based, the synthesized "Thought-Action Aaaighi-AAttoo-Obseaattioo trajectories provide a foundational resource for training future thinking-with-viin
models that can directly query visual encoders.
This process continues until the Reasoner produces a final answer. The core actions Acl, simulate visual lookup:
segment_retrieeee (query): Finds the timestamp of an event from natural language query. aggment_querytiimsttaae Retrieves the detailed description of an event from a timestamp.
This entire text-based simulation is highly efficient. The structured execution trajectory T {(T), A Obsihhss.................... (Aima)) is then converted by LLM Mconven into a natural language CoT string R with explicit <action> and <observation> tags, making ready for supervised fine-tuning and O&R reward calculation.
3 POST-TRAING ON REWATCH DATASET
As shown in Figure 4 we use the SFT+RL paradigm to train Qwen2.5-VL. VL. In the SFT stage, we use multi-task objectives to train to obtain -eWS..h-RI-SFT. In the RL stage, based on the GRPO [[8] algorithm and a novel O&R reward mechanism we propose, obtain WeWatch-R1.
3.1 SUPERVISED FINE-TUNING STAGE
In this stage, we perform multi-task SFT on a base LVLM using our three datasets: Re ReWatch-Capae Caption 10k (Deup), Re Watth--Q--177 (Doa), and Waath--CoT-1335 (Dcot). The goal is jointly instill three core abilities: foundational video-text alignment, direct question-answeriin mode), and step-by-step reasoning ("thinking" mode). Crucially, we train the model to switch between these response modes using distinct instruction prompts. For detailed prompt setting during SFT, please refer Appendix E.2
("non-thinking"
The SFT objective is to minimize a composite loss function, LSFT, which is the sum of the losses from these three tasks. Let the LVLM be denoted by a policy we with parameters 0. The total loss is defined as:
LSFT (0)
Lcap + Lox + LOST
(11)
where each component corresponds to specific learning objective:
Video-Text Alignment.
We train the model to generate detailed captions (Coenat) from videos (V).
Lcup
EVy,Caaaa E [log mo (Ceeaal VV]
(12)
5

---

Published as a conference paper at ICLR 2026
(a) SFT
(b) RT.
Capties-100
Reference Model
Computation
Base
QA-170%
SFT Model
LYLM Policy
Calculation
ReWatch- CoI-135k
Rtlllut
Reward
Advantage
Q4-170k
(c) Rollout (Purcly Textual
Reward Calculation
Action
Ressoning Reward
Question
Video
Observation Reward
Caption
LYLM
Thought
Action
Observation
Answer
Observation
Accuracy Reward
Answer
Answer
Question
Format Reward
Figure 4: Our two-stage Post-Training framework. (a) Base Model is first fine-tuned (SFT) on all Wetath datasets, (b) then further refined as policy via Reinforcement I Learning (RL. using the ReWatch-QA dataset. (c) The "Rollout" panel illustrates the generative process the policy: producing a purely textual chain-of-thought that simulates
Ceciht--AActiooObbereeeee loop through self- eeff-geneatte text segments. (d) We employ four verifiable reward mechanisms.
reasoning
Direct Qusstion-Answering (Non-thinking). We train the model output a concise answer (A)
when given direct-answer instruction Liirect.
for -(V.. [lo (A(A direet, Q]]
(13)
Chain-of-Thoupee Reasoning (Thinking). We train the model to generate the full reasoning trace (R) when given think-step-by-ster instruction Inink-
CcSt -Ev.. Te (R Iuink: Q)]
(14)
By optimizing these objectives concurrently, we produce a versatile SFT Model that is proficient in both direct answering and complex reasoning, This model then serves the proficient initial policy for the subsequent Reinforcement Learning stage.
3.2 REINFORCEMEN LEARNING STAGE
Previous LVLMs of video reasoning Io: 16) directly utilize the accuracy the final answer Tace as the reward signal for reasoning enhancement through reinforcement learning. Formally,
Pace Miuudge Agt);
(15)
where Mjnage()) is the judge model used to assess the consistency of inputs, which can be rule-based verifier or an LLM. However, the foundation of video reasoning lies in the ability to reason grounded video content. Such reward for mere accuracy overlooks the capabilities of video content-oriented reasoning, which may lead potential visual linguistic hallucinations, To address this limitation, we design the Observation Reasoning (O&R) reward mechanism, which encourages the model to perform appropriate reasoning grounded in the accurate understanding of video content, rather than relying on potential visual or linguistic hallucinations. Specifically, we model the video reasoning QA process sequential flow:
Video+Question + Observatiios==Reasnnin
Answer
On one hand, the model should base its reasoning on accurate observations of the video content. Thus, we first assess the accuracy of video observations in CoT by comparing them with the detailed video caption, and use this evaluation as the observation reward. Formally,
{Act,, Obs,) N = Parse(R),
(16)
mcan (.M (..Man((MMMMaaa (Cdetail. {Act,, Obs; Obs,1)]
(17)
Here, Parse(-) denotes parsing the actions and observations from the model output.
On the other hand, the model should reason out appropriate observational actions according to the question. Therefore, design the reasoning reward by evaluating the accuracy of directly answering

---

Published as a conference paper at ICLR 2026
questions using the actions and observations. If the model can provide a correct answer based on these actions and observations, the reasoning process deemed valid and sufficient. This reward guides the model to reason appropriate observation actions that effectively address the question. Formally,
Aeo MQn (Act,, Obs, (bst)e.))
(18)
Trea = =MMudgee ( (Aao: Agt).
(19)
Here, Manfer()) is an LLM used to answer the question based on the given actions and observations. The final reward can be expressed as,
rO&R Tace x (1 + Tobs + Trea) + Tymss
(20)
1, correct format
Tfmm
(21)
0. otherwise
Here, rfmt denotes the format reward, enabling the model to output responses in the for- mat we desire. For example, we expect the model to enclose its actions and observations with <action>... ... </action> and <observation>. </observation> tags, and the answer with <answer>... </answer> < tag. Finally, we employ the GRPO [8] algorithm for model optimization.
4 EXPERIMENTS
We train Qwen2.5-VL-77 on the Re Wetch dataset to obtain Rewatch-R1, and then compare with other LVLMs on five video reasoning and four video understanding benchmarks. For detailed experimental settings, please refer the Appendix C.I
4.1 MAIN RESULTS
Table shows the superior video reasoning performance of our model, yielding following key insights.
SOTA Performance among models of comparable size. In both 192-frame and 384-frame settings, the average scores of ReWatch-R1 across five reasoning benchmarks significantly surpass those of all other comparison models. This validates the effectiveness of our dataset and training methodology.
High-Quality CoT Data is Critical. The SFT-only model ReWatch-RI-SFT (33.25%) already surpasses most competitors like Video-RI-SFT (29.74%) and Long VideoReasoon (26.31%), which use the same training configuration. This proves the superiority of our CoT training data.
RL Unlocks Further Potential. Reinforcement learning further boosts performance. Our final ReWatch-R1 model improves upon the SFT version (33.25% 35.51% This shows that while SFT teaches the form of Co',, our RL. phase imparts the spirit, enabling more logical and factually grounded reasoning.
The Efficacy of "Thinking" is Contingent on Learning "How to Think", Enabling CoT ("Think- ing" mode) is detrimental for untrained base model (27.54% vs. 30.71%), it can induce hallucinations. In contrast, our fully trained Re Watch-RI excels with COT. This proves our method successfully teaches the model how reason.
We further evaluate performance on video understanding benchmarks in Table 4 and performance on videos of varying durations Figure For detailed analysis, please refer Appendix C.2 and C.3
4.2 ANALYSIS RESULTS
High-Qualtn SFT Data Foundational for RL. An ablation study in Figure 5a shows two key findings. First, SFT is an indispensable prerequisite for RL, training without it (w/o SFT) causes a catastrophic performance drop, as RL needs a strong initial policy. Second, high-quality CoT data is vital. Replacing Watcch-- Watchh-ooT data with that from Video-RI significantly degrades performance. This validates that our multi-agent framework produces a superior training corpus for complex reasoning.
7

---

Published as a conference paper at ICLR 2026
Table Performance comparison on Video Reasoning tasks. indicates that reproduced the model using training configuration with 192 frames. indicates that reinforcement learning is conducted using exactly the same data as R. The best results among models of the same size are indicated in bold.
VCR Bench
Video Holmes
Video MathQA
CG-AV Counting
Models
Thinking
MINERVA
Average
192 Frames
Qwen2.5-VL-32B
39.85
38.15
43.28
33.33
23.95
35.71
Qwen2.5-VL-7B
36.75
33.19
38.87
24.76
19.96
30.71
Qwen2.5-VL-7B
34.72
29.15
34.78
24.52
14.51
27.54
GL.M..
34.53
33.75
38.98
27.38
21.32
31.19
VL3.5SV
30.17
33.12
35.11
27.86
22.30
29.71
Video-R1
32.69
32.36
41.97
25,95
22.01
31.00
Video-Chat-RI
32.79
30.33
36.31
22.62
14.51
27.31
VideoRFT
32.22
41.37
25.00
VersaVid-RI
36,56
31.45
39.09
24.05
23.27
30,88
TW-GRPO
26.11
34.38
42.19
26,90
19.47
29.81
GRPO-CARE
35.49
31.87
38.27
25.48
19.57
30.14
Video-R1-SHI"
33.85
31.45
37.29
26.43
19.67
29.74
Video-R1-RI."
34.24
31.45
37.18
27.38
21.13
30.28
LongVideoRenson--S
24.37
29.71
38.60
23.10
15.77
26.31
LongVideenResoo-RR..
35.30
35.01
43.49
23.57
20.55
31.58
ReWatch-RI-SFF
35.78
35.43
39.52
30,00
25.51
33.25
ReWatch-R1
40.14
35.70
43.00
30.71
24.73
+ O&&
40.43
36.05
43.88
31.67
25.51
35.51
384 Frames
Qwen2.5-VL-32B
X
39.75
38.63
44.04
33.81
25.71
36.39
Qwen2.5-VL-7B
X
34.91
34.59
39.90
24.76
20.16
30.86
Qwen2.5-VL-7B
/
32.45
31.10
34.89
24.00
16.57
27.80
GI.M4.1V-9H
38.59
36.54
41.10
33.10
23.08
34.48
Intern VL3.5-8B
30.56
29.43
32.55
28.57
23.27
28.88
Video-R1
32.40
35.77
41.37
23.57
20.84
30.79
Tideo-Chat-R R1
31.72
31.66
36.47
22.62
14.61
27.42
VideoRFT
34.62
34.38
41.26
25.24
20.93
31.29
VersaVid-Ri
33.46
33.75
39.74
23.57
21.32
30.37
TW-GRPO
25.82
35.43
42.24
27.86
19.96
30.26
GRPO-CARE
36.46
33.05
38.11
25.00
20.64
30.65
Video-RI-SF7
33.95
35.56
37.29
25.24
21.91
30.79
Video-RI-RL""
35.69
32.29
37.83
26.67
20.06
30.51
LongVideoReason-SSPP
24.18
30.20
38.49
23.33
6.04
24.45
LongVideeRReason-RR.
34.91
37.24
43.88
24.29
22.01
32.47
ReWarch-R RI-ST
36.17
35.50
39.09
30.48
22.78
32.80
ReWatch-RI
39.56
38.15
43.98
30.95
25.32
35.59
+ O&&
38.78
36.54
44.26
32.62
26.68
35.78
High-quality QA data is crucial for RL. A comparative analysis in Figure 5b shows that the quality of QA data used for RI. determines final performance. Training on only baseline QA data (Video- RI-QA [16) (10k) and Long VideoReaaoon-Qs [10] (10k)) yields the lowest scores (42.0% all, 34.3% reasoning, 51.7% understanding). whereas our Re ReWatch-QA data provides notable improvements. This confirms that Re ReWatch-QA. due its challenging nature, offers more potent reward signal that guides the model toward robust reasoning abilities instead of overfitting to simpler patterns.
Dataset Complexity & Video Dependency. Figure 6a presents quantitative analysis of the com- plexity comparison between the ReWarch-OA QA and Video-RI RI-QA datasets. The detailed experimental design can be found in Appendix C.4 The results show that the Re ReWacch-CL QA dataset elicits more
8

---

Published as a conference paper at ICLR 2026
Rwthnd-Rt
Vite-RthdoT
SFT
baadine detaet
50.
40
20
DAT
Reasoning
Understanding
A3
Understand
(a) Ablation results using different CoT data.
(b) Ahlation results using different QA data.
Figure 5: Ablation results of our synthesized data against baselines.
100
Reasoring Bendhmatks
Un Urderstarding Benehmmanis
20-
(a) A comparison of QA complexity.
(b) Evolution of number of actions and accuracy.
Figure 6: Analysis on QA complexity and Evolution of action count.
profound reasoning than Video-RI-QA. ReWatch requires nearly double the reasoning steps (3.31 vs. 1.82) and significantly longer responses (398.75 vs. 205.74). Critically, Video-RI has a high Text-Only Accuracy of 68.9%, indicating questions are often solvable from text alone. In contrast, the accuracy of Re Watch is only 29.4%, near the 25% random- mudos-guess guess baseline. This proves our three-stage filtering effective, eliminating textual shortcuts and forcing genuine video understanding.
RI optimizes the reasoning process, leading to more efficient yet more accurate responses. Figure (6b) shows two-stage evolution. First, SFT teaches the model a structured reasoning format, increasing action counts and accuracy. Then, during RL, accuracy continues to improve while the average number of actions decreases. This indicates RL refines the policy to be more effective and efficient, pruning redundant steps to focus on critical actions. The model thus transitions from learning reasoning's 's form (SFT) to mastering its function with efficiency (RL).
The thinking mode, while converging more slowly during training, ultimately achieves significantly higher performance ceiling than the non-thinking mode. As shown in Figure the two modes exhibit different learning dynamics. During the SFT phase (solid lines), the direct-answer "non-thinking" mode improves rapidly, whereas the "thinking" mode develops slowly. This suggests SFT primarily teaches the format of reasoning, not its logic. The subsequent RL phase (dashed lines) acts as a catalyst, causing dramatic performance leap in the thinking mode by forcing the model to learn the causal links between reasoning and correct answers. Ultimately, the final model's 's "thinking" performance surpasses the mon-thinking" mode in all tasks. This empirically proves that an explicit, step-by-step reasoning process, cultivated via our SFT-RL regimen, optimal for complex video tasks.
Scalability of the ReWatch Framework. To verify whether our proposed pipeline generalizes to larger parameters, we scaled up the base model to Owen2.5-VLL-32B. As presented in Table 2 the performance trajectory remains consistent with the 7B experiments. The post-trained ReWatch- R1-32B achieves average accuracy of 38.08% reasoning benchmarks, surpassing both the strong base model (35.71%) and the SFT variant (36.17%). Notably, the inclusion of the O&R reward mechanism continues to yield performance gains (improving from 37.66% to 38.08%) further validating that our data synthesis pipeline and SFT+RLVR strategy are model aagl---gnostic and effective at unlocking reasoning capabilities in larger-scale LVLMS.

---

Published as a conference paper at ICLR 2026
ReWorch RI SFT (hon thinking
RoNatth R! SFT (einking)
Tasks
Understanding Tasks
36
preddhiking
53
60
81
50
30
40
20
86
3.7
57T
Figure 7: Impact of SFT and RL on different prompting methods. The plots show the accuracy of our Re Watch--R model with "thinking" (ReAct) vs. "non-thinking" (direct answering) prompting. Solid lines show performance progression during the SFT phase, dashed lines show the final perfor- mance after RL.
Table 2: Performance comparison on Video Reasoning tasks of Qwen2.5-VI-32BB The best results among models of the same size are indicated in bold.
VCR
Video Holmes
Video MathQA
CG-AV Counting
Models
Thinking
MINERVA
Average
Bench
Qwen2.5-VL.
x 39.85
38.15
43.28
33.33
23.95
35.71
ReWatch-RR RI RI-SFT-32B
V
40.81
37.52
43.11
34.29
25.12
36.17
ReWatch-R1-32
/
44.68
38.08
45.56
36.43
23.56
37.66
+ O&&
45.55
38.35
45.78
37.62
23.08
38.08
CONCLUSION
In this work, we address the critical data bottleneck in complex video reasoning by introducing Re Watch, large-scale dataset synthesized via a novel multi-stage agentic pipeline that generates temporally-dense captions, challenging multi-hop questions, and andeo-grounded Chain-of-Thhughe traces. We then develop ReWatch- by post-training strong LVLM using an SFT and RLVR framework, featuring our innovative Observation & Reasoning (O&R) reward that uniquely evaluates both the correctness of the final answer and the factual grounding of the reasoning process itself. The resulting model establishes a new tfate-of-the-an on five challenging video reasoning benchmarks. This demonstrates that our integrated approach of superior data synthesis and process reinforcement learning provides a robust and effective paradigm for complex temporal reasoning in LVLMs. In future work, plan extend framework to thinking-with-videe paradigm, where the model and agents directly interact with visual encoders to retrieve information, further closing the gap between textual reasoning and visual perception.
process-oriented oriented
ETHICS STATEMENT
The videos used to construct the ReWatch dataset sourced exclusively from publicly available academic datasets [24] 19 32 16 59), which are intended for research purposes. We do not collect any new data involving human subjects, and therefore, no Institutional Review Board (IRB) approval is required. We do not attempt to re-identify any individuals who may appear in these public videos.
REPRODUCIBILITY STATEMENT
We are committed to ensuring the reproducibility our research. To this end, we provide comprehensive description of methodology, data, and experimental setup.
10

---

Published as a conference paper at ICLR 2026
The source code for our data synthesis pipeline, the Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) training procedures, and our evaluation scripts will be released upon publication.
Code:
Dataset Construction: Our primary contribution, the ReWatch dataset, is synthesized using a novel pipeline. The complete methodology for this pipeline, including the multi-stage process for captioning. QA generation, and CoT synthesis, described in detail in Section 2 and illustrated in Figure 3 The specific foundation models used each stage of the synthesis process are explicitly listed in Appendix B3.2
Experimental Setup and Hyperparameters: All experimental details required to reproduce our results are provided in the appendix. Appendix C.I contains complete breakdown of the training parameters for both the SFT and RL stages, including learning rates, batch sizes, context lengths, and the specific models used for reward calculation.
Our evaluation protocol clearly defined to ensure fair and consistent comparison. We detail the benchmarks used in Appendix [ the exact prompts used to elicit "thinking" and "non- thinking" responses from all models in Appendix and E.2 and the prompt for our GPT-4.I-based answer judging in Appendix E.3
Evaluation:
REFERENCES
[1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, 1lge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023.
[2] Anurag Arnab, Ahmet Iscen, Mathilde Caron, Alireza Fathi, and Cordelia Schmid. Tem- poral chain of thought: Long-video understanding by thinking in frames. arXiv preprint arXiv:2507.0200 2025.
[3] Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang. Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-v) technical report. arXiv preprint arXiv:2502.1392: 2025.
[4] Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang. Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025.
[5] Guo Chen, Zhiqi Li, Shihao Wang, Jindong Jiang, Yicheng Liu, Lidong Lu, De-An Huang, Wonmin Byeon, Matthieu Le, Tuomas Rintamaki, al. Eagle 2.5: Boosting long-context post-training for frontier vision- language models. arXiv preprint arXiv:2504. 15271, 2025.
[6] Jr-Jen Chen, Yu-Chien Liao, Hsi-Che Lin, Yu-Chu Yu, Yen-Chun Chen, and Frank Wang. Rex- time: A benchmark suite for reasoning- Processing Systems, 22222666222677, 2024.
ceasoning-across-teee across-time in videos. Advances in Neural Information
[7] Xinlong Chen, Yuanxing Zhang, Yushuo Guan, Bohan Zeng, Yang Shi, Sihan Yang, Pengfei Wan, Qiang Liu, Liang Wang, and Tieniu Tan. Versavid-rl: A versatile video understanding and reasoming model from question answering to captioning tasks. arXiv preprint arXiv:2506.09079 2025.
[8] Yi Chen, Yuying Ge, Rui Wang, Yixiao Ge, Junhao Cheng, Ying Shan, and Xihui Liu. Grpo- Consistency-awant reinforcement learning for multimodal reasoning. arXiv preprint arXiv:2506.1614: 2025.
[9] Yi Chen, Yuying Ge, Rui Wang, Yixiao Ge, Lu Qiu, Ying Shan, and Xihui Liu. Exploring the effect of reinforcement learning on video understanding: Insights from seed-bench-rl. arXiv preprint arXiv:2503.245: 24376, 2025.
[10] Yukang Chen, Wei Huang, Baifeng Shi, Qinghao Hu, Hanrong Ye, Ligeng Zhu, Zhijian Liu, Pavlo Molchanov, Jan Kautz, Xiaojuan Qi, et al. Scaling rl to long videos. arXiv preprint arXiv:2507.07966 2025.
11

---

Published as a conference paper at ICLR 2026
[11] Junhao Cheng, Yuying Ge, Teng Wang, Yixiao Ge, Jing Liao, and Ying Shan. Video-holmes: Can mllm think like holmes for complex video reasoning? arXiv preprint arXiv:2505.21374 2025.
[12] Xu Chu, Xinrong Chen, Guanyu Wang, Zhijie Tan, Kui Huang, Wenyu Lv, Tong Mo, and Weiping Li. Qwen look again: Guiding rision-languaa reasoning models to re-attention visual information. arXiv preprint arXiv:2505.23558 2025.
[13] Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality. long context, and next generation agentic capabilities. arXiv preprint arXiv:2507 266666,577.0261, 2025.
[14] Jisheng Dang, Jingze Wu, Teng Wang. Xuanhui Lin, Nannan Zhu, Hongbo Chen, Wei-Shi Zheng, Meng Wang, and Tat-Seng Chua. Reinforcing video reasoning with focused thinking. arXiv preprint arXiv:2505 20000022711.... 18, 2025.
[15] Yue Fan, Xiaojian Ma, Rujie Wu, Yuntao Du, Jiaqi Li, Zhi Gao, and Qing Li. Videoagent: memory-augmentedd multimodal agent for video understanding. In European Conference on Computer Vision, pp. 75-92. Springer, 2024.
[16] Kaituo Feng, Kaixiong Gong. Bohao Li, Zonghao Guo, Yibing Wang. Tianshuo Peng, Junfei Wu, Xiaoying Zhang, Benyou Wang. and Xiangyu Yue. Video-rl: Reinforcing video reasoning in milms. arXiv preprint WrXiv:2503.21776 2025.
[17] Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang. Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal lims in video analysis. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 24108-24118 2025.
[18] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint arXiv:2501. 12948, 2025.
[19] Songhao Han, Wei Huang, Hairong Shi, Le Zhuo, Xiu Su, Shifeng Zhang, Xu Zhou, Xiaojuan Qi, Yue Liao, and Si Liu. Videoespresso: A large-scale chain-of of-tthught dataset for fine-grained video reasoning via core frame selection. In Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR), pp. 26181-228222 1619112 June 2025.
[20] Jian Hu, Zixu Cheng, Chenyang Si, Wei Li, and Shaogang Gong. Cos: Chain-of-shot prompting for long video understanding, arXiv preprint arXiv:2502.06428, 2025.
[21] Kairui Hu, Penghao Wu, Fanyi Pu, Wang Xiao, Yuanhan Zhang, Xiang Yue, Bo Li, and Ziwei Liu. Video-mmmu: Evaluating knowledge acquisition from multi-discipline professional videos. arXiv preprint arXiv:2501. 13826, 2025.
[22] Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. Vision-rl: Incentivizing reasoning capability in multimodal large language models. arXiv preprint arXiv:2503.06749 2025.
[23] Pu Jian, Junhong Wu, Wei Sun, Chen Wang. Shuo Ren, and Jiajun Zhang. Look again, think slowly: Enhancing visual reflection in vision-language models. arXiv preprint 2212:22509.11322 2025.
[24] Xuan Ju, Yiming Gao, Zhaoyang Zhang, Ziyang Yuan, Xintao Wang. Ailing Zeng, Yu Xiong, Qiang Xu, and Ying Shan. Miradata: A large-scale video dataset with long durations and structured captions. Advances in Neural Information Processing Systems, 37:48955-48970. 2024.
[25] Somnath Kumar, Yash Gadhia, Tanuja Ganu, and Akshay Nambi. Mmetagent: Multi-modal crit- ical thinking agent framework for complex visual reasoning. arXiv preprint N1Vv:24005.1355. 2024.
12

---

Published as a conference paper at ICLR 2026
[26] Yilong Lai, Jialong Wu, Zhenglin Wang, and Deyu Zhou. Adarewriter: Unleashing the power of prompting-basee conversational query reformulation via test-time adaptation. arXiv preprint arXiv:2506.01381 013881 2025.
[27] Yilong Lai, Jialong Wu, Congzhi Zhang, Haowen Sun, and Deyu Zhou. Adacqr: Enhancing query reformulation for conversational search via sparse and dense retrieval aligmment. In Proceedings of the 31 international conference on computational linguistics, pp. 7698-7720,
2025.
[28] Yilong Lai, Yipin Yang, Jialong Wu, Fengran Mo, Zhenglin Wang, Ting Liang, Jianguo Lin, and Keping Yang. Crmweaver: Building powerful business agent via agentic rl and shared memories. arXiv preprint arXiv:2510,25333, 2025.
[29] Dacun Lee, Jachong Yoon, Jaemin Cho, and Mohit Bansal. Video-skill-cot: Skill-based chain- of-thoughts for domain-adaptive video reasoning. arXiv preprint arXiv:2506.03525 2025.
[30] Xinhao Li, Ziang Yan, Desen Meng. Lu Dong, Xiangyu Zeng, Yinan He, Yali Wang, Yu Qiao, Yi Wang, and Limin Wang. Videochat-r1: Enhancing spatio-temporal perception via reinforce- ment fine-tuning. arXiv preprint arXiv:2504.0665 066588 2025.
[31] Zongxia Li, Wenhao Yu, Chengsong Huang. Rui Liu, Zhenwen Liang, Fuxiao Liu, Jingxi Che, Dian Yu, Jordan Boyd-Graber, Haitao Mi, al. Self-rewarding vision-language model via reasoning decomposition. arXiv preprint arXiv:2508. 1tt.::550..1655555 19652, 2025.
[32] Jingyang Lin, Jialian Wu, Ximeng Sun, Ze Wang. Jiang Liu, Yusheng Su, Xiaodong Yu, Hao Chen, Jiebo Luo, Zicheng Liu, et al. Unleashing hour-scale video training for long video- language understanding. arXiv preprint arXiv:2506.05332, 2025.
[33] Weihuang Lin, Yiwei Ma, Jiayi Ji, Xiaoshuai Sun, and Rongrong Ji. Cir-cot: Towards inter- pretable composed image retrieval via end-to-end chain-of-thouu of-tohugttttht reasoning. arXiv preprint arXiv:2510.08003 2025.
[34] Weihuang Lin, Yiwei Ma, Xiaoshuai Sun, Shuting He, Jiayi Ji, Liujuan Cao, and Rongrong Ji. Hrseg: High-resolution visual perception and enhancement for reasoning segmentation. In Proceedings of the 33rd ACM International Conference Multimedia, pp. 3202 2211,,11,111 2025.
[35] Ye Liu, Kevin Qinghong Lin, Chang Wen Chen, and Mike Zheng Shou. Videomind: A chain-of-lora agent for long video reasoning. arXiv preprint ariiv:2503.134 2025.
[36] Lidong Lu, Guo Chen, Zhiqi Li, Yicheng Liu, and Tong Lu. Av-reasoner: Improving and bench- marking clue-groud -reuudeo audio-visual counting for milms. arXiv preprint arXiv:2506.05328, 2025.
[37] Juhong Min, Shyamal Buch, Arsha Nagrani, Minsu Cho, and Cordelia Schmid. Morevqa: Exploring modular reasoning models for video question answering. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 13235-13245 2024.
[38] Arsha Nagrani, Sachit Menon, Ahmet Iscen, Shyamal Buch, Ramin Mehran, Nilpa Jha, Anja Hauth, Yukun Zhu, Carl Vondrick, Mikhail Sirotenko, et al. Minerva: Evaluating complex video reasoning. arXiv preprint (tl::2505.0068) 2025.
[39] Jinyoung Park, Jeehye Na, Jinyoung Kim, and Hyunwoo J Kim. Deepvideo-rl: Video rein- forcement fine-tuning via difficulty- awware aware regressive grpo. arXiv preprint Miii:25506007666 2025.
[40] Yukun Qi, Yiming Zhao, Yu Zeng. Xikun Bao, Wenxuan Huang, Lin Chen, Zehui Chen, Jie Zhao, Zhongang Qi, and Feng Zhao. Ver-bench: A comprehensive evaluation framework for video chain-of-thouge reasoning. arXiy preprint arXiv:2504.07956. 2025.
[41] Hanoona Rasheed, Abdelrahman Shaker, Anqi Tang, Muhammad Maaz, Ming- Ming-Hsuan Hsuan Yang, Salman Khan, and Fahad Shahbaz Khan. Videomathga: Benchmarking mathematical reasoning via multimodal understanding in videos. arXiv preprint 25335.2506.05349 2025.
13

---

Published as a conference paper at ICLR 2026
[42] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint a:Xiv:2402.03300, 2024.
[43] Yudi Shi, Shangzhe Di, Qirui Chen, and Weidi Xie. Enhancing video-llm reasoning via agent-of-thoughts distillation. arXiv preprint 21w:: 2024.
[44] V Team, Wenyi Hong. Wenmeng Yu, Xiaotao Gu, Guo Wang, Guobing Gan, Haomiao Tang, Jiale Cheng. Ji Qi. Junhui Ji, Lihang Pan, Shuaiqi Duan, Weihan Wang. Yan Wang, Yean Cheng, Zehai He, Zhe Su, Zhen Yang, Ziyang Pan, Aohan Zeng, Baoxu Wang, Bin Chen, Boyan Shi, Changyu Pang. Chenhui Zhang. Da Yin, Fan Yang. Guoqing Chen, Jiazheng Xu, Jiale Zhu, Jiali Chen, Jing Chen, Jinhao Chen, Jinghao Lin, Jinjiang Wang. Junjie Chen, Leqi Lei, Letian Gong, Leyi Pan. Mingdao Liu, Mingde Xu, Mingzhi Zhang, Qinkai Zheng, Sheng Yang, Shi Zhong, Shiyu Huang, Shuyuan Zhao, Siyan Xue, Shangqin Tu, Shengbiao Meng, Tianshu Zhang, Tianwei Luo, Tianxiang Hao, Tianyu Tong, Wenkai Li, Wei Jia, Xiao Liu, Xiaohan Zhang, Xin Lyu, Xinyue Fan, Xuancheng Huang. Yanling Wang, Yadong Xue, Yanfeng Wang, Yanzi Wang, Yifan An, Yifan Du, Yiming Shi, Yiheng Huang, Yilin Niu, Yuan Wang, Yuanchang Yue, Yuchen Li, Yutao Zhang, Yuting Wang, Yu Wang, Yuxuan Zhang, Zhao Xue, Zhenyu Hou, Zhengxiao Du, Zihan Wang, Peng Zhang, Debing Liu, Bin Xu, Juanzi Li, Minlie Huang, Yuxiao Dong, and Jie Tang. Glm-4.5v and glm-4. glm-4.1v-thine I--thinking: Towards versatile multimodal reasoning with scalable reinforcement learning, 2025. URL https: http://.rggg/bb//////o 01006
[45] Qi Wang, Yanrui Yu, Ye Yuan, Rui Mao, and Tianfei Zhou. Videorft: Incentivizing video reasoning capability in milms via reinforced fine-tuning. arXiv preprint arXiv:2505.12434 2025.
[46] Weihan Wang. Zehai He, Wenyi Hong, Yean Cheng, Xiaohan Zhang, Ji Qi, Xiaotao Gu, Shiyu Huang, Bin Xu, Yuxiao Dong, al. Lvbench: An extreme long video understanding benchmark. arXiv preprint wrXiv:2406.08035. 2024.
[47] Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing. Shenglong Ye, Jie Shao, et al. Internv13.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv preprint arXiv:2508. 18265, 2025.
[48] Xiaohan Wang, Yuhui Zhang, Orr Zohar, and Serena Yeung-Levy. Videoagent: Long-form video understanding with large language model as agent. In European Conference on Computer Vision, pp. 58-76. Springer, 2024.
[49] Yan Wang, Yawen Zeng, Jingsheng Zheng, Xiaofen Xing, Jin Xu, and Xiangmin Xu. Videocot: video chain-of-thougee dataset with active annotation tool. arXiv preprint arXiv:2407.05355, 2024.
[50] Zikang Wang. Boyu Chen, Zhengrong Yue, Yi Wang. Yu Qiao, Limin Wang, and Yali Wang. Videochat-al: Thinking with long videos by chain-of-shot reasoning. arXiv preprint arXiv:2506.06097, 2025.
[51] Lai Wei, Yuting Li, Kaipeng Zheng, Chen Wang, Yue Wang, Linghe Kong, Lichao Sun, and Weiran Huang. Advancing multimodal reasoning via reinforcement learning with cold start. arXiv preprint arXiv:2505.222. 223344, 2025.
[52] Yana Wei, Liang Zhao, Jianjian Sun, Kangheng Lin, Jisheng Yin, Jingcheng Hu, Yinmin Zhang, En Yu, Haoran Lv, Zejia Weng, al. Open vision reasoner: Transferring linguistic cognitive behavior for visual reasoning. arxiv preprint arXiv:2507..0255. 2025.
[53] Yongliang Wu, Xinting Hu, Yuyang Sun, Yizhou Zhou, Wenbo Zhu, Fengyun Rao, Bernt Schiele, and Xu Yang. Number it: Temporal grounding videos like flipping manga. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 13765, 2025.
[54] Yongliang Wu, Zonghui Li, Xinting Hu, Xinyu Ye, Xianfang Zeng, Gang Yu, Wenbo Zhu, Bernt Schiele, Ming- Hsuan Yang, and Xu Yang. Kris-bench: Benchmarking next-level intelligent image editing models. arXiv preprint 1iX::2505.16707 2025.
14

---

Published as a conference paper at ICLR 2026
[55] Yongliang Wu, Shiji Zhou, Mingzhuo Yang, Lianzhe Wang, Heng Chang, Wenbo Zhu, Xinting Hu, Xiao Zhou, and Xu Yang. Unlearning concepts in diffusion model via concept domain cor- rection and concept preserving gradient. In Proceedings of the AAAI Conference Artificial Intelligence, volume 39, pp. 8496-8504, 2025.
[56] Yongliang Wu, Yizhou Zhou, Zhou Ziheng, Yingzhe Peng. Xinyu Ye, Xinting Hu, Wenbo Zhu, Lu Qi, Ming-Hsuan Yang, and Xu Yang. On the generalization of sft: reinforcement learning perspective with reward rectification. arXiv preprint arXiv:2508.0566 2025.
[57] Yongliang Wu, Wenbo Zhu, Jiawang Cao, Yi Lu, Bozheng Li, Weiheng Chi, Zihan Qiu, Lirian Su, Haolin Zheng, Jay Wu, et al. Video repurposing from user generated content: large-scale dataset and benchmark. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 8487-8495, 2025.
[58] An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.0935 2025.
[59] Dongjie Yang. Suyuan Huang, Chengqiang Lu, Xiaodong Han, Haoxin Zhang, Yan Gao, Yao Hu, and Hai Zhao. Vript: A video is worth thousands of words. Advances in Neural Information Processing Systems, 37:57240-574
2024.
[60] Zeyuan Yang. Delin Chen, Xueyang Yu, Maohao Shen, and Chuang Gan. Vea: Video curious agent for long video understanding. arXiv preprint arXiv:2412. 10471, 2024.
[61] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023.
[62] Huaying Yuan, Zheng Liu, Junjie Zhou, Hongjin Qian, Yan Shu, Nicu Sebe, Ji-Rong Wen, and Zhicheng Dou. Think with videos for agentic long-video understanding, 2025. URL
https://arxiv. hwtp:/wwworggbb////// 2506. 10821
[63] Huaying Yuan, Zheng Liu, Junjie Zhou, Hongjin Qian, Ji-Rong Wen, and Zhicheng Dou. Videodeepresearch: Long video understanding with agentic tool using. arXiv preprint (tl::250610822)) 2025.
[64] Xiangyu Zeng, Kunchang Li, Chenting Wang, Xinhao Li, Tianxiang Jiang, Ziang Yan, Songze Li, Yansong Shi, Zhengrong Yue, Yi Wang, et al. Timesuite: Improving milms for long video understanding via grounded tuning. arXiv preprint arXiv:2410.199 19702, 2024.
[65] Congzhi Zhang, Jiawei Peng. Zhenglin Wang, Yilong Lai, Haowen Sun, Heng Chang, Fei Ma, and Weijiang Yu. Vrest: Enhancing reasoning large vision-language models through tree search and self-reward mechanism. arXiv preprint mtXiv:2506.0866 2025.
[66] Congzhi Zhang, Linhai Zhang, Jialong Wu, Yulan He, and Deyu Zhou Causal prompting: Debiasing large language model prompting based on front-door adjustment. In Proceedings of the AAAI Conference Artificial Intelligence, volume 39, pp. 25842-25850, 2025.
[67] Haoji Zhang, Xin Gu, Jiawen Li, Chixiang Ma. Sule Bai, Chubin Zhang, Bowen Zhang, Zhichao Zhou, Dongliang He, and Yansong Tang. Thinking with videos: Multimodal tool-augmented reinforcement learning for long video reasoning. arxiv preprint arXiv:2508.044: 2025.
[68] Shuyi Zhang, Xiaoshuai Hao, Yingbo Tang, Lingfeng Zhang, Pengwei Wang, Zhongyuan Wang, Hongxuan Ma, and Shanghang Zhang. Video-cot: A comprehensive dataset for spatiotemporal understanding of videos based on chain-of-thought. arXiv preprint arXiv:2506.08811 2025.
[69] Xiaoyi Zhang, Zhaoyang Jia, Zongyu Guo, Jiahao Li, Bin Li, Houqiang Li, and Yan Lu. Deep video discovery: Agentic search with tool use for long-fon form video understanding. arXiv preprint arXiv:2505.1880 18079, 2025.
[70] Yongheng Zhang, Xu Liu, Ruihan Tao, Qiguang Chen, Hao Fei, Wanxiang Che, and Libo Qin. Vitcot: Video-text interleaved chain-of tfthought for boosting video understanding in large language models. arXiv preprint arXiv:2507.0988 2025.
15

---

Published as a conference paper at ICLR 2026
[71] Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Ziwei Liu, and Chunyuan Li. Llava- video: Video instruction tuning with synthetic data. Transactions on Machine Learning Research, 2025.
[72] Yilun Zhao, Haowei Zhang, Lujing Xie, Tongyan Hu, Guo Gan, Yitao Long, Zhiyuan Hu, Weiyuan Chen, Chuhan Li, Zhijian Xu, et al. Mmvu: Measuring expert-level multi- discipline video understanding. In Proceedings the Computer Vision and Pattern Recognition Conference, pp. 8475-8489, 2025.
[73] Ziwei Zheng, Michael Yang, Jack Hong, Chenxiao Zhao, Guohai Xu, Le Yang, Chao Shen, and Xing Yu. Deepeyes: Incentivizing" thinking with images" via reinforcement learning. arXiv preprint arXiv:2505. 14362, 2025.
16

---

Published as a conference paper at ICLR 2026
A LLM USAGE STATEMENT
We disclose that Google's Gemini 2.5 Pro is utilized during the preparation of this manuscript. Its role was strictly limited to that general-purposs writing assistance tool. Specifically, the LLM is employed for tasks such translating initial drafts and polishing the English text to improve grammar, clarity, and conciseness. All core research ideas, including the conceptualization of the ReWatch dataset, the design of the state synthesis pipeline, the development of the O&R reward mechanism, the experimental setup, and the analysis of the results, are conceived and executed entirely by the human authors. The LLM does not contribute to the intellectual content the scientific contributions of this paper and is therefore not considered a contributor.
B DETAILS OF DATASET CONSTRUCTION
B.1 DATASET STATISTIC
Table 3 and Figure provide detailed statistical and distribution information of our dataset. Table defines the 10 types of questions that we have manually defined.
Table 3: Statistics of our dataset.
Distribution of Video Duration
Statistic
Number
Total Videos
10994
of
Video Source
MiraData
1748 (15.9%)
16
VideeEspresso
1977 (18.0%)
VidcoMarathon
3296 (30.0%)
Video-R1
1982 (18.0%)
20.g
304g
Vript
1991 (18.1%)
Duration (Mnutes)
Video Duration
Short (< min)
3970
Medium 20 min)
5473
Distribution of Question and CoT Lengths
Long (20 60 min)
1551
Caption Token (avg/max)
4375.2/68279
Number Samples
12.5k
GoT Takera
Summary Token (avg/max)
504.8/16370
2.5k
Total Questions
170944
Dimensions
Event Localization
21121 (12.4%)
2.5k
Temporal Localization
17765 (10.4%)
Counting
18756 (11.0%)
Cause and Effecc
16296 (9.5%)
Reading
14480 (8.5%)
Spatial Perception
16425 (9.6%)
Object Recognition
18342 (10.7%)
Distribution Reasoning Steps
State Changes
15184 (8.9%)
Numerical Reasoning
19260 (113))
Number of Samples
Counterfactual Reasoning
13315 (7.8%)
Types
Multiple-hhoice
85833 (50.2%)
208
Open-ended
8511 (49.8% 8%)
Question Token (avg/max)
70.6/256
Answer Token (avg/max)
6.2/256
Numbor <action>
Total Chain of Thought
135400
Reasoning Steps (avg/max)
2.3/11
Reasoning Token (avg/max)
332.5/2045
Figure 8: Distribution of our dataset.
B.2 MODEL SETTINGS FOR DATA SYNTHESIS
When synthesizing Re ReWatch-Caption, the Semantic Segmentation model Masy and the Detailed Description Generation model Meup are all Gemini2.5-Flash -hinknink) [3
When synthesizing Re ReWatch-QA, the Summary Generation model Mssu is Gemini2.5-Flash-L (Non-Thee Thinking) [3. The Contrastive QA Generation model Mmg is Gemini2.5 5-Fassh (Thinking) [ The Answer Verification model Muerity GPT4.1 11. The L.LMs set M M for Text Bias Elimination
17

---

Published as a conference paper at ICLR 2026
and Summary Bias Elimination includes Bwen3-233B-A22BB Instruct [58] and Owen2.5-VL-722. Instruct [4]. Threshold ftezt and Suum are equal to 1. The rewritten model Mewwie for multiple-choice questions is Gemini2.5- F-aahhhhh (Non- (Non-Thinking).
When synthesizing ReWatch-CoT, Reasoner model Ar is Gemini2.5 (Thinking) 13). and Observer model Ao is 1PT4.1 []] The model Meonvni used for converting structured trajectories is Gemini2.5-Flashh NNn-TTinninng
C DETAILED EXPERIMENTS
C.I EXPERIMENTAL SETUP
Benchmarks We evaluate the model on five video reasoning benchmarks (VCR Bench [40], MIN-
ERVA (38), Video Holmes []]. Video MathQA [4]] CG-AV Counting (36)) and four video general
understanding benchmarks (MMVU [72]. LVBench [46), VideoMME [17], VideoMMMU (21).
We classify these benchmarks based on the task definitions provided in their original papers and the performance characteristics of base models.
Reasoning Benchmarks: Focus on complex, multi-step temporal logic, such as: Causality & Counterfactuals (e.g.. VCR-Bench), State Tracking & Counting (e.g., CG-AV Counting), Information Retrieval across long contexts (e.g., Video Holmes), Characteristic; Base models typically exhibit low accuracy (often <30%), indicating failure of logic rather than just perception.
Cennera/Unnerstanddin Benchmarks: Focus on broad capabilities, primarily: Holistic summa- rization (e.g., VideoMME Synopsis), Entity Recognition & Attribute Perception (e.g., MMVU), Characteristic: Base models already perform relatively well, relying on pattern matching and semantic recognition.
Training Dataset Configuration Our primary model, ReWatch-R1, is derived from Qwen2.5-VL. LL- 7B- 4 via a two-stage training pipeline. First, we create an intermediate model. Re Watch- RI-SFT, by performing SFT using a mixture of three datasets: Re Retach-CCaption Caption, Re ReWatch-QAA and Re Watch-Coll Subsequently, ReWthhh-RI-SFT is further refined using RL to produce ReWatch-R1. The RL phase leverages total of 40k QA pairs, which are randomly sampled from Re ReWatch-CL (20k), Video-RI -QA [6] (10k), and Long VideoReason- QQA [10 (10k).
Training Parameter Configuration In the SFT stage, the length of the model context is 16k. The default fps 2.0, with a maximum sampling of 192 frames, and the maximum resolution of each frame is 128*28*22 The train batch size (per device) to be I and the gradient cumulative to be 4. The learning rate is 1c-6, max grad norm 1.0, and the optimizer is AdamW. The number of epochs is 10. 16 H800 Gpus used. In the RL stage, the length of the model context is 16k. The default fps is 2.0, with a maximum sampling of 192 frames. The maximum resolution of each frame is 228*28*20
The number of rollouts is 8. The sampling temperature is 0.8 and top_p is 0.9. Both train ciii_batch_size and ppo_mini het_mminnbbachhh__. batch_size are 14. ppo_micro hto__mccoo_bathhhseeeee_ batch_size size__eee_ee____eeeee_e_eeeee Bace__e___pp__ is 1. The learning rate is le-5, max_grad Lor-_grrd_oorr _orr 5.0, and the optimizer is Adam W. The number of epoch is 1. 16 H800 Gpus are used. In the reward mechanism of reinforcement learning, we use Qwen3-30B-A3B -3B-B---3--nstruu Instruct 588 inference model Minfer and judge model Miudge-
Baselines We compare the performance with that of the most advanced video reasoning models in the current literature, including Qwen2.5. D0wen2.5-VL-BB B3, GLM4. IV-9B [44, Intern VL3.5-8B (7). Video- R1 [16], Video-Char Chooo-ChaRRRR R1 [30]. VideoRFT
45 Versa Vid-R1 7Z) TW-GRPO 14) GRPO-CARE []. In addition, We also use two open-source datasets, Video-RI-CoT 16 and Long RideoReason- CoT [00, to reproduce Video-RI RI-TT and Long VideoReason-SFF under the same training configu- ration of BeWatch-RI-SPF The RL stage for Video-RI -I-L and Vongoioooo RL utilizes an identical dataset of 40k QA pairs with ReWatch-R1.
Evaluation We employ GPT-4.1 || to assess if model responses align with ground truth using Prompt20 with accuracy the metric for all benchmarks. During inference, the maximum resolution for each frame limited to 228*2**28 pixels, and the maximum number of frames is 192 or 384. Greedy decoding is used for V Video-R1, Video-Chat-RI, VideoRFT, Video-RI-SFT,
18

---

Published as a conference paper at ICLR 2026
Table 4: Performance comparison Video Understanding tasks. indicates that reproduced the model using training configuration with 192 frames. indicates that reinforcement learning is conducted using exactly the same data as ReWatch-R1. The best results among models of the same size are indicated in bold.
Models
Thinking
MMVU
LVBench
VideoMME
VideoMMMU
Average
192 Frames
Qwen2.5-VL-32B
62.30
43.83
68.52
61.56
59.05
Qwen2.5-VIL-7B
53.10
41.19
63.59
49.67
51.89
Qwen2.5-VL-7B
52.20
36.93
58.19
50.78
49.53
GLM4.
57.90
40.99
61.81
$4.67
53.84
VL3.55
50.70
36.86
61.19
55.00
50.94
Vidco-R1
53.20
40.28
64.41
50.33
52.06
Vidco-Chat-R
50.70
37.83
60.07
46.44
48.76
VideoRFT
55.30
42.48
64.81
49.89
53.12
VersaVid-R1
52.90
40.15
61.67
45.11
49.96
TW-GRPO
43.40
41.96
64.48
49.56
49.85
GRPO-CARE
55.50
36.67
63.93
52.56
52.17
Video-RI-SC R-SFT"
53.50
37.31
58.59
47.67
49.27
Video-R1-RI
55.40
37.64
63.89
50.00
51.73
Long LongVideoRcasoon-9
37.90
35.96
55.67
45.56
43.77
Long-VideoRaaso--R...
57.20
41.12
61.59
$1.00
52.73
ReWatch-RI RI-SFT
53.40
41.58
62,41
46.33
50.93
ReWatch-R1
55.80
42.74
64.96
52.22
53.93
+ O&R
57.80
42.54
64.93
51.33
54.15
384 Frames
Owen2.5-VL-32B
62.20
46.22
68.89
60.44
59.44
Qwen2.5-VL-7B
53.70
42.80
64.19
48.11
52.20
Owen2.5-VL-7B
51.33
36.22
57.50
48.33
48.35
GLM4. V-99B
57.60
44.35
66.44
57.33
56.43
Intern VL3.5-8
48.20
38.02
56,41
45.89
47.13
Video-R1
52.90
40.61
64.19
49.11
51.70
Iido--Chtt-R]
50.90
37.38
59,52
45.67
48.37
VideoRFT
55.30
40.74
64.15
48.67
52.22
VersaVid-Ri
52.00
40.67
62.85
44.33
49.96
TW-GRPO
42.80
42.74
65.41
50.89
50.46
GRPO-CARE
55.00
37.06
65.52
$2.00
52.40
Video-R1-SED
53.90
38.02
59.96
48.44
50.08
Video-RI-RD -1
55.40
38.35
65.41
$1.67
52.71
Long LongVidcoeaason-SS SFF'
38.10
36.54
57.33
47.67
44.91
Long LongVideoRcuson-RI..
56.60
41.19
62.56
51.56
52.98
ReWatch-RI-SFT ReWatch-R1
54.80
42.22
62.22
48,22
51.87
54.90
42.87
64.48
51.22
53.37
+ O&R
57.70
43.25
65.56
$1.89
$4.60
Video-RI-RL LongVideoReaaon-SFTT Long LongViidoReasoo-RLL. ReWatch-RI-SFT, and ReWatch-RI. The decoding temperature is set to 0.8 for GLM4.IV-9B and 0.6 for LL3e5nV33.5-88 Models utilize different prompts in "Thinking" and "Non-Thinkii modes, as detailed in the Appendix E.1
C.2 PERFORMANCE COMPARISON VIDEO UNDERSTANDING BENCHMARKS
Table presents a comparative analysis of the performance our model against other models on video understanding benchmarks. The key experimental findings and insights as follows.
19

---

Published as a conference paper at ICLR 2026
Table Performance comparison Video Understanding tasks of Qwen2.5-VL-32B. The best results among models of the same size are indicated in bold.
Models
Thinking
MMVU
LVBench
VideoMME
VideoMMMU
Average
Qwen2.5-VI-32B
X
62.30
43.83
68.52
61.56
59.05
ReWatch-RI-SFT
/
59.90
43.51
66.26
57.44
56.78
ReWatch-R1
/
62.60
45.97
69.33
62.44
60.09
+O&R
/
62.40
46.68
69.44
62.89
60.35
Synergistic Improvement in Reasoning and Understanding Without Catastrophic Forgetting. ReWatch-R1 achieves state-of-the-art (SOTA) performance among models of a comparable size, with an average score of 54.15% at 192 frames across four general video understanding benchmarks. This demonstrates that specialized training for complex reasoning does not impair the model's foundational abilities. On the contrary. it enhances general understanding by facilitating a more profound analysis video content. This positive outcome likely attributable to the multi-task learning design implemented during the Supervised Fine- Tuming (SFT) phase. The Re ReWathhCarg Captinn task preserves the model's ss fundamental video-text alignment, while the ReWatch-QA (direct-answer (reasoning mode) tasks train distinct response pathways. Together, these tasks cultivate a comprehensively capable model rather than one with a specialized or biased skill set.
mode) and
RL-driven Alignment of "Thinking" and "Non-thinking" Performance. After SFT with Chain- oh-Thhugh,, the performance of the ReWatch-R1-St variant still lags behind the dinect-answer ("non-thinking") performance of the base model. However, with the application of RI., the resulting ReWatch-R1 model not only exhibits further performance gains on video understanding tasks but also surpasses the direct-answer performance of the base model. This indicates that the enhancements in reasoning capabilities successfully generalize to foundational understanding tasks. This finding suggests that "deep reasoning" and "shallow understanding" are not entirely discrete processes. model proficient in complex logical thought may consequently develop more reliable fundamental observation and recognition abilities.
Generalization to Larger Models. Table 5 details the performance of the Qwen2.5 VI- model on video understanding tasks. Consistent with our findings on the 7B model, the RL stage proves critical for larger models well. While the intermediate SFT model TReWatch-R1-5 experiences a slight performance regression compared to the base model 78% vs. 59.05%), the subsequent Reinforcement Learning stage effectively recovers and enhances these foundational capabilities. The final ReWatch-R1-32 model achieves a offte--f--he-ate average score of 60.35% outperforming the base model. This confirms that RLVR framework, augmented with the O&R reward, successfully balances the trade-off between complex reasoning and general understanding, ensuring robust performance across model scales.
C.3 PERFORMANCE COMPARISON ACROSS DIFFERENT VIDEO DURATIONS
Figure D presents a comparative analysis of model performance on videos of varying durations. The findings highlight two primary conclusions regarding long-video reasoning.
Superior Performance in Long- Video Reasoning, The proposed method demonstrates significant advantage in long-via oong-vieoo reasoning. ReWatch-R1 substantially outperforms all other models of comparable size on reasoning tasks for long videos (>20) min). For instance, ReWatch-RI achieves 27.46%, an absolute improvement over 3.4 percentage points compared to the next-best model, L VeseoRReSon-RL (24.03%). This result provides strong evidence for the efficacy of the overall methodology. The ReWatch dataset, with its hierarchical subtitles and contrastive QA, is specifically designed to create challenges that require reasoning across extended temporal spans. The model's success indicates that this specialized training endows with a superior ability to locate, associate, and reason with key information embedded within lengthy and often noisy video streams.
Robustness to Performance Degradation on Long Videos. An analysis of all models reveals consistent trend: performance reasoning tasks declines video duration increases. This
20

---

Published as a conference paper at ICLR 2026
ReWosh KI
Video R1-BL
LongVinsRasou.B
Quen25 VL-77
Reasoning Benchmarks
Understanding Benehmarks
Accuracy (%)
15
Accuracy (%)
69
30
25
45
20
Long
Long
Video
Video
Figure 9: Performance comparison across different video durations. Short: 0-3 minutes, Medium: 3-20 minutes, Long: over 20 minutes. We averaged the performance of the benchmarks for reasoning and understanding respectively, and all results were evaluated at 192 frames.
observation confirms that long-viden video reasoning is a pervasive and yet-umsolved challenge for current LVLMs, a phenomenon that can be described as "Long Video Tax." However, the key advantage of ReWatch-R1 lies in its more attenuated rate of performance degradation. For example, while its own performance drops from 40.38% (short videos) 27.46% (long videos), its decline is less severe relative to its high baseline. This indicates that the model not only establishes a superior starting performance but also demonstrates greater resilience when confronted with the challenges of extended durations, further substantiating the robustness of the proposed method in handling long-term temporal dependencies.
C.4 COMPARATIVE ANALYSIS OF DATASET- REASONING COMPLEXITY AND VIDEO DEPENDENCY
Figure 6a presents quantitative analysis of the reasoning characteristics elicited by the ReWatch and Video-R RT datasets. The experiment involves using the Re ReWatch- RI-SFT model to perform inference on the Re training set and the multiple-choice subset of the Video-RI training set. From the outputs for each dataset, 5,000 correctly answered samples are randomly selected for analysis. Three metrics are computed for these samples: the average number of reasoning steps (<action> tags), the average response length, and the degree of video dependency. Video dependency is specifically quantified as "Text-Only Accuracy" the accuracy the powerful Qwen2.5-Vi VLL-BB model when answering questions with only textual input and no video. The results show that the ReWatch dataset demands more profound, multi-step inference, eliciting nearly double the number of reasoning steps (3.31 vs. 1.82) and significantly longer responses (398.75 vs. 205.74 characters). Most critically, the Text-Only Accuracy for Video-RI is 68.9%, indicating that questions can often answered from textual cues alone. In stark contrast, the accuracy for the Re Watch dataset is merely 29.4%, figure close to the 25% uundom-guessing baseline. This provides compelling evidence that the dataset's s three-stage filtering mechanism is highly effective, successfully eliminating spurious shortcuts and ensuring that problems are solvable only through genuine video understanding.
C.5 PERFORMANCE ANALYSIS OF DIFFERENT TASK TYPES
To provide deeper understanding of where our method yields the most significant gains, we analyze performance across specific task types on VCR-Bench (Table and VideoMME (Table 7
On VCR-Bench, ReWatch-RI equipped with the O&R reward demonstrates exceptional proficiency tasks requiring precise evidence retrieval. Most notably, in Video Temporal Grounding, our model achieves a substantial improvement, jumping from the base model's 25.87% to 37.76%. This sharp increase validates that the "re- rrewwachhngg mechanism and O&R reward successfully teach the model to verify intermediate reasoning steps against specific video segments. We also observe strong gains in Video Temporal Counting (improving from 40.99% to 49.07%), suggesting that the model's ' ability to track state changes time is significantly enhanced.
21

---

Published as a conference paper at ICLR 2026
On VideoMME, the results illuminate the distinction between "reasoning" and "perception." ReWatch- RI excels in categories demanding logical inference, achieving its highest gains in Spatial Reasoning (rising from 71.43% 78.57%) and Temporal Reasoning (rising from 48.59% to 53.11%). This confirms that our SFT+RLVR pipeline specifically boosts the model's deductive capabilities. However, performance holistic tasks such as Information Synopsis remains unchanged (79.57% for both base and ours), and Spatial Perception sees no improvement. This indicates that while our method significantly unlocks complex reasoning potential, tasks relying purely on global video summarization or static spatial awareness remain challenge or have reached a saturation point with the current base model architecture.
Table The detailed performance of different models on the VCR- VCR-Bench dataset. Performance is presented according different task types. All the models in this table evaluated at 192 frames.
Qwen2.5- VL--BB
ReWatch RI-SFT
ReWatch-Rl R1
w/ O&R
Thinking
X
Temporal Spatial Reasoning
48.89
42.22
44.44
48.89
Video Plot Analysis
40.29
38.13
45.32
42.45
Fundamental Temporal Reasoning
49.69
46.54
49.06
48.43
Video Temporal grounding
25.87
31.47
35.66
37.76
Vidco Temporal Counting
40.99
37.27
49.69
49.07
Video Knowledge Reasoning
49.67
52.94
54.25
54.25
Overall
36.75
35.78
40.14
40.43
Table 7: The detailed performance of different models the VideoMME dataset. Performance is presented according to different task types. All the models in this table are evaluated at 192 frames.
Qwen2.5-VL-7B
ReWatch-RI-SFT
ReWatch-R1
w/ O&R
Thinking
x
Counting Problem
43.66
42.54
45.52
45.90
Information Synopsis
79.57
79.88
79.57
79.57
Object Recognition
69.21
68.08
72.03
69.77
Action Reasoning
56.84
52.63
53.33
55.09
Object Reasoning
59.69
57.93
60.79
60.35
Temporal Perception
74.55
72.73
76.36
78.18
Attribute Perception
76.13
76.13
75.68
76.58
Temporal Reasoning
48.59
51.98
55.37
53.11
Action Recognition
61.66
59.11
65.50
65.81
OCR Problems
70.50
69.78
71.22
71.94
Spatial Perception
70.37
64.81
66.67
70.37
Spatial Reasoning
71.43
73.21
78.57
78.57
Overall
63.59
62.41
64.96
64.93
C.6 CASE STUDY
To qualitatively demonstrate the superiority of ReWatch-R1, we present two case studies comparing its reasoning process with baseline models.
Mitigating Hallucinations via Active Retrieval. Figure 10 illustrates scenario where the model must determine how character monitors scene. The baseline Video-R1 1 relies on internal "thinking" driven by textual priors and common sense, incorrectly hallucinating "surveillance camera a desk lamp" simply because it aligns with common spy tropes. In contrast, ReWatch-RI leverages its agentic capability to actively query the video content. By executing a targeted retrieval action (locating the segment 00:29-00:32
correctly observes the fine-graine -neeggraned visual detail of "transparent smartphone screen" displaying live feed. This demonstrates that our "re-watching" mechanism enables the model bypass textual shortcuts and ground its reasoning in specific visual evidence.
22

---

Published as a conference paper at ICLR 2026
Enhancing Faithfulness with O&R Reward. Figure I1 highlights the critical impact of our Observation & Reasoning (O&R) reward on complex temporal tasks. In this counting problem. the model must track objects appearing over time. The model trained without the O&R reward (ReWatch-R1 w/o O&R) adopts the correct reasoning format but fails to observe the subtle addition of "brown puppy statue," " leading to an incorrect count. However, the full ReWatch-RI model, refined by the O&R reward, exhibits superior visual faithfulneess. It explicitly notes the object's appearance timestamp 00:01 and updates its count 4 accordingly. This confirms that the O&R mechanism effectively incentivizes the model to perform rigorous, evidence- basee verification, significantly reducing visual oversight.
Video-R1-7B: ethinio think about this for moment. The question is asking how the man holding the phone saw the situation on site. Let's break it down step by step. First, let's consider the options: A: Installed surveillance in others' eyes. - This seems quite extreme and unlikely, It's not something that would be easily done without lamp on
detection. B: A survelliance camera was installed site. - This is plausible option. It's common for surveillance cameras to hiddlen in everyday objects like desk lamps. C: Through television broadcast- While possible, it's not specified in the scene that there was television broadcast. It's likely than other options. phone can connect to global satellites. - This is also possible, but it's mentioned
Question: How did the man holding the phone see situation site?
A: Installed surveillance in others' eyes,
surveillance camera installed the desk lamp site.
C: Through television broadcast. His phone can connect to gi global satellites.
of stretch without any evidence. Now, let's focus on the most logical and detailed option. The scene shows desk lamp, which It's small, inconspicaous abject that could be easily hidden. The man holding the phone likely using his phone view the footage from the camera in the desk lamp. Hmm, it seems that the most reasonable The surveilance camera was installed in the
Ground Truth Answer:
place for surveiliance
ReWatch--B -1-hchh----7:: need determine how
phone
observed the situation on site. First, I'll examine his interaction with the phone, cactions look
segment 00:00 to 00:30, focusing on his phone's function. </actions cobservation- obsarve that betwsen 00:00 and 00:30, the man holding phane, but its specific function isn't detailed; it's just present, </abservations Since the phone's function wasn't clear, need find out how he saw the situstion. cactions I'll search for any indication of how viewed the scene communicated with someone else. </action> cobservations notice that from 00:29 to 00:32, the man is shown looking through a transparent smartphone screen, which clearly displays live feed of the scene. This suggests his phone being used as surveillance device. c/obsenvation This directly answers the question. Therefore,
and detailed option desk lamp on site. This aligns with the common practice of using everyday objects for surveillance purpases. Oh, I see. The most logical and detailed answer based on the scene and common practices is B. Let's verify one last time. The scene does not show any signs of global soteilite connection, so less likely. presence a desk lamp suggests B is the most plausible. Final answer: B Let's break down last time ensure. The desk lamp a common place for surveillance cameras, and the man is using his phone to view the footage. This aligns perfectly with option Bis/think>> Banswer>B-/anneee
<answersAc/answe /answer>>
Figure 10: Case Visualization. Our model can notice more fine-grained information during the reasoning process.
Video-R1-7Bt cthinlotat me think about canefully. The question asis number of objects already on the table before the book is picieed up to show the back. First, let's
identify the objects that visible the table: There a knfe with blue handle, 2. There
toy car. That's Now, consider there are any other objects might present but not immediately obvious. Himm, don't see anything else that stands out. The tabl ecloth iiseff ion't considened an abject, and the persoe's hands aren't objects aither. So, we have three Walt, let's break
There bunch
objects right there.
Question: in the video, how mony different objects are already on the table before the book is picked up to show the back?
distinct objects: knllo, the
make sure. The knife clearly there, the lettuce there, the toy there. No other abjects are mentioned or visually apparent. Oh,, sea. The question is moking for the count objects that are already on the table before the book picked up. The book the table when the question being asked, doesn't count count. Therefore, the correct answer should
A:D
8:
C:4
inttial
6:6
count the objects that alreacly
Ground Iruth Answer:
on the table, which is Let's verifly one last time. Yes, the count is indeed 3. The book is not part of the initial count because it hasn't been placed on the table yet. Final answer: Let's double check options. which incomeet. 2, which also incomeet. The correct answer indoed </thinko canswer-B- B>/aaswee>>>>
R1-8 To determine the number of iteess on the table before picking the book.1 first need determine the time when the
which incorrect C which incorrect.
book was picked up to show the back" </actiono cobsrrvtionn observed that this action began approcimately at 00:03, dotservationn New that I kiow the time frame, need to calculate the different objects that appeared will review
ocourred, castions
search
ReWatch- R1-78 w/o OBR: To determine the number of objects on the table before the book is picked I first
caction- I'l search for pinpoint when that event </action> <observat "the book picked up
table before moment. focus on the unique items placed the
segment
from 00:00 00:03, table. s/actions cobservation- observed that at the beginning (00-00), there were three objects on the table: 00:01,
observe that this happens from 00:03 to 0007 s/obeevcationn Now that know the timeframe, need to count the distinct objects present on the table during that period. cactions I'll examine the segment from 00:00 to 00:07, facusing on unique items the table. </actions cobservatione observe that there three distinct objects: green lettuce, small red toy car and a blue-handlen knife. These are clearly visible and remain stalionary throughout this time frame, c/abservations on Therefore, the total
letture, small red toy
blue handhaddddd added the table. e/observation: These four objects are still on table. Therefore, before picking up the book, the total namber of
pupay
different objects on the table was 4. C//////wwerrr
number different objects the table before the book picked three. canswers Bc//nweer/s
Figure 11: Case Visualization. Adding O&R rewards makes the reasoning process more faithful to the original video.
23

---

Published as a conference paper at ICLR 2026
D RELATED WORK
D.1 VIDEO QA DATASETS AND BENCHMARKS
growing body of video reasoning benchmarks reveals that current LVLMs struggle on complex, multi-step temporal reasoning. Recent evaluations [4] 38: 36 56
54
57
55
[
target causal attribution, temporal ordering, state tracking, counting. and cross cross-modal grounding, and consistently report large performance gaps even for strong models B 44
T6 30 45].
further underscore the challenge by emphasizing hour-scale contexts and dense event structure. Collectively, these benchmarks confirm that multi-hop, evidence-driven video reasoning remains underdeveloped in LVLMs.
Long-video understanding suites 722 46 17 21
In contrast, the available training corpora offer limited support for developing such capabilities. Large open sources provide long videos and captions but predominantly yield holistic or coarse descriptions that lack precise temporal annotations 24 19
perception-centric QA that only requires simple single-step reasoning 771 9 68 64]. Recent video-reasoning efforts augment these resources with step-by-step traces, yet their Phain-of-Thoughe (CoT) is typically distilled from text-only LLMSs and often resorts to commonsense elimination rather than verifiable, video-gu iieeo-grounded retrieval [6 45 49]. Such supervision ill-suited for Reinforcement Learning with Verifiable Reward (RLVR), which requires challenging, multi-hop questions and checkable, content-gee grrenddgrouneed processes to produce reliable reward signals [12] 23). This mismatch leaves RL methods data-starved: they can optimize answer formats and surface patterns but struggle to learn siddence-lnkee temporal reasoning [3]].
59
To close this gap. synthesize Re Watch, dataset that couples (i) temporally precise, hierarchical captions preserving event order, (ii) nighdiiffiullty QA generated by contrasting detailed captions against summaries to remove shorteuts, and (iii) multi-agent, video-grounded CoT that explicitly records retrieval and verification steps. This design aims to provide the process-level supervision and question difficulty necessary to unlock RLVR for complex video reasoning.
D.2 VIDBO REASONING IN LARGE VISION-LANGUAGE MODELS
Reinforcement Learning for video reasoning emerges as a complementary path. Recent works [6] 30 45
I0. [3) adopt RL/RFT-style training to improve reasoning, generally using final-answer accuracy as the primary reward and relying on the above training data. While promising, these pipelines inherit the limits of their supervision: weakly grounded CoT and shortcut- parouut--rone& QA. Rewards remain coarse, focusing on outcomes rather than verifying intermediate observations or the sufficiency of the reasoning process. As a result, models can overfit to answer patterns, exhibit hallucinations, and fail to align intermediate steps with evidence in the video.
Agentic methods integrate reasoning with tool use to improve grounding. Recent work extends agentic paradigms like ReAct [61] long video understanding, enabling models to dynamically interact with video during inference to produce grounded reasoning chains [63] 69 660 [5] 48) 25
37
50
70 65 [6]. However, these methods are often training-free failing internalize
such reasoning abilities within the base model. Other approaches
43 35
29; 28: 27 26
33 34
data and then train models with SFT, but they typically generate fixed tool-use trajectories from a single planning phase, lacking the iterative "think-and-ace capability. Concurrently, the "think with video" paradigm emerges (67 62). which dynamically retrieves and injects video segments into the model's 's context. This strategy, however, places excessive demands on context length and involves complex model context management and agentic RL training, severely limiting training efficiency.
use agents to synthesize video-based
Our work combines the strengths of the above lines while addressing their limitations: we couple agentic data synthesis with RLVR, and while maintaining dynamic interaction with long videos and evidence verification, we internalize efficient, grounded reasoning into the multimodal model, thereby overcoming key limitations of current video reasoning.
24

---

Published as a conference paper at ICLR 2026
E PROPMTS
E. THINKING PROMPTS
We use different prompts to activate the thinking mode of different models. The detailed Settings are follows: Qwen2.5-VL is not reasoning model, has the thinking mode enabled by default, so we use the direct QA Prompt 18 InternVL3.5 requires additional hints to activate the thinking mode,
the CoT Prompt T2 GLM4.1V itself
use the Prompt 17, Video-RI and VideoRFT
use the Prompt [4 Video-Chat-R uses the Promp 15 LongVideoRcassee uses the Prompt [6 Our
model Re Watch--RR uses the Prompt 13
Prompt for the Thinking mode of Qwen2.5-VL VL
You are a video understanding expert. You are given a video and a question. You need to answer the question based on the video content. Please provide step-by-step solution the given question. And provide the final answer in the end.
Question: (question)
Figure 12: Prompt for the Thinking mode of Qwen2.5-Vi VL.
Prompt for the Thinking mode of Re Watch-RI
You video understanding expert. You given video and question. You need to answer the question based on the video content. Please answer the question step by step. When you need more video details, you will re-watch the relevant clips and <action> and </action> to mark the actions, and use <observation> and </observation> to mark the visual details you observe. When you have enough information determine the final answer, you will wrap the final answer in <answer> and </answer>.
Video Information and Question:
Video Duration: ]
Question: (question)
Figure 13: Prompt for the Thinking mode of Re R1.
Prompt for the Thinking mode of Video-R1 and VideoRFT
{Question]
Please think about this question as you were a human pondering deeply. Engage in internal dialogue using expressions such 'let think', 'wait', 'Hmm', 'oh, see', 'let's break down', etc, or other natural language thought expressions It's encouraged include self-reflection verification the reasoning process. Provide your detailed reasoning between the <think> and </think> tags, and then give your final answer between the <answer> and </answer> tags. {Output_ {Output_Templan Template)
Output_Tenne Template:
"multiple choice": Please provide only the single option letter (e.g., A, B, C. etc.) within the <answer> </answer> tags.",
"numerical": Please provide the numerical value (e.g., 42 3.14) within the <answer> </answer> tags.".
"OCR": Please transcribe text from the image/video clearly and provide your text answer within the <answer> </answer> tags.", "free-form": Please provide your text answer within the <answer> </answer> tags.".
25

---

Published as a conference paper at ICLR 2026
"regression": Please provide the numerical value (e.g., 42 3.14) within the <answer> </answer> tags."
Figure 14: Prompt for the Thinking mode of Video-R1 and VideoRFT.
Prompt for the Thinking mode of Video-Chat-RI
{question)
Output your thought process within the <think> </think> tags, including analysis with either specific timestamps (xX.xx) or time ranges (xx.XX xx.Xx) in <timestep> </timestep> tags.
Then, provide your final answer within the <answer> </answer> tags.
Figure 15: Prompt for the Thinking mode of Video-Chat-RI.
Prompt for the Thinking mode of Long VideoReason
You are a helpful assistant. The user asks a question, and then you solves it.
Please first think deeply about the question based on the given video, and then pro- vide the final answer. The reasoning process and answer are enclosed within <think> </think> and <answer> </answer> tags, respectively, i.e., <think> reasoning process here </think> <answer> answer here </answer>.
Question: (question)
Figure 16: Prompt for the Thinking mode of Long VideeReasonn
Prompt for the Thinking mode of Intern VL3.5
You are Al assistant that rigorously follows this response protocol:
First, conduct detailed analysis of the question. Consider different angles, po- tential solutions, and reason through the problem step-by-step. Enclose this entire thinking process within <think> and </think> tags.
After the thinking section, provide clear, concise, and direct answer the user's question. Separate the answer from the think section with newline.
Ensure that the thinking process is thorough but remains focused on the query. The final answer should be standalone and not reference the thinking section.
You are given a video and question. You need to answer the question based on the video content. Please directly provide your answer.
Question: (question)
Figure 17: Prompt for the Thinking mode of Intern VL3.5.
26

---

Published as a conference paper at ICLR 2026
E.2 NON-THINKING PROMPTS
In the evaluation, all the models in this paper use the same Prompt when applying the non-thinking mode.
When training ReWatch-RI-SFF, we apply Prompt 19 Prompt 18
and Prompt 13 on datasets
@WWtch-Cappion ReWatch-QA, and ReWatch-CoT respectively.
Prompt for the Non- Non-Thinking mode
You are a video understanding expert. You are given a video and a question. You need to answer the question based the video content. Please directly provide your
Question: (question)
Figure 18: Prompt for the Non-Thinking mode of all models in this paper.
Prompt for the video-text aligmment
Analyze the provided video and generate brief, chronologically ordered set of dense descriptions. Divide the video into some meaningful segments based on its storyline. Each segment should be long possible and encompass relatively complete event scene. Each segment must be accompanied by its corresponding start and end timestamps. **mppotantll**** ensure that the timestamps for all segments are continuous and cover the entire duration (i ([duration)) )) of the video, from beginning to end.
For each segment:
Provide a precise start and end timestamp (format: MM:SS))
2. Write concise but informative description of what is happening in that segment. Focus on actions, key objects, and interactions.
Please format the output
(MM:SS-MM:SS] Description of the segment.
[MM:SS-MM:SS] Description of the next segment. (and so on, until the end of the video)
Figure 19; Prompt for the video-text alignment.
E.3 ANSWER JUDGE PROMPT
Prompt for Answer judge
You are AI assistant who will help me to judge whether the answer generated by model consistent with the standard answer.
Input Illustration:
Standard Answer is the standard answer to the question
Model Answer is the answer generated by a model to this question.
Task Illustration:
Determine whether Standard Answer and Model Answer are consistent.
Consistent Criteria: the meaning is expressed in the same way, also considered consistent.
27

---

Published as a conference paper at ICLR 2026
Output Format:
If they are consistent, output 1; if they are different, output O. DIRECTLY output or 0 without any other content.
Question: (question)
Model Answer: fextract_ang answer)
Standard Answer: (gt_answer)
Your output:
Figure 20: Prompt for Answer judge.
Table 8: Definitions of the 10 synthesized QA types.
Task Type
Definition
This task requires the LVLM output precise start and end times a specific event in the video, based on a natural language query.
Event Localization
This task provides timestamp or interval from video and requires the LVLM to describe what happened within that specific time.
Temporal Localization
This task requires the LVLM to calculate the frequency of events or actions and to perceive the number of occurrences of specific objects.
Counting
This task requires the LVLM to identify direct causal relationships between specific events in the video, meaning one event directly led to the occur- rence of another.
Cause and Effeet
This task requires the LVLM to identify temporal changes in the attributes, position, behavior, emotions of specific objects characters in the
State Changes
video.
This task requires the LVLM to identify and understand textual information appearing in the video frame (e.g.. signs, subtitles, screen displays, docu- ment content).
Reading (OCR)
This task requires the LVLM to understand the relative spatial positions, distances, and movement trajectories between objects, people, and their environment within the video.
Spatial Perception
This task requires the LVLM perform all mathematical operations other than simple counting, including speed, estimating time, calculating proportions, etc.
Numerical Reasoning
limited to comparison, calculating
This task requires the LVLM identify name specific objects, people, or animals appearing in the video.
Object Recognition
This task requires the LVL.M, given the video context, to hypothesize scenario where a certain event did not occur or occurred differently, and then infer the likely objective, verifiahle consequences. This does not involve subjective feelings or pure speculation but based on physical laws, logic, or established patterns shown the video.
Counterfactual Reasoning
28


Reviews:
[{"review_id": "gSNBwjmODT", "reviewer": "Reviewer_tQx8", "summary": "This submission presents ReWatch-R1, a framework for advancing complex video reasoning in large vision-language models (LVLMs). The core contributions include the creation of the large-scale, multi-stage ReWatch dataset aimed at challenging multi-hop video reasoning, a multi-agent data synthesis pipeline generating temporally precise captions, high-difficulty QAs, and video-grounded CoT traces, and a new Observation & Reasoning (O&R) reward mechanism for RL. By post-training an LVLM backbone with these elements through SFT and RL, the authors demonstrate state-of-the-art performance across five video reasoning benchmarks, alongside ablation studies.", "questions": "-Can the authors provide concrete ablation or error-type breakdowns specifically isolating the effect of the O&R reward on hallucination rates or logical errors, perhaps through qualitative examples or confusion matrices? It is unclear if this reward is the main driver for reduced hallucinations, or if SFT data quality dominates.\n-Given the extensive use of LLMs at nearly every phase, is there a risk that ReWatch-R1’s strong performance partly reflects an overfit to LLM-specific language or annotation artifacts in the data/reward pipeline?\n-How sensitive is the model (and pipeline) to coarse versus fine semantic segmentation in the captioning stage? For example, does segment over-segmentation harm long-term reasoning due to context fragmentation?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "-The topic of video reasoning is of community’s interest and timely.\n-The hierarchical segmentation and multi-agent CoT pipeline used to generate the ReWatch dataset is methodically crafted and shown to have yielded data with deeper temporal grounding and higher question complexity.\n-The baselines compared are pretty up-to-date, which verifies the state-of-the-art achievement on the benchmarks tested in the paper.", "weaknesses": "-The incremental benefit of the O&R reward mechanism is not fully dissected independently from other RL contributions. In Table 1, the improvement from RL (+O&R) appears relatively modest, and there is limited analysis delineating whether specific reasoning failures or hallucinations are directly ameliorated by the O&R reward in practice.\n-The paper leans heavily on LLMs (Gemini, GPT-4.1, Qwen, etc.) for practically all phases - data synthesis, answer verification, reward calculation, and benchmarking. Although this is common in the area, the cumulative propagation of LLM biases and potential “meta-overfitting” is insufficiently analyzed; for example, does repetitive LLM-based data filtering introduce subtle data shortcuts or annotation artifacts? A systematic error/robustness analysis would be welcome.\n-There is little systematic exploration of failure modes, such as where ReWatch-R1 still hallucinates, or which reasoning subtypes remain unsolved - as could be illustrated by extensive qualitative error analysis or confusion matrices per reasoning dimension (section 4.2 or appendix). This is important to inform the future community on limits.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1ar0QNFfC5", "reviewer": "Reviewer_kZ7P", "summary": "This paper mainly contributes to the following three points:\n\n1). Proposing a novel multi-stage synthesis pipeline to synthesize ReWatch, a large-scale dataset which includes ReWatch-Caption, ReWatch-QA, and ReWatch-CoT three components, for fostering advanced video reasoning. \n\n2). Proposing a new Observation & Reasoning (O&R) reward for RLVR that improves reasoning by rewarding both final-answer correctness and the factual grounding of intermediate steps in video content.\n\n3). Developing ReWatch-R1 by post-training a strong baseline LVLM with Supervised Fine-Tuning (SFT) and O&R framework, achieves state-of-the-art average performance on five challenging video reasoning benchmarks.\n\nThe dataset construction pipeline and the two-stage post-training framework proposed in this paper put the description of video change into the CoT, providing a new idea for the research of video reasoning from the perspectives of dataset and training.\n\nIn summary, this paper shows a high level in terms of method description (data construction pipeline,the O&R methods, etc.), experimental setup and writing, but for the situation that pipeline may cause error accumulation, some experimental results are not analyzed (this part will be described in detail in \"Weaknesses\"), which leads to the lack of analysis completeness. The quality of this paper will be improved after analyzing the missing parts.", "questions": "1) Why are the number of ReWatch-QA and ReWatch-CoT inconsistent? Shouldn't one QA data correspond to one CoT data?\n\n2) In Section 2.3 (specifically line 247), why does a structured execution trajectory end with A_final instead of observation? (Thought-Action-Observation loop)", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper proposes a dataset construction pipeline for video reasoning, which is enlightening for the dataset construction method in this field. The detailed description of video in different time steps is introduced based on semantic segmentation, which combines with the video summary to enhance the reasoning ability. In Addition, this paper uses three-layer filtering to screen the data that can best reflect the reasoning ability. The two-stage post-training framework simulates the \"Thought-Action-Observation loop\", and uses the improved O&R reward for RLVR to enhance the reasoning performance of ReWatch-R1.\n\nTherefore, this paper provides a new idea for the subsequent dataset construction method in this field and the improvement of reasoning performance.", "weaknesses": "The weaknesses of this paper focus on the method, experiment and writing：\n\nIn method and experiment：\n\n1). Error accumulation: the stage of semantic segmentation and detailed description generation in the dataset construction pipeline designed in this paper may cause error accumulation. The error caused by segmentation may cause one semantic of the video to be put into multiple segments. The description generation may result in the missing description between multiple time steps, which may result in the description of an item appearing in the previous time step, but the description of the item is missing in the next step, resulting in error accumulation. This paper does not discuss and analyze this situation.\n\n2). In Table 1, the results of ReWatch-R1-SFT and ReWatch-R1+O&R on the CG AV counting are the same, which are not analyzed in the paper.\n\n3). Lack of comparative analysis between the vanilla RLVR and the improved O&R method under the dataset constructed in the paper.\n\n4). In the analysis in Section 4.2 (specifically line 411), the performance degradation caused by Video-R1 replacing ReWatch-CoT does not consider the reason that SFT data and RL data is mismatch, which seems to be somewhat contradictory to the previous sentence \"SFT is an independent prerequisite for RL\".\n\n5). Also in the analysis in Section 4.2 (specifically 414 lines), it seems that ablation study using different data combinations (e.g. only using ReWatch-Caption or ReWatch-QA or ReWatch-CoT) have not been carried out, and the conclusion that \"The Quality of QA data used for RL determines final performance.\" is somewhat far fetched.\n\n6). In Appendix C2, the performance degradation of 384 frame training is not analyzed compared with 192 frame training.\n\nIn writing：\n\n1). In Figure 1, the annotation order of the legend is inconsistent with the display order of each LVLM in the figure.\n\n2). The two marking symbols not appear in Table 3 of appendix C2.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kFWMHLn917", "reviewer": "Reviewer_UAaJ", "summary": "This paper introduces **ReWatch-R1**, which tackles the challenge of video reasoning from two aspects: \n1. Data. A large synthetic dataset called **ReWatch** is collected, which includes detailed video captions, challenging question-answer pairs, and high-quality, video-grounded reasoning traces (COT data). These are generated using a multi-agent ReAct system. \n2. Model. Authors post-train QwenVL by SFT and GRPO. They introduce a new Observation & Reasoning (O&R) reward, which evaluates the accuracy of video observations and the validity of reasoning process. \nReWatch-R1 achieves promising performance compared with other 7B models.\nAnalysis shows that high-quality reasoning data is crucial for RL and RL on \"thinking\" mode improves the reasoning efficiency.", "questions": "1. Equation (20): the design of non-format rewards lacks motivation. For example, why not use the (weighted) summation of all rewards. The design has no explanation or experimental results.\n2. RL on 7B model shows promising improvements. However, the performance still lags behind the larger 32B model. Therefore, whether the same RL on 32B leads to improvement is questionable.\n\nSome typos:\n1. L255: \"a novel O&R reward mechanism we propos\" -> \"propose\"\n2. L771: \"Tabale 4\" -> \"Table 4\"", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The proposed multi-agent COT data synthesis pipeline is scalable to curate large-scale video-grounding reasoning data.\n2. The reward design shows the emphasis on explicit observation and reasoning is beneficial for video reasoning.\n3. The data and model design achieves state-of-the-art performance on video reasoning and understanding benchmarks in 7B-scale models. The extensive analysis shows insights on the role and importance of SFT and RL.", "weaknesses": "1. The information source for data synthesis is semantic segmentation and detailed video description from Gemini. The accuracy of this multi-step hierarchical captioning is not validated. Therefore there is no direct quality assessment of the synthetic data.\n2. All results are based on a 7B model. The benefits of high-quality COT data and O&R reward mechanism are not validated on larger-scale models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Ivv1eEfmEN", "reviewer": "Reviewer_ymDU", "summary": "The paper proposes ReWatch-R1, a framework that enhances complex video reasoning in large vision-language models through agentic data synthesis and verifiable reinforcement learning. It introduces ReWatch, a large-scale, high-quality dataset built via a three-stage pipeline—hierarchical video captioning, high-difficulty QA generation, and multi-agent chain-of-thought synthesis—ensuring strong temporal grounding and reasoning diversity. Furthermore, the authors design an Observation & Reasoning (O&R) reward that jointly evaluates answer correctness and factual grounding of intermediate reasoning steps. Combined, these innovations enable ReWatch-R1 to achieve state-of-the-art performance on multiple challenging video reasoning benchmarks.", "questions": "1. Does the proposed Observation & Reasoning (O&R) reward mechanism introduce additional reasoning overhead or inference latency compared to conventional reward designs? It would be helpful if the authors could provide quantitative results or analysis on the computational cost and efficiency trade-offs, as well as discuss potential optimization strategies to mitigate these issues.\n2. The paper currently compares ReWatch-R1 with a limited set of baseline models. Have the authors considered evaluating against stronger closed-source models (e.g., Gemini 2.5 Pro/Flash, GPT-4o) or recent RL-based approaches such as TW-GRPO, VersaVid-R1? Such comparisons would help better position ReWatch-R1’s performance and highlight its relative advantages within the broader landscape of contemporary video reasoning models.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper introduces a novel multi-stage data construction pipeline that enables the creation of ReWatch, a large-scale and high-quality dataset specifically designed for video reasoning. The dataset comprehensively includes caption, QA, and chain-of-thought (CoT) annotations, providing valuable and versatile resources that can benefit future research and community development in multimodal reasoning.\n2. The authors offer detailed experimental descriptions and implementation settings, ensuring strong reproducibility and facilitating independent verification and extension of their results.", "weaknesses": "1. The paper lacks qualitative experimental results, such as example responses generated by ReWatch-R1 for specific video reasoning cases. Including such examples would help readers intuitively understand the model’s reasoning style, output quality, and advantages over baseline models.\n2. The selection of baseline models is relatively limited, lacking comparisons with stronger closed-source systems such as Gemini 2.5 Pro/Flash and GPT-4o, as well as with recent RL-based approaches like TW-GRPO. Including these baselines would provide a more comprehensive evaluation of ReWatch-R1’s effectiveness and competitiveness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Otng8FCwwb", "reviewer": "Reviewer_bNUV", "summary": "This paper introduces novel datasets about captions, QAs, and CoTs, with a training pipeline to improve complex video reasoning. Additionally, based on the synthesized CoT data, this paper further designs a new reward for RL training. The trained model achieves state-of-the-art performance on several video reasoning benchmarks, outperforming models trained on other open-source datasets.", "questions": "- As shown in Tab.3, why does the trained model show marginal improvement on the most popular, general video question-answering datasets with reasoning?\n- How do you distinguish reasoning video benchmarks and general video benchmarks?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- This paper finds drawbacks in the current synthesized long CoT datasets, e.g., Video-R1, and synthesizes a quality-improved video reasoning dataset.\n- This paper redesigns the reward shaping for video understanding.\n- The trained model sets a new SOTA on several benchmarks.", "weaknesses": "- The contribution of this paper mainly lies in the dataset construction. Can you break down the improvement of each part, like adding timestamps into the captions, adding question difficulty, and the proposed reward design?\n- The introduction about the observation is not clear. How do you define observation formerly? In common, observation is visual content, while in your settings, the observation is processed captions. Would you consider implementing the thinking-with-image style training?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Aj5J9x9qxo", "reviewer": "Reviewer_w6PG", "summary": "The paper proposes ReWatch-R1, a novel approach to improve complex video reasoning in LVLMs by addressing the critical data bottleneck in existing methods. The authors introduce ReWatch, a large-scale dataset synthesized via a multi-stage agentic pipeline that includes temporally dense captions, high-difficulty multi-hop QA pairs, and video-grounded CoT traces generated through a Multi-Agent ReAct framework simulating human-like “re-watching.” They further develop an Observation & Reasoning (O&R) reward mechanism for RL, which jointly evaluates answer correctness and the factual grounding of intermediate reasoning steps.", "questions": "1. Line 207: Why does the original set of 85K QA pairs yield over 170K multiple-choice QA pairs?  \n\n2. From the Chain-of-Thought (CoT) example shown in the bottom-right corner of Figure 2, is retrieving explicit timestamps truly necessary? Could the reasoning path be simplified—for instance, as follows:  \n> *\"... So, I’ll \\<action\\> retrieve segments focusing on the man with blonde, curly hair on a jet ski interacting with a passenger \\</action\\>. \\<observation\\> The man on the jet ski passes a sandwich to the passenger, who then takes a bite. \\</observation\\> This directly answers the question. The food item passed was a sandwich...\"*", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. Novel QA curation methods: ReWatch is carefully designed to enforce video dependency and multi-step reasoning through contrastive QA generation and rigorous filtering, effectively eliminating textual shortcuts and hallucination-prone supervision.\n\n2. Innovative O&R reward mechanism: By evaluating both final answers and the fidelity of intermediate observations and reasoning steps, the O&R reward explicitly discourages hallucination and promotes evidence-based reasoning.", "weaknesses": "1. Formula 4 merges captions across each time interval independently, which leads to a loss of referential consistency. For example, the caption for 0–10s might be “A man…”, while that for 10–20s is again “A man…”, even though it refers to the same individual as in the earlier segment. This inconsistency compromises the overall quality of the generated captions.\n\n2. The observation mechanism introduces inference latency. Despite yielding performance gains on certain video reasoning benchmarks, it provides only marginal improvements on most general video understanding benchmarks.\n\n3. The paper lacks comparisons with more recent and advanced baselines, such as VersaVid-R1 and GRPO-CARE.\n\n4. Quantitative analysis is missing: the paper does not present any concrete inference results or case studies from ReWatch-R1.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)