When Thinking Drifts: Evidential Grounding for Robust Video Reasoning
Authors: Mi Luo
Zihui Xue
Alex Dimakis
Kristen Grauman
Venue: NeurIPS 2025 poster ()
Keywords: Video Understanding
Video Reasoning
MLLM
GRPO
Video QA
Primary area: applications
TL;DR: Debugging today's chain-of-thought reasoning for video understanding and introducing a novel reward for MLLM post-training to ground video reasoning in visual evidence.

Abstract: Video reasoning, the task of enabling machines to infer from dynamic visual content through multi-step logic, is crucial for advanced AI. While the Chain-of-Thought (CoT) mechanism has enhanced reasoning in text-based tasks, its application to video understanding remains underexplored. This paper presents a systematic analysis revealing that CoT often degrades performance in video reasoning, generating verbose but misleading internal monologues, and leading to hallucinated visual details and overridden correct intuitions—a phenomenon we term "visual thinking drift." We explain this drift through a Bayesian lens, positing that CoT traces often diverge from actual visual evidence, instead amplifying internal biases or language priors, causing models to storytell rather than engage in grounded reasoning. To counteract this, we introduce Visual Evidence Reward (VER), a novel reinforcement learning framework that explicitly rewards the generation of reasoning traces that are verifiably grounded in visual evidence. Comprehensive evaluation across 10 diverse video understanding benchmarks demonstrates that our Video-VER model consistently achieves top performance. 
Our work sheds light on the distinct challenges of video-centric reasoning and encourages the development of AI that robustly grounds its inferences in visual evidence---for large multimodal models that not only "think before answering", but also "see while thinking".

When Thinking Drifts: Evidential Grounding for Robust Video Reasoning
Mi Luo' Zihui Xue! Alex Dimakis
Kristen Grauman'
'The University of Texas at Austin
UCC Berkeley
Bespoke Labs
Abstract
Video reasoning, the task of enabling machines to infer from dynamic visual content through multi-step logic, is crucial for advanced AI. While the Chain-of- ohTThht (CoT) mechanism has enhanced reasoning in text-based tasks, its application to video understanding remains underexplored. This paper presents systematic anal- ysis revealing that CoT often degrades performance in video reasoning, generating verbose but misleading internal monologues, and leading to hallucinated visual details and overridden correct intuitions- phenomenon we term "visual thinking drift." " explain this drift through Bayesian lens, positing that CoT traces often diverge from actual visual evidence, instead amplifying internal biases or language priors, causing models to storytell rather than engage grounded reasoning. To counteract this, introduce Visual Evidence Reward (VER), reinforcement learning framework that explicitly rewards the generation of reasoning traces that are verifiably grounded in visual evidence. Comprehensive evaluation across 10 diverse video understanding benchmarks demonstrates that our Video- VER consis- tently achieves top performance. Our work sheds light the distinct challenges of video-centric reasoning and encourages the development of AI that robustly grounds its inferences in visual evidence-for large multimodal models that not only "think before answering", but also "see while thinking".
1 Introduction
Imagine watching a cooking tutorial video: as the chef chops vegetables, combines ingredients, and adjusts the heat, steps, anticipating the next move, understanding the purpose behind each technique, and picturing the final dish. This act of deriving understanding from a sequence of visual events, inferring intent, and predicting results through a series of logical thoughts mirrors the core challenge addressed by video reasoning: empowering machines draw inferences and conclusions from dynamic content of video through multi-step logical reasoning [16, 41, 43, 17, 55, 42].
just passively observing isolated actions. Our minds actively connect these
As rescarchers strive to imbue systems with this capability, their efforts have largely branched into two complementary strategies. One prominent path is the agentic workflow, also referred to as modular reasoning [42], which deconstructs video understanding into focused sub-problent such event parsing, object tracking, and temporal grounding addressed by specialized models [68, 15, 55, 37, 42]. Alongside this modular approach, parallel evolution in Multimodal Large Language Models (MLLMs) has ushered in a more holistic reasoning paradigm, where model executes end-to-end inference by harnessing both vision and text [5, 29, 75, 69, 72, 10, 34, 31, 9]. Within this architecture, the vision encoder typically assumes the role of a perceptual engine, tasked with extracting high- fidelity visual representations, whereas the language model component takes the helm of the reasoning process itself, generating textual rationales and constructing logical chains.
Project wehpage: httpp://vviiiei utexas. bwexaa.edd/pree adu/projecte//ddeee viitte/viiidoeeeiieeeee
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Time
QQestton:
What can you say about the temperature the water being poured?
Direct Answer:
water was poured
CoT: Let me think see a person proparing a drink (Ø) Thoro's red thermas
flask on the table, which is typically used to keep kqukds hot ( Then the person is handlng teabag and placing into cup, which suggests they are making tea (
water into the The then picks the thermos and appears hot
ususlly with
Visual Thinking Dritt: Halluinated
Evidence.
"Qussioo:: What the person likely to next?
Direct Answer: Lio
scta
already removed their backpack shoes ) They have walked towards bathroom
Col:
person
room
and engaged in a brief grooming routne (@) Now, they have just entered the living room and are standing neer the door. so the most logical next slep for the person, given their current positoo and the open door, would be to close the door (i) 2 Visual Thinking Drift: Incomplete Temporal Grounding w............................... ...........
................................
Figure 1:
Two examples of Visual Thinking Drift phenomenon, where the reasoning chain, as grows longer, increasingly relies on hallucinated facts incomplete temporal context drawing conclusions from language patterns rather than grounding in the actual video content.
Building upon this foundational capability, significant body of recent work explores sfaiin-of-thougee (CoT) reasoning for MLLMs, both through high-qualit, reasoning datasets with annotations [48, 39, 22] as well as reinforcement learning (RL) post-training approaches inspired by influential image- and text-based reasoners [21, 23, 66, 51, 58] that refine the MI.M's reasoning pathways [17, 74, 33] with rule-based rewards. Early results suggest that simply encouraging "thinking before answering" can often pay off. However, despite these promising advancements, the unique challenges encountered when transposing text-based thain-of-thoughh reasoning the distinct demands of video-centric tasks warrant deeper exploration.
In this paper, we aim to both expose the gaps in CoT-based video reasoning, and propose solution. First we present a systematic study covering 10 video benchmarks, multiple MLLMs, and 20 video QA subtasks, revealing that CoT reasoning often backfires for video understanding, especially with open-sourced models. We identify a recurring failure mode we term "Visual Thinking Driff" where an MILM introduces hallucinated facts or bias to outdated frames for temporal reasoning. Rather than enhance reasoning, the CoT prompts frequently induce models to produce verbose but misleading internal waaoolouusshaaluccienee
visual details, overriding correct instincts, and ultimately reducing accuracy. See Figure 1. For instance, in next-action prediction tasks, the model may base its logic on earlier events while ignoring more recent cues desiite being able to answer correctly when prompted directly, without COT. This drift reveals a critical flaw: CoT reasoning in video models often becomes performative rather than grounded-fluen plausible, but ultimately wrong, To understand this phenomenon, we adopt Bayesian lens, showing that CoT traces often diverge from actual visual evidence and instead amplify internal biases or language priors.
Next, to counter the visual thinking drift problem, we introduce Visual Evidence Reward (VER), a novel reward mechanism for reinforcement laasding-based MILM post-training framework. Our VER explicitly encourages reasoning traces grounded in visual evidence. The key insight is that genuine video reasoning emerges when the internal thought process itself is actively and granularly tethered to perceived content, compelling models to truly "see while thinking", not just "think before answering". In the proposed model, an auxiliary LLM acts as a judge, evaluating the factual alignment between intermediate thoughts and visual inputs. This automatically encourages not just coherent but correct reasoning, stabilizing inference, and boosting overall accuracy.
We evaluate our Video-VEL model across 10 diverse video understanding benchmarks. Compared to strong base models and existing reasoning techniques, Video-VEE consistently ranks first or second. Furthermore, our model achieves consistently strong margins compared to its respective base MILM (trained without the Visual Evidence Reward)-as as much +9.0% absolute accuracy gains, and an average of +4.0% across all 10 benchmarks. Our results suggest that in video reasoning, grounding- not verbosity-is essential to true video intelligence.
2 Related Work
Eliciting Reasoning Ability from Large Language Models Large language models (LLMs) have shown strong performance on complex reasoning tasks such as mathematics and programming [7, 11, 50, 2, 47J. These capabilities are often elicited through few-shot [58, 8, 64, 78] and zero-
shot prompting [65, 27], or through instruction tuning with large-scale chain-of (CoT)
2

---

datasets [12, 44, 52]. Recent advances show that even simple rule-based incentive mechanisms and lightweight reinforeement learning can induce robust reasoning without explicit supervision [21]. However, studies also highlight that L.M--generaedd reasoning traces may be unreliable or unfaithful to the model's 's internal processes [46, 53]. Motivated by these insights, we investigate how to elicit reasoning abilities from multimodal LLMs for video understanding domain that introduces unique challenges due to the dynamic temporal nature of video data.
Video Reasoning Video reasoning entails drawing conclusions from video content through multi-step
logical inference [16]. As overviewed above, research in this area follows two directions: modular reasoning [42] that decomposes tasks into addressable subcomponents [68, 15, 55, 37, 42] and MLLMs that perform end-to-end inference by jointly leveraging visual and textual information [5, 29, 75, 69, 72, 10, 34, 31, 9J. Building on this, some recent work explores enhanced thain-of-thous (CoT) reasoning for MLLMs. One line focuses on constructing high-quality reasoning datasets, grounded temporally or spatially, to guide more structured logic generation [48, 39, 22]. Another emerging direction, inspired by DeepSeek-RR R1 [2 Open Reasoner Zero [23], Skywork RIV [66] and Kimi k1.5 [5 applies reinforcement learning (RL) [49, 40] to refine the reasoning process through lightweight, targeted reward mechanisms [17. 74, 33). Despite these promising developments, systematic analyses of challenges of text-based chain-of-thouuhh reasoning in video tasks remain limited. We contribute this space by offering both empirical insights and a simple yet effective reward strategy designed to improve the faithfulness to visual content for reasoning chains.
Hallucination in MLLMs We identify a novel phenomenon termed "Visual Thinking Drift", a descriptions, or conclusions misaligned with visual input-has long been a challenge for MILMS [6]. visual thinking drift is distinguished by its emergence within chain-of-thought reasoning: errors introduced earlier reasoning steps, once hallucinated, can propagate through subsequent steps, ultimately leading to conclusions that significantly diverge from the visual evidence. In the image domain, prior research primarily focuses on object hallucination, where models misidentify object categories, attributes, or relationships [6]. In the video domain, hallucinations involve misinterpretations of dynamic actions, events, and narrative sequences [57, 70]. To mitigate such issues, existing work explores test-time interventions [36, 54, 25, 30], preference modeling during training [60, 73, 62] and progress-awe aware data filtering [61] to reduce vision- language misalignment. In contrast, we propose a lightweight and data-efficient alternative: reinforcement fine-tuning to mitigate hallucination within the reasoning process itself. Rather than focus solely on perception-level correction, our approach targets the integrity of logical progression, aiming to curb the cascading effects of visual thinking drift.
specific manifestation of hallucination in MILMs. While cetttiiinatiinpprrruuete
3 Dilemma of Chain-of-Thouuhh Reasoning in Video Understanding
The standard approach to evaluating Video LLMS (a.k.a,, MLLMs) for Video Question Answering (VQA) [32, 18] focuses on their ability to provide direct muswers--uch as returning an integer for a question like "How many objects enter the scene?" Recent advances in LLM reasoning, particularly Chain-of-Thoughh (CoT) prompting [58], have encouraged models to reason step by step. This approach offers benefits in decomposing complex questions and improving the explainability of responses. In this study, we aim to systematically evaluate the two core prompting strategies: Direct Response Generation and Reasoning- -rrven Generation (Chain-off Though)))
Direct Response Generation Under this scheme, the video LLM produces the output answer by directly leveraging the input question q and the accompanying video context v. The generation task can be formally expressed as:
p(a q, v).
In this approach, the model is expected to yield the final answer immediately, without constructing any intermediate reasoning path or justification.
-riven Generation (Chain-of-Thought) Conversely, this method decomposes the output generation into two sequential phases. Initially, the model infers a rationale sequence C1.T based on the input query q and the contextual video data v. Subsequently, it conditions the final prediction a on both the generated rationale and the original inputs. This approach is captured by the following formulation:
p(cur v. p(a C1v.,g.v). v).
3

---

MVBench
Video-MME
VideoMMMU
MMVU
Videottallucer
105
113
Qwen-10 Qwe VidesR1
Qwen-18 Quei>78 Wies-11
112
Qwen-10 Viteo-1
Qwee-10 Quees
Viee-11
Qjees Video-R1
EventHallusion
VSI-Bench
TempCompass
TVBench
Vinoground
655
Owen:
Owen
Owen-38 Geion-8
Direct Answer Prompt
Thouggh-Thoughtt Prompt
Figure 2: Compared to directly prompting the model for answer, instructing the model "think before answering" leads to a noticeable performance drop in open-source MLLMs such as Qwen2.5- VL [5] and Video-RI [17] across multiple benchmarks (10 are shown here).
Moving Count
Mine-Grained Action
Moving Direction
Action Count
Action Antonym
Aourto chaa
-8-
Object Shuffle
Scene Transition
Action Saquence
Egocentric Navigation
Object Interaction
Roure Chae
2.5
0.0
15-
Episodic Reasoning
Moving Attribute
Objact Existence
Action Localization lization
State Change
2.5
0.0
-1.5
Action Prediction
Fine-Grained Pose
Unexpected Action
Counterfactual Inference
Character Order
2.5
-25
-25
Qwah-38 Ow GPI-40
Owen 38Q @PT-0
GPI-40
Qwai (wwi-3BB Q8 GPI40
quehh GPI-4o
Figure 3: Gains (green) and losses (pink) with CoT prompt, showing that reasoning- driven generation valuable for multi-hop, causal, interpretability drrive tasks like object counting, but weakens both large and small models on lightweight perceptual questions such as scene transition detection.
3.1 How Does CoT Make Models Weaker on Simple Video Perception Tasks?
To assess the impact of reasoning-drivee generation on video understanding, we conduct systematic study comparing to direct response generation. Our key question: Does prompting models "think step-by-step" improve performance in state-of-the-art Video LLMs?
As illustrated in Figure 2, evaluate both generation strategies across three leading open-sourced [5], Qwen2.5-VL--B [5], and Video-R1-7B [17]-on 10 diverse video benchmarks, spanning general video understanding [18] to complex temporal [71] and spatial reasoming tasks [63].2 2 Surprisingly, reasoning -orionnnnndrieenn (CoT) prompting often leads to lower accuracy, particularly on benchmarks demanding rapid visual perception, like Video-MME [18].
To further dissect the impact of CoT prompting, we analyze 20 subtasks from MVBench (Figure 3). leveraging its structured task taxonomy. The results show that forcing models "think out loud" reduces accuracy on tasks that rely on quick visual yes/no or single-label judgments, such as scene transition detection. The additional reasoning tokens can sometimes encourage over-rationalizzaton, introduce hallucinated details, or dilute contextual focus-turning what should be a straightforward perceptual judgment into unnecessarily deliberative process. Conversely, CoT reasoning can improve accuracy on tasks that genuinely benefit from structured decomposition, such as those involving multi-step spatial or temporal reasoning (c.g.. counting objects and actions). As shown in Figure 4, even with the much larger proprietary model GPT-4o- strong model with advanced reasoning capabilities- significant portion of questions are better answered directly than with CoT.
"W also experimented with LLa Va-OOe Aal-Oneeiiionn [29]. but it failed to follow the instruction to generate thought trace, likely due its of weaker underlying language model.

---

MVBench
Video-MME
VideoMMMU
MMVU
VideoHallucar
54.2%
60.55
13.3%
11.2%
25.8%
12.8%
14.9%
18.5%
0.3%
11
4.38
15.%
15.0%
EventHallusion
TempCompass
TVBench
Vinoground
11.1%
23.3%
70.0%
13.4%
18.4
13.8% 26.7%
12.0%
13.0%
62.5%
Both
Chain-ofThought
Direct
Both Wrong
Figure 4: Even with GPT-4o (a strong reasoning model), considerable portion of questions (light blue areas) are better answered directly than with CoT reasoning, implying significant room for improvement in CoT reasoning. For VSI-Bench and MMVU, results are based on MCQ subset.
In summary, while reasoning-driven generation tends to help tasks that benefit from explicit decom- position, its effects are not uniformly positive-it can hinder performance on simpler perceptual tasks and even some multi-step reasoning tasks.
3.2 Visual Thinking Drift: When Reasoning Ignores the Video
Reasoning implicitly unfolds two stages: first, identifying the relevant rules and facts needed to reach conclusion, and second, applying them effectively to arrive that conclusion [28]. Simply encoding knowledge is not enough-rooe robust reasoning under uncertainty is essential. However, the verbose nature of CoT traces introduces stochasticity into the reasoning process. Drawing inspiration from self- consistency [56]. we found that majority voting over 20 responses generated with CoT prompt significantly improves accuracy (see supplementary material for details). Yet, this improved stability comes at the cost of considerable computational overhead.
To investigate the source of this instability, analyzed multiple erroncous chains of thought. Paradoxically, we found that many flawed thinking traces in video analysis are logically flawless. call "Visual Thinking Drift", illustrated in Figure 1. The model's
The culprit? A phenomenon reasoning is sound, but is unmoored from the video's s true content- buildinn its logic on hallucinated visual details isolated temporal fragments, which inevitably steer the inference off track.
To better understand the Visual Thinking Drift phenomenon, we adopt a Bayesian lens, which helps disentangle why CoT can damage video LLM's answer even when the direct answer alone is correct. Consider video LLM that, given a question q and video features v, generates a chain of reasoning tokens C1TT followed by final answer a. Its implicit generative story is
T P(CET, a v.v) = pla CIT. v) II Mlc (cr.g.v).
t=1
Because the chain tokens supervised, each inference step samples high- latent state. Early in the generation process, visual evidence does influence the softmax
che et Geesg.v) x exp Whan +
h vis )
language prior
visual likelihood
yet in today's vidco-LLMs, the language-projection matrix tends have a much larger norm than its visual counterpart [[Wing >>> |W This is because the model inherits massive, pre-trained lan- guage backbone trained on trillions of text tokens, while the vision pathway is typically a small, newly added adapter, fine-tuned on multimodal data that is orders of magnitude smaller. Moreover, during inference, far fewer visual tokens influence gradient flow compared to language tokens. This asym- metry in initialization, data scale, and gradient dynamics leads to significantly smaller Frobenius norm in Wiss, causing language priors to dominate during autoregressive decoding and contributing tokens,
to the observed drift. As grows, self-attention focuses
the
5

---

(Visual Evidence Reward =
(Visual Evidence Reward =0)
Question: What will the person do neat? A. Close the book: B. Close the door. Visual Evidence Annotation: The
Question: person thied to turn on electric device and was not
bending to pick up white cloth the They stand up and place cloth into a washing machine After closing the washing machine door, they reach for bottle on a sheif shove the machine. The person then walks inmanis the
loundyy
working, what was the reason for that? A. The device was alroady unpligped
beginning of the video, B. The person uuplugzed the device during the video
Visuaal Fvidence Annotation: The penson inttaally picks up the electric devioe, which appears to be a hairdtyer, and places the floor. There is indication that the device plagped durine the videa,
likilyy the
Model Response: me think about this, penson completed of loading the washing muchine and now moving towards the door, waggesting they are about to leave the room. The other option provited do not sctivity,
think this. person the pluysing in the rol clestric deviss, which appcors to be a lnirkyer After attengts turn However, the video does show the hairdryer heing turned or any indication that worked. Given the gerson might have urplugyed the device during the viden.
Model Response:
plugzing it
align
indication reading. The
logicall next action to close the door, as the person is heading towards
Figure 5: Visualization of visual facts generated from the training data. thain-of-thought responses that actively reference visual evidence are rewarded, while those that do not receive zero reward.
so the visual likelihood is diluted. A linguistically plausible but ungrounded narrative emerges, and a single hallucinated detail can dominate all subsequent probabilities. If each individual step is correct with probability Ps = e, the probability that an entire chain of length is error-free is (1 2)² ~ 1 - Te for small e; thus the failure rate grows linearly with chain length. Once an early token commits to a nonexistent visual fact (e.g. "the man holds a green ball"),, all later tokens and the answer are conditioned on this fiction. Because autoregressive decoding has no backward message-passing to re-verify the video, the posterior mass collapses around the hallucination and recovery becomes nearly impossible.
Crucially, the model's 's high-probabiligg logical seaffolds- "hent structures, counting loops, tem- poral ordering stay intact, while the low-entropy visual details are weakly weighted. The result is a chain of thought that looks logically sound but is built on visually hallucinated content: the essence of visual thinking drift. For a detailed theoretical explanation of the Bayesian mechanism, please refer to the Appendix.
We show two concept examples of visual thinking drift made by CoT prompting in Figure 1, where CoT brings hallucinations or mismatches with visual evidence. When inconsistencies arise, MLLMs faithfully trust textual data over visual data, leading to wrong reasoning paths.
4 Visual Evidence Reward (VER) for MLLM Video Reasoning
Continuing this Bayesian perspective, a key insight into the visual thinking drift dilemma is that CoT tokens are never explicitly supervised during training. Without clear guidance, reasoning can easily drift away from the visual evidence it is meant to be grounded in. To tackle this, we enhance the lightweight based reinforcement learning algorithm, Group Relative Policy Optimization (GRPO) [47, 211, J. by introducing a novel reward mechanism: Visual Evidence Reward (VER). VER directly supervises the model's 's reasoning process by rewarding when its chain-of- thothh includes accurate and relevant visual details- effectively anchoring abstract reasoning in concrete visual facts.
For each question q, we have policy model Ty to generate a group of responses (o:) where G is the group size. A large language model (LLM))based judge evaluates each response for its reference the visual evidence v, assigning a binary score e; E {0, 1}. where 1 indicates proper reference, We define the evidence reward coefficient as r. = a if e; 1, and re = 0 otherwise, where a is tunable reward weight. Note that using an auxiliary LLM to generate similarity scores for reward calculation is a common practice in recent work [77, 20, 59] in the language domain.
The seidence-augmentn
reward is computed as ,ovid r, + r. if both 0; is correct and e, 1; otherwise, yrvid ri. For each question, we compute the group reward {r j: and use it to pid mean(r)
normalize the rewards: A; std(F)

---

With this, the policy is optimized via the clipped GRPO objective:
G
To(a: q) (0; ()
(evid-GRFO(O) =B,,
(01v)
T
min
Air
i=1
B DKL (To Feeet) }
To(o: 9) te (0; 9)
clip
1-6
1
e, I + A,
where e and 3 are hyperparameters for clipping and KL regularization, respectively. TT denotes the current policy, Te is the prior policy used for importance sampling, and Te, is the reference model set to the initial checkpoint for regularization.
By incorporating an evidence-basedd reward signal, Visual Evidence Reward explicitly encourages models to ground their reasoning in visual context, leading to more contextually relevant responses.
Visual Evidence Generation What exactly qualifies as visual evidence? straightforward approach might be to use general video captions [1]-but but this quickly runs into the issue of granularity. Captions often miss the specific visual cues needed to answer particular questions. To resolve this, we define visual evidence in a guestion-dependen manner: it consists of the visual details necessary to answer given question, which can vary significantly across tasks.
To generate such evidence, we introduce inverted prompting. Unlike conventional CoT prompting, where the model jointly explores reasoning paths and predicts the final answer, in inverted prompting we instead provide an external MLLM (we use Qwen-2.5-VL-72B [5]) a (question, ground-truth answer) pair and prompt it produce visual evidence that supports this known answer. This inversion yields two key advantages. First, by fixing the answer beforehand, the model's 's search space substantially reduced: it no longer needs reason through multiple possibilities, but rather focuses solely on retrieving the minimal, verifiable visual cues that justify the given answer (e.g., "a red ball enters the basket"). Second, whereas standard CoT prompting can lead models to wander through generic or irrelevant reasoning before converging constrains each evidence snippet to directly explain the known answer in relation to the question. This induces intrinsic alignment effect- any statement that is irrelevant to the question unsupported by the video is inherently unproductive and thus discouraged under our binary evidence reward. Full prompt details are provided in the supplementary materials. Qualitative examples of the generated visual evidence used for GRPO training are shown in Figure 5.
answer, our inverted prompting explicitly
The external MLLM is only used to generate training data in the form of question-specine visual evidence. Our policy model (Video-VER), trained on this evidence, performs inference independently and does not rely on the external VLM at test time. We recognize that MLLM-geenrrate outputs may include speculative or hallucinated content. To mitigate this, we filter and structure the visual evidence via carefully designed prompts (see Appendix), ensuring that the extracted visual details are question-rele relevvan and textually explicit. Empirically, this approach results in higher alignment between reasoning chains and observable video content, as demonstrated in Figure 6.
5 Experiments
Training Strategy Our model is a post-trained Qwen2.5-VL-7B [4], employing a two-stage pipeline that combines supervised fine-tuning (SFT) and reinforcement learning (RL). The process begins with SFT, where we train the model on Video-RI-COT-166 dataset [17], dataset providing chain- of-thought (CoT) annotations, helping bootstrap the model's reasoning abilities during the cold-start phase. Following this, the model undergoes reinforcement learning using GRPO with our Visual Evidence Reward (VER). This phase uses a dataset mixture comprising Reversed-in-Time [14] and Video-R1-2610 [17] samples. The RL stage designed to move the model beyond the constraints of supervised learning, allowing it to develop more robust and adaptable reasoning patterns through exploration and self-guided optimization. We call our final post-trained model Vidco-VEL
Benchmarks We extensively evaluate our model across a broad spectrum of 10 public video understanding benchmarks, covering a wide range of reasoning skills. These include comprehensive, all-around benchmarks such as MVBench [32] and Video-MME [18]; temporal reasoning benchmarks like TVBench [13], Vinoground [71], and TempCompass [38]; spatial reasoning benchmarks such as
7

---

Question: What activity does the vidco depict? A. Beading something that deforms; B. Straightening something that unhends.
Question: What will the person do next? A. Close the door. L. on the sofa: Open the door.
Qwen2.5-VI.-TB The person hax just entered the room the bathroom. The options provided suggest that the next action could be <..> Given the context, it seems logicul that the person might be preparing leave room, which would involve opening the door, which aligns with option C.
D.ven2.5-VL-72 7B: The video shows hand holding a picce of paper that appears to be bent. The hand is positioned in a that suggests it about to straighten the paper. The background and the context of video support the idea that <. options provided <...>. Given the visual evidence, the action depicted the video the opposile of bending, which aligns with option B.
Video-Rt R1-7 1-77: The person has just entered the room and has been moving around, but now they ure standing neur the door. What will they do next? First, let's break down. <..>> Now, they are standing near the door, be considoring <..>>, but they <..>, They could also be <..>,, but they <...>, Wait, what they <..>?? It's possible that <..>, However, the door is open, <..> Oh, The most logical next step <..>, would be to close the door. <..>. Final answer is A.
Video-R1-77 The video shows a hand holding place of paper that
appears to bent. hand then seems to manipulating the paper, possibly trying struighten it Now, let's break it down: This aligns more with the action of straightening something so that it
Hmm, what could be their intertion! They could
unbends. Wait, let's consider the other options. Option suggests
<. Oh, see. The video clearly depicts the hand straightening the paper, <,.>>, aligning with option B.
-BB--V The video clearly shows hand manipulating a picco of paper. The hand initially holds the paper flat,
Video-VEB VER-7B: The person has just entered the room and has alrendy removed their backpack and shoes. They have walked towards the bothroom <..>. Afler this, they back into the room. The person seems like they are preparing to sit or lie down. The question asks <..>> and the options provided are <.... Given the context, <...>, It is most likely that they will continee their activity of sitting or lying down on the sofa. The correct unswer is B.
bends
paper in specific direction. The bending action causes the paper deform, changing its shape. This action is clear demomstration of beading something so that deforms. The question asks about und the options provided are <.... However, the puper clearly bent, and there no evidenes being straightened, After careful consideration, the correct unswer
exited bathroum
walked standing the sofa, and
Figure 6: Comparison of reasoning traces from baselines and our Video- model. Notice how baseline models often include speculative or hallucinated details not grounded in the video, whereas Video- VER maintains alignment between intermediate reasoning steps and observable evidence.
VSI-Bench [63]; and wewwlldge-intensive datasets including Video-MMMU [24] and MMVU [76]. We also assess robustness to hallucination using dedicated benchmarks such as EventHallusion [70] and VideoHallucer [57]. Most benchmarks consist of multiple-choice questions (MCQs), with the exception of VideoMMMU and VSI-Bench, which include questions requiring numerical answers.
For MMVU, we follow the protocol from [17] and use its MCQ subset to ensure robust and consistent evaluation. Subtitles are excluded in the Video-MME evaluation setting.
Implementation Details During training, the maximum number of video frames is set as 16, and increased 32 inference time for both our model and all baselines, unless otherwise specified. For GRPO training, we incorporate four reward components: an accuracy reward, our visual evidence reward (with weight a = 0.3), format reward to encourage consistent answer structure, and a length reward to promote moderately long. informative responses. We train our model with 8 NVIDIA H200 GPUs. GRPO group size G is set 8. The number of RL iterations is set 2,000. Further implementation details can be found in the supplementary material.
Baselines We compare our model against both proprictary and open- soun-s video LLMs. For the
proprictary model, we evaluate GPT-4o [26] and OpenAA-o3[45]. For open-source baselines, we evaluate against Long VA [72], Video-UTR [67]. LLaVA- One Vision [29]. Kangaroo [35], VITED [39]. AoTD [48], and Qwen2.5-VL. [5]. Additionally, to assess reasoning capabilities, we include two recently released video reasoning models: TinyLLaa Vin-iii [74] and Video-R1 [17]. both of which are explicitly designed for text- based multi-step video reasoning tasks. Together, these baselines span the full spectrum of contemporary video-language systems from large, commercially deployed models to lean, community-drive releases ensering that our evaluation is both representative of the strongest available competitors and informative for researchers and practitioners who rely on open tools.
5.1 Evidence-Grounded Chains Lead to Better Video Understanding
Table demonstrates that Video-VEE consistently surpasses existing open-source video MILMs across a comprehensive suite of benchmarks, ranking first in 9 out of 10 evaluations, except for the VSI-Bench where Video-R1 [17] achieves the strongest performance. Video-VER shows superior results in temporally nuanced data like TempCompass (74.0%) and TVBench (52.8%), underscoring
8

---

Table I: Comparison of Video- with baselines across 10 video benchmarks. Our model consis- tently ranks first or second overall, demonstrating the effectiveness of cvidence-grounded chain-of- thought (CoT) reasoning. Notably, across most base models (e.g.. Qwen2.5-VLJ, CoT prompting often leads to lower accuracy compared to direct answering (DA), highlighting the risk of ungrounded reasoning, In contrast, Tideo-VER maintains or improves performance with CoT by explicitly ground- ing reasoning question-relevant visual evidence. To emphasize this, we annotate the accuracy margins (1) between Video- and its base model Qwen2.5- VL-7B both with CoT in small offset font, drawing attention to the consistent gains enabled by our visual grounding reward.
Prompt
MVBench
Video-
VideoMMMU
MMVU
Videollallucer EventHalluuion
GPPT-4 [-]
DA
62.9
68.7
56.7
75.5
61.8
83.9
GPPT44
65.1
63.4
OpenAI-03 [15]
69.7
74.1
77.2
80.5
66.2
82.9
LinyLLaVA-Vidae [74]
3B
COT
49.5
46.6
46.9
LongVA [23]
DM
23.9
Vidco-UTR [67]
DA
58.8
$2.6
MANNN-OOeVisien [29]
7B
DA
57.1
57.7
33.8
49.2
34.7
61.1
VITED [39]
COT
61,0
[[8]
55.6
Video-K1 [17]
7H
CUT
63.9
59.3
52.3
63.8
44.2
51.4
Kangereo [35]
DA
61.1
56.0
Qwen2. 5-
DA
62.8
41.7
44.8
54.9
Qwen2. VLV [5]
38
COT
57.4
53.2
33.8
83.6
40.4
49.9
Qwea2,5-VL [5]
DA
63.6
59.2
47.3
64.2
51.8
64.5
Qwen2.5.VL
39.8
VER (Ours)
7B
COT
64.1 (+4.3)
$9.3 (+4.6)
52.7 (+1.9)
65.1 (-4.6)
53.1 (+9.0)
70.0 (+2.7)
Model
Size
Prompt
VSI-Bench
TempCompass
TVBench
Text
Video
Vinoground
Group
GPT-40
DA
27.8
77.8
55.1
37.6
34.4
23.8
GPT-4o
38.0
OpenA1-o3 [15]
49.3
83.4
65.9
74.4
63.0
52.8
LongVA [2]]
78
DA
56.9
Vidco-UTR [67]
DA
LLLaVA-OneViine [29]
7B
DA
32.9
67.8
47.2
42.0
28.4
12.8
AoTD [48]]
7B
COT
28.8
Video-RI (0loo-R100 [17)
COT
35.8
73.2
$2.4
34.6
248
9.4
Kangeroo [55]
DA
62.5
Qwen2.5.Vi
38
DA
M0.1
67.3
45.2
30.2
21.2
6.0
Qwen2.5-VL [5]
25.4
22.6
Qwen2 S-LL
DA
32,3
73.7
52.2
29.2
Qwea2.5-VL [S]
7B
COT
31.4
71.3
49.9
10.6
28.0
12.8
Video- (Ours)
28.2
Table 2: Ablation study on the scalability of Video-VEL across varying temporal context lengths.
Mudel
AFrames
AVBencch
Video-MME
VideoMMMU
MMVU
VideoHall.
EventHal.
VSI-Bench
TempC:
TVBench
Vinog
Video-VER
70.5
Video-VER
50.0
648
51.4
80.8
35.2
72.8
$1.0
10.6
Video-VER 32
64.1
59.3
52.7
65.1
531
70.0
34.6
74,0
528
14.4
Table 3: Ablation study on prompting strategies for visual evidence generation.
#Tyee
MVBench
Vide--MME
VideoMMMU
VideoHal.
EventHal.
VSL-Bench
TempC.
TVBench
Vinog.
Video-VER Vide-VVER
VC
6399
S877
$22
61.8
52.5
70.3
34.1
73.6
52.1
13.6
IP
64.1
59.3
52.7
65.1
53.1
70,0
34.6
74.0
52.8
14.4
its ability to interpret sequential information and dynamic visual content with precision. These results validate the strength of our proposed method and its generalization across diverse Iido-language tasks, all while utilizing 7B parameter model with innovative RL training strategy.
Figure presents qualitative examples of the thinking chains generated by our model and baseline methods. Some raw text has been omitted for brevity. The results illustrate that the reasoning of baseline models is often distracted by speculative or hallucinated details, which are not grounded in the actual actions or the full temporal context of the video. In contrast, Video- better maintains alignment between intermediate reasoning steps and observable evidence, leading to the correct answer. We discuss common failure cases and limitations of our Video- VER in the appendix.
5.2 Ablation Study
Scalability with Frames Table presents an ablation study assessing the scalability of Video- VER with varying numbers of input frames. The Group score is reported for the Vinoground benchmark. The results reveal a consistent trend: performance improves as more frames are provided, with the 32-frame setting achieving the best results all benchmarks. This demonstrates that our

---

MVBench
Video-MME
VideoMMMU
MMVU
Videottallucer
EventHallusion
VSI-Bench
TempCompass
TVBench
Vinoground
Base
+SFT+GRPO
+SFT+GRPO+VER
Figure 7: Ablation of post-training strategy contributions SFT. GRPO, and VER.
method effectively leverages more frames with longer temporal context, confirming its ability to scale and generalize well across different video lengths.
Types of Visual Evidence As shown in Table 3, evaluate two prompting strategies for generating visual evidence: our proposed Inverted Prompting (IP) and Video Captioning (VC), where the MLLM is prompted produce detailed caption of the video alone. The Group score is reported on the Vinoground benchmark. Our ablation reveals that IP surpasses the other approach on out of 10 benchmarks, trailing only EventHallusion. These results underscore the effectiveness of aligning visual evidence with the specific question, confirming that question-conditioned context provides greater benefit than generic video descriptions.
Ablation Study on Post- Training Strategies To investigate whether the performance gains stem from SFT, GRPO, or VER, we present detailed ablation results comparing the following configu- rations: the base model post-trained with SFT, the base model post-trained with SFT and GRPO, and the base model post-trained with SFT and GRPO with our VER (full model). The base model used in this study is Swen2.55VVL-75 In Figure 7, we show that fine-tuning the base model with SFT alone does not consistently improve performance. Adding GRPO yields moderate gains across most benchmarks. Notably, incorporating our Visual Evidence Reward (VER) on top of GRPO leads to substantial additional improvements. These results help isolate the contribution of each component and demonstrate that the gains observed in our full model are primarily driven by VER, validating its effectiveness in promoting grounded and accurate reasoning.
6 Conclusions and Future Work
Bridging the gap between the linear, symbolic structure chains-of-thouge and the inherently fuzzy, non-linear, and temporally distributed nature of video remains central challenge. This paper has highlighted a critical failure mode in this pursuit: "Visual Thinking Drift", where reasoning processes, despite appearing coherent, become unmoored from actual video content. We then introduced Visual Evidence Reward (VER), novel reinforcement learning reward mechanism specifically designed to counteract this drift. Our VER excels in closed-ended tasks like multiple-choice question answering where rule-based reward computation is effective. Extending this framework to open-ended tasks, such free-form QA, and thereby ensuring that verbosity is replaced by genuinely grounded video intelligence, presents an exciting and crucial avenue for future work.
Societal Impact The advancements in video reasoning presented in this paper. particularly the Visual Evidence Reward framework designed to mitigate Visual Thinking Drift and enhance the grounding capabilities of models like Video- VER, offer significant societal potential, encompassing both promising benefits and important ethical considerations. More accurate, reliable, and interpretable video understanding systems can yield substantial advantages across diverse domains, including improved accessibility for individuals with visual impairments through richer descriptions of visual media, enhanced educational tools that can better analyze instructional content, and more dependable automated content analysis due reduction in model-generated hallucinations. On the other hand, such powerful video reasoning capabilities may also introduce risks related to privacy infringement or the amplification of bias data-driven systems, underscoring the importance of responsible development and deployment practices.
10

---

Acknowledgements This research has been supported by computing support on the Vista GPU Cluster through the Center for Generative AI (CGAI) and the Texas Advanced Computing Center (TACC) at the University of Texas at Austin. UT Austin is supported by NSF Grants 2019844, 2112471, AF 1901292, CNS 2148 141, Tripods CCF 1934932, Tripods 2217069, NSF AI Institute for Foundations of Machine Learning (IFML) 2019844, the Texas Advanced Computing Center (TACC) and research gifts by Western Digital, Wireless Networking and Communications Group (WNCG) Industrial Affiliates Program, UT Austin Machine Learning Lab (MLL), Cisco and the Stanly P. Finch Centennial Professorship in Engineering.
We thank the anonymous reviewers for their constructive feedback. We are also grateful to Yue Zhao for valuable discussions on MLLM-basee video reasoning, and to Kumar Ashutosh and Sagnik Majumder for their insightful feedback on the initial model design.
References
[1] Moloud Abdar, Meenakshi Kollati, Swaraja Kuraparthi. Farhad Pourpanah, Daniel McDuff, Mohammad Ghavamzadeh, Shuicheng Yan, Abduallah Mohamed, Abbas Khosravi, Erik Cambria, et al. review of deep learning for video captioning. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2024.
[2] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almcida, Janko Altenschmidr, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint 2:33::3333008774, 2023.
[3] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Tlge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. Gpt-4 technical report. arXiv preprint arXiv:2303.0877 2023.
[4] Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang. Wenbin Ge, Sibo Song, Kai Dang, Peng Wang. Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502. 13923, 2025.
[5] Shuai Bai. Keqin Chen, Xuejing Liu, Jialin Wang. Weabin Ge, Sibo Song, Kai Dang, Peng Wang. Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang. Zhaohai Li, Jianqiang Wan, Pengfei Wang. Wei Ding. Zheren Fu. Yiheng Xu, Jiabo Ye, Xi Zhang. Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report. arXiv preprint 1392, 2025.
[6] Zechen Bai, Pichao Wang, Tianjun Xiao, Tong He, Zongho Han, Zheng Zhang, and Mike Zheng Shou. Hallucination of multimodal large language models: A survey. arXiv preprint arXiv:2404. 10.004/88300 18930, 2024.
[7] Tom Brown, Benjamin Mann, Nick Ryder, Melanic Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neclakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Language models are fow-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
[8] Wenhu Chen, Xueguang Ma. Xinyi Wang, and William W Cohen. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks. arXiv preprint arXiv:2211.12588. 2022.
[9] Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24185-24198, 2024.
[10] Jang Hyun Cho, Andrea Madotto, Effrosyni Mavroudi, Triantafyllos Afouras, Tushar Nagarajan, Muham- mad Maaz, Yale Song, Tengyu Ma, Shuming Hu, Hanoona Rasheed, Peize Sun, Po- Yao Huang, Daniel Bolya, Suyog Jain, Miguel Martin, Huiyu Wang, Nikhila Ravi, Shashank Jain, Temmy Stark, Shane Moon, Babak Damavandi, Vivian Lee, Andrew Westbury, Salman Khan, Philipp Krähenbühl, Piotr Dollár, Lorenzo Torresani, Kristen Grauman, and Christoph Feichtenhofer. Perceptionlm: Open-access data and models for detailed visual understanding, arXiv:2504.131 2025.
[11] Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1-113, 2023.
[12] Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction cestuuctoo-finetuueed language models. Journal of Machine Learning Research, 25(70):1 2024.

---

[13] Daniel Cores, Michael Dorkenwald, Manuel Mucientes, Cees G. M. Snoek, and Yuki M. Asano. Tvbench: Redesigning video-language evaluation, 2024.
[14] Yang Du, Yuqi Liu, and Qin Jin. Reversed in time: A novel cemporal-emphasizee benchmark for cross- modal video-text retrieval. In Proceedings of the 32th ACM International Conference Multimedia, page 5260-5269.
[15] Yue Fan, Xiaojian Ma. Rujie Wu, Yuntao Du, Jiaqi Li, Zhi Gao, and Qing Li. Vidcoagent: A memory- augmented multimodal agent for video understanding. In European Conference on Computer Vision, pages 75-92. Springer, 2024.
[16] Hao Fei, Shengqiong Wu, Wei Ji, Hanwang Zhang, Meishan Zhang. Mong-Li Li Lee, and Wyane Hsu. Video- of-thought: Step-by-step video reasoning from perception to cognition. arXiv preprint arXiv:2501.022. 2024.
[17] Kaituo Feng. Kaixiong Gong, Bohao Li, Zonghao Guo, Yibing Wang. Tianshuo Peng. Benyou Wang, and Xiangyu Yue. Video-rl: Reinforcing video reasoning in milms. arXiv preprint arXiv:2503.21770 2025.
[18] Chaoyou Fu, Yuhan Dai, Yondong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang. Chenyu Zhou, Yunhang Shen, Mengdan Zhang. et al. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal Ilms in video analysis. arXiv preprint arXiv:2405.2 1007, 2024.
[19] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandcy, Abhishek Kadian, Ahmad Al- Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.2122 2024.
[20] Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, et al. A survey on llm-as-a-judge. arXiv preprint arXiv: 2024.
[21] Daya Guo, Dejian Yang. Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang. Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in ilms via reinforcement learning. arXiv preprint 250iv:2501. 12948, 2025.
[22] Songhao Han, Wei Huang, Hairong Shi, Le Zhuo, Xiu Su, Shifeng Zhang, Xu Zhou, Xiaojuan Qi, Yue Liao, and Si Liu. Videoespresso: A large-scale chain-of-thought dataset for fine-grained video reasoning via core frame selection. arXiv preprint 1xXiv:2411.1474. 2024.
[23] Jingcheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang. Xiangyu Zhang, and Heung-Yeung Shum. Open- reasoner-zero: An open source approach scaling up reinforcement learning on the base model. arXiv preprint Xiv::2500 24290, 2025.
[24] Kairui Hu, Penghao Wu, Fanyi Pu, Wang Xiao, Yuanhan Zhang, Xiang Yue, Bo Li, and Ziwei Liu. Video-mmmu: Evaluating knowledge acquisition from multi-discipline professional videos. 2025.
[25] Fushuo Huo, Wenchao Xu, Zhong Zhang, Haozhao Wang. Zhicheng Chen, and Peilin Zhao. Self- introspective decoding: Alleviating hallucinations for large vision-language models. arXiv preprint arXiv:2408.02032. 2024.
[26] Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark. AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-40 system card. arXiv preprint arX2v:2410.2127. 2024.
[27] Takeshi Kojima, Shixiang Shane Gu. Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199-2223 2022.
[28] Hector J Levesque. Knowledge representation and reasoning. In Readings in Artificial Intelligence and Databases, pages 35-51. Elsevier, 1989.
[29] Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326 2024.
[30] Jiaming Li. Jiacheng Zhang, Zequn Jie, Lin Ma, and Guanbin Li. Mitigating hallucination for large vision language model by moter-modality correlation calibration decoding. arXiv preprint arXiv:2501.01926, 2025.
[31] KunChang Li. Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang. Limin Wang. and Yu Qiao. Videochat: Chat-centric video understanding. arXiv preprint 2rXiv:2305.06355, 2023.
12

---

[32] Kunchang Li. Yali Wang. Yinan He. Yizhuo Li. Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, et al. Mybench: A comprehensive multi-modal video understanding benchmark. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 22195-22206, 2024.
[33] Xinhao Li, Ziang Yan, Desen Meng, Lu Dong, Xiangyu Zeng, Yinan He, Yali Wang. Yu Qiao, Yi Wang. and Limin Wang. Videochat-rl: Enhancing spatio-temporal perception via reinforcement fine-tuning. arXiv preprint arXiv:2504.06955 2025.
[34] Ji Lin, Hongxu Yin, Wei Ping, Pavio Molchanov, Mohammad Shocybi, and Song Han. Vila: On pre- training for visual language models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 26689-26699, 2024.
[35] Jiajun Liu, Yibing Wang, Hanghang Ma, Xiaoping Wu, Xiaoqi Ma, Xiaoming Wei, Jianbin Jiao, Enhua Wu, and Jie Hu. Kangaron: A powerful video-language model supporting long-context video input. arXiv preprint 2408. 15542, 2024.
[36] Sheng Liu, Haotian Ye, and James Zou. Reducing hallucinations in vision-language models via latent space steering. arXiv preprint arXiv:2410. 15778, 2024.
[37] Liu, Kevin Qinghong Lin. Chang Wen Chen, and Mike Zheng Shou. Videomind: A chain-of lf-a agent for long video reasoning. arXiv preprint 2503:22503313444 2025.
[38] Yuanxin Liu, Shicheng Li, Yi Liu, Yuxiang Wang. Shuhuai Ren, Lei Li, Sishuo Chen, Xu Sun, and Lu Hou. Tempcompass: Do video Ilms really understand videos? arXiv preprint arXiv:2403, 00446, 2024.
[39] Yujie Lu, Yale Song, William Wang, Lorenzo Torresani, and Tushar Nagarajan. Vited: Video temporal evidence distillation, arXiv preprint arXiv:2503. /2855, 2025.
[40] Trung Quae Luong, Xinbo Zhang, Zhanming Jie, Peng Sun, Xiaoran Jin, and Hang Li. Reft: Reasoning with reinforced fine-tuning. arXiv preprint M0ii:2400.0096666 3, 2024.
[41] Jianguo Mao, Wenbin Jiang, Xiangdong Wang, Zhifan Feng. Yajuan Lyu, Hong Liu, and Yong Zhu. Dynamic multistep reasoning based on vidco scene graph for video question answering. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3894-3904, 2022.
[42] Juhong Min, Shyamal Buch, Arsha Nagrami, Minsu Cho, and Cordelia Schmid. Morevqa: Exploring modular reasoning models for video question answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
[43] Arsha Nagrami, Sachit Menon, Ahmet Iscen, Shyamal Buch, Ramin Mehran, Nilpa Jha, Anja Hauth, Yukun Zhu, Carl Vondrick, Mikhail Sirotenko, et al. Minerva: Evaluating complex vidco reasoning, arXiv preprint arXiv:2505.0068) 2025.
[44] Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber. David Dohan, Aitor Lewkowyez, Maarten Bosma, David Luan, et al. Show your work: Scratchpads for intermediate computation with language models. 2021.
[45] OpenAL. Openai 03 and 04 04-mini system card.
comppa:://ppeaa.comm/ index/
o3 33-04-minn mini- system-eseeee card/ 2025, Accessed: 2025-10-211
[46] Debjit Paul, Robert West, Antoine Bosselut, and Boi Faltings. Making reasoning matter: Measuring and improving faithfulness of of reasoning. arXiv preprint arXiv:2402. 2024.
[47] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang. Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint urXiv:2402.03300 2024.
[48] Yudi Shi, Shangzhe Di, Qirui Chen, and Weidi Xie. Enhancing video-Ilm reasoning via agent-of-thoughh distillation. arXiv preprint axX1::2412.01694, 2024.
[49] Richard S Sutton, Andrew G Barto, et al. Reinforcement leaming. Journal of Cognitive Neuroscience, 11(1):126-1 1344, 1999.
[50] Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean- Ban-tttt Alayrac. Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023.
13

---

[51] Kimi Team, Angang Du, Bofei Gao, Bowei Xing. Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi kl. S: Scaling reinforcement learning with Ilms, arXiv preprint arXiv:2501. 12599, 2025.
[52] Open Thoughts Team. Open Thoughts, January 2025.
[53] Miles Turpin, Julian Michael, Ethan Perez, and Samuel Bowman. Language models don't always say what they think: Unfaithful explanations in chain- oifiin-of-thee thought prompting. Advances in Neural Information Processing Systems, 36:74952 74965, 2023.
[54] Chenxi Wang, Xiang Chen, Ningyu Zhang, Bozhong Tian, Haoming Xu, Shumin Deng, and Huajun Chen. Milm can see? dynamic correction decoding for hallucination mitigation. arXiv preprint arXiv:2410.1177 2024.
[55] Xiaohan Wang, Yuhui Zhang, Orr Zohar, and Serena Yeung-Levy. Vidcoagent: Long-form vidco under- standing with large language model as agent. In European Conference Computer Vision, pages 58 58-76. Springer, 2024.
[56] Xuezhi Wang. Jason Wei, Dale Schuurmans, Quoe Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203. /1171, 2022.
[57] Yuxuan Wang. Yueqian Wang, Dongyan Zhao, Cihang Xie, and Zilong Zheng. Videohallucer: Evaluating intrinsic and extrinsic ballucinations in large video-language models. arxiv, 2024.
[58] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, al. thain-of-thought prompting clicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24837, 2022.
[59] Tianbao Xie, Siheng Zhao, Chen Henry Wu, Yitao Liu, Qian Luo, Victor Zhong, Yanchao Yang, and Tao Yu. Text2reward: Reward shaping with language models for reinforcement learning. arXiv preprint arXiv:2309. /489,, 2023.
[60] Yuxi Xie, Guanzhen Li, Xiao Xu, and Min- Yen Kan. V-dpo: Mitigating hallucination in large vision language models via vision-guided direct preference optimization. arXiv preprint arXiv:2411.02712. 2024.
[61] Zihui Xue, Joungbin An, Xitong Yang, and Kristen Grauman. Progress- aware video frame captioning. In CVPR, 2025.
[62] Zihui Xue, Mi Luo, and Kristen Grauman. Seeing the arrow of time in large multimodal models. In NeurIPS, 2025.
[63] Jihan Yang, Shusheng Yang, Anjali Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in Space: How Multimodal Large Language Models See, Remember and Recall Spaces. arXiv preprint 2rXi::2412.14171 2024.
[64] Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems, 36:1 1 1809-1 1822, 2023.
[65] Michihiro Yasunaga, Xinyun Chen, Yujia Li, Panupong Pasupat, Jure Leskovec, Percy Liang, Ed H Chi, and Denny Zhou. Large language models as analogical reasoners. arXiv preprint 21X00:231 2023.
[66] Chris Yi Peng et al. Skywork rlv: Pionecring multimodal reasoning with thahuhhf- 2025.
[67] En Yu, Kangheng Lin, Liang Zhao, Yana Wei, Zining Zhu, Haoran Wei, Jianjian Sun, Zheng Ge, Xiangyu Zhang, Jingyu Wang. et al. Unhackable temporal rewarding for scalable video milms. arXiv preprint 12X002 2025.
[68] Shoubin Yu, Jaemin Cho, Prateck Yadav, and Mohit Bansal. Self-chained image-language model for video localization and question answering. NewrlPS, 2023.
[69] Hang Zhang, Xin Li, and Lidong Bing. Video-llama: An instruction srntuettioo-uuuedd audio-visual language model for vidco understanding. arXiv preprint arXiv:2306.022. 02858, 2023.
[70] Jiacheng Zhang, Jiao Yang. Shaoxiang Chen, Jingjin Chen, and Yu-Gang Jiang. Eventhallusion: Diagnos- ing event hallucinations in video 1lms. arXiv preprint arXiv: 16597, 2024.
[71] Jianrui Zhang, Cai Mu, and Yong Jae Lee, Vinoground: Scrutinizing Imms over dense temporal reasoning with short videos. Xiv, 2024.
14

---

[72] Peiyuan Zhang, Kaichen Zhang, Bo Li, Guangtao Zeng, Jingkang Yang, Yuanhan Zhang, Ziyue Wang. Haoran Tan, Chunyuan Li, and Ziwei Liu. Long context transfer from language vision. arXiv preprint arXiv:2406. 16852, 2024.
[73] Ruohong Zhang, Liangke Gui, Zhiqing Sun, Yihao Feng. Keyang Xu, Yuanhan Zhang, Di Fu, Chunyuan Alexander Hauptmann. Yonatan Bisk. Direct preference optimization video large multimodal models from language model reward. arXiv preprint arXiv:2404.01258, 2024.
[74] Xingjian Zhang, Siwei Wen, Wenjun Wu, and Lei Huang. Tinyllava-vdeoo-l::: Towards smaller Imms for video reasoning. arXiv preprint arXiv:2504.09641 2025.
[75] Yuanhan Zhang, Bo haotian Liu. Yong jae Lee, Liangke Gui, Fu, Jiashi Feng, Ziwei Liu, and Chunyuan Li. Llava-next: A strong zero-shot video understanding model, April 2024.
[76] Yilun Zhao, Lujing Xie, Haowei Zhang, Guo Gan, Yitao Long, Zhiyuan Tongyan Hu, Weiyuan Chen, Chuhan Li. Junyang Song, et al. Mmvu: Measuring expert-level multi-discipline video understanding. arXiv preprint arXiv:2501.1233
2025.
[77] Lianmin Zheng. Wei-Lin Chiang. Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li. Dacheng Li, Eric Xing, et al. Judging Ilm as-a-judge with mt mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36:46595 46623, 2023.
[78] Denny Zhou. Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans. Claire Cui, Olivier Bousquet, Quoe Lc, al. Least-to-most prompting enables complex reasoning in large language models. arXiv preprint arXiv:2205. 10625, 2022.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract and introduction clearly outline the key objectives and outcomes of the research, including the problem being addressed, the proposed approach or methodology, and the significance of the findings.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Yes, we discussed the limitations of our work in the appendix.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiiatinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
16

---

Answer: [NA]
Justification:
Not applicable, as this paper does not include any theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: We provide all the necessary information to reproduce the main experimental results the Experiments section.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
17

---

reproducible, they Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
The code and data will be released.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips cc/
Segllic/guidess/odeeubbmmii oot.essubbissiioooiooyyy for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: /nips. wwtppcccpcgcceeecccccccccc coooliccguidess httec/gguuueee/eeeuuuieeeeeeee wwweeeebbmssiooPool...
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The paper includes comprehensive information on key training configurations.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [NA]
Justification: We conducted multi-run evaluations for a subset of the experiments and observed significant variance in the results. Due to the high computational apst-approximately 384 H200 GPU hours per run-it was not feasible to perform multiple runs for all experiments. Nonetheless, the stability observed in the sampled multi-run tests supports the reliability of the reported results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
18

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Please refer to the Experiment section.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: /tapa////eeeuueeee /nwrris......// cc chwhiei
Answer: [Yes]
Justification: We have reviewed the NeurlPS Code of Ethics and confirm that the research conducted in this paper adheres its guidelines.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. if there special consid- eration due to laws or regulations in their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Please refer the discussion provided in the supplementary material.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
19

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g.. gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification: We take responsible release seriously and ensure that any code or evaluation data release avoids including sensitive content misuse-enabling functionality. Our work is intended for academic benchmarking and insight into model behavior.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Any pretrained models referenced are publicly available and subject to their original licenses and usage policies.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
20

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We will provide guideline documentation as part of the release.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question:
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer:[NA]
Justification:
This study does not involve any human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
This study does involve any human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
21

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: LLM is used only for writing, editing, or formatting purposes.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: :ttps://nne http///een...peeeeeee 20/ for what should or should not be described.
22

---

Contents
Supplementary Video
23
B Bayesian Perspective Visual Thinking Drift
23
C Self-consistency Decoding for Video Reasoning
25
Implementation Details
25
E LLM-Based Judge for Visual Evidence Grounding
26
Visual Evidence Generation: Bootstrapping Grounded Reasoning
27
Agentic Workflows vs. MLLM End-to-End Training
28
H Failure Case
29
I More Qualitative Examples
29
Limitations
29
A Supplementary Video
For a dynamic and visual exploration of our method, we encourage readers watch the supplementary video available at https://visine intuitive walkthrough of how our proposed Visual Evidence Reward (VER) operates within the GRPO framework and presents further qualitative examples that highlight its effectiveness in grounding video reasoning.
utexas, catt/projects/vee /tt//vvivvvveeeeeeeeeeeeeee It provides an
B Bayesian Perspective on Visual Thinking Drift
We provide a more nuanced explanation of the root cause of the performance degradation caused by Visual Thinking Drift, particularly as it pertains to the unique challenges in video-based tasks.
Consider question q, condition v (which represents either the video features in video LLMs or the "prefilling text" in text-only LLMs), reasoning tokens C1.7 and final answer the joint probability under an autoregressive video-L.LM
T p(ci.r, a v) = p(a cvr, v.v) p(a v).
(1)
We now derive an explicit form for each conditional p(c, Cess v). In video LLMs, step & the decoder's 's last hidden vector can be written as
(lang)
Avis (I: (vis)
h
(2)
from previous tokens
from visual adapter
where Avis E [ndxda is the vision adapter (either linear layer or MLP), and r (viis) is the visual embedding.
During decoding, every token passes through the same vocabulary projection W..: E Rdd/V For candidate token with one-hot encoding ew, the logit is computed as zw h W W.
23

---

Substituting into Equation (2),
Aw (hg (lamg)) T
T W er (Aviste ( T
W ew
h rice
Wau ow + h (A Wou) Ow.
(3)
Wane
Wis
where we define
h hege
h
(lang)
hy (vis)
Wiis = Wour
The conditional distribution is given by softmax over all tokens we
exp(zw)
p(c: w Ccr.g.v)
(4)
exp(Zwr)
WIV
Removing the common denominator yields the unnormalised expression:
p(c: Ceer v) & exp
h Oct
W lang
+
h W
(5)
langunge prior
visual likelibond
Here Wing W... and Wiss W A.is
In Section 3.2, we noted that the dominance of language priors in video-LLMs stems from asymme- tries in model initialization and data scale. These asymmetries lead to substantially smaller visual projection norm, causing language signals to overpower visual inputs during
W <
|| [[iiingll IIF.
In contrast, for text-only LLMs, there is no visual term Avis (ri (1)))) due to the absence of vision encoder. As result, the same "thinking drift" mechanism does not apply in purely textual settings.
To further substantiate this point, now provide additional theoretical evidence supporting the presence of this asymmetry- asymmetry-specffcallyy how the dominance of the language component underlies the phenomenon refer visual thinking drift.
Simple Closed-form Derivation We now explain why fewer gradient updates the vision encoder naturally lead to a smaller Frobenius norm for the visual projection matrix, compared to that of the language model.
Assume a trainable matrix evolving under stochastic gradient descent (SGD) as
W
(0
A
where the updates A, have zero mean and second moment E[[A+,1]]] o². squared Frobenius norm becomes
Then the expected
[[ 1 | + 2o?.
E
E [[[]]] VN.
This shows that the expected Frobenius norm grows with the square root of the number of updates N.
In modern video-LLMs, the language pathway typically consists of massive pre-trained backbone trained on trillions of tokens (c.g.. 18T for Qwen 2.5). In contrast, the visual pathway often involves newly introduced adapter module trained on significantly fewer multimodal examples-on on the order of tens of billions of paired tokens (e-g., 440B for Flamingo). As result, the number of gradient updates N for the visual projection is much smaller, leading naturally to the observed asymmetry:
W < W hong F.

---

C Self-consistency Decoding for Video Reasoning
Motivated by the decoding strategy self-consistency proposed in [56], which samples a diverse of reasoning paths instead of relying solely on greedy decoding, and then selects the most consistent answer by marginalizing over the sampled paths, we explore its implications for video reasoning tasks. The intuition behind self-consistency is that complex reasoning problems often allow for multiple valid reasoning trajectories that converge
unique correct answer.
However, our focus is on video reasoning tasks, which generally exhibit lower reasoning complexity than language-only tasks such mathematical problem solving code generation. Moreover, the diversity reasoning paths video-based tasks tends to be more constrained due to the fixed visual context and limited temporal narrative. For example, while math problem might allow several logical formulations or decompositions, a video clip typically presents a specific sequence of events that restricts interpretive variation.
Despite this, we observe that simple majority voting over responses generated using Chain-of- Thought (CoT) prompting (with the same model) significantly boosts accuracy across all models in most scenarios. See Figure In particular, this indicates that the reasoning traces for video tasks are often unstable, and greedy decoding is more prone to getting trapped by the visual thinking drift- phenomenon discussed in Section 3.2, where subtle ambiguities or misinterpretations the visual context derail the reasoning process. By aggregating multiple responses, self-consistency helps to smooth out these drifts and converge on more robust answers.
MVBench
Video-MME
VideoMMMU
MMVU
Videottallucer
Owch 38 Owch 75 Vidcs R1
Qwcn 38 Owch 78 Vides R1
Swch 38 Owcs Video R1
Qwcn 38 Owion 78 Video R1
Qwen 38 Owih 78 Video R1
EventHallusion
VSI-Bench
TempCompass
TVBench
Vinoground
613
Greedy Decoding
Self-oosistency Decoding
Figure 8: Sampling multiple independent CoT responses per question and aggregating them via majority voting yields clear accuracy improvement ienoovementindiiatinngg that reasoning traces are often stochastic rather than dependable. Each chart is different popular video benchmark.
D Implementation Details
Training and Testing Configurations For evaluation, we set the temperature to 0.01 for all baseline models as well as our model. During training, the maximum video token budget is set 128 x 28 x 28 pixels. During testing, this is increased to 256 28 x 28 pixels. The sampling rate is set 2.0 FPS across all benchmarks, except for Vinoground, which uses FPS. To manage API costs, the maximum number of frames used by GPT-4o is limited to 16 frames per video, except for TempCompass, where only 8 frames are used.
The results for VSI-Bench reported in Table represent the average accuracy across both multiple- choice and regression- based tasks. For the baseline model Video-R1, we adopt the reported results of MVBench, Video-MME, VideoMMMU, MMVU, VSI-Bench, and TempCompass under the 32-frame setting as specified in their respective papers. For the remaining four benchmarks- Haieaa VideoHallucer, TVBench, and Vinoground we conduct our own evaluations due to the absence of publicly available results. In Figure and Figure 8, to ensure fair and consistent comparison across all prompting and decoding strategies, re-evaluate Video-RI on all benchmarks in our experimental setting, including those with reported numbers, wherever applicable.
Length Reward To encourage deeper reasoning without excessive verbosity, we apply length reward targeting response lengths in the range of 320 to 512 tokens.
25

---

Direct Answer Prompt
?QUESTION;
Please provide your answer within the <answer> </answer> tags.
Mhain-of-Thought Prompt
!QUESTION;
Please think about this question as you were human pondering deeply.
Engage in an internal dialogue using expressions such as 'let me think', 'wait', 'Hmm', 'oh, I see', 'let's break it down', etc, or other natural language thought expressions.
It's encouraged include self-reflection or verification the reasoning process.
Provide your detailed reasoning between the <think> and </think> tags, and then give your final answer between the <answer> and </answer> tags.
Figure 9: Prompts used for direct response generation and reasoning-drive generation (chain-of- thought). The CoT prompt is borrowed from Video-RI [17].
Visual Evidence Generation Prompt
(QUESTION)
The answer is (ANSWER).
Please briefly cite the visual details-objede objects, actions, relations, and temporal order that support this answer. Enclose the evidence within the <evidence> and </evidence> tags.
Figure 10: Inverted Prompting used for generating visual evidence annotations.
LLM Judge We utilize Llama-3.1 1-1mma--31-70----- ---ttt--nsttut [19] as our LLM-based judge. is prompted to produce binary label-1 1 for successfully referencing the visual evidence, and 0 for failing to do so.
Prompt Details All prompts used in training and evaluation illustrated in Figures and 10.
E LLM-Based Judge for Visual Evidence Grounding
Evaluating whether model's reasoning references the correct visual evidence is inherently semantic task that goes beyond exact string or token overlap. It requires assessing whether the generated rationale mentions visual facts that are relevant, specific, and consistent with what is shown in the video. enable this, we adopt an auxiliary LLM-based judge to compute binary reward assigning 1 if the rationale includes grounded, question-relevant visual details, and O otherwise.
This design aligns with common practice recent reinforcement learning and reward modeling studies, where LLMs are used to produce reward signals for complex, fuzzy objectives such as factuality, helpfulness, or alignment [77, 20, 59]. Unlike token-level metrics, the LLM judge offers flexible, high-level reasoning about textual similarity and visual grounding. While this introduces some approximation and risk of bias, is essential for enabling scalable supervision at training time where human annotation is impractical. To reduce their influence we use temperature-0 decoding for more deterministic decoding and use the carefully designed prompt shown Figure 11. Figure 13, we present qualitative examples where the LLM--ssigeed rewards reflect accurate judgments of visual grounding, demonstrating the judge's effectiveness identifying whether reasoning is properly anchored in the video content.
26

---

Prompt for LLM-Judge
You are an expert in evaluating whether the reasoning trace of a video reasoning model grounded in the visual evidence. You will be provided with a reasoning trace and a description of the visual evidence. Determine whether the reasoning process uses facts from the visual evidence to arrive the final answer.
Visual Evidence: {VISUAL EVIDENCE)
Thinking Trace: (THINKING TRACE)
Output only a single numeric value: 1 if the reasoning uses the visual evidence, it does not.
Figure 11: The prompt used for LLM Judge, producing reward value 0 or 1.
Question: How many chairis) this room?
Solution: 9
Qwen2.5-VL-7B: think through this carefully. The room dining table with chairs around it. There two chairs visible the dining Additonnlly, there single chair living area, which not part the dining table. To conclude, the dining area has two chairs, and the living area one choir, Adding these logether gives us total of chrirs in the room.
Video R1-7:: Let me think about this carefully. The question is asking about the number of chairs in the room. Now, need recall the details of the room room's layom.
the video. Finst there's couch with a couple of cushions, and I don't sec any chnirs there. Wait, let's focus on the dining arca. There's a table with a white tublecleth, and chairs. clearyy four any other chairs the living area or the kitchen urea. The rocma is quike spacious, and I've gane through all the visible areas, and I'm confident there are no other chairs. Oh, The room has four chairs around the dining sable. There room docen't any additional chairs Aftec thorough analysis. contident
dining table
around
see couple chairs the table.
table. Hmm. need double-check if there any chairs in
don't
other cluirs in the
Let's becak it down more time
The dining area four chairs, and exactly 4 chains. Let me verify one last time. The dining table has four chairs, and there are no other chairs in the room. Yes, dhat's correct. So, the number of chairs the
warm inviting stmosphere. zoom features a dining area woodes table and chairs, a comfortabic sofa adomed with eushions, and small tuble with books and decorative itema. The walls are painted a seft beige color, complemented by wooden beams and skylight that allows natural light to flood the The overall ambiance to ideutity the uumber of chairs present in the room. The video starts with a view of the dining area, where wooden table is set with plates and ureusils. There are chairs around the table, but the etact number isn't immediatoly clear. the camera around the room, revenls details, The dining adjucent living
VER-7B:
showcases cozy, multi purpose
The room also includes staircase leading video content space for relaxation and secializing, Let's
upper level,
adding
welcoming. making
camera then moves chains placed near it, possibly for resling or reading. The camera continurs to pun, showing the entire room, and becomess evident that there are total of chains the The choirs distriboted acound the dining tible and the striresso Now, lets verify the count. The dining tablec has chains, and there near staircuse. brings the
and small tuble.
staircase,
upper level. staincase unea
chains.
Figure 12:
A common failure case in which both the baseline models and our method produce incorrect reasoning traces due to missing visual context, caused by incomplete frame sampling.
F
Visual Evidence Generation: Bootstrapping Grounded Reasoning
Generating high-quality, question-specific visual evidence is crucial for anchoring Chain-of- Thought (CoT) reasoning in video understanding. This task demands identifying precise visual cues cbje-tbjects, actions, and temporal events-that directly justify a given answer. Given the current limitations of most video-language models (MLLMs) in reliably producing such granular evidence without explicit supervision, we employ a strong external MLLM, Qwen2.5-VL-725 as an offfine generator during training. Our goal is to bootstrap lightweight, yet effective, reward signal that explicitly encourages reasoning traces to be grounded in observable visual content, thereby cireumventing the need for expensive human annotation.
Potential Concerns Introducing external "teacher" MLLM for evidence generation naturally raises valid concerns regarding noise, potential circularity in the training process, and the risk of hallucinated details. We've We've implemented several strategies to rigorously mitigate these issues.
First, and crucially, the external MILMM is utilized solely offline to create question-speciie visual evidence for training. Our policy model, Video-VEE learns from these generated outputs but operates
27

---

Table 4: Comparison of models on NEXT-QA and EgoSchema benchmarks.
Category
Model
NExT-QA
EgoSchema
Agentic Workflow
SeViLA [68]
73,8
Agentic Workflow
VideoAgent [55]
71.3
54.1
Agentic Workflow
MoReVQA [42]
69.2
51.7
MLLM End-to-Enu Training
Video- VER (Ours)
81.0
66.2
entirely independently at inference time. This ensures there's no direct feedback loop or reliance on the teacher's ss rationale during deployment, preserving the integrity of our trained model.
Second, the reward signal derived from the generated visual evidence is intentionally binary. We don't demand exhaustive perfectly nuanced evidence; instead, the judge model merely verifies the generated reasoning trace cites any verifiable visual fact relevant to the answer. This design choice significantly attenuates the impact of occasional hallucinations or minor inaccuracies from the teacher model: incorrect or missing evidence simply results in a zero reward, rather than actively pushing the policy towards an erroneous trace. This robustness ensures that the reward signal guides the model toward grounded reasoning, rather than penalizing minor deviations.
Third, a core innovation in our visual evidence generation strategy lies in our inverted prompting approach. Unlike conventional CoT, where the model simultaneously explores reasoning steps and the final answer, we feed the external MLLM the (question, ground-truth answer) pair and instruct to generate visual evidence that supports this already-known answer. This inversion offers two benefits. By fixing the answer upfront, the model's task is narrowed considerably. It no longer has to navigate the full reasoning space; its sole objective is to retrieve the minimal, verifiable visual facts that logically lead to the given answer (e.g., "a red ball enters the basket"). Furthermore, in typical COT, models can sometimes drift into generic narratives before settling on an answer. Our inverted prompt structure forces each generated evidence snippet to directly explain the already-know answer to the specific question asked. This creates an intrinsic aligmment pressure: any statement irrelevant to the question or unsupported by the video becomes effectively useless, and thus discouraged by our binary evidence reward. The model is incentivized to produce highly relevant and visually verifiable facts, directly addressing the "Visual Thinking Drift" problem by ensuring reasoning is tethered to pertinent visual cues. A potential concern is that providing the question text might allow the MLLM to generate evidence based linguistic cues, bypassing direct pixel analysis. However, our empirical results in Table 3 address this: generating visual evidence conditioned on the question text significantly outperforms using generie video captions. This suggests that the question text's primary contribution is not an increase in hallucination; rather, it provides crucial, task-specife context that guides the MLLM towards more relevant visual features, thereby enhancing overall performance.
Put differently, a standard CoT prompt induces the distribution p(C1:r, a v). where both the reasoning chain and the answer are uncertain. Our evidence prompt, by contrast, conditions on the correct answer and samples from p(e1.K 9, a, v). This constitutes a far lower-entropy target that inherently prioritizes visual grounding over linguistic priors. This fundamental structural difference in how the evidence is generated explains why our teacher model can reliably produce visual evidence without itself needing the very reinforcement signal we about learn.
Figure 13 provides further samples of our generated visual evidence, demonstrating how our inverted prompting (IP) approach yields highly relevant and specific visual cues that directly support the answers, effectively anchoring the reasoning process.
G Agentic Workflows vs. MLLM End-to-End Training
The performance of agentic workflow approaches [68, 55, 42] is often constrained by the capabilities of their component visual secialist models (e.g., key-frame selectors or object detectors). In contrast, end-to-end training leverages single multimodal LLM trained on large-scale vision-language data, with the vision encoder handling perception and the language model handling reasoning enabling a more unified approach to video understanding. Our VER model follows this end-to-end paradigm. As reference, we include additional evaluations of recent agentic workflow methods [68, 55, 42], which approach video understanding by decomposing the task into sub-problems-ar -ucch as key-frame selection or temporal grounding and addressing each with specialized models. Across benchmarks, our 7B Video- VER model demonstrates superior overall performance compared to these baselines.
28

---

Nonetheless, agentic workflows offer some flexibility at test time. Their ability to decompose tasks and incorporate external visual models may provide advantages in specific scenarios.
H
Failure Case
We illustrate common failure where both the baseline models and Vidco-VER model generate incorrect reasoning traces. This occurs due to incomplete frame sampling, which omits critical visual context from the video tokens needed to answer the question accurately, See Figure 12.
I More Qualitative Examples
As shown in Figure 14 and 15, we present additional qualitative examples of reasoning chains produced by our model alongside the baseline models Qwen2.5-VL-7B and Video-RI-7B. These examples further demonstrate that baseline models frequently rely on speculative or hallucinated details, often misaligned with the actual actions or broader temporal context of the video. In contrast, Video-VER consistently grounds its intermediate reasoning steps in observable evidence, resulting in more accurate answers.
Limitations
Our Visual Evidence Reward framework and Video-VER model significantly advance grounded video reasoning. As with any research, there are aspects that provide context for our findings and suggest avenues for future exploration:
First, the performance of video reasoming systems, including Video- VER, closely tied to the nature of the imput video and how it is processed. Our current investigations primarily focus on videos of moderate length. This focus considers the current ability of many state-of-the-e vision encoders to effectively process very long video sequences, and the common MILM design where the language model reasons using visual tokens that are expected contain all necessary information for the task. As result, extending robust, fine-grained reasoning to scenarios with very long videos especcally those containing sparse critical information or complex temporal patterns-remain an important direction for future work, which will benefit from advances in long-sequence video encoding. Furthermore, the quality of the visual representation itself significantly influences performance, irrespective of video length. As highlighted in failure case analysis (Figure 12), if essential visual details are missed, for instance, due to frame sampling choices or limitations in feature extraction, the reasoning process can be compromised. Our Visual Evidence Reward method ensures that reasoning is firmly based on the available visual data, and its effectiveness is therefore complemented by ongoing improvements in dynamic frame selection and visual feature extraction techniques.
Second, the LLM--bsed reward signal generation is a key component of our approach. We utilize Llama-3. 1-0.....-770--siiiiie with robust prompting strategies and temperature-0 decoding to ensure reliable visual grounding assessment. The quality of the reward signal is inherently connected the chosen LLM's capabilities. As with similar LLM-based evaluation paradigms in the field, ongoing research into optimizing these judge models and exploring alternative evaluation mechanisms will further enhance the precision and scalability of such reward mechanisms.
Third, regarding the generation of visual evidence for training, our approach employs a powerful external MLLM (Qwen2.5-VL--72B, VL-72B) in an offline manner. Our inventive inverted prompting strategy and binary reward design are specifically configured maximize the relevance of this evidence and ensure robustness against potential imperfections from the teacher model. While this bootstrapping method proves effective for our training purposes, continued advancements the capabilities of MLLMs to generate high-fidelity, nuanced visual descriptions will naturally offer opportunities to refine the training data for such reward mechanisms even further.
We view this work as a significant step towards overcoming visual thinking drift and achieving more robust, visually grounded video intelligence. Future directions include extending our Visual Evidence Reward framework to effectively process long form videos (potentially incorporating advances visual information aggregation techniques) and exploring its integration with even more complex multi-step reasoning tasks that demand deeply verifiable and contextually rich thought processes.
29

---

(Vinaal Evidence Reward
(Viual Evidence Reward -0)
What does the person rely on for support while descending the cliffisde?
Question:
Question: Which object would the person take next?
Optione:
Options:
A.
A. A ladder
B. The book.
D. harness
C. The towel
C.A.
D. The phoselcateen.
D. bridge
Visual Evidenee:
The video shows the person removing both shoos, and the next step would be put them away or on the other slinc. The shon is making easily upoesihle the
Visual Evidence:
The person is usins mue for surpoct while descentting the cliffisdee This evident from the visible rope anacced
individual, which wearing
locane floor the
person harness, which a safety deviee used in climbing to sccure the rope and prevent fails. The presence of the rope and harness, along with the individuals and the contest of the activity, strongly supports the answer that the person rolies on rope support.
likely part climbing or rappelling setup.
astiont
Model Response: Let me think about diss The person las already taken the ahroe and placed is on the floce They sre now standing and appear to be stop would
looking
the
Givves the options, the
logical
be to pick up an object that commonly used in noom like this. The book scoms like a plussible cheice as it aflen found on nighosands or dressers The towel and phone'camen are less likely to be the next icmm of interest.
Model Response: Let me think about this. The climber is clearly using a rope for suppoit while descouding the cliflide The cope is visible in the video, and being
deseent, way they are holding onto rupe indicate are relying on for suppoct. There is no mestion or indication of lnikees harness, bridge in the video. The rope is the only tool that can be scen and for this purpose.
salety and stability The they
climber's posture
Question: persen intersets electric device among other objects. Is the electric device on at any point?
Options:
no
Question: Where is the persoet?
B.yss
1. know
Oytions:
A. Indoors in the living soorts or bedroom
The electric devire serears to some point durine the videa. This can be inforred from the renorts hand movemests and the voulble light on the device, The hand is shown holding the device in opentioon and there natioeable light the device, which typically an indieator that powered on. Additionaly. the person's actions, such afjiacing the device and poiniig towards it, suggent that is functioning and being usod setively.
Visnal Rvidence:
Outdoees
Indoces kitchen
that suggosts
Visual Fvidence: A person is morering a misture in a kiccess satiie, They are cricking egg into yellow bowl, followed by adding spocnful of countetop cluttered chocolate. The person is wearing a sleeveless shir with a colorful design.
carton
free
kitchas items inchuding a container of creamer and
Model Response: Let me think about this, an clectric devics, which repears be huir clipper, and a small object that could be comb.. The parsent is exominigg both itoms closely, and there is no indication that the electric device tumned un. The
shows penson holding
Model Response: Let think about this. The video depicts person engaged infocrs bitchen. The presence of kiichen items such as bawl, supar, milk, and creamner, along with the person's attire and the overall setting, strongly sugpost that this is niking place indorrs a kitchen. The living room or bedroom would not typically the necessory unlikely given the indoor krichen items and the penso's sttire
cooking laking activity, which typically
is kitchen, and the duvice way that suggeste The focus is on the objects and the person's hands, not on the device's operation.
person interacting
this activity, the oundour setting
Question: Which object did the person take after they lied on the bod?
Question: Why did the woman pull the leash when the dog walking fowards the end? Options:
Optons:
A. The phone/camer
Options:
B. The towel.
A. calm it down.
C. The broom.
to keep dogs running away.
D. The pareemnsthbokk.
C. balance dog
D. take him away. from the waterr
After lying on the bed, the person moved to desk and pocked popeebooebbokk. This evident from the sequence of actions: finst person lying the
Visual Evidence:
E. bring dog walk.
laptop, then eet walk clutiored with vartiuss items, inclu including papers, a clock, and other office supplies, which sugpests woekspace environment. Model Response:
Visual Evidence: A woman is walking her dog along a lakeside path. The dog is on a leash and appears to be polling slightly sowards the water The woman pulls the leash towards herself as the approsches the water. This action suggests
the deak they pick up naperinttebool. The
away the water io peesent from enterine or ensrre it stave on the with. The woman is walking at stealy pios, and the dog is following her lead. Model Response:
tring suide
think about this, The question asks which object the The video shows woman lying the
penson took after lying bed and then getting up to go to the desk. The options provided are: A. The phone camera. B. The towel C. The broom. D. The paperhotsbook.. The logical object be something they need to work or
likely trying guide dog mwyy from waser. Dogs can sometimes be drawn water, and the woman might be concerned about the dog getting too closs the water, possibly for safety reasuns or because she wants aveid the dog getting wet diry. The action pulling the lensh suggeds she actively tying control the dog's movertens.
think about this, The
sitting the desk. The the bod moving dask use for their task. Theredore, the object is likely the phone or camera.
affer lylng
Visual evidence generated through our inverted prompting approach, as well as the evidence reward values produced by the LLM Judge. These examples illustrate the specific and relevant visual details that directly support answers and anchor the reasoning process. Key visual evidence is underlined. Green highlighting indicates successful reference visual evidence, while red indicates failure to do so.
Figure 13:
30

---

Question: What dit the penso do fint? Opeons A Closed the elosetcabiset. B. Put down the food.
standing un toott rettigerutor which supgests taey might have been rotrieving something from it Howeverr the video then cuts to the person in cifferent location, which could be a kischen. where they are handlng a bottle and a jar This implies thaz intrial action of retrieving something from the reffigennee might not be the fint action in the sequence shown in the video. The inswer is B.
Qwen2.5.VLL-78
tunk. The person
Video-Ri-TIB TI-: Let me think about this for moment, The question asking what the person did first and we have two options to consider: A. Closed the clour/cabinet, and B, Por down the food. Fint, let's broik it Jown and mnayyr the focus on
of actions the video. The pemon is interacting with action. person was initially stnddnn the
various objects the kitchen, and refigecator, and then they moved lowards the counter. They picked up bottle and jur and then they started to progace sometiing Houn, secuss like the person distt inseracted with the reftigenator <..>, However, the question specifically about <..>. Oh. The person first intoracted with the refrigerator, and then they counter. S.... Therefore, the first action was selated relligenter.
clear timeline evens.
the
didrtt close the eloscticabinc immediately afler pesson lirs interacted with reliigesttr, whish
the kitchen. They
focused
on the reftgerator implics they put down the food ofter tiking somethnn out puiting somating in the refrigaatoo stor. Thersfore, the corret snswer is
Vidro-VERR-7:: Las me think shout this for moment The parson in the video is seen interacting with various objects in kitchen senting Find, they appioe to be reaching into cabinct close, which megests they are retrioving something Then, they scan be handling bottle, posiibly pouring contants into container. Afher that, they are holding piece food, which they appere be proparing esamining, The quesion anks the penson the options provided are A. Closed the closet cabinet and B. Put down the food. To answer this, I need carefully analyze the sequence actions in the video. liist, the person reaches into cabinet or closet, indicating they are rerriving something. Second, they then handle a botil, which implies they are some purpose. Finally, cabinet or closet restieve somsething. This aligns option A. Hmm, verily The person not put down
and
hokiing puece
which
peoparing esamiting, Givea sequetie, the flist
pesson took reaching any food at the beginning, they were still in the groces of ietrieeing Boo the cabinet. This confinn that the correct miswer A
Question:
show complete motorcycle? A no
Qwe-2.5-VLL-7B: Let me think about this. The image shows a motorcycle, but it appears to be close up shot, focusing on the front part the bike. The front wheel, headlight, and part of the body are visible, but the rear wheel and other parts of the motorcycle are not shown. This suggests that the mighh not show complete motorcycle The inswer is B
Videa-RI-7B.
18: Lei think about
questies
lat' break down, vidaa illuminated bright light. motoreycle is
focus motorcycle,
extetly shown. the video shows motorcycle in gamge, clearly visible, and il appais to a scontcr, gives its devigs and festures. The lighting hig highlight the detnis of the motoreyek, such is the heallights, the bodywork, and the wheels. Howeves, the question is <.... A complete motoreycle would typically include all parts and Reatures that make at tully functional vchicke this vides the motoncycle is shown
impoctant consider
position, and while is ready for use or has all its parts assembled. Wait, lets consider the comext. The video libely a promotional or showcase vićeo for the motorrycle. Such videos ofhen focus on the design and festures of the metoncycle rither than showing in complete store The abomer of any indication of the assembled suggests that
and well-lit, it
that suggests
mosorcycle ready
having
complete motorcycle. S..> Final
Let me think shout this, The video showesscs a sleek, modem motoncycle prominantly displayd in dimly li garage The motoncycle is and black color scheme and intricate design features Lets beeak down the questio and brightly illuminated by spotligtt, highlighnng its vibcant complete motorcycle, video content carefiilly. The question noks the close up of the motorcyole, focusing on its headlight and from design. The camera thes pans avound the motorcycle. showing different angles and dotuils. This suggests that the valuo indaad showing the mottrrycllc from various parapacivves which commion practize when showessing complete vehicle. Nest, the includes wider shot the motorcycle bases background da not detract from the completenss the metoncycle; they simply provide context for setting video daes not away to any other objects scenes that would suggest partial view of the moloncyclc. Instead maintains a focus on the molossycle throughone highlighting its design and festuros in comprihensive manner Gliver this malyysss, is clear that the video doee show complete monoreyele. Amswrr Ai
Video-VV V.R-TR:
this,
consider
the video with
garage, which futther euphasizes
the motoreyele. ganae setting the presence
Question: How many times did the penson launch objects on the sable! A B.3 C.5 D.4
Qwen2.5-V1-72 Let think about this. The person scan launchigg objects on the table the vidao. I can soe that they lave bunchel the objest twiee. The first time, they launch it and bounces off tible. The secend time, they launch aqain and it bounces off tuble well. There are other launches vooble video.
Let me about carefully, The question adking how many times person launched objects on recall sequence of actions the video. First, let' beeak it down: 1. The person Inunched an object, possibly a ball, onto table. 2. The penson then lamnched anocher object, possibly a different ball oc similar obsect, onto the table. New, let' count the launches; The fasst launch was clearly visible. two distine: laucches. <..> Oh,
Video- RI-7B:
answer this,
The second launch was also elearly visible. 11mm,
like there
The
launched objects times. The
question is tsking for <.... Let's beeak down one more time sure. The persos lounched object, then launched anotier object. That's launclies. Aften this reflection, am confident that the cornec answer is tos. Lefs check the optioss A 2... 3 D. The coneet snswer is
objects objects roll scroe the table different dircctions. video captunes the motion and mnjectory of the objects as they are lounched. Let's break down the sequence of events in the video to detormine the nimber of times the person lounched objects
Video- V VRR
think The video shows person interacting with objects
tablle persen their
across the sable The persor launches objects multiple times
places tblle The person fist places small object
sable then their hand launch
first launch. person places thind small object the and launches This
then
object on
the dhidd lnunch. Afher sevicwing the video. is elear that the person lounched objects three limes. The person did not lnunch any objects more than throe times, and there were additional launches beyoad these three. Now, lets consider the options: A. This is incoerect because the person launched objects rimex. D.
the persos lnunched objects three This incorvect bocame the person did not lmunc objocts four times The cornec amswer is B, as the penson burchhe objects siree times,
B.
5
bessuse penson
leunch objects
Figure 14: Comparison of reasoning traces from baseline models and our Video-VEI model reveals that baselines often introduce speculative or hallucinated details not grounded in the video. In contrast, Video VER consistently maintains alignment between reasoning steps and evidence. Text highlighted in red indicates incorrect reasoning steps, while text highlighted in green marks the key logical steps that lead the correct answer.
31

---

Quesiion: How many buttors did you see the person successiully buttoning up and cocrecily aliguing the buttons with die holes? A. B.3 C.5 D.4
Qwen2.5-11-78t. Let think about this. The penson soen humoning up shin, and appears that they are succosfailly aligning the buttons with holes. However, the video only shows few memosts of the battoning pracess. It's not clear how many butions ware actuailyy buttoned up and poriently aligned. The options provided are 2, S, and 4, Without secing the cutire process, it's difficult to desenmine the csact mumber. Hom, the video only shows would option we
being buttoned few moments, Option seems uniikely based the Sootage. Answer &s
break down;
would be
4,
ing about number aligned with the holes. I need recall the details of the video and count the button, Fint,, let's break down. The person was seen buztoning up shirt, and need to count the buttons that were suocessfully aligned. From the video, remember the person buttoning up the shirt in spucific sequence. Whit, let me verif: The person started from the bottocs and worked their
Video-RI-TR: Le think about carefillly The quection
the person susecssfullly buttoned up and concetly
courted the buttons
one. The first button
successfully aligned, the second button
button was aligned,
was also aligned, aligned too. Oh, I see. the pensen successfully buttoned up dive buttons in sotal. <.... That's five buttees in sotal. After this reflection. I am condidens that the person succesffully bationed up live buttons The excnect snswer should he the option that conceponnl five butlons, Lef's vailf lavi time. The passor incorectly,
the
button aligned
dumble-check.
founh
up my
aligned conrecly. Therefore,
Vides-VERR- 7E:-7R: Let me think. The video shows a persor scated at table, weasing maroon bluse with V-ncclline The person is seen bullosing up the blouse, caredtilly aligning ench bution with its corresonninng hole. The process methodicnl with the person using both hands to ensure the buttons are property aligned and secured. The background inclades
the sable, adding contest the setting. The person successfully number buttons person successfully buttoned up and corectly aligned with holes I. The person starts by holding the blouse open, reonaling the humons The penoo then begins to button up the blouse, one button at time The person carefally all gign each button with its corrapording hale and presess it down 4. The person cominuss this proczsi usa bluse is (illy buttoned up. From the video appears that person sucooedully butioned up the blous, and the manhhr of butlons visible the and presed down, ensuring bloss fully buttoned up. careful
and plastic
up the blouse, completing the task precision. Let's
down video step by step decemine
video The aligned each button correctly consideratoon correct answer D
Question: W hich description a more suitable manch for the video? A. The worton closest to the camer mosing at fastes seeed.
R. The woman closest to the camera is moving se the slowrst speed.
Veve2,5-LL-7BB The vadeo shows group of women running on a track. The woman closest to the camera appenes be running at stoudy paes, but there's other
the group also rurning similar pace, there's clear evidence that the woman clesest camera moving the slowest speed. Therefore, the most sutabll manch video
indication that she
apton
Video-RI-7B:
cues video. The woman closest the camera apocars to be running at consisient pace, but there's is no clear indication that she is moving the listest speed. lact, the other runners seem to be maintanning a similar speed, and there's evidence that the woman closest to the camera is lagging behind speeding Now. let's consider the second option, <..>. limm, seems that the woman closest the camera moving the fastest speed: the is moving at similar optior. Final
Let me about
musseat. question asking >..... Fast, lefs beeak dows and cossider
pace other rumners, which aligns with
Video VER-7B: Let me think about this. The video showcases a group of female ahieees participating a track and Beld event, specitically a burcles race. The rumners are soen in vanius stnges of this stride, with their bodics lessing Seward and their amms swinging thy Rylllmically. s... The nunners moving spetial detals. clouast the camera 2... repeness to he slightly ahead of others < Nest, la's consider the temporal detsils. video eapearos the rurner in varions stages of thair stride, and the wonan closest to the cameea not lagging behind s... is maintaining a sicady pass, which is typical ix competiive race <..>, Therefono, the more suitsble match for the video that the wornnn closent the camera moving freter spord Firal answer
pocs,
slightly shead ottherrs suggesting competiiiee
The
First,
Question: la the person in the video holding metal wires? A. yes B.
hands working with tool, there sparks flying. which suggests kind of are manipulating something, but not explicidy s..>. The person seen handling a metal container, which could be relaned the activity but doeen't necessanly indicate <.>> The vidro does not provide clear evidence the person holding metal wires. The activily seems to invelve meniserkingg gig.. but the specific object being monipubnted is not soentifeed as metal wires
Quea2,5.VV Lwen2.5-VL-7BB:
think about this, The video
metalwoeking activity, flowever, the video does clearly
<..h.
Vidlo-RI-7B: Let me think about for moment. question asking 2.... anower this, in the video First, let's break it down. The video shows a person working with some kind soel, and there are sporks flying around. This suggess that the penan likely using tool that involves cutting grinding metal. The person also hulding cylincrical object, which aprears be metal container part the tool New, consider the wires. The wires memioned wines that are clearly visible. The focus on the tool and the sporks, which are indicative metalworking Hmm,, seems that the person not holding metal wires. The wines are not prominent feature in the video, and the person not manipalating any wires. The penson working with a tool that involves am confiden
carefully antlyve the visual information provided
question are explicitly shown
video. The person not holding any
the
the videa
person holding metal
Fanal
think. <... The video elculy slows a persen warking handling wires, and wires uppear metallic based their sheen and the way they being muniplaated The pesson's hands cucfully groppng wires, and there is no indication that the wincs are not mesallic The workshop setting funther supoots, the idea +.... Now, lats consider the options. Option contradicts holding metall wines, which sligne with the visual evidence video. Option suggests that person is not holding metal strngly support
ERR--V
and the question adks the pesson The pesson indeed
suggests that the
The pensnt' detalled analyis of the video and the question, the anover is A.
metallic
Figure 15: Comparison of reasoning traces from baseline models and our Video- VER model reveals that baselines often introduce speculative or hallucinated details not grounded in the video. In contrast, Video- VER consistently maintains alignment between reasoning steps and evidence. Text highlighted in red indicates incorrect reasoning steps, while text highlighted in green marks the key logical steps that lead to the correct answer.
32


Reviews:
[{"review_id": "NPR0BqTmhM", "reviewer": "Reviewer_JHn6", "summary": "\"Visual thinking drift\" is the phenom. where CoT generates misleading internal descriptions and steps which hinder performance in visual understanding tasks. The authors propose a \"visual evidence reward\" RL framework to overcome this.\n\nFirst they motivate this drift problem with an analysis of the success rate of 4 models on 20 video QA tasks with and without using CoT. In many cases, using CoT causes a performance **drop**, demonstrating that visual thinking drift is a real issue. Even with GPT4o, it's better to adopt the direct answer strategy over the CoT strategy for 50% of the video QA benchmarks, contra the strict gain under CoT we observe in the text domain.\n\nTheir analysis centers the idea that a \"latent state\" of each step of a reasoning chain can drift away from the semantics of the vision tokens as the influence of the vision tokens is overwhelmed by influence of prior latent states.\n\nVisual evidence reward is based on this idea: each reasoning step is checked against the visual info using an LM judge to train a policy model. This VER is added to the GRPO reward during RL. Specifically, this judging is performed by comparing the text of the step to Qwen2.5-VL-72B-generated descriptions of the visual input conditioned on the known question for the training example. They train the model using Video-R1-COT-165k for SFT followed by GRPO with VER on Video-R1-260k and Reversed-in-Time.\n\nTheir model achieves significant gains over open baselines, particularly in the COT setting.\n\n(Deleted the first review post because I accidentally submitted a review for a different paper)", "questions": "Revisiting the demonstrations in Figures 2, 3, and 4 which motivate the problem of visual thinking drift, can you report numbers for Video-VER *without* chain-of-thought? If your hypothesis is true, I would expect to see more consistent performance gains with CoT.\n\nNitpick: Consider revising to remove  LM-generated flowerly language, such as: \"As the chef **deftly** chops vegetables...**envisioning the delicious outcome**.\" (bolded) Such superfluous adjectives and editorializing are distracting \n\nBeyond this I found the paper quite easy to understand and have no other significant questions or complaints", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Solid demonstration of the initial phenomenon (visual thinking drift) which their intervention is intended to fix, with analysis of existing models.\n\nSimple but effective fix to this problem using VER.\n\nComprehensive set of baselines evaluated.\n\nI see no glaring reasons to reject.", "was_revised": true, "final_justification": "The authors have slightly increased the quality of the results by adding an ablation on CoT. I think the paper remains a clear Accept.", "rebuttal": "Thank you very much for your valuable comments. We are pleased that you appreciated our **“solid demonstration of the initial phenomenon”**, **“analysis of existing models”**, **“simple but effective fix to this problem”**, and **“comprehensive set of baselines evaluated”**.\n\n---\n\n> **Q1. Results of Video-VER without *chain-of-thought*.**\n\nAs shown in the table below, we evaluated our Video-VER model using two types of prompts: a chain-of-thought (CoT) prompt, which encourages the model to reason step by step before answering, and a direct answer (DA) prompt, which requires only a final answer without intermediate reasoning. Both the base model Qwen2.5-VL and our Video-VER model have 7B parameters.\n\nOverall, as the reviewer anticipated, Video-VER achieves more consistent performance improvements when prompted with CoT, which aligns with its training objective. While DA prompting also yields performance gains over the base model on most benchmarks, the improvements are less consistent and generally smaller than those observed with CoT. These results support our hypothesis that VER rewards in GRPO training encourage grounded, step-by-step reasoning.\n\n\n| Model      | Prompt | MVBench     | Video-MME   | VideoMMMU   | MMVU        | VideoHallucer   |\n|------------|--------|-------------|-------------|-------------|-------------|-------------|\n| Qwen2.5-VL | DA     | 63.6        | 59.2        | 47.3        | 64.2        | 51.8        |\n| Qwen2.5-VL | COT    | 59.8        | 54.7        | 47.8        | 60.5        | 44.1        |\n| Video-VER  | DA     | 63.6 (+3.8) | 57.5 (+2.8) | 48.6 (+0.8) | 64.5 (+4.0) | 51.9 (+7.8) |\n| Video-VER  | COT    | 64.1 (+4.3) | 59.3 (+4.6) | 52.7 (+4.9) | 65.1 (+4.6) | 53.1 (+9.0) |\n\n| Model      | Prompt | EventHallusion   | VSI-Bench   | TempCompass      | TVBench     | Vinoground      |\n|------------|--------|-------------|-------------|-------------|-------------|-------------|\n| Qwen2.5-VL | DA     | 64.5        | 32.3        | 73.7        | 52.2        | 14.8        |\n| Qwen2.5-VL | COT    | 67.3        | 31.4        | 71.3        | 49.9        | 12.8        |\n| Video-VER  | DA     | 65.5 (-1.8) | 32.7 (+1.3) | 72.9 (+1.6) | 52.6 (+2.7) | 13.8 (+1.0) |\n| Video-VER  | COT    | 70.0 (+2.7) | 34.6 (+3.2) | 74.0 (+2.7) | 52.8 (+2.9) | 14.4 (+1.6) |\n\n> **Q2. Writing nitpick.**\n\nThank you for the insightful suggestion! We will revise the text to remove superfluous adjectives."}, {"review_id": "ZnXMEOZnWb", "reviewer": "Reviewer_Z3Pb", "summary": "This paper presents a systematic analysis revealing that CoT often degrades performance in video reasoning, which is related to a phenomenon called \"visual thinking drift.\" Then, the authors introduce a reinforcement framework, “Visual Evidence Reward (VER)”, to explicitly rewards the generation of reasoning traces with visual evidences. 10 diverse video understanding benchmarks demonstrates the Video-VER achieves top performance compared to other baseline models. This work pointed out the importance of visual grounding in the thinking process of multimodal models, and may encourage the development of large models in robust visual grounding-based rational thinking process.", "questions": "- If visual evidence reward can help improve model performance, which performs better between using agentic workflow and end-to-end training? And what is the foundational reason to lead to the difference?\n- If the large model can control the behavior in visual space, will there also be hallucinations? Is it consistent with the cause of visual perception illusion?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths：\n- Present a phenomenon called visual thinking drift, which is consistent with the well-known AI hallucination problem. This paper further analyze this problem from the perspective of visual grounding, and then propose corresponding RL solutions. \n- Conduct extensive experiments to show that Video-VER achieves better performance. Also, the ablation study provides more useful information to show the effects on different models types and model sizes.\n\nWeakness:\n- Maybe more reasoning models should be added as baselines, such as OpenAI o1/o3.", "was_revised": true, "final_justification": "The author's response resolved most of the issues and the scores were appropriately adjusted.", "rebuttal": "Thank you for the valuable feedback. We appreciate the acknowledgment of our **analysis of the visual thinking drift phenomenon**, the **further exploration from a visual grounding perspective**, and the **proposed reinforcement learning solutions**, as well as the **extensive experiments** and **ablation study**.\n\n---\n\nTo respond to your questions:\n\n> **Q1: “Maybe more reasoning models should be added as baselines, such as OpenAI o1/o3.”**\n\nThank you for the suggestion. As shown in Table 1, our baselines span the full spectrum of video-LLM capabilities: proprietary (GPT-4o) and open-source; generalist (LongVA, Video-UTR, LLaVA-OneVision, Kangaroo, Qwen2.5-VL) and specialist video-reasoning models (TinyLLaVA-Video-R1 and Video-R1). Nonetheless, we appreciate the reviewer’s interest in advanced proprietary reasoning models such as OpenAI’s o1/o3. To address this, we evaluated o3 (released April 2025) under both direct-answer (DA) and chain-of-thought (CoT) prompting across all ten video benchmarks. As the table below shows, CoT prompting induces a consistent performance drop—despite o3’s strong reasoning capabilities—**mirroring the trends in Figure 2 and reinforcing our conclusion that CoT does not reliably boost video-LLM performance**. \n| Prompt | MVBench | Video-MME | VideoMMMU | MMVU  | VideoHal. | EventHal. | VSI-Bench | TempC. | TVBench | Vinog. |\n|--------|---------|-----------|-----------|-------|----------------|----------------|-----------|--------------|---------|------------|\n| DA     | 69.7    | 74.1      | 77.2      | 80.5  | 66.2           | 82.9           | 49.3      | 83.4         | 65.9    | 52.8       |\n| COT    | 66.4    | 71.0      | 71.0      | 78.9  | 63.8           | 82.1           | 37.7      | 81.3         | 62.1    | 36.6       |\n\n(Note: OpenAI’s “Hiding the Chains of Thought” policy prevents o3 from exposing its internal reasoning; in some CoT runs the model interjects messages like “Sorry, I can’t share my private reasoning,” which may contribute to the accuracy loss.)\n\n> **Q2: Which performs better between using agentic workflow and end-to-end training?**\n\nThe performance of agentic workflow approaches is often constrained by the capabilities of their component visual specialist models (e.g., key-frame selectors or object detectors). In contrast, end-to-end training leverages a single multimodal LLM trained on large-scale vision-language data, with the vision encoder handling perception and the language model handling reasoning—enabling a more unified approach to video understanding. Our VER model follows this end-to-end paradigm.\n\nAs a reference, we include additional evaluations of recent agentic workflow methods [64, 53, 41], which approach video understanding by decomposing the task into sub-problems—such as key-frame selection or temporal grounding—and addressing each with specialized models. Across benchmarks, our 7B Video-VER model demonstrates superior overall performance compared to these baselines.\n\n|   Category     |         Model           | NExT-QA | EgoSchema |\n|-----------------------------|---------|-----------|------------------|\n| Agentic Workflow | SeViLA [64], NeurIPS 2023   | 73.8    | -         |\n| Agentic Workflow | VideoAgent [53], ECCV 2024  | 71.3    | 54.1      |\n| Agentic Workflow | MoReVQA [41], CVPR 2024     | 69.2    | 51.7      |\n| End-to-End Training         |   Video-VER (Ours)      | 81.0    | 66.2      | \n\nNonetheless, agentic workflows offer some flexibility at test time. Their ability to decompose tasks and incorporate external visual models may provide advantages in specific scenarios.\n\n> **Q3: If the large model can control the behavior in visual space, will there also be hallucinations? Is it consistent with the cause of visual perception illusion?**\n\nThank you for the thoughtful question. If a large model could directly manage visual information—for example, by maintaining spatially accurate representations or consistently referring to visual evidence throughout the reasoning process—it would likely help reduce hallucinations. However, hallucinations can still arise due to inherent challenges in video perception, such as low visual resolution, ambiguous scenes, or limited temporal grounding.\n\nAs discussed in Section 3.2, current video language models (video-LLMs) often rely heavily on language-based priors because visual signals tend to fade or weaken during the model’s step-by-step reasoning process (autoregressive decoding). As a result, the model may generate outputs that are only loosely grounded in the actual video content.\n\nThis is where our approach becomes *especially important*. A key focus of our work is to ensure that the model's reasoning actively refers back to visual evidence at intermediate steps. By reinforcing this connection between reasoning and visual input, we aim to reduce hallucinations and improve the model’s alignment with the video. This aligns with our hypothesis that \"visual thinking drift\" occurs when reasoning becomes unanchored from the visual content, leading the model to fabricate visual details due to insufficient fidelity and control over the visual stream."}, {"review_id": "kxBlFJkAct", "reviewer": "Reviewer_9tCR", "summary": "This paper addresses the issue where Chain-of-Thought (CoT) often degrades performance in video reasoning tasks because the model overlooks crucial visual content during the reasoning process. To address this, the paper introduces a new reward mechanism, called Visual Evidence Reward (VER), which penalizes the model when it fails to infer to actual visual details. Specifically, ground truth evidence is generated by prompting the Qwen2.5-VL-72B model with both a video and a question, and an auxiliary LLM compares this ground truth evidence with the policy model’s response to provide a binary signal indicating whether the two are aligned. This VER is integrated into the GRPO framework. Experimental results on various video reasoning benchmarks show improvements over the baseline.", "questions": "Unfair Comparison Setup [W1] is the most significant issue affecting my decision. As it stands, I don't believe readers can accurately assess the experimental impact of the proposed method.\n\nQ1. See [W1]\n\nQ2. See [W2]\n\nQ3. **Regarding Evidence Generation as Distillation for VideoQA**: The paper uses a significantly larger model (Qwen2.5-VL-72B) to generate the visual evidence that guides the training of Video-VER-7B. Could the authors discuss this process from the perspective of knowledge distillation – where the 7B model is learning to better ground its VideoQA reasoning by aligning with the evidence identified by the 72B model? Is the 72B model expected to perform clearly better overall due to its size, or does the VER training enable the smaller 7B model to achieve comparable or even superior performance in VideoQA reasoning on the target benchmarks?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "## Strengths\n\nS1. **Interesting and Sound Motivation**: Section 3 presents interesting and meaningful take-away to readers that CoT does not exactly help VideoLLMs to perform temporal reasoning because it actually ignores videos.\n\nS2. **Clear Presentation:** The paper is well-written and easy to follow, with clear explanations that make the context easy to understand.\n\n---\n## Weaknesses\n\nWhile this paper tackles an interesting problem, I have major concerns regarding the method and experimental validations:\n\nW1. **Unfair Comparison Setup:** To provide a fair evaluation, there should be an ablation study comparing the following setups: baseline+SFT, baseline+SFT+GRPO, and baseline+SFT+GRPO+VER. In the current experimental setup, it is unclear whether the observed gains are due to GRPO or VER, making it difficult for readers to assess the individual contributions.\n\nW2. **Reliability of Visual Evidence Source**: Figure 4 indicates that even strong reasoning models, such as GPT-4o, can disregard visual facts. However, the paper then uses Qwen2.5-VL-72B to generate the visual evidence that grounds the training. This raises natural questions about how Qwen2.5-VL-72B is free from the same issue of potentially overlooking or hallucinating visual information when generating this critical evidence.", "was_revised": true, "final_justification": "My major initial concern was unfair comparison setup, and this was well addressed in the authors' rebuttal. Comparing individual gains indeed have shown that VER can present unique gains compared to naive GRPO training. Other remaining concerns were about whether the proposed method ties the learning to the teacher model or can learn beyond their visual evidence. This was also well addressed in the rebuttal phase. Therefore, with these major concerns resolved, I am inclined to revise my rating to 4: borderline accept.", "rebuttal": "Thank you for appreciating the **interesting and well-founded insight into the “visual thinking drift” phenomenon** presented in our work, as well as our overall **clear presentation**.\n\n---\n\nWe understand that the reviewer's main concern lies in the *less well-explained cause of the observed accuracy improvements* in our experimental results. To address this, we provide the following response.\n\n> **Q1. Comparison Setup: Is the gain from GRPO or VER? – “most significant issue affecting my decision”**\n\nThanks for the thoughtful feedback! We agree that providing a clearer breakdown of the contributions from each model component is important for evaluating the effectiveness of our approach. In response, we present detailed ablation results below that compare the requested configurations: Baseline + SFT, Baseline + SFT + GRPO, and Baseline + SFT + GRPO + VER (full model). Baseline here is Qwen2.5-VL-7B model.\n\nOur findings show that fine-tuning the baseline model with SFT alone does not consistently improve performance. Adding GRPO yields moderate gains across most benchmarks. Notably, incorporating our Visual Evidence Reward (VER) on top of GRPO leads to substantial additional improvements. All performance deltas are reported relative to the original baseline model.\n\nThese results help isolate the contribution of each component and demonstrate that the gains observed in our full model are primarily driven by VER, validating its effectiveness in promoting grounded and accurate reasoning.\n\n| Method                   | MVBench     | Video-MME   | VideoMMMU   | MMVU        | VideoHallucer |\n|--------------------------|-------------|-------------|-------------|-------------|----------------|\n| Baseline                     | 59.8        | 54.7        | 47.8        | 60.5        | 44.1           |\n| Baseline + SFT              | 60.5 (+0.7) | 55.4 (+0.7) | 48.2 (+0.4) | 63.5 (+3.0) | 46.9 (+2.8)    |\n| Baseline + SFT + GRPO       | 61.2 (+1.4) | 54.8 (+0.1) | 49.6 (+1.8) | 63.4 (+2.9) | 47.8 (+3.7)    |\n| Baseline + SFT + GRPO + VER | 64.1 (+4.3) | 59.3 (+4.6) | 52.7 (+4.9) | 65.1 (+4.6) | 53.1 (+9.0)    |\n\n| Method                   | EventHallusion | VSI-Bench   | TempCompass | TVBench     | Vinoground  |\n|--------------------------|----------------|-------------|--------------|-------------|-------------|\n| Baseline                     | 67.3           | 31.4        | 71.3         | 49.9        | 12.8        |\n| Baseline + SFT              | 65.5 (-1.8)    | 32.7 (+1.3) | 68.8 (-2.5)  | 50.0 (+0.1) | 10.4 (-2.4) |\n| Baseline + SFT + GRPO       | 67.5 (+0.2)    | 33.2 (+1.8) | 69.7 (-1.6)  | 51.3 (+1.4) | 12.6 (-0.2) |\n| Baseline + SFT + GRPO + VER | 70.0 (+2.7)    | 34.6 (+3.2) | 74.0 (+2.7)  | 52.8 (+2.9) | 14.4 (+1.6) |\n\n\n> **Q2. Reliability of Visual Evidence Source** \n\nWe understand that the reviewer's concern lies in the reliability of Qwen2.5-VL-72B for generating visual evidence, particularly given its potential to overlook or hallucinate visual details—an issue also observed in models like GPT-4o. We address this concern in Supplementary Section E. While large models can indeed hallucinate during visual reasoning, our approach is specifically designed to mitigate this issue. We employ several mechanisms to ensure that the generated visual evidence is grounded and visually anchored:\n\n- Offline Use of Teacher Model (supp L381-384): The external 72B MLLM is used only during training to generate question-specific visual evidence. Our Video-VER model learns from this data but operates independently at inference time, ensuring that any potential errors from the teacher model are not propagated or relied upon during deployment.\n\n- Binary Reward Signal (supp L385-391): Instead of requiring exhaustive annotations, we use a binary reward that simply checks whether the generated reasoning includes any verifiable visual fact. If the evidence is missing or incorrect, the model receives a zero reward. This effectively discourages hallucinations without reinforcing incorrect details.\n\n- Inverted Prompting Strategy (supp L392-410): Unlike standard chain-of-thought prompting, we condition the 72B model on both the question and the correct answer, prompting it to produce concise, verifiable visual facts that support the answer. This tightly constrains generation, encouraging relevance and factual grounding over generic or speculative content.\n\nTo demonstrate the effectiveness of our visual evidence extraction method, we provide an ablation study comparing the use and non-use of our Inverted Prompting Strategy (IPS) in generating visual evidence. The results show that IPS significantly improves the quality of visual evidence, which in turn enhances the overall performance of VER across all benchmarks.\nIn conclusion, through these design choices, our model is trained to prioritize visual grounding over reliance on linguistic priors, directly addressing the root cause of “visual thinking drift.”\n\n| Model          | MVBench | Video-MME | VideoMMMU | MMVU  | VideoHal. | EventHal. | VSI-Bench | TempC. | TVBench | Vinog. |\n|----------------|---------|-----------|-----------|-------|----------------|----------------|-----------|--------------|---------|------------|\n| Qwen2.5-VL        | 59.8    | 54.7      | 47.8      | 60.5  | 44.1           | 67.3           | 31.4      | 71.3         | 49.9    | 12.8       |\n| Ours (w/o IPS) | 62.7    | 57.5      | 50.9      | 64.6  | 51.9           | 69.0           | 33.2      | 72.9         | 51.9    | 13.6       |\n| Ours (w/ IPS)  | 64.1    | 59.3      | 52.7      | 65.1  | 53.1           | 70.0           | 34.6      | 74.0         | 52.8    | 14.4       |\n\n\n\n> **Q3. Regarding Evidence Generation as Distillation for VideoQA** \n\nThank you for the insightful question! Distillation is an interesting interpretation, and indeed the larger 72B model aims to ensure the most reliable visual evidence signal as possible.  Nonetheless, our method goes well beyond standard knowledge distillation paradigms. Specifically, our model is not trained to mimic the final outputs or answer distributions of the 72B model through supervised fine-tuning (SFT). Instead, the 72B model is used once, offline, to generate minimal, question-specific visual evidence that supports correct answers. This evidence is then used to define a binary, fact-level reward signal that guides the training of the 7B Video-VER model via reinforcement learning.\n\nThis setup differs fundamentally from conventional knowledge distillation in several key ways. **First**, there is no direct imitation of the teacher’s answers. The 7B model is not learning to reproduce the 72B model’s outputs or full reasoning traces. It is rewarded only when its own reasoning includes verifiable visual facts aligned with the evidence. **Second**, the supervision is task-specific and grounded. The visual evidence highlights what matters visually in the context of each question, teaching the model how to reason rather than simply what to predict. **Third**, the 72B model’s role is limited to only helping the smaller model identify and attend to relevant visual cues. There is nothing more.\n\nAs a result, the 7B Video-VER model learns to ground its answers in concrete visual evidence, enhancing both interpretability and robustness. In short, our method uses the 72B model as a generator of visual signals that shape the reward function, enabling the smaller model to learn how to reason visually. This distinction is central to the effectiveness of our VER training approach."}, {"review_id": "jKVrNKW0Q1", "reviewer": "Reviewer_1B4M", "summary": "This paper investigates the limitations of Chain-of-Thought (CoT) prompting in video reasoning tasks for multimodal large language models (MLLMs). The authors identify a critical issue termed \"visual thinking drift,\" where CoT-generated reasoning chains diverge from actual visual evidence, leading to hallucinations and degraded performance. They propose a Bayesian-inspired explanation for this phenomenon, attributing it to the amplification of language priors over visual signals during autoregressive decoding. To mitigate this, the authors introduce Visual Evidence Reward (VER), a reinforcement learning framework that explicitly rewards reasoning traces grounded in verifiable visual evidence. Extensive experiments across 10 video understanding benchmarks demonstrate that VER consistently improves accuracy over baseline models.", "questions": "1.  How does Video-VER perform on videos exceeding 32 frames? While the ablation study (Table 3) demonstrates improved performance with increased frame counts, real-world applications often involve much longer sequences.\n2. How does Video-VER compare to modular reasoning approaches (e.g., [46] or [38]) that explicitly decompose video understanding into subtasks like action recognition or temporal grounding?", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "### Strengths\n1. The concept of *visual thinking drift* is an interesting and underexplored challenge in video reasoning. The paper provides empirical findings (Figure 2) and theoretical insights (Bayesian analysis) into such problem.\n2. The experiments span 10 diverse benchmarks, demonstrating robustness across tasks like temporal reasoning, object counting, and hallucination mitigation.\n\n### Weaknesses\n1. The Bayesian analysis in Section 3.2 attributes *visual thinking drift* to the dominance of language priors over visual evidence during autoregressive decoding. However, this explanation is overly generic and fails to address why CoT prompting works relatively better for text-based LLMs (where \"prefill context\" acts as the analog of v) compared to video reasoning tasks. This suggests that the root cause of drift may be more nuanced and specific to video reasoning challenges. The authors should explicitly investigate why CoT fails for video tasks even when the same Bayesian mechanism applies to both modalities.\n2. The experiments in this paper exclusively use `Qwen2.5-VL-7B` as the base model for evaluating the Video-VER framework. While the method demonstrates improvements on this specific architecture, the lack of testing across diverse model architectures (e.g., LongVA, LLaVA-OneVision) or parameter scales (e.g., smaller or larger models) raises concerns about the generalizability of Video-VER.\n3. The generation of visual evidence relies on a strong external MLLM (`Qwen2.5-VL-72B`), which could introduce biases or hallucinations itself. While the paper mentions filtering, the robustness of this process is not fully validated.", "was_revised": true, "final_justification": "The authors have addressed my major concerns. In light of these improvements, I have revised my rating accordingly and now recommend an acceptance.", "rebuttal": "Thank you for the valuable feedback. We are happy that you appreciated our identification of the **“visual thinking drift” phenomenon as an interesting and underexplored challenge**, as well as our **empirical findings**, the **theoretical insights**, and **the breadth of our experimental evaluation across diverse tasks**.\n\n---\n\nTo address your concerns, we provide the following response:\n\n> **Q1. “explicitly investigate why CoT fails for video tasks even when the same Bayesian mechanism applies to both modalities”**\n\nThank you for the insightful comment! The “visual thinking drift” phenomenon is indeed specific to video reasoning and deserves clearer explanation. \n\nIn L168, we note that language priors outweigh visual likelihoods in practice, but did not elaborate further. In today’s video-LLMs, the language-projection matrix  $W_{lang}$ tends to have a much larger norm than its visual counterpart $W_{vis}$. This is because the model inherits a massive, pre-trained language backbone trained on trillions of text tokens, while the vision pathway is typically a small, newly added adapter, fine-tuned on multimodal data that is orders of magnitude smaller. Moreover, during inference, far fewer visual tokens influence gradient flow compared to language tokens. This asymmetry in initialization, data scale, and gradient dynamics leads to a significantly smaller Frobenius norm in $W_{vis}$, causing language priors to dominate during autoregressive decoding and contributing to the observed drift.  \n\nAnother important point is the modality mismatch in visual reasoning. In text tasks, the question, answer, and reasoning all happen in the same modality—language—so the model \"thinks\" in the same space as it reads and answers. But in video tasks, the model must reason about visual content using language as a conduit. This indirect reasoning makes it easier for the model to drift away from the actual video, especially when visual grounding is weak.\n\nWe will update this explanation in the text accordingly.\n\n> **Q2. Generalization test across diverse model architectures (e.g., LongVA, LLaVA-OneVision) or parameter scales**\n\nThank you for the constructive feedback. As noted in [1], LLaVA struggles to generate high-quality, coherent reasoning paths—an observation consistent with our own findings (noted in the footnote under L145). Per the reviewer’s suggestion, we also tested LongVA and observed a similar issue. We conjecture that this limitation stems from the use of a weaker underlying language model (Qwen2), which appears insufficient for reliably supporting structured reasoning prompts.\n\nTo our knowledge, Qwen2.5-VL is currently the most stable open-source model for following structured prompts and generating reasoning traces, as supported by recent literature [17, 32] and frequently used open-source projects [2, 3]. This capability is crucial for applying RL-based fine-tuning, which relies on consistent intermediate reasoning.\n\nThat said, we appreciate the reviewer’s concern regarding the generalizability of Video-VER. To address this, we conducted additional experiments using the Qwen2.5-VL base model with different parameter scales, 3B. As shown in the table below, our VER method still provides consistent accuracy improvements over the 3B model, demonstrating that our approach generalizes well across model scales.\n\n| Model            | MVBench | Video-MME | VideoMMMU | MMVU  | VideoHal. | EventHal. | VSI-Bench | TempC. | TVBench | Vinog. |\n|------------------|---------|-----------|-----------|-------|----------------|----------------|-----------|--------------|---------|------------|\n| Qwen2.5-VL-3B          | 57.4    | 53.1      | 33.8      | 53.6  | 40.4           | 49.9           | 25.4      | 59.8         | 47.2    | 5.8        |\n| Qwen2.5-VL-3B+VER | 62.7    | 54.5      | 42.3      | 58.2  | 45.6           | 55.7           | 30.3      | 63.4         | 48.2    | 6.8        |\n\n[1] Yang, L., et al. (2025). GRPO-for-Llava. GitHub repository. \n\n[2] Wang, X., et al. (2025). Open-R1-Video. GitHub repository.\n\n[3] Feng, K., et al. (2025). Video-R1. GitHub repository.\n\n> **Q3. Potential hallucination introduced by the 72B visual evidence generator**\n\nWe understand that the reviewer's concern lies in the potential biases or hallucinations introduced by the external Qwen2.5-VL-72B as a visual evidence generator. We address this concern in Supplementary Section E. While large models can indeed hallucinate during visual reasoning, our approach is specifically designed to mitigate this issue. We employ several mechanisms to ensure that the generated visual evidence is grounded and visually anchored:\n\n- Offline Use of Teacher Model (supp L381-384): The external 72B MLLM is used only during training to generate question-specific visual evidence. Our Video-VER model learns from this data but operates independently at inference time, ensuring that any potential errors from the teacher model are not propagated or relied upon during deployment.\n\n- Binary Reward Signal (supp L385-391): Instead of requiring exhaustive annotations, we use a binary reward that simply checks whether the generated reasoning includes any verifiable visual fact. If the evidence is missing or incorrect, the model receives a zero reward. This effectively discourages hallucinations without reinforcing incorrect details.\n\n- Inverted Prompting Strategy (supp L392-410): Unlike standard chain-of-thought prompting, we condition the 72B model on both the question and the correct answer, prompting it to produce concise, verifiable visual facts that support the answer. This tightly constrains generation, encouraging relevance and factual grounding over generic or speculative content.\n\nTo validate the robustness of our visual evidence extraction method, we provide an ablation study comparing the use and non-use of our Inverted Prompting Strategy (IPS) in generating visual evidence. The results show that IPS significantly improves the quality of visual evidence, which in turn enhances the overall performance of VER across all benchmarks.\n\nIn conclusion, through these design choices, our model is trained to prioritize visual grounding over reliance on linguistic priors, directly addressing the root cause of “visual thinking drift.”\n\n\n| Model          | MVBench | Video-MME | VideoMMMU | MMVU  | VideoHal. | EventHal. | VSI-Bench | TempC. | TVBench | Vinog. |\n|----------------|---------|-----------|-----------|-------|----------------|----------------|-----------|--------------|---------|------------|\n| Qwen2.5-VL        | 59.8    | 54.7      | 47.8      | 60.5  | 44.1           | 67.3           | 31.4      | 71.3         | 49.9    | 12.8       |\n| Ours (w/o IPS) | 62.7    | 57.5      | 50.9      | 64.6  | 51.9           | 69.0           | 33.2      | 72.9         | 51.9    | 13.6       |\n| Ours (w/ IPS)  | 64.1    | 59.3      | 52.7      | 65.1  | 53.1           | 70.0           | 34.6      | 74.0         | 52.8    | 14.4       |\n\n\n> **Q4. Performance on videos exceeding 32 frames**\n\nWe appreciate the reviewer’s interest in evaluating our method on longer video sequences. While our main experiments (Table 3) focus on input lengths of 8, 16, and 32 frames, we have conducted additional experiments with 64-frame inputs to assess scalability. These results, presented in the table below, show that our method effectively leverages extended temporal context and continues to perform well, demonstrating strong scalability and generalization to longer sequences.\n\nDue to computational constraints, we are limited to evaluating up to 64 frames. However, we agree that handling longer videos is crucial for real-world applications. As highlighted in our failure case analysis (Figure 14), missing key visual cues due to limited sampling or encoding can impair reasoning. Thus, the VER mechanism not only enhances visual reasoning but also stands to benefit from continued advances in long-context video modeling. \n| #Frames   | MVBench | Video-MME | VideoMMMU | MMVU  | VideoHal. | EventHal. | VSI-Bench | TempC. | TVBench | Vinog. |\n|-----------|---------|-----------|-----------|-------|----------------|----------------|-----------|--------------|---------|------------|\n| 32 | 64.1    | 59.3      | 52.7      | 65.1  | 53.1           | 70.0           | 34.6      | 74.0         | 52.8    | 14.4       |\n| 64 | 65.0    | 61.7      | 52.7      | 65.6  | 53.5           | 71.5           | 36.1      | 74.1         | 53.3    | 15.4       |\n\n\n> **Q5. Comparison with [46] or [38]**\n\nAs noted in L38–39, both [46] and [38] focus on curating extensive spatio-temporal annotations to support supervised fine-tuning (SFT), where the model learns from human-labeled examples for specific subtasks. In contrast, our work centers on reinforcement fine-tuning (RFT), which directly optimizes visually grounded reasoning through an RL reward signal. This approach enables the model to learn end-to-end reasoning without relying on handcrafted subtask-specific annotations.\n\nWe have included additional evaluations of [46] and [38] across four benchmarks. For both baselines, we report their best published results. Our Video-VER model consistently outperforms these methods, suggesting that our visually grounded GRPO framework offers a competitive—and potentially more generalizable—alternative to these modular, annotation-heavy approaches.\n\n| Method            | MVBench | VSI-Bench | NExT-QA | Video-MME |\n|-------------------|---------|-----------|---------|-----------|\n| VITED [38]        | 61.0    | -         | 73.4    | -         |\n| AoTD [46]         | 55.6    | 28.8      | 77.6    | 45.0      |\n| Video-VER (Ours)  | 64.1    | 34.6      | 81.0    | 59.3      |"}]

Author rebuttal:

Meta-review: This paper studies visual reasoning in multimodal large language models (MLLMs). The key problem and motivation is that Chain-of-Thought (CoT) mechanism in video understanding remains underexplored. This work found a visual thinking drift phenomenon, CoT often degrading performance in video reasoning and leading to hallucinated visual details, and further explained the visual thinking drift via a Bayesian lens. A reinforcement learning framework Visual Evidence Reward (VER) is further proposed to explicitly reward reasoning steps aligned with verifiable visual evidence.

The main strengths are that (1) the visual thinking drift problem is novel, (2) the theoretical and empirical analyses are insightful, (3) the proposed visual evidence reward is simple but effective, (4) the evaluation on 10 benchmarks is comprehensive. The main weaknesses or concerns were that (1) comparison with more reasoning models are to be included, (2) a clearer ablation setup to evaluate VER, (3) reliability conresponding to Qwen2.5-VL-72B usage. After rebuttal, the concerns have been addressed and all the final ratings are positive. AC agree with reviewers and recommend accepting this paper.
Decision: Accept (poster)