VideoMind: A Chain-of-LoRA Agent for Temporal-Grounded Video Reasoning
Authors: Ye Liu
Kevin Qinghong Lin
Chang Wen Chen
Mike Zheng Shou
Venue: ICLR 2026 Poster (2026)
Keywords: Multi-modal Agent
Video Understanding
Video Temporal Grounding
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: An agentic solution for long video understanding and video temporal grounding through LoRA switching.

Abstract: Videos, with their unique temporal dimension, demand precise grounded understanding, where answers are directly linked to visual, interpretable evidence. Despite significant breakthroughs in text-based reasoning with large language models, multi-modal reasoning - especially for videos - remains limited. In this work, we fill this gap by introducing VideoMind, a novel video-language agent for temporal-grounded video reasoning. Our method involves two key innovations: (1) We identify four essential capabilities for grounded video reasoning and propose a role-based agentic workflow, comprising a planner to coordinate roles, a grounder for temporal event localization, a verifier to assess event candidates, and an answerer for question answering. (2) To efficiently integrate these roles during inference, we propose a novel Chain-of-LoRA mechanism, where a unified base model with multiple LoRA adapters is leveraged to enable seamless role switching, balancing efficiency and flexibility. Extensive experiments on 15 benchmarks across Grounded VideoQA, Video Temporal Grounding, and General VideoQA tasks demonstrate the effectiveness of the proposed scheme in advancing video agent, test-time scaling, and long-form video reasoning. Code, models, datasets, and demos are available at https://videomind.github.io/.

Published as a conference paper at ICLR 2026
VIDEOMIND: A CHAIN-OF-LORA AGENT FOR TEMPORAL-GROUNDED VIDEO REASONING
Ye Liu'*, Kevin Qinghong Lin²*, Chang Wen Chen 1s, Mike Zheng Shou ts
The Hong Kong Polytechnic University NNational University of Singapore coco. ye. liu@connect...... wow.eechttoolu......
ABSTRACT
Videos, with their unique temporal dimension, demand precise grounded under- standing, where answers are directly linked to visual, interpretable evidence. De- spite significant breakthroughs in text-based reasoning with large language mod- els, multi-modal reasoning - especially for videos remains limited. In this work, we fill this gap by introducing VideoMind, novel video-language agent for video reasoning. Our method involves two key innovations: (1) We identify four essential capabilities for grounded video reasoning and pro- pose role-based agentic workflow, comprising p anner to coordinate roles, a grounder for temporal event localization, veri fi er to assess event candi- dates, and an answerer for question answering. (2) To efficiently integrate these roles during inference, we propose novel Chain-of-LoRA mechanism, where a unified base model with multiple LoRA adapters is leveraged to enable seamless role switching. balancing efficiency and flexibility. Extensive experiments on 15 benchmarks across Grounded VideoQA, Video Temporal Grounding, and General VideoQA tasks demonstrate the effectiveness of the proposed scheme advane- ing video agent, test-time scaling, and long-form video reasoning. Code, models, datasets, and demos are available at ht https: htttp://viddeming //wideomiiddd.cccomom github. io/.
remporal-groundee
INTRODUCTION
Recent advancements in large language models (LLMs) have demonstrated remarkable success in text-based reasoning (Wei et al., 2022; Yao al., 2023a; Shinn et al., 2023), significantly improving both accuracy and interpretability in complex problem-solvine scenarios (Yao et al., 2023b). Fol- lowing these breakthroughs, efforts have been devoted extending these reasoning capabilities to multi-modal domains (Seed, 2026; Xu et al., 2025; Thawakar et al., 2025) such as vision-centric science (Lu al., 2022) and math (Ma al., 2025) understanding.
Among multi molal signals, videos pose a unique challenge due to their temporal dimension, in- troducing complexities absent in images or text. Effective video reasoning requires not only rec- ognizing visual appearances but also understanding how they evolve over time (Xiao et al., 2024; Chen et al., 2024a; Liu et al., 2024e; Wu al., 2025). While recent visual Chain-of-Thouglee (CoT) methods (Zhang et al., 2023e; Xu et al., 2025; Thawakar et al., 2025) excel generating detailed thoughts for static images, they struggle with long videos as they cannot explicitly localize or revisit earlier parts of the sequence, as presented in Figure/ 1 (left). Humans, by contrast, can reason long videos with ease (Xie et al., 2025b; 2024b; 2022): they break down complex problems, identify relevant moments, revisit them confirm details, and synthesize their observations into coherent an- swers. This natural proficiency motivates the development of an AI agent that emulates this process flexibly coordinating multiple capabilities to achieve advanced, vision-centrie reasoning.
In this work, we introduce VideoMind, a video-language agent with enhanced temporal-grounded reasoning capabilities. To meet the demands of diverse tasks, we define four essential roles for un- derstanding complex long-form videos: (1) a planner to decompose tasks and coordinate other roles, (2) a grounder for precise moment localization, (3) verifier for moment candidates assessment, and (4) an answerer for moment-aware response generation. Each role is carefully
"Equal contribution. "Corresponding authors.

---

Published as a conference paper at ICLR 2026
questiin related "bunnies gathering it ancwect the guactton
higher spatial watching contident Ive figured out answez.
Why are the busstey
Pataee
Ansonncer
pethining
Action Plan
Tannstomps:
Confdencee:
Final Answec
Sally
Hone
0.20
Kos Mowwraly hecting artidd
LaBA
LARA
0.66
Rase
30 asrwer ali quantes aced to cuumtrrche of ando and Vicioo
I'm localizing the moment about "sunnies gathering the table" Here coadidate conteining
There are four candidate moments. Let verify them one by by--om, focusing the temporal boundaries semantic similantties.
Gioundas
Varifir
Text-boeed GoT
VideoMind: Multi-madal Chain-of-LoRA Receoning
Figure I: Illustration Video Chain-of-LoRd reasoning mechanism. The problem is de- composed by the planner and distributed to the grounder, verifier, and answerer systematically localize, verify, and interpret the relevant video moments.
designed to deliver strong performance, for example, the grounder is equipped with timestamp decoder to ensure accurate temporal grounding. To enable efficient integration of these roles, we also propose novel Chain-of-LoRA mechanism, where all the roles are implemented based unified I.MM backbone with role-specific LoRA adapters (Hu et al., 2022). Therefore, role-specific capabilities be trained separately on tailored datasets. During inference, all the LoRA param- eters are cached into the memory, so that each role could be activated by simply switching to the corresponding LoRA, as shown in Figure (right). This approach reflects minimalist yet flexible design philosophy, facilitating seamless transitions and interactions among roles without incurring the memory overhead of maintaining multiple full models. As result, VideoMind achieves both efficiency and flexibility on diverse video understanding tasks.
We conduct extensive experiments on 15 public benchmarks, including on Grounded VideoQA, 6 on Video Temporal Grounding, and on General VideoQA, to evaluate the effectiveness of our approach. VideoMind exhibits strong adaptability in addressing diverse reasoning tasks by jointly providing accurate responses and temporal-groundee evidence. Notably, our 2B model surpasses GPT-4o (OpenAI, 2024a) and Gemini-1.5-Pre (Reid et al., 2024) on several long video benchmarks such as CG-Bench (Chen et al., 2024a), MLVU (Zhou et al., 2024), and LVBench (Wang et al., 2024c). State-of-the-ane art performance is also achieved on temporal grounding datasets including QVHighlights (Lei et al., 2021) and Charades-STA (Gao et al., 2017). We further conduct ablation studies to justify our design choices, particularly the Chain-of-LoRA mechanism for enhancing flexibility while preserving efficiency. Our contributions summarized follows:
1. We propose VideoMind, a multi-modal agentic framework that enhances video reasoning by emulating human cognitive processes, including task decomposition, moment localiza- tion and verification, and answer synthesis. It addresses the unique challenges of long video reasoning a progressive and structured manner.
2. We introduce Chain-of-LoRA gle model to seamlessly switch among multiple roles. This approach enhances Video- Mind's flexibility without incurring additional memory overhead.
efficient test-time scaling mechanism that enables sin-
3. Our method demonstrates strong performance across three scenarios: Grounded VideoQA, Video Temporal Grounding, and General VideoQA. Notably, VideoMind-25 outperforms GPT-4o and Gemini- 1.5-Po 1.5 on several long video benchmarks.
2 RELATED WORK
rempoorl-grounnded Video Understanding Significant advances in video understanding have propelled tasks such as video retrieval (Lin et al., 2022; Lin & Shou, 2025) and captioning (Zhao et al., 2023; Lin et al., 2024b; Chen et al., 2024b). However, these models often lack visually grounded correspondence and interpretability, particularly for long-form videos. The task Video Temporal Grounding (Gao et al., 2017; Krishna al., 2017) tackles this issue by requiring precise
2

---

Published as a conference paper at ICLR 2026
temporal localization for diverse queries, though regression-based models (Liu et al., 2022; 2024d) excel at localization but fall short in providing textual interpretabilityy. Recent benchmarks (Xiao et al., 2024; Chen et al., 2024a; Liu et al., 2024e) intensify this challenge, demanding both reason- ing for complex questions and fine-grained temporal correspondence. Previous baselines for these tasks typically rely on multi-task objectives modular agents composed of distinct components (Wang et al., 2024d; Fan et al., 2024), often yielding sub-optimal performance or overly complex systems, which constrain their efficiency and flexibility. Our VideoMind is an agentic workflow built upon unified LMM, seamlessly integrating multiple functionalities while enhancing localization and interpretability, thus surpassing the limitations of prior methods.
Multi-modal Reasoning Large Multi-moda Models (Liu et al., 2023; 2025) exhibit generalized capabilities such as free-form question answering. However, they fall short in addressing complex challenges that often require reasoning (Wei et al., 2022). One approach to overcome this is to develop ageentbbsed interfaces (Zhang et al., 2023a; Kahatapitiya et al., 2024), which integrates tex- tual outputs from visual tools to enable reasoning via LLMs. Advanced methods (Suris et al., 2023; Yang et al., 2023; Gao et al., 2023) invoke visual APIs through progressive execution and reasoning. Alternatively, pure text-based reasoning (OpenAI, 2024b; Guo et al., 2025) has been dominant paradigm in LLMs, exemplified by training with long CoT processes using reinforcement learning, which provides detailed step-by-step reasoning, with some works (Chen et al., 2025b; Feng al., 2025) extending this mechanism to the visual domain. Despite these advances, extending reason- ing to videos remains an open challenge. Given the long-context nature of informative videos, we believe that a vision-centric CoT should incorporate a human-like (Xie et al., 2023; 2024a; 2025a; Wen al., 2025) re-watching strategy and self-validation of intermediate observations, leading us to introduce novel Chain-of-LoRA framework for video reasoning.
Inference-time Searching Inference-time searching has emerged as critical technique for tack- ling complex reasoning challenges in different domains. The advent of OpenAl ol (OpenAI, 2024b) has advanced these inference-time techniques within L.LMs by integrating sampling strategies such as controlled decoding (Chakraborty et al., 2024), Best-of-N sampling (Lightman et al., 2023), and Monte Carlo Tree Search (MCTS) (Wang et al., 2024c), allowing LLMs to iteratively refine outputs and achieve superior performance without altering their underlying weights. However, the poten- tial of inference-time searching remains largely untapped in video understanding, where temporal reasoning introduces unique challenges. In framework, we explore how such strategy can be tailored for video temporal reasoning. observing that models are highly sensitive to the selection of temporal segments, often producing unreliable predictions when segment choices To address this, we propose a moment-level searching approach where a grounder generates multi- ple candidates, followed by a verifier that evaluates and determines the correct correspondence. The framework also supports flexible role switching with minimal memory overhead.
sub-optimal.
3 METHOD
Overview Figure 2 provides overview of VideoMind. Our model derives from the Qwen2-Ve VI. (Wang et al., 2024b) architecture, consisting of an LLM backhone and a ViT-based visual encoder support dynamic resolution inputs. Given video input V and a text query Q, the model performs step-by-step reasoning by adaptively calling different roles: (1) Planner: Dynamically coordi- nates the following roles based on the query. (2) Grounder: Identifies and localizes relevant video moments. (3) Verifies Evaluates the validity of the moments identified by the grounder, refining them through a zoom in process with boolean outputs. (4) Answerer: Generates the final response in natural language. This mechanism enables the models to revisit the videos several times (with varying temporal segments & spatial resolutions) to derive the final response.
3.1 PLANNER
An agent must be flexible enough to handle diverse tasks and efficiently determine which functions (roles) to call. To achieve this, we design the planner, which dynamically coordinates all the other roles for each query. It decides the sequence of function calls based on the multi-modal context. We utilize JSON-style object {"type" " "<role>" ", "value": " Karguumnt>> function call. In this way, a sequence of roles can be succinetly represented as a list of such objects. Three reasoning plans for different tasks are pre-defined and illustrated in Figure 3.
to denote
3

---

Published as a conference paper at ICLR 2026
8GA
1ch
Tiseflange
Decadee
Yae
N
Werker
Crpe Servent
VideoMind
Sell Aoll-Altentin Black
LoRA-P
LoRA-G
LoRA-V
OFF
"hhy bunnies gathering on the table?"
ldeealyy the best mertin
Groud the minoar Teasiio paliciiy
Vadly windie Sa and Jainty
Why bwen gahhetin
Hhis Rrndo
Planner
Grounder
Verifer
Answetet
The overall workflow of VideoMind. Given a video and query, it adaptively activates different roles (e.g. Planner Grounder Veri fier Answerer in this case) and performs step-by-step reasoning by calling individual modules.
Figure 2:
Grounder
Verifier
Answerer
Planner
Grounder
Verifier
Answerer
Figure 3: Planner coordinates all the other roles based on the video and query context, offering three reasoning plans and query rephrasing mechanism address diverse demands.
This plan requires the agent to generate both a textual response and corresponding temporal moment. For example, Grounded VideoQA scenarios (Xiao et al., 2021), to answer the question "What is the boy doing when the baby is crying?", the agent should identify the moment of "baby is crying". and then investigate the boy's s activity.
(1) Grounding & Verifying & Answering:
(2) Grounding & Verifying: This plan is designed for grounding-only tasks such as moment re- trieval (Lei et al., 2021; Gao et al., 2017). For questions like "When does the woman go down- stairs?"), the model should provide precise timestamps directly the answer. Since the grounding results could potentially be unreliable, an extra zoom-in verification step is necessary.
(3) Answering Only: If the question is straightforward (e.g., "Summarize this video") the video very short (e.g., less than 10s), it could be unnecessary to perform grounding. Instead, the model should watch the entire video and answer the question directly.
Query Rephrasing When the user query lacks sufficient detail for accurate moment localization, the planner is allowed rephrase the question into question "What the person sitting on the bed doing as the baby plays?" may confuse the grounder it contains multiple events ("person sitting on the bed" and "baby plays"). ). It can be rephrased to "the baby is playing" as an accurate scene description.
descriptive version. For instance, the
To train the planning and query rephrasing capabilities, curated a dataset of 39K samples (shown in Table 1) from public benchmarks. For planning, we aligned each reasoning plan with correspond- ing question types: temporal questions from NExT-QA (Xiao et al., 2021) are assigned to Plan- 1, moment queries from QVHighlights (Lei et al., 2021) are for Plan-2, and causal & descriptive ques- tions from NEXT-QA (Xiao et al., 2021) are for Plan-3. For query rephrasing, we leverage GPT-4o mini (OpenAI, 2024a) to generate synthetic video + quest ion + query samples for training.
3.2 GROUNDER
The grounder aims to localize relevant moments (i.e., predicting start and end timestamps) based on text queries, thereby supporting the reasoning process by identifying visual cues. This require- ment calls for the development of an LMM with robust temporal grounding capabilities.
4

---

Published as a conference paper at ICLR 2026
Timestamp Decoder Instead of directly predicting timestamps through language modeling (Ren et al., 2024) or special tokens (Huang et al., 2024a; Liu et al., 2024e), we develop a timestamp decoder to maximize the LMM-based grounding performance. Specifically, introduce <REG> token to facilitate this process. When the <REG> token is generated, the last-layer hidden states of it and all the visual tokens will be sent into the decoder for timestamp prediction, obtaining tuple [fatarts tend] representing the normalized start and end timestamps.
As shown in Figure the decoder accepts the hidden states of the visual tokens h, R H X Dr and the <REG> token h, E R1 as inputs, where T, H. W, Dr, are the downsampled number of frames, height, width, and hidden dimensions of the LLM, respectively. We apply a 1D aver- age pooling with kernel size and stride equal to H W to compress the visual tokens to one token per frame.
C
Lreg
CLS Head
REG Head
P.
T/A
Temporal Pyramid
h AvgPool(h.) E R TxPL
(1)
L.o.
Then, h2 and h, are projected by two linear layers E,, and E,, to reduce the hidden dimension
Self self-Attention FFN
= E,h,,) E TRDD
= E.
(2)
The resulting e, and e, serve as consolidated representa- tions of the video frames and the query', respectively. To effectively integrate their information, we concatenate them along the sequence dimension and send them into a three- layer transformer encoder (Vaswani al., 2017).
LN Linear E.
LN Linear E,
h,
[e'; e = Transformer((es + m. + h, + m.))
(3)
Element Wise Add
Positional Encoding
Here, modality indicators m. E IRIxD and m, E 1x are randomly initialized learnable embeddings. my is expanded to T x D before being added with ev. ep is normalized si- nusoidal positional encoding (Vaswani al., 2017) for pre- serving temporal awareness. The output sequence is split back into e', and e/,, indicating the contextualized frame and query embeddings, respectively.
Modality Embedding
O
-REG> Token
Video Frame Taken
Multi-madal Taken
Figure 4: Detailed architecture of the timestamp decoder.
To improve the model's adaptability to videos and moments of vary- ing lengths, we map e', into four-level temporal feature pyramid (Liu et al., 2024d; Zhang et al., 2022). Each level is produced by a ConvlD - LayerNore + SiLU block, where the ConviD employs a kernel size and stride of 2. Therefore, the resulting four levels retain 1, 1/2, 1/4, and 1/8 of the original sequence length, respectively. To accelerate the prediction, nate the sequences from all pyramid levels along the temporal dimension to form P. with length L + 7/2 T/4 + T/8, allowing parallelized prediction across temporal resolutions.
Temporal Feature Pyramid
concate-
We introduce two heads for timestamps prediction: (1) classification head is designed for frame-level boreground-background classification. This is instantiated by two-layer ConviD module with kernel size 3 and padding 1, followed by a Sigmoid activation. The outputs are frame-level confidence scores f-o indicating whether each frame falls inside the desired moment. A binary focal loss (Lin et al., 2017) is utilized to optimize these scores.
Prediction Heads
Cats a 1 - 2:)7 log((:)
Here, a = 0.9 and > = 2.0 are hyperparameters of the focal loss, and dets is the loss reweighing term. (2) A boundary regression head is adopted to predict the frame-level temporal offsets for
start and end boundaries {1o 6 This also a two-layer ConvlD block (with output chan- nels), followed by an exponential activation. Predictions from different pyramid levels are further modulated by different learnable scaling factors. These outputs are supervised by an L1 loss.
Creg Areg 6: |6 6/ -
(5)
In order to realize better alignment between e, and e/. we incorporate an additional contrastive loss encourage learning more diseriminative representations. Specifically, calculate the cosine
We the "query" denote the features of <REG> token.
5

---

Published as a conference paper at ICLR 2026
Table I: Training datasets for different roles. Source datasets were repurposed for training planner and verifier. mr and step denote the moment retrieval and step localization subsets, respectively.
Role
ASamples
Source Dutasels
Planner
39K
NeXT-QA (34K), QVHighlights (5K)
OVHighlights (5K), DiDeMo (35K), TACoS (9K), Intem (54K), CosMo-Cap (87K),
Grounder
210K
QuerYD (19K), HiREST (8K), HiRESTap (4K)
Verifier
232K
DiDeMo (165K), TACoS (43K), QVHighlights
similarities among all frame-query pairs (denoted as (s: then sample a positive frame (falling within the ground truth boundary) and apply the following optimization objective:
cxp(sp/T)
Lcon -con log
(6)
exp($p/T) Eiee exp(s:/T)
Here, θ is the set of frame indices with Sp > and T = 0.07 is the temperature parameter. The final loss for the timestamp decoder is the sum of these losses at all layers with Acts 5.0, Areg 1.0, and Acon = 0.05. The training datasets for the grounder are listed in Table 1.
3.3 VERIFIER
Key moments are crucial for providing visual cues, yet they might be unreliable due to grounding errors. Thus, further verifications are necessary. We let the grounder generate top-5 predictions, then employ the verifier select the most reliable one. This process presented below.
Recap by Zooming-in For each candidate moment, we apply a zoom-in strategy by expanding the bound- aries by 50% on both sides and temporally cropping the enlarged segment. The resulting segment and the orig- inal text query are sent to the verifier to assess whether the queried event exactly occurs within the temporal boundaries. To enhance boundary awareness, we adopt two special tokens, <SEG-START> and <SEG-END>, explicitly mark the beginning and end of the mo- ment. These tokens are inserted among the visual to- kens the corresponding frames, effectively guiding the model in recognizing moment boundaries.
Grounder: Find the candidate moments
10.
15,
51.97
42, 89
Zoom-in
Verifier: Yes or No?
0.52
0.47
Figure The grounder generates multiple candidate moments, which are then refined by the verifier via zooming-in investi- gate and select the best one.
Boolean Judgement The verifier's responses are bi- nary, i.e, either "Yes" or "No". To train this role, we sample predictions from the grounder and assign binary labels based on an loU threshold of 0.5. The model then fine-tuned via SFT predict these labels. During inference, for each candidate mo- ment, we employ teacher forcing to obtain the likelihoods of the <Yes> and <No> tokens, denoted as Ly and Ln. respectively. The confidence score then computed as Sigmoid(Ly, Ln). The moment with the highest score is selected and passed to the answerer.
3.4 ANSWERER
The answerer responds to the given question based on the cropped video segment (w/ grounder) or the whole video (w/o grounder). Since the objective of this role is strictly aligned with existing LMMs, we employ the original model directly without fine-tuning or architectural modifications.
3.5 CHAIN-OF-LORA
The four roles introduced above demonstrate distinct yet complementary capabilities, collaborating to achieve advanced vision-centric reasoning. However, simply integrating these roles into a single model poses challenges, their core functionalities efficiently implementing them as multiple models while still accommodating diverse demands, we propose novel Chain-of-LoRR. mechanism enable flexible and efficient role switching.
interfere with one another. To avoid in-

---

Published as a conference paper at ICLR 2026
Table 2: Performance comparison Grounded VideoQA CG-Bench (Chen et al., 2024a).
Method
Size
long-acc.
mloU
acc.@IoU rec.@loU
GPT-4o (OpenAI, 2024a)
5.62
Gemini-1, 1,5 Pro (Reid et al., 2024)
37.2
3,95
5.81
2.53
(Antrooic, 2025)
40.5
3.99
5.67
2.79
Video-LLaVa (Lin et al., 2023a)
7B
16.2
1.13
1.96
0.59
(Zhang
2023b)
7R
VideoChar2 (Li et al. 2024b)
7B
19.3
1.28
1.98
0.94
ST-LLM (Liu al., 2024c)
7B
23.8
2.23
2.86
1.13
VhudeGPT4Video (Chen
2024J)
16B
26.7
1.85
2.65
1.01
Chat UniVi (Jin et 2023)
13B
25.9
2.07
2.53
1.21
8B
1.56
LongVA (Zhang al., 2024)
76
28.7
2.94
3.86
1.78
LLaVA-Oee
al., 2024n)
7R
1.63
Vidco-CCAM (Fel ot 2024)
14B
29.7
2.63
3,48
1.83
Kangaroo (Liu et al., 2024b)
8B
30.2
2.56
2.81
1.94
VIIA (Fu 2024b)
8x7B
33.3
3.06
3.53
2,06
Qwen2-VI. (Wang al., 2024h)
72R
41.3
3.58
5.32
3.31
InlemVL2 (OpenCVVaah, 2024)
42.2
VideoMind (Ours)
2B
31.0
5.94
8.50
4.02
VideoMind (Ours)
7B
38.4
7,10
9.93
4.67
Table 3: Performance comparison on Grounded VideoQA on ReXTime (Chen et al., 2024c). FT
indicates fine-tuning on the target dataset.
Method
Size
FT
R@0.3
R@0.5
mloU
Ace
Acc@loU
V.imeILMM (Huang al., 2024a)
X
28.84
17.41
20.14
36.16
TimeChat (Ren et al., 2024)
7B
X
14.42
7.61
11.65
40.04
(Huang 2024b)
29.49
16.29
21.49
34.44
VTimeLLLM LM (Huang al., 2024a)
/
43.69
26.13
29.92
17.13
TimeChat (Rea al., 2024)
7B
/
40.13
21.42
26.29
49,46
10.92
VideoMind (Ours)
X
34.31
22.69
24.83
69.06
17.26
VideoMind (Ours)
7B
x
38.22
25.52
27.61
74.59
20.20
In greater detail, roles are based on shared LMM backbone and are augmented with different LoRA adapters (Hu et al., 2022). Note that additional timestamp decoder is used exclusively by the grounder. During inference, the framework dynamically activates role-specific LoRA adapters according to the planner, thereby maximizing the strengths of each role while minimizing the mem- ory consumption and architectural modifications to the base model.
4 EXPERIMENTS
We evaluate the effectiveness of VideoMind through extensive experiments across public bench- marks. Specifically, we study the following research questions.
Q1. Whether VideoMind is flexible and effective on diverse video understanding tasks com- pared to the corresponding baselines with task-specific designs?
Q2. Compared with (1) training single agent on multiple tasks or (2) distributing all roles to different models, what advantages does Chain-of-LoRA offer?
Q3. What effects does each individual design contribute? More importantly, whether each role is necessary for building such video reasoning system?
Detailed information about the benchmarks, evaluation settings, implementation details, and more experimental results can be found in the appendix.
4.1 Q1: COMPARISON WITH STATE-OF-TT THEE--R-TTTETTSS
Grounded Video Question Answering Table compares the Grounded VideoQA performance
on CG-Bench (Chen al., 2024a), challenging video benchmark with an average duration of 27 minutes. On temporal grounding metrics (mloU and rec. @ @IOU), our lightweight 2B model outper- forms all the baselines, including GPT-4o (OpenAI, 2024a) and Gemini 1.5 Pro (Reid al., 2024). Our 7B model further setups a new state-of-the- art on clue- gruungro QA (acc. @IoU). In Table 3 and Table 4, we further present the comparison results on ReXTime (Chen al., 2024c) and NEXT-GQA

---

(Xiao et al., 2024). Despite the chal- lenges posed by the causal event re- lationships ReXTime, our model can successfully identify the cor- rect moment, resulting in significant performance boosts compared with zero-shot baselines. On NEx T-GQA. compared to agent-based solutions such as LLoVi (Zhang al., 2023a) and LangRepo (Kahatapitiya et al., 2024) and end-to-end methods like Published as a conference paper at ICLR 2026
Table 4: Performance comparison Grounded VideoQA NEXT-GQA (Xiao al., 2024).
IoU
IoP
Method
Size
Acc&GQA
R@0.5
R@0.3
R@0.5
FroeeBBLM NG+ (Yang et 2022)
890M
13.5
6.1
9.6
28.5
23.7
24.2
17.5
SeViLA (Yu al., 2023)
4B
29.2
13.8
21.7
34.7
22.9
29.5
16.6
LangRepo (Kahatapitiya ot al., 2024)
8B
12.2
18.5
28.7
31.3
17.1
VideoStreaming (Qian al., 2024b)
31.0
17.8
LLoVi (Zhang et al., 2025a)
1.T
15.3
20.0
36.9
37.3
24.3
HawkBye (Wang al., 20241)
7B
37.0
19.5
25.7
VideoChat- ChotCChaTPPP... (Yan al., 2024)
7B
41.2
23.4
27.7
47.5
32.8
35.6
25.5
VideoMind (Ours)
2B
23.2
28.6
32.6
36.4
25.2
VideoMind (Ours)
71
50.2
25.8
31.4
56.0
35.3
39.0
28.2
Table 5: Performance comparison on video temporal grounding on Charades -Trra (Gao et al., 2017)
and ActivityNet--Cptions (Krishna al., 2017). FT means fine-tuning on the target dataset.
Charades-STA
ActivityNet Nelaaityee-Caattions
Method
R@0.3
R@0.5
R@0.7
mloU
R@0.3
R@0.
R@0.7
mloU
VM (Huang ot ali, 20244)
7B
51.0
27.5
11.4
31.2
44.0
27.8
14.3
30.4
TimeChat (Ren 2024)
Momentor (Qian et ial,, 2024a)
7B
X
42.6
26.6
11.6
28.5
42.9
23.0
12.4
29.3
Cha al., 2024)
VidcoChat-TPO (Yan et 2024)
7B
58.3
40.2
18.4
38.1
E.T. Chat (Liu et al., 2024e)
45.9
20.0
42.3
24.1
12,8
6.1
18.9
Vouunded-ideoLLLMMMM (Wang 2024a)
54.2
36.4
19.7
36.8
TRACE (Guo al., 2024)
7B
X
40.3
19.4
LLaVA-ST
2025a)
UniTime (1.i el al., 2025b)
7B
59.1
31.9
52.2
22.8
14.1
27.3
VideoMind (Ours)
67.6
51.1
26.0
45.2
44.0
26.5
12.6
30.1
VideoMind (Ours)
73.5
59.1
31.2
50.2
48.4
30.3
15.7
33.3
Table 6: Performance comparison General VideoQA Video-MME (Fu et al., 2024a), MLVU (Zhou et al., 2024), and LVBench (Wang al., 2024c).
Video-MME
MLXU
LXBench
Method
Size
Long
M.Avg
GPT-4o (OpenAL, 2024a)
71.9
65.3
54.5
30.8
Gemini Pro (Reid et 2024)
75.0
67.4
33.1
Video-LLaVA
2023a)
29.3
TimeChat (Ren et al., 2024)
7B
34.3
32.1
30.9
22.3
MovieChat (Song al., 2023)
PLLaVA (Xu ot al., 2024)
34B
40.0
34.7
53.6
26.1
VideoChat- TPO (Yan et al., 2024)
7B
48.8
41.0
54.7
VideoChat-TPO (Yan et al., 2024), VideoMind demonstrates its effec- tiveness both key event grounding and question answering.
LongVA (Zhang al., 2024)
7B
52.6
46.2
56.3
(Ours)
46.3
VideoMind (Ours)
7B
58.2
49.2
64.4
40.8
Video Temporal Grounding We also evaluate the grounder and verifier on video temporal ground- ing datasets. The results on Charades-STA. (Gao et al., 2017) and ActivityNet-Caption (Krishna et al., 2017) shown in Table 5. Benefiting from (1) the timestamp decoder design, and (2) verifier that refines the results by focusing on critical segments, our model surpasses all LLM--based temporal grounding methods and yields competitive results compared to fine-tuned experts.
General Video Question Answering We are also interested whether our temporally augmented design can improve general VideoQA tasks. In Table 6, we evaluate model on three long video benchmarks to determine the Chain-of-Lofe design generalizes to common settings. Our designs effectively help the model localize segments before answering the question.
4.2
Q2: THE ADVANTAGES OF CHAIN-OF-LORA
Table 7 studies the effect of role integration VideoMMnnd-2B. First, text-based CoT does not im- prove the base model, highlighting the need for a cisioiccentric reasoning strategy. Second, the key capabilities of roles may conflict with another, thus only sub-optimal performance can be
8

---

Published as a conference paper at ICLR 2026
Question: Why dad the adult squat down and opened his arm at the end of the video?
Options: (A) Want bug child (B) Dancing (C) Doing squats (D) Want bug dog (C) Stretching his armus
Goounder Quary: aguas
Thought
VideoMind (Ours)
Stevidao
Grouzd Truli Manani
while bunkingg dae chall, This augpodtr that (ie PARROst The olleer splaan di LNt poace be
2 Venter AnnwerTr
Grasndel
The moseene miters
Figure 6: Visualization of the reasoning process of VideoMind. Through chaining the p anner, grounder, veri fier, and a answerer, our model accurately localizes the critical moment and selects the correct answer, avoiding confusion from incorrect segments.
Table 7: Performance and efficiency comparison of Table Effects of individual roles. A. G. V. different test-time scaling and role integration strate- gies. Mem indicates the peak GPU memory con- sumption. Notably, Chain-of-L achieves the best performance with minimal memory cost.
P. G% denote the answerer, grounder, veri- fier, planner, and the percentage of samples processed with the grounder, respectively.
Roles To Use
ReXTime
Charades-STA
NEXT-GQA Charades-STA Video-MME
mloU
R@0.5 R00.7 mloU
Method
mloU
Ace
R@05 mloU
All
Long
0%
/
100%
24.5
68.8
Qwen2-VIL-28
4.1G
69.6
_
$3.0
43.1
/
100%
24.8
69.1
+CoT
4.1G
69.7
43.3
100%
24.7
/
40%
26.7
70.0
All-in-One
4.2G
20,5
47.8
42.1
+All -ill-iitribeted
16.6G
28.6
71.4
51.1
45.2
55.4
36.3
47.2
21.7
42.0
-Chain-of-LoRA
4.2G
28.6
71.4
51.1
45.2
55.4
46.3
51.1
26.0
45.3
achieved via joint training. Compared the ail-distributed approach that requires multiple copies (4x) of weights, Chain-of-LoRA offers the best balance between effectiveness and efficiency.
4.3
Q3: KEY ABLATION STUDIES
Effect of Individual Roles The contributions of different roles are studied in Table 8. Our observa- tions follows: (1) Grounder: : By identifying visual cues, the grounder can slightly improve QA accuracy, indicating that the grounder is especially effective on long videos. (2) Verifier: Selecting the best candidate through the verifier improves grounding performance, yielding a con- sistent gain of 3.2 mloU on Charades-STA. (3) Planner: Coordinating roles via the planner even when performing grounding on only 40% samples (the remaining 60% are directly processed by the answerer) boosts the accuracy from 69.2 to 70.0. This highlights the model's 's flexibility to adaptively determine whether to perform grounding under different temporal contexts.
4.4 VISUALIZATION
In Figure 6, we illustrate how VideoMind applies all roles to progressively derive the correct answer while avoiding potential mistakes. The planner determines what roles are needed, then calls the grounder to generate candidate moments. The verifier selects the most relevant segment (highlighted in yellow), which is then zoomed-in and passed to the answerer for further reasoning.
CONCLUSION
In this work, introduced VideoMind, video-language agent designed for cemporal-grounder video reasoning. Our approach employs an agentic workflow consisting of four carefully designed roles along with Chain-of-LoRA strategy to flexibly switch among them. Extensive experiments on Grounded VideoQA, Video Temporal Grounding, and General VideoQA tasks demonstrate the effectiveness and significance our method, particularly in long video reasoning by providing evidence-based answers. We hope this work inspires future advancements in agentic reasoning.
We acknowledge that our method requires careful optimization of individual designs and preparation of training data. In our future work, we will investigate (1) the possibility of ootiiooptim of multiple roles and (2) the integration of audio modality.
Limitations & Future Work

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
This research is supported by The Hong Kong Research Grants Council (GRF-15229423) and by Tencent's research donation. also acknowledge The University Research Facility in Big Data Analytics (UBDA) at The Hong Kong Polytechnic University for providing computing resources that have contributed to the research results reported within this paper. Mike Shou does not receive any funding for this work.
ETHICS STATEMENT
This study focuses on algorithmic innovations for improving the visual reasoning capabilities of large multi-modal models. It does not involve human subjects, private data, or any potentially harm- ful insights. All datasets used are publicly available and widely adopted in the community. We acknowledge the potential risks of misuse associated with LLMs and LMMs, including the bias propagation and harmful content generation. However, this study does not directly address the de- ployment or generation. Instead, contributes the understanding of the model architecture and the reasoning mechanism. To the best of our knowledge, research complies with the ICLR Code of Ethics and does not involve any known violations or harms.
REPRODUCIBILITY STATEMENT
We committed ensuring the full reproducibility of this study. To achieve this, have pro- vided key hyperparameters settings in Section 3, formulation of inference pipeline in Section A. implementation details in Section A.2, evaluation metrics Section B.I, and prompt templates in Section C.I. We also open-source all the code, model checkpoints, data, and training logs in this study facilitate future research in this direction.
REFERENCES
Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Awadalla, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Harkirat Behl, et al. Phi-3 technical report: A highly capable language model locally your phone. arXiv:2404. 14219, 2024.
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. Gpt-4 technical report. wwXiv:2303.08772 2023.
Anthropic. Claude 3.5 sonnet model card, 2025. URL /claude- 3-5 5- sonnet.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2.5-vl technical report. 2.XY::2502.13923 2025.
Souradip Chakraborty, Soumya Suvra Ghosal, Ming Yin. Dinesh Manocha, Mengdi Wang, Am- rit Singh Bedi, and Furong Huang. Transfer q star: Principled decoding for 1lm alignment. arXiv:2405.20495, 2024.
Guo Chen, Yicheng Liu, Yifei Huang, Yuping He, Baoqi Pei, Jilan Xu, Yali Wang, Tong Lu, and Limin Wang. Cg-bench: Clue-grounded question answering benchmark for long video under- standing, 242. 2024a.
Joya Chen, Zhaoyang Lv, Shiwei Wu, Kevin Qinghong Lin, Chenan Song, Difei Gao, Jia- Wei Liu, Ziteng Gao, Dongxing Mao, and Mike Zheng Shou. Videollm-online: Online video large lan- guage model for streaming video. In CVPR, pp. 18407-18418, 2024b.
Jr-Jen Chen, Yu-Chien Liao, Hsi-Che Lin, Yu-Chu Yu, Yen-Chun Chen, and Frank Wang. Rextime: A benchmark suite for wwsoonnng-acros-ttnn in videos. In NewrIPS, pp. 28662-28673 2024c.
10

---

Published as a conference paper at ICLR 2026
Lin Chen, Xilin Wei, Jinsong Li, Xiaoyi Dong. Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Zhenyu Tang, Li Yuan, et al. Sharegpt4video: Improving video understanding and genera- tion with better captions. In NeurIPS, pp. 19472-19495, 2024d.
Qirui Chen, Shangzhe Di, and Weidi Xie. Grounded multi-hop videoqa in long-form egocentric videos. In AAAI, pp. 2159-2167, 2025a.
Shimin Chen, Xiaohan Lan, Yitian Yuan, Zequn Jie, and Lin Ma. Timemarker: A versatile video-Ilm for long and short video understanding with superior temporal localization ability. arXiv:2411.1821, 2024e.
Yukang Chen, Wei Huang, Baifeng Shi, Qinghao Hu, Hanrong Ye, Ligeng Zhu, Zhijian Liu, Pavlo Molchanov, Jan Kautz, Xiaojuan Qi, et al. Scaling rl to long videos. arXiv:2507.07966 2025b.
Google DeepMind. Gemini 2.5: Our most intelligent ai model, 2025. URL
waacchooooyy/ooooo ooologgg///ooooo /goglloollooooo deepmii nd/ gemi n -oodmmooee--- sondeel-thinee wtwpell-tiiiiiin-nnnnn--a---e-ee
cch2025/.
Yue Fan, Xiaojian Ma, Rujie Wu, Yuntao Du, Jiaqi Li, Zhi Gao, and Qing Li. Videoagent: A memory-augmented multimodal agent for video understanding. arXiv:2403.11481, 2024.
Jiajun Fei, Dian Li, Zhidong Deng, Zekun Wang. Gang Liu, and Hui Wang. Video-ccam: Enhanc- ing video-language understanding with causal cross-attention masks for short and long videos. arXiv:2408. 14023, 2024.
Kaituo Feng, Kaixiong Gong, Bohao Li, Zonghao Guo, Yibing Wang, Tianshuo Peng. Junfei Wu, Xiaoying Zhang, Benyou Wang, and Xiangyu Yue. Video-rl: Reinforcing video reasoning in mllms. arXiv:2503.21777 2025.
Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Video-mme: The first-ever comprehensive evalua- tion benchmark of multi-mode modal iims in video analysis. arXiv:2405.2 1075, 2024a.
Chaoyou Fu, Haojia Lin, Zuwei Long, Yunhang Shen, Yuhang Dai, Meng Zhao, Yi-Fan Zhang, Shaoqi Dong, Yangze Li, Xiong Wang, et al. Vita: Towards open-source interactive omni multi- modal 1lm. arXiv:2408. 05211, 2024b.
Difei Gao, Lei Ji, Luowei Zhou, Kevin Qinghong Lin, Joya Chen, Zihan Fan, and Mike Zheng Shou. Assistgpt: A general multi-modal assistant that can plan, execute, inspect, and learn. arXiv:2306.0866 2023.
Jiyang Gao, Chen Sun, Zhenheng Yang, and Ram Nevatia. Tall: Temporal activity localization via language query. In ICCV, 5267-5275, 2017.
Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Gird- har, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, et al. Ego4d: Around the world in 3,000 hours of egocentric video. In CVPR, pp. 18995-19012, 2022.
Daya Guo, Dejian Yang, Haowei Zhang. Junxiao Song. Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv:2501.12943 2025.
Yongxin Guo, Jingyu Liu, Mingda Li, Qingbin Liu, Xi Chen, and Xiaoying Tang. Trace: Temporal grounding video I1m via causal event modeling. arXiv:2410.05643. 2024.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 2022.
Bin Huang, Xin Wang, Hong Chen, Zihan Song, and Wenwu Zhu. Vtimellm: Empower Im grasp video moments. In CVPR, pp. 14271-14288 2024a.
De-An Huang, Shijia Liao, Subhashree Radhakrishnan, Hongxu Yin, Pavlo Molchanov, Zhiding Yu, and Jan Kautz. Lita: Language instructed temporal-locee localization assistant. arXiv:2403. 19046, 2024b.
11

---

Published as a conference paper at ICLR 2026
Dongfu Jiang, Xuan He, Huaye Zeng, Cong Wei, Max Ku, Qian Liu, and Wenhu Chen. Mantis: Interleaved multi-image instruction tuning. arXiv:2405.01483 2024.
Peng Jin, Ryuichi Takanobu, Caiwan Zhang, Xiaochun Cao, and Li Yuan. Chat-univi: Unified visual representation empowers large language models with image and video understanding. arXiv:2311.08045 2023.
Kumara Kahatapitiya, Kanchana Ranasinghe, Jongwoo Park, and Michael S Ryoo. Language repos- itory for long video understanding. arXiv:2403. (4T2:2403.11622,, 2024.
Ranjay Krishna, Kenji Hata, Frederic Ren, Li Fei-Fei, and Juan Carlos Niebles. Dense-captioning events in videos. In ICCV, pp. 706-715, 2017.
Hugo Laurencon, Leo Tronchon, Matthieu Cord, and Victor Sanh. What matters when building vision-language models? In NeurlPS, pp. 87874-87907, 2024.
Jie Lei, Licheng Yu, Tamara L Berg, and Mohit Bansal. Tvr: A large-scale dataset for video-subtitle moment retrieval. In ECCV, pp. 447-463, 2020.
Jie Lei, Tamara L Berg, and Mohit Bansal. Qvhighlights: Detecting moments and highlights in videos via natural language queries. In NeurlPS, 2021.
Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. arXi::2408.0332. 2024a.
Hongyu Li, Jinyu Chen, Ziyu Wei, Shaofei Huang, Tianrui Hui, Jialin Gao, Xiaoming Wei, and Si Liu. Llava-st: A multimodal large language model for fine-grained spatial-temporal under- standing. In CVPR, pp. 3592-8603, 2025a.
Kunchang Li, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo. Yali Wang. Limin Wang, and Yu Qiao. Videochat: Chat-centric video understanding. arXiv:2305.066. 06355, 2023a.
Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, et al. Mybench: A comprehensive multi-modal vidco understanding benchmark. In CVPR, pp. 22195-22206, 2024b.
Pandeng Li, Chen-Wei Wei Xie, Hongtao Xie, Liming Zhao, Lei Zhang, Yun Zheng, Deli Zhao, and Yongdong Zhang. Momentdiff: Generative video moment retrieval from random to real. In NeurlPS, pp. 65948-65966, 2023b.
Zeqian Li, Shangzhe Di, Zhonghua Zhai, Weilin Huang, Yanfeng Wang, and Weidi Xie. Universal video temporal grounding with generative multi-modal large language models. MrXiv:2506.18883 2025b.
Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. In ICLR, 2023.
Bin Lin, Bin Zhu, Yang Ye, Munan Ning, Peng Jin, and Li Yuan. Video-llava: Learning united visual representation by alignment before projection. arXiv:2311.10122. 2023a.
Lin, Hongxu Yin, Wei Ping, Pavlo Molchanov, Mohammad Shoeybi, and Song Han. Vila: On pre-training for visual language models. In CVPR, pp. 26689-26699, 2024a.
Kevin Qinghong Lin and Mike Zheng Shou. Vlog: Video-language models by generative retrieval of narration vocabulary. In CVPR, pp. 3218-3228, 2025.
Kevin Qinghong Lin, Jinpeng Wang, Mattia Soldan, Michael Wray, Rui Yan, Eric Z Xu, Difei Gao, Rong-Cheng Tu, Wenzhe Zhao, Weijie Kong, et Egocentric video-language pretraining. In NewrlPS, pp. 7575-7586, 2022.
Kevin Qinghong Lin, Pengchuan Zhang, Joya Chen, Shraman Pramanick. Difei Gao, Alex Jin- peng Wang. Rui Yan, and Mike Zheng Shou. Univtg: Towards unified lideo-language temporal grounding. In CVPR, pp. 7794-8804, 2023b.
12

---

Published as a conference paper at ICLR 2026
Kevin Qinghong Lin, Pengchuan Zhang, Difei Gao, Xide Xia, Joya Chen, Ziteng Gao, Jinheng Xie, Xuhong Xiao, and Mike Zheng Shou. Learning video context as interleaved multimodal sequences. In ECCV, pp. 375-396, 2024b.
Tsung- Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In ICCV, pp. 2980000-2222000 1980-2988, 2017.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lec. Visual instruction tuning. In NeurlPS, pp. 14892-34916, 2023.
Haotian Liu, Chumyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang. Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, ocr, and world knowledge, 2024a. URL
g. tugt..u.... /o/blll//
Jiajun Liu, Yibing Wang, Hanghang Ma, Xiaoping Wu, Xiaoqi Ma, Xiaoming Wei, Jianbin Jiao, Enhua Wu, and Jie Hu. Kangaroo: A powerful video-language model supporting long-context video input. arXiv:2408. 15542, 2024b.
Ruyang Liu, Chen Li, Haoran Tang, Yixiao Ge. Ying Shan, and Ge Li. St-llm: Large language models are effective temporal learners. In ECCV, pp. 1-18, 2024c.
Ye Liu, Siyuan Li, Yang Wu, Chang Wen Chen, Ying Shan, and Xiaohu Qie. Umt: Unified multi- modal transformers for joint video moment retrieval and highlight detection. In CVPR, pp. 3042- 3051, 2022.
Ye Liu, Jixuan He, Wanhua Li, Junsik Kim, Donglai Wei, Hanspeter Pfister, and Chang Wen Chen. , ru-innng: Efficient image-to-video transfer learning for video temporal grounding. In ECCV, 2024d.
Ye Liu, Zongyang Ma, Zhongang Qi, Yang Wu, Ying Shan, and Chang W Chen. E.t. bench: Towards open-ended event-level video-language understanding. In NeurIPS, pp. 32076-321 10, 2024e.
Ye Liu, Zongyang Ma, Junfu Pu, Zhongang Qi, Yang Wu, Shan Ying, and Chang Wen Chen. Unip- ixel: Unified object referring and segmentation for pixel-level visual reasoning. In NeurIPS, 2025.
Iyy Loshchilov and Frank Hutter. Decoupled weight decay regularization. In ICLR, 2019.
Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Wei Chang. Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering. In NeurlPS, pp. 2507-2521, 2022.
Zongyang Ma, Yuxin Chen, Ziqi Zhang, Zhongang Qi, Chunfeng Yuan, Shaojie Zhu, Chengxiang Zhuo, Bing Li, Ye Liu, Zang Li, Ying Shan, and Weiming Hu. Visionmath: Vision-form mathe- matical problem-solving. In ICCV, 2025.
Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Shahbaz Khan. Video-chatept: Towards detailed video understanding via large vision and language models. arXiv:2306.05224, 2023.
Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Khan. Videogpt+: Integrating image and video encoders for enhanced video understanding. arXiv:2406.00411 2024.
WonJun Moon, Sangeek Hyun, SangUk Park, Dongchan Park, and Jae-Pil Heo. Query-dependent video representation for moment retrieval and highlight detection. In CVPR, pp. 23023-23033, 2023.
OpenAl. Gpt-4v(ision) system card, 2023. URL http://tpe////ppaaaac..co/////g/ ystem- card/.
OpenAl. Gpt-4o system card, 2024a. URL wtt.h//gss//ppppaa.....o///////a m- card/.
OpenAl. Openai ol system card, 2024b. URL S/RU:tppp///peeaa..co//////ss/sp 1 --systemm-card/ rd/.
13

---

Published as a conference paper at ICLR 2026
OpenAl. Gpt-5 system card, 2025. URL /tt.httos//ppppa....c///a/////// m- card/.
OpenGVLab. Internv12:
Better than the best expancxpan performance boundaries of open-source multimodal models with the progressive scaling strategy, 2024. httpt::t/ttteeeeeeeeeettet
Athub.io/ble /2//0///222222222 2024 07-00---22000000 02 InternVL 2.0/.
Long Qian, Juncheng Li, Yu Wu, Yaobo Ye, Hao Fei, Tat-Seng Chua, Yueting Zhuang, and Siliang Tang. Momentor: Advancing video large language model with fine-grained temporal reasoning.
arXiv:2402.11435 2024a.
Rui Qian, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Shuangrui Ding, Dahua Lin, and Jiaqi Wang. Streaming long video understanding with large language models. In NeurIPS, pp. 119336- 119360, 2024b.
Mengxue Qu, Xiaodong Chen, Wu Liu, Alicia Li, and Yao Zhao Chatvtg: Video temporal ground- ing via chat with video dialogue large language models. In CVPR, pp. 1847-1856, 2024.
Santhosh Kumar Ramakrishnan, Ziad Al-Halah, and Kristen Grauman, Naq: Leveraging narrations as queries supervise episodic memory. In CVPR, pp. 6694-6703, 2023.
Michaela Regneri, Marcus Rohrbach, Dominikus Wetzel, Stefan Thater, Bernt Schiele, and Manfred Pinkal. Grounding action descriptions in videos. Transactions of the Association for Computa- tional Linguistics, 1:25-36, 2013.
Machel Reid, Nikolay Savinov, Denis Teplyashin, Dmitry Lepikhin, Timothy Lillicrap, Jean- baptiste Alayrac, Radu Soricut, Angeliki Lazaridou, Orhan Firat, Julian Schrittwieser, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv22403.05330, 2024.
Shuhuai Ren, Linli Yao, Shicheng Li, Xu Sun, and Lu Hou. Timechat: A time-sensitive multimodal large language model for long video understanding. In CVPR, pp. 13313-1432, 2024.
ByteDance Seed. Seed2.0 model card: Towards intelligence frontier for real-world complexity, 2026. URL https: bytedance com/ en/ seed2.
Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. In NeurlPS, pp. 8634-8652. 2023.
Enxin Song, Wenhao Chai, Guanhong Wang, Yucheng Zhang, Haoyang Zhou, Feiyang Wu, Xun Guo, Tian Ye, Yan Lu, Jenq-Neng Hwang, et Moviechat: From dense token to sparse memory for long video understanding. arXiv:2307.1644 2023.
Didac Suris, Sachit Menon, and Carl Vondrick. Vipergpt: Visual inference via python execution for reasoning. ICCV, pp. 11888-11899 11898, 2023.
Omkar Thawakar, Dinura Dissanayake, Ketan More, Ritesh Thawkar, Ahmed Heakl, Noor Ahsan, Yuhao Li, Mohammed Zumri, Jean Lahoud, Rao Muhammad Anwer, et al. Llamav-ol: Rethink- ing step-by-step visual reasoning in 1lms. arXiv:2501.06186, 2025.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, pp. 5998-6008,
2017.
Haibo Wang, Zhiyang Xu, Yu Cheng, Shizhe Diao, Yufan Zhou, Yixin Cao, Qifan Wang, Weifeng Ge, and Lifu Huang, Grounded-videellmm: Sharpening fine-grained temporal grounding in video large language models. arXiv:2410.03290, 2024a.
Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model's 's perception of the world at any resolution. arXiv:2409. 12/9/, 2024b.
Weihan Wang, Zehai He, Wenyi Hong. Yean Cheng, Xiaohan Zhang, Ji Qi, Xiaotao Gu. Shiyu Huang, Bin Xu, Yuxiao Dong, et al. Lvbench: An extreme long video understanding benchmark. arXiv:2406.n 08035, 2024c.
14

---

Published as a conference paper at ICLR 2026
Xiaohan Wang, Yuhui Zhang, Orr Zohar, and Serena Yeung-Levy. Videoagent: Long-form video understanding with large language model as agent. arXiv:2403 2024d.
Xiyao Wang, Linfeng Song, Ye Tian, Dian Yu, Baolin Peng, Haitao Mi, Furong Huang, and Dong
Yu. Towards self-improvement of Ilms via mets: Leveraging stepwise knowledge with curriculum preference learning. arXiv:2440.0653 2024e.
Yueqian Wang, Xiaojun Meng, Jianxin Liang. Yuxuan Wang, Qun Liu, and Dongyan Zhao. Hawk- eye: Training video-text Ilms for grounding text in videos. arXiv:2403. 10228, 2024f.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe Le, Denny Zhou, et al. Chain-of- thothu prompting elicits reasoning in large language models. In NeurIPS, pp. 24824-24837, 2022.
Junjian Wen, Linshan Xie, Qiuyi Lyu, Zhangjin Zhang, Ed X Wu, and Alex TL. Leong. Simultaneous rsfmri and eeg reveal the brain-wide actions of delta-gamma neural oscillations coupling. In ISMRM, 2025.
Haoning Wu, Dongxu Li. Bei Chen, and Junnan Li. Longvideobench: A benchmark for long-context interleaved video-language understanding. In NeurlPS, pp. 08828-28857, 2024.
Jianlong Wu, Wei Liu, Ye Liu, Meng Liu, Liqiang Nie, Zhouchen Lin, and Chang Wen Chen. A survey video temporal grounding with multimodal large language model. arXiv:2508. 10922, 2025.
Junbin Xiao, Xindi Shang, Angela Yao, and Tat-Seng Chua. Next-qa: Next phase of question- answering to explaining temporal actions. In CVPR, pp. 9777-9786, 2021.
Junbin Xiao, Angela Yao, Yicong Li, and Tat-Seng Chua. Can i trust your answer? visually grounded video question answering. In CVPR, pp. 13204-13214, 2024.
Linshan Xie, Xunda Wang, Teng Ma, Pit Shan Chong, Lee Wei Lim, Peng Cao, Pek-Lan Khong, Ed X Wu, and Alex TI. Leong. Are topographically segregated excitatory neurons in visual thalamus functionally diverse? optogenetic fmri study. In ISMRM, 2022.
Linshan Xie, Xunda Wang, Teng Ma, Hang Zeng, Junjian Wen, Peng Cao. Ed X Wu, and Alex TL Leong. Short single pulse optogenetic fmri mapping of downstream targets in thalamo-cortical pathways. In ISMRM, 2023.
Linshan Xie, Xuehong Lin, Xunda Wang, Junjian Wen, Teng Ma, Jiahao Hu, Peng Cao, Alex TL Leong, and Ed X Wu. Simultaneous eeg-fmri reveals spontaneous neural oscillatory activity in cingulate cortex underlying transient rsfmri network dynamics. In ISMRM, 2024a.
Linshan Xie, Xunda Wang, Xuehong Lin, Teng Ma, Junjian Wen, Peng Cao, Alex TL Leong, and Ed X Wu. Single-pulse optogenetic perturbation of thalamo-cortical networks reveals functional architecture of rsfmri networks. In ISMRM, 2024b.
Linshan Xie, Xunda Wang, Xuehong Lin, Junjian Wen, Alex TL Leong, and Ed X Wu. Intra- and inter-regional neural activity synchronizations drive rsfmri network dynamics upon single thalamic input. ISMRM, 2025a.
Linshan Xie, Xunda Wang, Xuehong Lin, Junjian Wen, Teng Ma, Alex TL Leong, and Ed X Wu. Brain-wide resting-state fmri network dynamics elicited by activation of single thalamic input. Nature Communications, 2025b.
Guowei Xu, Peng Jin, Li Hao, Yibing Song. Lichao Sun, and Li Yuan. Llava-cot: Let vision language models reason step-by-step. ICCV, 2025.
Lin Xu, Yilin Zhao, Daquan Zhou, Zhijie Lin, See Kiong Ng, and Jiashi Feng. Pllava: Parameter- free llava extension from images to videos for video dense captioning, arXiv:2404.1699 2024.
Ziang Yan, Zhilin Li, Yinan He, Chenting Wang, Kunchang Li, Xinhao Li, Xiangyu Zeng, Zilei Wang, Yali Wang, Yu Qiao, et al. Task preference optimization: Improving multimodal large language models with vision task alignment. arXiv:2412. /9326, 2024.
15

---

Published as a conference paper at ICLR 2026
Antoine Yang, Antoine Miech, Josef Sivic, Ivan Laptev, and Cordelia Schmid. Zero-shot video question answering via frozen bidirectional language models. Advances in Neural Information Processing Systems, 15:124-141, 2022.
Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Ehsan Azarnasab, Faisal Ahmed, Zicheng Liu, Ce Liu, Michael Zeng, and Lijuan Wang. Mm-react: Prompting chatgpt for multimodal reasoning and action. 1rXiv:2303.11381, 2023.
Shunyu Yao, Dian Yu, Jeftrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. In NeurIPS, pp. 1889--18822, 2023a.
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In ICLR, 2023b.
Shoubin Yu, Jaemin Cho, Prateek Yadav, and Mohit Bansal. Self-chained image-language model for video localization and question answering. In NeurIPS, 76749-76771, 2023.
Ce Zhang, Taixi Lu, Md Mohaiminul Islam, Ziyang Wang, Shoubin Yu, Mohit Bansal, and Gedas Bertasius. simple IIm framework for long-range video question-answering arXiv:2312.17255. 2023a.
Chen-Lin Zhang, Jianxin Wu, and Yin Li. Actionformer: Localizing moments of actions with transformers. In ECCV, pp. 492-510, 2022.
Hang Zhang, Xin Li, and Lidong Bing. Video-llama: An instruction-tuned audio-visual language model for video understanding.
Stti:23006002858, 2023b.
Hao Zhang, Aixin Sun, Wei Jing, and Joey Tianyi Zhou. Span-based localizing network for natural language video localization. arXiv:2004.1333 2020a.
Peiyuan Zhang, Kaichen Zhang, Bo Li, Guangtao Zeng, Jingkang Yang, Yuanhan Zhang, Ziyue Wang, Haoran Tan, Chunyuan Li, and Ziwei Liu. Long context transfer from language to vision. arXiv:2406. /6852, 2024.
Songyang Zhang, Houwen Peng, Jianlong Fu, and Jiebo Luo. Learning 2d temporal adjacent net- works for moment localization with natural language. In AAAI, pp. 12870-12877, 2020b.
Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis, and Alex Smola. Multimodal chain-of-thought reasoning language models. arXiv:2302. 00923, 2023c.
Yue Zhao, Ishan Misra, Philipp Krahenbuhl, and Rohit Girdhar. Learning video representations from large language models. In CVPR, pp. 6586-6597, 2023.
Junjie Zhou, Yan Shu, Bo Zhao, Boya Wu, Shitao Xiao, Xi Yang, Yongping Xiong, Bo Zhang, Tiejun Huang, and Zheng Liu. Mlvu: A comprehensive benchmark for multi-task long video 2024.
understanding. arXiv:2406.04200
Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Yuchen Duan, Hao Tian, Weijie Su, Jie Shao, et al. Internv13: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv:2504.1042 2025.
16

---

Published as a conference paper at ICLR 2026
APPENDIX
In this appendix, we provide more details about the model inference pipeline and implementation details to complement the main paper. Additional experiments, detailed analysis, and discussions are also incorporated. Below is the table of contents.
A. Model
1. Inference Pipeline
2. Implementation Details
B. Experiments
1. Benchmarks and Settings
2. More Experimental Results
3. More Detailed Analysis
C. Miscellancous
1. Prompt Templates
D. The Use of LLMs Statement
A MODEL
A.1 INFERENCE PIPELINE
The formulation of VideoMind's inference pipeline is illustrated Algorithm 1. Given a video V and a question Q, the planner dynamically calls different roles demand to analyze the multi- modal context and generate the answer.
Algorithm VideoMind's Chain-of-LoRA Pipeline
1: Input: video V and a question Q
2: Output: An answer A to the question with temporal moment T = [L,, Le]
3: Plan P + Planner(V, Q)
4: if Grounder E P then
5:
{ [t,, f ] + Grounder Q)
6: for all do
7:
1: Zoo.InIVV (2, (d)
8:
Score; + Verifier Q)
9: end for
10: i i arg max (Score.)
11: end if
12: if Answerer E P then
13: A (-Answerer()) Q)
14: end if
15: return (A.T)
A.2 IMPLEMENTATION DETAILS
We leverage the 2B and 7B versions of Qwen2-VL (Wang et al., 2024b) as our base models, and apply LoRA adapters with rank 64 and alpha 64 to the planner, grounder, and verifier. The hidden size of the timestamp decoder is set to 256. The maximum number of tokens per frame and maximum number of frames for the planner, grounder, verifier, and answerer are set as [64, 100], [64, 150], [64, 64], and [256, 32], respectively. We train different roles separately on different datasets and load them together during inference, so that the model can efficiently switch roles by activating different LoRAs. During training, set the global batch size to 32, and utilize the AdamW optimizer (Loshchilov & Hutter, 2019) with learning rates of 2e-5, le-4, and 5e-5 for planner, grounder, and verifier, respectively. All the roles were trained for epoch their specific datasets, with linear warmup in the first 3% steps. During inference, we apply an NMS with ToU 0.75 reduce duplicated moments from the grounder.
17

---

Published as a conference paper at ICLR 2026
Table 9: Details of the evaluation benchmarks. The datasets encompass three representative tasks, le., Grounded VideoQA, Video Temporal Grounding, and General VideoQA, with video durations ranging from several seconds more than one hour.
Duration
Domain
Main Metries
Grounded Video Question Answering (Groanding QA)
CG-Bench (Chen et al,, 2024a)
1624,4s
Diverse Vlog. News, Activity
rec. @oU, acc. @ loU mloU, Acc 0.5) mfoP. Acc
ReXTime (Chen 2024c)
NEXT-GQA (Xiao ol., 2024)
39.5s
Reasoning
Video Temporal Grounding (Grunding only)
Charades
al., 2017)
30.1s
Indeor Activity
(w.3)) mloU
ActivityNes Capliionn (Krishna et al., 2017)
111.4s
RØ (0.3 -.7)) mloU
QVHighlights (Lei et al., 2021)
Vlog. News
(0.5,0.7), 0.7). mAP
TACoS (Regneri et al., 2013)
358.2s
Cooking
RØ (0.3 0.7),, mloU
Ego4D-NNLQ (Gramman al. 2022)
379.0s
Egocentric
R (0.3 0.7) mloU
ActivityNet RTL (Iuang
2024b)
111.4s
Reasoning
P@0.5, mloti
General Video Question Answering (QA only)
Vidco-MME (Fu et al,, 2024a)
1017.9s
Diverse
Acc (w/o subx)
MINU (Zhou al,, 2024)
930s
Diverse
Ace
LVBench (Wang et ol., 2024c)
4101s
Diverse
Acc
MVBench (Li 2024b)
15s
Diverse
LongVidee VideoBBench (Wu et al., 2024)
473s
Diverse
Acc
Table 10: Performance
(Chen et al., 2025a). FT means fine-tuning the
target dataset. Sent. Sim, denotes sentence similarity computed by al all-Ming LM-66V2..
Temporal Grounding
Question A Answering
Method
Size
FT
loU@0.3
mloU
Scat, Sim.
Score
Human
87.0
61.8
74.3
7,5
GPT 4o (OpenAI, 2024a)
X
12.0
12.2
73.7
5.4
InternVL2 (OpenGiVLabs 2024)
8B
X
6.3
6.6
71.9
4.5
LLaVA Video (Lii al,, 2024a)
62.1
TimeChat (Ren et al,, 2024)
7B
X
3.0
3.6
58.9
3.3
VTimeLLM (Huang et al., 2024a)
7B
8.8
9.2
70.5
4.3
GeLM (Chen et 2025a)
7B
18.2
16.7
75.0
4.8
VideoMind (Ours)
2B
X
23.2
17.8
58.8
3.5
VideoMind (Ours)
7B
X
25.1
19.0
77.3
4.9
B EXPERIMENTS
B. BENCIIMARKS AND SETTINGS
The experiments are extensively designed across 15 diverse benchmarks. The statistics are listed in Table 9. The major benchmarks are introduced below.
CG-Bench (Chen et al., 2024a) is designed for long video grounded question answering, featuring diverse domain and various evaluation metrics. It includes 1.2K manually curated videos, ranging from 10 to 80 minutes, with total of 12K QA pairs. The dataset is categorized into perception, rea- soning, and hallucination question types, and introduces clue-based evaluation methods like white box and black box assessments to ensure models provide answers based on accurate video reasoning.
ReXTime (Chen al., 2024c) models on complex temporal reasoning, using an automated pipeline for QA pair generation, significantly reducing manual effort. It includes 921 validation and 2.1K test samples, each manually curated for accuracy, and highlights a 14.3% accuracy gap between SoTA models and human performance. This benchmark is crucial for evaluating models on cause-and-effeed relationships across video segments.
NEXT-GQA (Xiao et al., 2024) aims to challenge models to reason about causal and temporal actions, supporting both multi-choia and open-ended tasks. This is an extension of NEXT-QA (Xiao et al., 2021) comprising 10.5K manually labeled video QA pairs with temporal segments. The samples in this benchmark are from "causal" and "temporal" classes, while the "descriptive" questions in NExT-QA are discarded.
18

---

Published as a conference paper at ICLR 2026
Table 11: Video temporal grounding TACoS (Regneri al., 2013). FT means fine-tuning on the larget dataset. Note that our method was co-trained on this dataset.
Method
Size
FT
R@0.3
R@0.5
R@0.7
mlot
LoneLLMM-baeee Specialists
2D-TAN (Zhang 20206)
40.0
28.0
12.9
27.2
Conent-DETR (Lei et al. 2021)
38.0
24.7
12.0
25.5
UniTGG al., 20236)
R². Tuning (Liu et al., 20244)
49.7
38.7
25.1
35.9
-ussd Generalisty
VideoMind (Ours)
38.6
26.9
15.5
27.4
VideoMind (Ours)
49.5
36.2
21.4
34,4
Table 12: Performance of video temporal grounding on Ego4D-NLQ (Grauman et al., 2022). FT means fine-tuning the target dataset. VideoMind is variant of our method trained with extra 67K egocentric grounding samples from NaQ (Ramakrishnan et al., 2023).
Method
Size
FT
R@0.3
R@0.
R@0.7
mloU
Mon-LLM-buxeed Specialists
2D-TAN (Zhang 20206)
4.3
1.8
0.6
3.4
VSLNet (Zhang et al., 2020s)
4.5
2.4
1.0
3.5
(Lei
2021)
UniVTG iI et al., 2023h)
7.3
4.0
1.3
4.9
R Tuning (Liu et al., 2024d)
7.2
4.5
2.1
49
UniVTG (Lin al., 20236)
3.5
based Generolists
VideoMind (Ours)
X
5.9
2.9
1.2
4.7
VideoMind (Ours)
3.7
VideoMMind-Ego (Ours)
3.9
5.3
Charades-STA (Gao al., 2017) contains 10K in-door videos, averaging 30.1 seconds each, with 16K temporal annotations spanning daily activity, alongside free-text descriptions. These rich an- notations make Charades-STA particularly suitable evaluating temporal grounding models under indoor environments.
SedivityNe-CCaption (Krishna et al., 2017) is a large-scale benchmark with 20K untrimmed YouTube vidcos with total of 849 hours, covering diverse activities from personal care to sports. This dataset contains high-quality dense video captioning annotations (3.65 temporally localized sentences per video), which we queries for video temporal grounding. Each query has average length of 13.5 words.
B.2 MORE EXPERIMENTAL RESULTS
Multi-Hop Grounded Question Answering To investigate the performance of our method on novel tasks that require hybrid or dynamically generated sequence of steps, we evaluate our method on MultiHop-EgoQA (Chen et al., 2025a), a Grounded VideoQA dataset highlighting multi-hop temporal reasoning, For each question, the model must ground and reason on multiple relevant mo- ments before answering, which is a paradigm that does not fit neatly into the pre-defined single-hop grounding pipeline. The evaluation results are shown in Table 10. Thanks VideoMind's archi- tectural design to produce multiple candidate moments in a single grounding step, it can effectively capture the multi-hop evidence required by this benchmark. As a result, our method achieves strong zero-shot performance, surpassing all open-source baselines and remaining competitive to closed- source GPT-4o (OpenAI, 2024a) across both grounding metrics and QA metrics.
Video Temporal Grounding We additionally compare VideoMind with representative methods on the challenging TACoS (Regneri et al., 2013), Ego4D-NLQ (Grauman et al., 2022), and QVHigh- lights (Lei et al., 2021) datasets in Table 11, Table 12, and Table 13, respectively. Our 2B model performs better than the strong task-specific baseline UTG (Lin et al., 2023b) on TACoS but slightly worse than Eg.QD-NL.Q. This is justifiable neither the grounder nor the verifier was
19

---

Published as a conference paper at ICLR 2026
Table 13: Fine-tuned video temporal grounding results on QVHighlights (Lei et al., 2021).
R1
mAP
Table 14: Comparison of performance
Method
Size
e0.5
00.7
@0.5
@.0.5
Avg.
on reasoning temporal localization on
Non: -aadedd Specialists
ActivityNet-RTL (Huang et al., 2024b). Our zero-shot VideoMind-78 outper-
XML (Lei et al., 2020)
41.83
30.35
44.63
31.73
32.14
XML+ (Lei et al, 2021)
46.69
33.46
47 89
34.67
34.90
forms the strong fine-tuned baseline
Momont-DETR (Lei et al. 2021)
59.78
40.33
60.51
35.36
36.14
LITA-13B (Huang et al., 2024b) by a
UMT (Liu et al., 2022)
60.83
43.26
57.33
39.12
38,08
considerable margin.
MofeetDiff (1.i al., 2023b)
58.21
41.48
54.57
37.21
36.84
QD-DETR (Moon et al, 2023)
62.40
44.98
62.52
39.88
39.86
UnIVTG (Lin et al., 2023b)
65.43
50.06
64.06
45.02
43.63
Method
|iize FT P@0.5
mloU
R² -Tninng (Liu et al., 2024d)
68.03
49.35
69.04
47.56
46.17
LITA (Hunng et al., 2024h)
7R
21.2
24.1
LLM-based Generaliats
LITA (Huang ot al., 2024b)
138
/
25.9
28.6
Video (Ours)
2B
75.42
59.35
74.11
55.15
$1.60
VideoMinn (Ours)
2B
20.1
22.7
VideoMind (Ours)
7B
78.53
61.09
76.07
58.17
54.19
VideoMind (Ours)
28.0
31.3
Table 15: Performance of VideoQA on Long VideoBench (Wu al., 2024) val split.
Durution Groups
Method
Size
Ace
(8, 151
(15, 601
(180, 600)
(00,36000
GPT-4o (OpenAI, 2024a)
66.7
71.4
76.7
69.1
60.9
GPT-4 Turbo (Achiam 2023)
59.0
68.2
62.4
50.5
Gemini P.S (Reid et al., 2024)
64.0
67.4
75.1
65.3
58.6
Gemini-1.5.Flaa (Reid al., 2024)
76.2
54.0
Ideficx2 (Laurencon et al., 2024)
8B
49.7
59.8
65.7
47.8
42.7
MisiinVisionn (Ahdin al., 2024)
4B
49.6
59.3
61.6
46.8
44.7
Mantis Idilii22 (Jiang et al., 2024)
8B
47.0
56.6
55.8
45.6
42.2
Mantis Bak1.I aVA (Jiang al., 2024)
43.7
57.6
40.3
38.7
VideoMind (Ours)
2B
48.8
59.3
59.3
49.3
41.7
VideoMind (Ours)
7B
56.3
67.7
67.4
56.8
48.6
Table 16: Performance comparison on general VideoQA on MVBench (Li et al., 2024b).
Model
Size
AP
AA
UA
OE
OT
0S
MD
AL
ST
AC
MC
MA
SC
FP
CO
ER
Avg.
GPT-4V (OpmAl, 2023)
55.5
63.5
72.0
46.5
73.5
59.0
29.5
12.0
40.5
3000
22.5
45.0
47.5
$2.0
59,0
43.5
Video ChatGPT (Mauz et 2023)
23.5
26.0
62.0
22.5
26.5
54.0
28,0
40.0
23.0
20.0
31.0
3055
25.5
39
18.5
29,0
33
29.5
26.0
35.5
32.7
Videe-LLMMA (Zhang
202.6)
40.5
21.0
VideeChar (1 at, 1023a)
33.5
56.0
40.
40.5
27,0
48
35.0
42.5
460
41
23.5
35.5
Vida-111VV (n
20234)
78
46.0
42.5
56.5
39.
53.5
53.0
48.0
410
20.0
31.5
82.5
45.0
26.0
530
415
33.5
41.5
27.5
58.5
31.5
45.0
TimeChat (Ren al., 2024)
7B
40.5
56.0
61.0
32,5
53.0
53.5
41.5
290
19.5
26.5
66.5
54.0
20.0
43.5
42.0
36.5
36.0
29.
35.0
35.0
38.5
PLLAVA (Ku al, 2024)
78
58.0
49.0
55.5
41.0
610
5600
61.0
360
23.5
36.0
820
39.5
47.0
52.0
450
42.0
53.5
305
48.0
31,0
46.6
StandiPT4Vidoo (Chen
20244)
54.5
35.5
ST-LLM (Lib ata 20242)
66.0
54,0
58.5
73.5
42.5
31,0
36.5
78.5
44.5
46.5
41.5
54.9
VideoGiPe (Mazz et al,, 2024)
3MB
69.0
60.
33.0
48.
66.5
85.5
75.5
36.0
44.0
38,0
89.5
39.5
71.
MIL5
45.0
53.0
50,0
29
44.0
60.0
58.7
VideoChac2 (Li et al. 20124b
78
75.5
58.0
53.5
50.5
60.5
57.5
24.5
45.0
47.5
46.0
82.5
37.0
64.5
87.5
51.0
66.5
47.0
35.0
37.0
72.5
60.4
VideoMind (Ouns)
78.5
76.0
75.5
46.0
69.5
90.5
71.5
33.0
65.0
40.0
92.5
52.
11.5
92.0
$1.5
61.5
61.5
37.5
51.0
57.0
62.5
VideoMind (Ouns)
7B
74.0
71.5
$1.0
50.0
77.0
93.0
75.0
38.0
48.5
46.0
91.0
30.0
50.0
94.5
49
55.5
70.0
40.5
57.0
61.0
64.6
trained on egocentric videos, while UniVTG was pretrained on 1.8M samples from Ego4D (Grau- man et al., 2022). To align the setting, we trained an additional VideoMind-2B variant with extra 67K grounding samples from NaQ (Ramakrishnan et al., 2023). To our best knowledge, VideoMind the first LLM-bbsed grounding model that supports multi-mome outputs, thereby being able to be evaluated on QVHighlights. Compared with task-specific experts, our VideoMind-2B significantly outperforms all previous methods and sets a new thth-ar.-
Reasoning Temporal Localization We also evaluate the generalizability of grounder and verifier on the more challenging reasoning temporal localization (Huang et al., 2024b) task, which is similar to video temporal grounding, but the queries are not directly describing the moment. The models are required to infer the actual event using their world knowledge. The results in Table 14 show that VideoMind can successfully generalize its zero-shot grounding capability to complex scenarios.
General Video Question Answering For the task of long VideoQA, we also provide evaluations on Long VideoBench (Wu et al., 2024) in Table 15, which further verifies the effectiveness of Video-
20

---

Published as a conference paper at ICLR 2026
Table 17: Comparison with representative video reasoning methods video QA/grounding tasks.
CG-Bench
MLVU
LVBench
Charades-STA
ActivityNet-aationns
Melhod
long-ace.
M-Avg
Ovemil
R@0.5
R@0.5
mlotr
Pure Text based Reaxoning Models
LongVILA-RI (Chen et al., 20256)
7B
26.7
56.5
34.7
30,3
30.0
16.4
21.4
Video (Feng 2025)
Vizion- Reasoning Models
VideoMind (Ours)
7B
38.4
64.4
40.8
59.1
50.2
30,3
33.3
Table 18: Performance of different timestamp modeling designs on Charades- -TAn STA (Gao et al., 2017).
Method
R@0.3
R@0.5
R@0077
mloU
Text-only (Ren et al., 2024)
56.8
39.5
14.3
36.
Special Tokens ct. 2024a)
56.4
39.2
35.7
Embeddinn Matching (Liu et al., 2024e)
59.6
43.5
17.0
38.2
Time Marker (Chen 2024e)
Timestamp Decoder (Ours)
64.1
47.2
21.7
42.0
Table 19: Case distribution on ReXTime (Chen et al., 2024c) and NEXT-GQA (Xiao et al., 2024).
Correct, Planning, Grounding, Verification, and Answering refers correct prediction, planning error, grounding error, verification error, and answering error, respectively.
ReXeime
NEXT-GQA
Method
Size
Correct
Planning
Grounding
Verification
Answering
Correct
Plamning
Grounding
Verification Answering
VideoMind VideoMind
28
69.1%
1.2%
18.3% 15.0%
5.7%
5.7%
71.2% 76.6%
1.9%
14.0%
6.9%
6.0%
78
74.0%
1.1%
4.6%
4.7%
0.7%
11.8%
5.8%
5.1%
3.9%
4.3%
3.0%
18.4%
18.5%
21.8%
18.4%
59.2%
18.1%
59.1%
24.0%
48.6%
24.8%
50.4%
ReXTime (2B)
(b) ReXTime (7B)
(c) NEXT-GQA (2B) (d) NEXT-GQA (7B)
Figure 7: Error distribution of our 2B and 7B variants on ReXTime (Chen et al., 2024c) and NExT- GQA (Xiao et al., 2024) datasets. The red, orange, blue, and green portions represent planning, grounding, verification, and answering errors, respectively.
Mind on videos scaling to one-hour long. Table 16 presents more results of VideoMind on MVBench (Li et al., 2024b), which is a benchmark with very short videos (around 15s). Our model can still achieve good performance on these short video scenarios.
Comparison with Text-based Reasoning Models In Table 17, we compare method with representative pure text-based video reasoning methods. Our method significantly outperforms both baselines on all benchmarks, demonstrating that vision-centric reasoning is superior to pure text- based reasoning on long/complex video reasoning tasks.
B.3 MORE DETAILED ANALYSIS
Timestamp Modeling Designs The grounder plays a crucial role in our proposed Chain-of-LoRA pipeline. The model's temporal grounding quality directly impacts the final QA accuracy. To demon- strate the necessity of this design, we implement and compare the following alternative timestamp modeling techniques based VideoMind-28 (Grounder):
21

---

Published as a conference paper at ICLR 2026
0.82
0.82
0.77
0.78
0.76
0.7-
0.63
0.70
0.72
0.7
0.65
0.70
0.72
0.69
0.0
0.6
0.5
Grounding IDU
Grounding
(a) ReXTime
(b) NEXT-GQA
Figure 8: The correlation between grounding IoU and the final QA accuracy of VideoMind-28 on ReXTime (Chen al., 2024c) and NEXT-GQA (Xiao al., 2024) datasets.
Table 20: Effect of the temporal feature pyramid on Charades- STA (Gao et al., 2017).
Table 21: Effect of different verifier styles on Charades STA (Gao et al., 2017).
Charades STA
Charades STA
#Pyramid Levels
Verifier Type
R
R@0.5
R@0,7
mlot
R@0.3
R@0.5
R@0.7
mlot/
60.55
44.57
15.82
38.13
Direct
60.42
45.28
19.32
39,84
61.51
46.90
19.36
40.43
Expand
65.10
48.70
23.15
43.57
41.27
Textual
63.55
47.23
21.69
42.02
Special Token
67.63
51.05
25.99
45.22
Table 22: Effect of the verifier Charades-STA (Gao al., 2017). IoU Raise means the percentage of the samples whose grounding loU is raised by the verifier.
Role(s)
Size
R@
R@0.5
R R@0.7
mloU
IoU Raise
Grounder
2B
63.2
46.9
20.5
41.7
Grounder Verifier
68.0
(47.6%)
51,2
(+9.2%)
24.3 (+18.5%)
44.8 (47.4%)
32.9%
Grounder
69.4
53.2
26.6
46.8
Grounder Verifier
73.8 (46.3%)
59.1 (+11.15)
(+13.2%)
49.8
(+6.4%)
31.3%
Table 23: The accuracy of planner with different input combinations.
Table 24: Comparison of average inference time CG- Bench (Chen et al., 2024a) (avg. duration: 27 min).
Inpul Video
Input Question
Planning Ace
Melhod
Size
Inference Time (s/ideeo)
0.42
LongVIIAARI (Chen et nl., 2025h)
7B
8.75
VideuMind
7B
9.53 (+8.9%)
0.93
VideoMind (w. Auto Planning)
7B
8.07 (-7.8%)
1. Text-only (Ren et al., 2024): Directly represent timestamps in text form (e.g., "2.3 seconds").
2. Special Tokens (Qian et al., 2024a): Define set of timestamp tokens (e.g., <TO>, <TI>). Embedding Matching (Liu et al., 2024e): Predict frame features retrieve the frame index.
4. Time Marker (Chen et al., 2024e): Explicitly insert textual timestamps among visual tokens.
Their zero-shot video temporal grounding results are shown in Table 18. The results clearly demon- strate that the timestamp decoder delivers the strongest temporal grounding capability. We attribute it two key factors: (1) It decouples continuous timestamp modeling from discrete token prediction, allowing the model to represent time with higher precision; (2) The direct regression supervision (I.1 Loss) further enhances time reasoning and stabilizes training. Moreover, the timestamp decoder naturally supports predicting multiple moments with corresponding confidence scores, supporting tasks like multi-moments retrieval (Lei et al., 2021) and facilitating moment re-ranking through the verifier. These advantages jointly enhance the reliability of temporal grounding, which ensures the correct moment could be localized for further reasoning.
Effect of the Temporal Feature Pyramid Table 20 studies the effectiveness of the temporal fea- ture pyramid. Our baseline model directly makes predictions on the last-layer transformer outputs. When adding more pyramid levels, the performance of video temporal grounding consistently im-
22

---

Published as a conference paper at ICLR 2026
Table 25: Controlled experiments with strictly aligned hyperparameter settings. Both MLVU (Zhou et al., 2024) and LVBench (Wang et al., 2024c) are downsampled to 300 samples cach.
MLVU (mini)
LVBench (mini)
Method
Size
M-Avg
Overall
4P (OpenAI, 2024a)
59.7
31.3
Gemini- Pro-Pro (Reid et al,, 2024)
60.3
36.3
VideoMind (Ours)
2B
59.3
35.7
VideoMind (Ours)
7B
62.7
40.3
Table 26: Performance comparison among the integration of our Chain-of-LoRA mechanism on
different representative base models.
CG-Bench
ReXTiime
Video-MME
MLVU
LVBench
Model
Size
@HoU
Acc
M-Avg
Overall
Qwen2 VL (Wang el al., 2024b)
4.0
17.3
55.4
58.7
35.4
7B
4.7
20.2
58.2
64.4
40.8
Qwen2.5.VL VL (Bai et al., 2025)
15.6
7B
5.7
19.8
65.9
66.3
45.2
VntemVL3 (Zhu et al., 2025)
28
4.1
17.5
58.2
61.4
38.1
8B
4.5
20.8
66.5
63.8
42.3
Table 27: Performance of the simulated multi-role pipelines on closed-soure models. Both MLVU (Zhou et al., 2024) and LVBench (Wang et al., 2024c) are downsampled to 300 samples each.
Multi-Role Pipeline
MLVU (mini)
LVBeuch (mini)
Method
M-Avg
Overall
GPT-4o (OpenAl, 2024a)
>
X
59.7
31,3 32.7 (+4.5%)
(+4.4%%
61.7
34,3
GPT-5 (OpenAl, 2025)
/
63.3 (+2.66%)
36.3 (45.8%)
Gemini-2.55 P.o-Pro (DeepMind, 2025)
65.7
76.3 (+4.1%%
68.7 (44.6%)
proves under metrics on Charades-STT STA (Gao et al., 2017) under zero-shot setting, suggesting the effectiveness of improving the robustness of the model when facing moments with different lengths.
Effeet of the Verifier for Zoom-in Evaluation To quantify the verifier's 's corrective gain, we pro- vide comparison between w. and w/o the verifier on Charades-STA (Gao et al., 2017) in Table 22. The results demonstrate that the verifier consistently enhances temporal grounding performance, es- pecially on high-quality predictions (e.g., 18.5% higher R@0.7 7 on the 2B variant), highlighting its importance in the overall pipeline.
Design Choices of Verifier In Table 21, examine various design choices for the verifier. The term "Direct" refers to the method where the grounded moment is directly sent into the model with- any expansion. "Expand" denotes expanding the temporal boundaries by 50%, while "Textual" involves adding supplementary textual information to indicate the length of the target event. "Spe- cial Token" represents our final approach, utilizing special tokens denote the grounded start and end timestamps. The comparison demonstrates that expanding the temporal boundaries effectively broadens the verifier's perceptual range, and the use of special tokens enhances the model's aware- ness of precise moment boundaries.
Reliability of the Planner We provide an in-depth investigation into the reliability of the planner. Specifically, we randomly split the planner's training dataset into an 80% training set and 20% test set, and then re-train the planner on the training set and evaluate as a three-way classification task on the held-out test set. The metric planning accuracy is defined as the proportion of samples for which the predicted reasoning plan correct. The comparison among different input combinations in Table 23 demonstrate that incorporating both video (even with low resolution) and question input substantially improves planning performance, and the resulting 93% accuracy reflects the considerable reliability of the planner.
23

---

Published as a conference paper at ICLR 2026
Inference-Tinee Efficiency In Table 24, we study the inference- time efficiency of our method on CG-Bench (Chen et al., 2024a). All experiments are conducted on a single NVIDIA RTX 6000 Ada GPU. Compared with the text-based reasoning baseline LongVILA-RI (Chen et al., 2025b), our full pipeline is approximately 8.9% slower. However, this gap can be easily bridged by activating the planner's aatoo-paannng capability. When the planner is allowed to choose the reasoning path, some easy questions are routed directly to the answerer, which substantially reduces the average inference time from 9.53s to 8,07s per video, resulting 7.8% faster inference speed than the baseline.
Overall Robustness and Error Accumulation We acknowledge that the proposed sequential rea- soning pipeline has the potential risk of error propagation and accumulation. To quantify this effect, we conduct systematic analysis of error propagation on two representative datasets: ReXTime (Chen et al., 2024c) (more temporal-related) and NEXT-GQA (Xiao et al., 2024) (more reasoning- related). For both datasets, each error case is categorized into one of the following types: (1) Plan- ning Error: The question can only be correctly answered by switching to another reasoning plan (e.g., from "all roles" to "answerer only"); (2) Grounding Error: All the top-5 predicted moments are incorrect (i.e., having temporal IoU < 0.5); (3) Verification Error: The moment selected after verification is incorrect; (4) Answering Error: The predicted answer is incorrect.
We present the case distributions in Table 19 and error distributions in Figure 7. Several conclusions can be drawn from the results: (1) The planner is highly reliable, accounting for less than 5% of the error cases on both datasets; (2) Grounding errors account for roughly half of all failures. This is aligned with our hypothesis that accurate temporal grounding plays a crucial role in the multi-role reasoning pipeline; (3) Verification and answering contribute comparably smaller portions of the failures, accounting for only about 20% error cases each.
Correlation between Grounding IoU and QA Accuracy We study the correlation between tem- poral grounding performance and QA accuracy in Figure 8. Specifically, we group the samples in ReXTime (Chen et al., 2024c) and NEXT-GQA (Xiao et al., 2024) datasets into different IoU buck- cts. and plot the average QA accuracy within each bucket. On ReXTime, which is more temporal- related, the results show a clear positive correlation between grounding IoU and final QA accuracy. On the more reasoning-related NEXT-GQA, such correlation is less significant.
Controlled Experiments on Closed-source APIs The results of closed-source models in Table 2 and Table are reported from the corresponding benchmark papers, without strictly aligned settings. Therefore, we provide controlled experiment to validate the advantages of our method. Specifi- cally, we select two challenging long video understanding benchmarks, i.e., MLVU (Zhou et al., 2024) and LVBench (Wang et al., 2024c), and randomly sample 300 QA pairs from each, forming MLVU (mini) and LVBench (mini). We align the key hyperparameters as follows:
1. Frame Rate: 1 FPS
2. Max Frame Count: 150
3. Frame Resolution: max 448 x 448 pixels with natural aspect ratio
4. Model Hyperparameters: temperat = 0, top.p 0, top = 0
The comparisons are presented in Table 25, clearly showing that our VideoMind-7B outperforms both GPT-40 (OpenAI, 2024a) and Gemini- 1---Po (Reid et al., 2024) on both datasets.
Integration with More Open-source LMMs In Table 26, we study whether the proposed Chain- of-LoRA pipeline provides a consistent benefit across different base models. The results show that when integrated with stronger base models like Qwen2.5-VL. (Bai et al., 2025) and Intern (Zhu et al., 2025), the performance of our Chain-of-LoRd, pipeline could be further enhanced on multiple long video benchmarks. highlighting our method's s generalizability.
Integration with Closed-source LMMs We are also interested in whether the proposed multi- role pipeline could be simulated via series of prompts on closed-soures source models. To investigate this, we evaluate the effectiveness of the multi-role reasoning prompt when applied to three models, i.e. GPT-4o (OpenAI, 2024a), GPT-5 (OpenAI, 2025), and Gemini-2.5-Pra (DeepMind, 2025), on the previously constructed MLVU (mini) and LVBench (mini). The results in Table 27 show that our pipeline consistently boosts the performance of different closed-source models, highlighting an interesting finding: the multi-role reasoning pipeline itself can systematically enhance long video reasoning, even without specispecific model designs or training.
24

---

Published as a conference paper at ICLR 2026
C MISCELLANEOUS
C.I PROMPT TEMPLATES
We present the prompts used in this work, including the input prompts for each role of VideoMind and the prompt for GPT-4o mini (OpenAI, 2024a) for data annotation.
Prompt for the Planner:
are acting as the planner now. Given a question about the video, your task is to analyze the question and identify the best way to answer this question. You have access to the following tools:
query and localizes the relevant video segment according the query. Verifier: tool supporting grounder by verifying the reliability of its outputs.
Grounder: Accepts
Answerer:
Answer given question directly based on the whole video or a cropped video segment.
Your response must be a list in JSON format. A valid plan for reasoning could be "grounder, ver- ifier, answer", "grounder, verifier", "answerer", depending the given question. Please example of format below.
[["type": "grounder", "value": "text "u ("type": "verifier"), ""type":
"answerer"H
Note that only the grounder accept argument called "value", which is the text query used for grounding. Now give you question: "[question)" )". Please think carefully and respond with your plan in ISON directly.
Prompt for the Grounder:
You acting the grounder now. Given video and text query, your goal is temporally localize the video moment described by the query. If query directly describing moment, simply localize according to its content. Otherwise, if the moment is described as "before/after a pivotal event", you need to determine the actual event it refers to. The localized moment should only cover the target event. Now I give you the query: "[query)". Please think carefully and provide your response.
Prompt for the Verifier:
You are acting as the verifier now. You will be presented a text query deseribing moment that potentialy happens in the given video. Your task is to identify whether the video segment between <SEG-STAA START> and <SEG-END> perfectly covers the moment. the described moment focus verifying whether the moment starts <SEG- SSEG-START> and ends <SEG-END END>, Respond with "Yes" you think the moment boundarics are correct, otherwise "No". the described moment cannot be seen in the video, respond with "No" directly. Now I give you the query: "{query)". Please think carefully and respond with "Yes" "No" dircctly.
be in the video, please
Prompt for the Answerer: When subtitles are considered, we only present the first 100 lines.
You are given vidco with {duration) scconds long. Subtitles: {subtitles)
{question]
Options:
(A) foption 1}
(B) foption 2
(C) foption 3}
(D) {option 4}
Please only give best option.
Prompt for Query Rephrasing Data Generation:
25

---

Published as a conference paper at ICLR 2026
You are an expert in rewriting questions into queries. I will give you question that requires to be answered based on specific moment in a video. Your task is to analyze the question and rewrite into declarative sentence, which could be used as text query to search for the relevant vidco moment. The query should concise, describing the key event key scene that the question asks for. Here are some examples:
Question: How does the male eyclist react when he sees the steep path?
Query: The male cyclist sees the steep path.
Question: What did the girl do at the end of the vidco?
Query: The end of the video.
Question: What did lady do was cycling off? Query: The lady is cycling off.
Question: What is the person with red shirt doing the yacht?
Query: The person with shirt stays on the yacht.
Now T give you the question: "(question)"
Please think carefully and respond with the query directly.
D THE USE OF LLMS STATEMENT
Large Language Models (LLMs) were used in this study aid in polishing the manuscript. Specif- ically, we used LLMs to assist in refining the language and detecting potential grammatical errors. This is improve readability and ensure clarity of the paper. We confirm that L.MSs not in- volved in research ideation, method exploration, and experiment designs. All research ideas, meth- ods, and analysis were produced by the authors. We take full responsibility for the content this paper, including the text generated or polished by the LLMs.
26


Reviews:
[{"review_id": "AVl77zcb9T", "reviewer": "Reviewer_vJ8R", "summary": "This paper introduces VideoMind, a novel video-language agent designed to address the challenges of temporal-grounded reasoning in long-form videos. The authors identify that existing methods struggle to explicitly localize and revisit relevant video segments, a process that is natural for human cognition. The system decomposes the complex task of video reasoning into four distinct roles: a Planner to coordinate the workflow, a Grounder to localize temporal moments, a Verifier to assess and refine these moments, and an Answerer to synthesize the final response. This structured approach mimics a human-like process of breaking down problems, identifying evidence, and confirming details. To efficiently implement this multi-role system, the authors propose a novel inference-time strategy. A single base Large Multimodal Model (LMM) is equipped with multiple, role-specific Low-Rank Adapters (LoRAs). During inference, the agent can seamlessly and efficiently switch between roles (Planner, Grounder, Verifier) by activating the corresponding LoRA adapter, avoiding the significant memory overhead of loading multiple full models.", "questions": "Why the authors miss the comparison with other video reasoning works, such as LongVILA-R1-7B[1], VIdeo-R1-7B[2]\n\n[1] Scaling RL to Long Videos\n[2] Video-R1: Reinforcing Video Reasoning in MLLMs", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1.The core idea of decomposing video reasoning into an agentic workflow of Planner, Grounder, Verifier, and Answerer is highly intuitive and effectively addresses the limitations of monolithic, end-to-end models. It formalizes a \"re-watching\" and \"self-verification\" loop that is crucial for complex video understanding.\n\n2. The Chain-of-LoRA mechanism is a standout contribution. It provides a highly practical solution to the problem of test-time scaling for multi-capability agents. By caching LoRA adapters in memory, it achieves the performance and flexibility of a distributed, multi-model system while maintaining the memory efficiency of a single model. The ablation study in Table 7 provides compelling evidence for this, showing it matches the performance of the \"All-Distributed\" approach with a fraction (≈1/4) of the memory cost.\n\n3. The individual components are well-designed. The Grounder's timestamp decoder, which moves beyond simple text-based timestamp generation, and the Verifier's \"zoom-in\" strategy with special boundary tokens (<SEG-START>, <SEG-END>) are thoughtful architectural choices that contribute directly to the model's strong grounding performance.", "weaknesses": "1. The Planner currently relies on selecting one of three pre-defined reasoning plans based on the query. This approach may be brittle and lack flexibility when faced with novel tasks or complex queries that require a hybrid or dynamically generated sequence of steps. The paper does not explore how the system would perform on tasks that do not fit neatly into these templates.\n\n2. The roles are trained separately on curated datasets, with the Answerer role not being fine-tuned at all. This multi-stage training process, while effective, may be complex to replicate and sensitive to the quality and composition of the data for each stage. A deeper discussion on the challenges of this approach versus a more integrated, joint-training strategy would be insightful.\n\n3.The pipeline is inherently sequential. An error in an early stage can propagate and lead to a complete failure. For example, if the Planner mischaracterizes the query and chooses the wrong plan, or if the Grounder fails to identify any plausible moments, the subsequent steps are rendered useless. The paper could benefit from a discussion of the system's robustness and any potential fallback mechanisms to handle such cascading errors.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LR2UzAg3Ud", "reviewer": "Reviewer_RtrU", "summary": "This paper presents VideoMind, a novel video-language agent designed for temporal-grounded video reasoning. The authors identify a key limitation in existing models: while strong at text-based reasoning, they struggle to explicitly localize, revisit, and reason about specific temporal moments in long videos.", "questions": "See Weaknesses.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper proposes the \"Chain-of-LoRA\", which is a highly practical solution to a major challenge in agentic AI. It directly addresses the need for multi-skilled agents without the prohibitive memory overhead of loading multiple full models.\n2. The paper's claims are backed by a comprehensive set of experiments across 14 benchmarks. The results are consistently strong.", "weaknesses": "1. While the 'Chain-of-LoRA' mechanism is shown to be highly memory-efficient, how does the total wall-clock inference time of the full, multi-step VideoMind pipeline (Planner $\\rightarrow$ Grounder $\\rightarrow$ Verifier $\\rightarrow$ Answerer) compare to an end-to-end baseline model that performs reasoning in a single forward pass? Does the paper provide any latency benchmarks (e.g., in seconds per video)?\n2. How dependent are the results on the Qwen2-VL architecture? Could the 'Chain-of-LoRA' training approach be applied to other strong open-source VideoLLMs (e.g., InternVL3 or Qwen2.5-VL) to validate that this agentic pipeline provides a consistent benefit across different base models?\n3. The workflow itself, separate from the LoRA implementation, could be a significant contribution. Could this multi-step reasoning pipeline (plan $\\rightarrow$ ground $\\rightarrow$ verify $\\rightarrow$ answer) be simulated via a series of prompts on closed-source models like GPT-4o, Gemini 1.5 Pro or newer models? This would help isolate the performance gain of the agentic workflow from the specific model training.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UEt5BQeo9j", "reviewer": "Reviewer_3Ldi", "summary": "The paper proposes VideoMind, a unified multimodal agent framework for long-video temporal grounding and question answering. It decomposes capabilities into four roles—Planner, Grounder, Verifier, and Answerer—and employs a Chain-of-LoRA strategy to efficiently invoke different roles by switching LoRA adapters on the same base large language model. The model is evaluated on 14 datasets spanning Grounded VideoQA, Video Temporal Grounding, and general VideoQA, reporting state-of-the-art or competitive results on multiple long-video benchmarks. The authors also commit to releasing the code and models.", "questions": "1.\tHave you attempted multi-task training that simultaneously optimizes the Planner/Grounder/Verifier on a shared backbone? If such attempts were unsuccessful, please share the main challenges and stabilization techniques. \n2.\tError sensitivity: Could you provide a sensitivity curve from the Grounder’s mIoU to the final QA accuracy, or bucket results on NExT-GQA by candidate-segment IoU to quantify the Verifier’s corrective gain? \n3.\tResource–performance trade-off: While Chain-of-LoRA has low VRAM usage, what is the time cost of repeated “zoom-out rechecks” and multi-candidate verification during inference? Can you report wall-clock comparisons and throughput curves on CG-Bench?\n4.\t Fair comparison with closed-source LMMs: Are frame sampling for long videos, maximum frame count, subtitle usage, temperature/sampling steps, etc., aligned? Could you include strictly controlled settings in the appendix to substantiate the stronger claim that the 2B/7B variants surpass GPT-4o/Gemini?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.\tThe method is structurally clear and engineering-efficient: it modularizes planning, temporal localization, verification, and answering, and realizes one model with multiple roles via LoRA switching, which reduces GPU memory while keeping flexible composition—an appealing design. \n2.\tThe Grounder includes a decoder specialized for temporal boundaries: a <REG>-triggered timestamp decoder and a multi-scale temporal pyramid, avoiding reliance on language tokens alone for time prediction; the technique is solid.\n3.\tThe Verifier adopts zoom-out recheck plus boundary markers: it expands both sides of a candidate segment and inserts <SEG-START/END>, performing binary validation to improve IoU and robustness, with clear training and inference definitions. \n4.\tEfficiency–effectiveness trade-off: compared with multi-model dispatch, Chain-of-LoRA achieves comparable accuracy while significantly reducing memory usage (4.2 GB vs. 16.6 GB), validating the design intent.", "weaknesses": "1.\tNovelty boundary: While the multi-role agent and LoRA switching are practical, their difference from existing modular tool-based agents with vision APIs/components lies mainly in implementation form rather than a new paradigm; stronger theoretical grounding or unified learning evidence is needed to highlight originality.\n2.\t Joint training and error propagation: The authors acknowledge in limitations/future work that better joint optimization is required. The current pipeline-style chaining of roles may accumulate upstream errors, and the paper lacks quantitative analysis of error propagation. \n3.\tReproducibility details and open-source dependencies: Although the appendix lists training hyperparameters and dataset splits, implementation details for data reuse/annotation across roles remain brief. It is recommended to provide role-level prompts, sampling, and filtering scripts in the supplementary material to ensure others can reproduce the key results in Tables 2/3/5. \n4.\tChoice of baselines and strength of claims: Comparisons to GPT-4o and Gemini-1.5-Pro emphasize advantages on portions of long-video benchmarks, but input/context configurations and sampling strategies for these closed models are not fully aligned. Controlled comparisons with equal frame sampling, total frame count, and resolution are suggested. \n5.\tCross-domain generalization and failure cases: On domain-specific datasets such as Ego4D-NLQ, performance lags behind strong pre-trained specialists; the paper provides only brief explanations and lacks targeted analyses and error visualizations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vaQveh8Sd0", "reviewer": "Reviewer_ocUG", "summary": "This paper proposes VideoMind, a video-language agent for temporal-grounded reasoning in long-form videos. The method's core is an agentic workflow comprising four distinct roles—Planner, Grounder, Verifier, and Answerer—that systematically decompose and address complex video queries. To efficiently integrate these roles, the authors introduce a novel \"Chain-of-LoRA\" mechanism, which leverages a unified base model with lightweight, switchable LoRA adapters for each role. The paper demonstrates the effectiveness of this approach through extensive experiments on benchmarks across three tasks: Grounded VideoQA, Video Temporal Grounding, and General VideoQA.", "questions": "Please see Weaknesses", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The \"Chain-of-LoRA\" mechanism is a key strength, enabling a single model to efficiently switch between specialized roles. This design achieves a strong balance between performance and memory efficiency, making it a more practical alternative to deploying multiple, full models.\n\n* The paper is supported by a thorough and comprehensive evaluation across 14 diverse benchmarks spanning three different video understanding tasks. The claims are well-substantiated by detailed ablation studies that validate the contributions of individual components and the overall framework design.", "weaknesses": "* The paper lacks a granular, independent evaluation of each module's reliability. For instance, there is no isolated analysis of the Planner's accuracy in task decomposition, the Grounder's precision, or the Verifier's classification performance. This makes it difficult to pinpoint sources of error or understand the individual limitations of each component.\n\n* The design of the Timestamp Decoder appears overly complex, and the paper fails to provide sufficient justification for this complexity. There is no direct, apple-to-apple comparison against simpler, more common alternatives (e.g., directly predicting timestamps through language modeling), making it difficult to assess the true benefit of the proposed decoder architecture.\n\n* The framework's reliance on an external model (GPT-4o) for query rephrasing undermines its contribution as a self-contained solution. This dependency introduces potential reproducibility issues and reliance on a proprietary API.\n\n* The paper overlooks several recently published and highly relevant papers in video temporal grounding [1-4].\n\n[1] Universal Video Temporal Grounding with Generative Multi-modal Large Language Models\n\n[2] TRACE: Temporal Grounding Video LLM via Causal Event Modeling\n\n[3] Grounded-VideoLLM: Sharpening Fine-grained Temporal Grounding in Video Large Language Models\n\n[4] LLaVA-ST: A Multimodal Large Language Model for Fine-Grained Spatial-Temporal Understanding", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)