Divid: Disentangled Spatial-Temporal Modeling within LLMs for Temporally Grounded Video Understanding
Authors: Yepeng Tang
Weining Wang
Longteng Guo
Tongtian Yue
Wenxuan Wang
Chunjie Zhang
Jing Liu
Venue: ICLR 2026 Poster (2026)
Keywords: Video understanding; Temporal grounding; VideoLLM
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Recent advances in Video LLMs have improved video understanding performance, but temporally grounded understanding in long-form videos remains challenging. Most models encode video frames into a flat sequence of visual tokens, which are then processed together with textual input by the LLM. While effective for short videos, this approach becomes inefficient for long-form videos due to lengthy token sequences that exceed context limits and incur high computational costs. Slow-Fast architectures partially address this by separating temporal and spatial features during encoding, but these features are still processed jointly within the LLM, lacking true spatio-temporal disentanglement. Moreover, spatial features are typically sampled in a query-agnostic manner, risking the loss of task-relevant content. To address these limitations, we propose Divid, a novel dual-branch framework that explicitly disentangles spatial and temporal modeling within the LLM decoder. Specifically, the temporal branch processes densely sampled, low-resolution frames to effectively capture long-range motion dynamics, while the spatial branch selects a sparse set of high-resolution keyframes guided by temporal attention. To unify the two branches, we design a lightweight spatio-temporal soft-router that adaptively fuses temporal and spatial cues at the token level, conditioned on the input query. This disentangled architecture not only improves temporal alignment accuracy but also leads to computational savings by minimizing redundant visual processing. Furthermore, we introduce TempGCap, a large-scale dataset consisting of 559K timestamp-grounded video-text pairs, providing rich temporal supervision. Extensive experiments on temporal grounding and grounded videoQA benchmarks demonstrate the superior performance and efficiency of our proposed Divid.

Published as a conference paper at ICLR 2026
DIVID: DISENTANGLED SPATIO-TEMPORAL MODELING WITHIN LLMS FOR TEMPORALLY GROUNDED VIDEO UNDERSTANDING
Yepeng Tang Weining Wang*., Longteng Guo".,, Tongtian Yue*,, Wenxuan Wang Chunjie Zhangl.21, Jing Liu
Instittte of Information Science, School of Computer Science and Technology, Beijing Jiaotong University
Visual Intelligence X International Cooperation Joint Laboratory of MOE, School of Computer Science and Technology, Beijing Jiaotong University Institute of Automation, Chinese Academy of Sciences "School of Artificial Intelligence, University of Chinese Academy of Sciences
ABSTRACT
Recent advances in Video LLMs have improved video understanding perfor- mance, but temporally grounded understanding in long-form videos remains chal- lenging. Most models encode video frames into a flat sequence of visual tokens, which are then processed together with textual input by the LLM. While effec- tive for short videos, this approach becomes inefficient for long-form videos due to lengthy token sequences that exceed context limits and incur high computa- tional costs. Slow-Fast architectures partially address this by separating temporal and spatial features during encoding, but these features are still processed jointly within the LLM, lacking true spatio-temporal disentanglement. Moreover, spa- tial features are typically sampled a query-agnostic manner, risking the loss of task-relevant content. To address these limitations, propose Divid, novel dual-branch framework that explicitly disentangles spatial and temporal modeling within the LLM decoder. Specifically, the temporal branch processes densely sam- pled, low-resolution frames to effectively capture long-range motion dynamics, while the spatial branch selects sparse set of high-resolution keyframes guided by temporal attention. To unify the two branches, we design a lightweight spatio- temporal soft-router that adaptively fuses temporal and spatial cues the token level, conditioned on the input query. This disentangled architecture not only im- proves temporal alignment accuracy but also leads to computational savings by minimizing redundant visual processing. Furthermore, we introduce TempGCapp a large-scale dataset consisting of 559K timestamp-groumded video-text pairs, pro- viding rich temporal supervision. Extensive experiments temporal grounding and grounded videoQA benchmarks demonstrate the superior performance and efficiency of our proposed Divid.
INTRODUCTION
Recent advances in video-language models with large language models (Video LLMs) have signif- icantly improved the performance of video understanding tasks (Li et al. 2023a Liu et al. 20024a] 2025). Despite these advances. temporally grounded video understanding in form videos mains a challenging problem. These tasks require models to reason about complex structures and align visual content with language instructions accurately at temporal granularity. A common paradigm in existing Video LLMSs is to encode video frames into a flat sequence of visual tokens via a visual encoder, which are then fed into the language model alongside textual input (Liu) et al. 2024a Lin et al.) 2023a). While effective on short video tasks, this approach encounters lim- itations when scaling to long- form videos. In long-form video scenarios, densely sampled frames
"Intered at the Institute of Automation, Chinese Academy of Sciences. "Corresonniigg authors. Emails: ejzhang www.ngg@hhi.uuu...n. jliu@nnpr.ia.acccm

---

Published as a conference paper at ICLR 2026
often result in overly long token sequences, leading to significant computational overhead and po- tentially exceeding the context window of LLMs. To mitigate this, recent studies (Xu et al. 2025] Huang et al. 2024b, Zhao et al. 2025) adopt the Slow-Fast (Feichtenhofer et al. 2019) architec- ture, which decouples video features into temporal and spatial tokens before concatenating them as input to the L.M.M However, this design remains limited. Temporal and spatial information processed jointly within the LLM. lacking explicit separation, and often leading to temporal confu- sion or hallucinations (Bac al. 2025). Moreover, spatial features are typically extracted through -ager-agnosticc uniform sampling, which lacks alignment with the language query and may overlook critical visual information relevant to the task.
still
To address these limitations, we propose Divid, novel framework that explicitly integrates spatio- temporal disentanglement inside the LLM decoder architecture. Inspired by humans first sean the global motion of video before attending to detailed video content, Divid employs two com- plementary branches, as shown in Figure [ The Temporal Branch processes densely sampled, low-resolution frames to effectively capture long-range video dynamics. Concurrently, the Spatial Branch dynamically selects a subset of high-resolution keyframes based on attention maps gener- ated by the temporal branch, enabling the model to perform query-awan Top-K key frame selection. Furthermore, we introduce a apatio-temporal soft-router embedded within each decoder layer. This component allows each language token to adaptively integrate spatial and temporal features through a lightweight, query-conditioned gating mechanism.
To further facilitate temporally grounded video understanding, we construct TempGCap, large- scale dataset consisting of 559K instruction-bar based, timestamp-groundee video-text pairs. The dataset combines human-annotated action boundaries, reassembled long-form videos, and curated stitched short videos, offering rich temporal supervision and diverse scene coverage. Compared to previous instruction-tuniin datasets such as Moment-10M (Qian et al. (024a), TempGCap offers significantly enhanced temporal precision and improved alignment between video content and lan- guage annotations, making particularly suitable for training video-language models on temporally grounded understanding tasks.
Extensive experiments on Charades-STA ReXTime, CG-Bench, and NEXT-GQA demonstrate that Divid consistently outperforms existing Video LLMs, achieving superior temporal grounding and video understanding. Notably, Divid achieves these enhancements while substantially reducing computational costs. In summary, our work makes the following key contributions:
We propose Divid, a dual-branch video-language framework that disentangles spatial and temporal modeling within the LLM decoder. It incorporates temporally guided keyframe selection and soft-router for dynamic spatio-temporal fusion, enabling fine-grained tem- poral grounding.
We construct TempGCap,, a large-scale instruction-tunin dataset with 559K timestamp- grounded vidco-text pairs. providing high temporal precision and diverse video coverage for training temporally grounded languaan models.
Extensive experiments on multiple temporal grounding and grounded VideoQA bench- marks demonstrate the effectiveness and generalization ability of our approach across di- verse temporal understanding tasks.
2 RELATED WORK
Video Large Language Models. Recent progress in large language models (LLMs) (Chiang et al. 2023 Touvron et al. 2023 Bai et al., 2025) has spurred the development of video LLMs, which integrate vision encoders with LLMs via projection bridges (Li et al. 2023a Zhang et al. 2023b). However, when applied to long videos, this paradigm leads to excessively long visual token se- quences that exceed LLMs' context limits and incur high computational costs. To address this limitation, several studies compress video features before passing them into the LLM (Song et al.) 2024 Shen et al. 2024 Zhang et al. 2025), thereby enabling the model to process a greater num- ber of frames within the available context window. However, such compression, particularly along the temporal dimension, negatively affects the model's temporal grounding capabilities. Inspired by the Slow-Fast paradigm (Feichtenhofer et al. 2019), several recent works (Huang et al.) 22024b]
Xu al. 2025 Zhao et al. 2025|
Bae et al. 2025) aim disentangle video representations into
2

---

Published as a conference paper at ICLR 2026
Output
The timestamp is from 31.0 seconds to 54.6 seconds.
FEN
Next LLM Layer
Temporal Perception Attention
Spatial Perception Attention
XN
Temporal Attention Scores
Key Frame Selection
Guidance
00:0
(T XXXXW)x
L d
x H XX
Large language Model
Temporal tokens
Spatial tokens
Text tokens
MLP
Spatial Pooling
Vision Encoder
Tokenizer
Input Video
Question
What timestamp of the moment when the man setting up the
Figure I: Overview of the proposed Divid framework.
fast tokens for capturing long-range temporal dependencies and slow tokens for preserving spatial details. LITA (Huang et al. 2024b) and ClowFast-Videoll (Xu et al. 2025) implement this idea by extracting temporal and spatial features separately, but simply concatenate them before feeding into the LLM, leading to limited spatio-temporal separation. VideoExpert (Zhao et al.) 2025) intro- (MoE) strategy, where temporal tokens regress timestamp boundaries via a dedicated head, while spatial tokens are used for caption generation. However, these methods still operate under a unified attention space, making them prone spatio-temporal entanglement and hallucination (Bae et al. 2025). To mitigate this, MASH-VLM (Bae et al. (0255) proposes to decouple attention via masking, yet relies heavily on predefined posítional encodings and remains incompatible with efficient attention mechanisms. Moreover, spatial tokens in these approaches are typically derived via uniform frame sampling, which disregards query relevance and may overlook task-critical visual content.
duces Mixture-of-Expeee
Temporally Grounded Video Understanding. Temporally grounded video understanding encom- passes two core tasks: temporal grounding and grounded video question answering (Grounded VideoQA). The temporal grounding task focuses on localizing the segment tically corresponds to given natural language description (Gao et al. 2017). Building upon this, Grounded VideoQA requires models not only to answer questions about the video, but also to locate the relevant temporal segments that serve evidence for their answers (Xiao et al.. 2024). Ex- isting Video LLM based approaches typically tackle the temporal grounding task via two primary strategies: (1) timestamp-as-ler generation, where the LLM decoder directly generates the tempo- ral boundaries as natural language text (Ren et al.) 2024 Zeng et al.) Chen et al. 2024d Li et al., 2024b); and (2) boundary regression, where an auxiliary decoder or additional network layers are employed to regress the timestamp boundaries precisely (Liu et al. 20241] 2025 Zhao et al. 2025) To enhance temporal grounding, recent studies introduce instruction-tuning datasets that provide timestamp-awar supervision (Huang et al.) 2024a Qian et al.) 2024a Wang et al.) 2024b). These datasets aim to bridge the gap between natural language queries and their corresponding temporal segments in video. However, they often face trade-off between annotation quality and scalability. Consequently, Video LLM based methods still exhibit limitations when applied to long-form videos and fine- grained temporal reasoning. In this work, investigate effective strategies for constructing scale, high-quality instruction-tunit datasets to enhance the temporal grounding capabilities of Video L.Mss.
video that seman-

---

Published as a conference paper at ICLR 2026
3 METHOD
Overview. We propose Divid, dual-branch framework that disentangles spatial and temporal mod- cling for long-fom forg-fomm video understanding within a large language model (LLM), As shown in Fig- ure given an input video and a textual query, vision encoder extracts frame-wise spatial features, which projected into the language embedding space. Spatial pooling is then applied to obtain low-resolution temporal tokens, while the original high-resolution features are retained for spatial reasoning. A text encoder encodes the query into sequence of language tokens. The core com- ponent of Divid is a disentangled spatio-temporal decoder, which comprises two parallel attention branches: the temporal perception branch and the spatial perception branch. The temporal branch processes temporally dense but spatially low-resolution features to capture global video dynamics and generates attention maps that guide the spatial branch in selecting key frames. The spatial branch then attends to high-resolution features of selected frames to extract fine-graines visual de- tails. To integrate the complementary information from both branches, a token-level spatio-temporal soft-router is introduced. It employs query-conditioned gating mechanism to adaptively fuse the text-side attended features from the spatial and temporal branches. The resulting fused representa- tion serves input to subsequent decoding layers, enabling joint spatio-temporal reasoning guided by language.
3.1 VIDEO AND TEXT ENCODING
Given a video V composed of T frames {1, I2,..., IT). each frame I, is processed by visual encoder frss followed by a multi-layer perceptron (MLP) to project the output into the shared em- bedding space:
X -x
(MLP(((((.)))
(1)
where H and W denote the spatial resolution of the feature map, and is the feature dimension. Then, spatial pooling is applied to the video features to obtain the lower-resolution video features:
X'
(2m)7
2, = fxon(x)), X'
xw''d
where (-) denotes a spatial average pooling operation. Without loss of generality, we assume H' = W" 1, and reshape X' E Txxixlxd into Txx which we still denote as X'. The low- resolution video features utilized temporal tokens X' for modeling temporal context, while the high-resolution video features serve as spatial tokens X for subsequent key frame selection and fine-grained spatial understanding.
Given a textual query Q consisting L tokens, we encode it with a text encoder fre
X = (Q), X E 2xd
(3)
where fext maps each token into the shared dl d-dimensional embedding space.
3.2 DISENTANGLED
LLM
The key component our Divid framework is the disentangled spatio-temporal decoder layer, which enables the large language model (LLM) to perform spatial and temporal modeling in decoupled yet collaborative manner. Each decoder layer consists of two parallel attention branches: the Tem- poral Perception Attention branch and the Spatial Perception Attention branch.
Temporal Perception Attention. Given the video temporal tokens X' e Rxx and the text tokens X R Lxd we concatenate them along the token dimension to form the joint input sequence Xtemp Concat(X' X E (x++))x where d is the dimension of the token embeddings shared across both video and text modalities.
The temporal attention branch applies self-attention to model long-range temporal dependencies: (Xtemmp W (X (Xtmmp WK)
A tmmp = Softmax
A temp E R (T++)
Vd
(X Wv). v). where We: WK, Wy E d are learnable projection matrices. To guide the spatial branch, we ex- tract the cross-modal attention map A ATY. E T.xT from Ayemps corresponding to the attention from text queries to video keys.
We obtain temporally-aware feature representation 7rop A
4

---

Published as a conference paper at ICLR 2026
Task Example
sub!
sub2
sucb3
Question: Describe content video from 3.6 seconds to 9.7 seconds detail. Answert A young child wearing blue life jacket and gray jacket is paddling yellow and white kayak on a calm body of water, surrounded by trees ond houses in the background. The child continues to paddle, moving...
136,284
318,323
104,693
Figure 2: The Tinestamp-Cuided Captioning and the proposed TempGCap dataset.
Key Frame Selection. To reduce computational cost and focus on the most relevant visual content, we select fixed number of key frames based on the cross-modal attention map A T2V kemp E BLxT
which indicates how strongly each text token attends to each video frame. Specifically, we first compute an importance score Se2v R for each video frame by averaging its attention weights across all text tokens. Then, we select the indices the top-k frames with the highest importance scores and extract the corresponding high-resolution spatial features from X RT*HXxxx
Xkey X[ RxxHxWxd
Twe (sp,( k).
(5)
Spatial Perception Attention. We flatten the selected spatial features Xkey E B and apply linear projection map them into the feature space of the current decoder layer. After flattening and projection, we obtain the spatial tokens Xkey E IRdoxd, where Iy
x x W. Then, the spatial tokens X are concatenated with the text tokens X to form a joint input sequence X R(L++L)xd This sequence is then processed through self-attention mechanism to effectively capture fine-grained spatial dependencies and interactions between spatial and textual information. Finally, we obtain spatially-awar feature representation Zapat E (l)++ The self-attention parameters are shared with the temporal attention branch.
Spatio-temporal Soft-Router. To adaptively integrate information from both temporal and spa- tial branches, introduce a token-level spatio-temporal soft-router that dynamically fuses the attended text features from the two branches through a query-dependent gating mechanism. Let Huemp, H E 2xd denote the text-token outputs extracted from the temporal and spatial branches, respectively (i.e., the text-aligned slices of zemp and Z We use the original textual representa- tion X E g. prior any cross-modar attention, as the input to a lightweight gating module. This module generates a routing score for each token:
G Softmax W, by) E <x2
where W, E Rddx2 and by E 2 are learnable parameters. The i-th row of G, denoted g:, represents the soft weights over the temporal and spatial branches for the i-th text token. We then perform weighted combination of the branch-specific attended features:
Z 9. (1) H! (2) H for
L.
where 9; and 9, (2) representation Ziex e gxxd adaptively integrates spatial and temporal cues at the token level and is used to update the temporal stream subsequent decoding layers.
the temporal and spatial routing weights from g The resulting fused
3.3 TEMPGCAP: INSTRUCTION DATA FOR TIMESTAMP-GUIDE CAPTIONING
Recent studies (Golkar et al. 2023 Zeng al. Jung et al. 2024) have shown that Video L.LMs often struggle with temporal understanding and are prone to hallucinations. Our proposed Divid model, which directly predicts timestamps using an LLM decoder, is particularly prone to such errors. Moreover, as Divid framework leverages temporal attention to guide spatial selection, strong tempo- ral perception is essential for accurate grounding. To address this issue, we introduce a timestamp- guided captioning task. As illustrated in Figure 2 this task requires the model generate a natural language description for a specific temporal segment of a video (c.g., from the 3rd to the 5th sec- ond). The model must accurately localize the descrihed segment while avoiding interference from
5

---

Published as a conference paper at ICLR 2026
irrelevant content outside the target window. This task is designed to enhance the model's tem- poral grounding capability and to promote a more precise understanding of temporal structure in videos. However, existing timestamp-guides captioning datasets often struggle to balance scale and annotation quality. For example, Momentor (Qian et al.) 2024a) provides 1.7M timestamp-guided captioning instances, but the alignment between video segments and textual descriptions is often imprecise. Additionally, the dataset is sourced from only around 65K YouTube videos, which lim- its content diversity. In contrast, the dataset used in VTimeLLM-Stage2 (Huang et al., 2024a) and Intern Vid-G (Wang et al., 2024e) contains only 95K samples, which is insufficient for learning gen- eralizable temporal grounding capabilities.
To this end, we construct a new instruction-tuniin dataset, TempGCap, to support the proposed timestamp-guided captioning task. The dataset is built from three complementary sources, as shown in Figure 2 Sub1: Reannotating untrimmed videos with manual temporal annotations. We first adapt existing temporal grounding datasets to align with our task setting. Additionally, we re- annotate action detection datasets (e.g., HACS (Zhao et al. 2019)), which typically include coarse action categories and their temporal boundaries but lack natural language descriptions. To obtain fine teeeggrined captions, we utilize the advanced video captioning model Tarsier (Wang et al. 2024c) to generate detailed descriptions for each annotated segment. Sub2: Recovering untrimmed contexts from captioned clips. We collect the original untrimmed videos corresponding captioned clips with high-quality textual annotations. Using the original timestamps of these trimmed segments, we construct timestamp-guls guided captioning samples. To improve annotation precision and diversity, we further refine the temporal boundaries by measuring visual similarity between boundary frames and their adjacent frames. Sub3: Synthesizing seeudo-untrmmedd videos. When original untrimmed videos are unavailable, we synthesize pseudo-untrimmed videos by concatenating several short clips into a longer video sequence. We then select one of the original clips and use its associated cap- tion along with its temporal location within the sequence to form timestamp-guider captioning instance. After filtering and random quality inspection, we construct a high-quality instruction- tuning dataset comprising 559K timestamp-guit -uidded captioning samples, sourced from 428K videos, which ensures wide coverage across diverse video domains. Additional details are included in the appendix.
4 EXPERIMENT
4.1 IMPLEMENTATION DETAILS
We use ViT-g/14 from EVA-CLIP (Sun et al., 2023) as the vision encoder and Qwen2-7B (Wang et al., 2024d) as the large language model. In the first pre-training stage, we use LLaVA-550
dataset (Liu et al. 2024b) with WebVid videos (Bain et al.] 2021). Following (Chen et al. 2024d), we prepend each frame with a textual token indicating its timestamp (e.g.. "3.0 second") ) where the textual tokens are directly encoded using the tokenizer of the LLM. During this stage, only the multi-modal projector and the LLM layer projectors of spatial tokens updated. The learning rate is set to 1 x 10-3, and the batch size is 256. For instruction tuning, we combine our TempG- Cap dataset (559k samples) with 77K timestamp-grouut pooouudd-g samples from DideMo (Anne Hendricks) jet al. 2017) and ActivityNet Captions (Krishna et al. (017). In addition, we incorporate V. 17deo-178KK (Zhang al., 2024b) and DeVE-QA QA QQin et al. 2024) for video QA instruction learn- ing. The learning rate during this stage is set to 2 x 10 and the batch size is 128. Throughout all training stages, the visual encoder is kept frozen. In our implementation, we adopt practical con- figurations to balance temporal coverage and spatial resolution. Specifically, the temporal attention branch operates on T 128 frames with a reduced spatial resolution of H' x W' 16, enabling efficient modeling of long-range dynamics. For the spatial attention branch, we select k 32 key frames based on temporal guidance, and retain high-resolution features with H x W = 64 to preserve fine-gree grained spatial details. Please refer to Appendix for additional hyper-parametert
4.2 EVALUATION SETUPS
Temporal Grounding. This task aims to localize a temporal segment within a video that cor- responds to given language query, by predicting the start and end timestamps. We adopt the Charades-STA (Gao et al.) 2017) dataset as the evaluation benchmark, which consists of 6.6K videos and 16.1 K query- pairs. The average durations of the videos and target moments are 30.6

---

Published as a conference paper at ICLR 2026
Table 1: Temporal Grounding on Charades- Table
Grounded VideoQA on CG-
STA (Gao et al. 2017) Dataset.
Bench (Chen et al. 2024a
Sine
Re00
Ree0
Mechod
Size
Retet
A
202T
(OpeaA)
UnVVO
35.6
37.0
Gemini-1.55P 1.5.Plo
2.53
GPT-4o
(OperAl
20240
35.7
Genini Flsl
Clando-1.--Sommn
3.99
5.67
2-45
2.79
Gnounded
Chai [in
MM
1Wugg et 20046
48
4B
36.4
20.0
19.7
36.8
VITA Zharks ot M000e
Qhwnn dan 20134
3:77
7735
3,06
3.58
1.53
5.32
2.33
7.54
VTisdLM
13B
34.3
14.7
Starc parvtt SINEE
D0240
168
L.85
2.85
1.01
Quen25.VV
738
148
1.76
Quer2.5-V1
76
ChusUlaVi-22
138
TineChar
Re Ccceviiee
7B
32.2
13.4
1.01
HoskEye
CaaVVTG
Momenur
Allaoe
78
78
76
31.4
26.5
11.8
14.5
26.8
34.7
VideoLLMM
Videschu??
791
1:38
1.58
0.94
0.84
78
Owen-Vi
3013
79
0.89
1.19
0.42
VolooCha-7
45.7
35.0
ST STL LLM Ligr A00e0)
73
2.23
2.86
1.13
VolooCh
7B
27.6
VILA Literaa 70A.
603
1.56
2.89
1.38
(ImeMarier
806
51.9
76.9
48.6
MiniCr poree
88
2.35
2.96
1.35
TinsScurch
7B
52.4
24.5
48.6
LongVA 2ane pilree
710
2.98
1.80
1.78
VideoMini
1.88
53.1
76.
45.2
Kangeno Lire
601
1.26
1:23
VideoMindd
2005
78
39.1
31.2
50.2
InterVL
20046
78
3.91
5,05
2.64
51.4
Divid
1.58
4.77
2.37
Table 3: Grounded VideoQA on NEXT-GQA (iaao et al. 2024).
Tall R@0.5
Method
LLM Size
R
mloUi
R@0.3
R
mloP
AccOGQA
M NG+ (ane et al.
2022
890M
13.5
9.6
28.5
23.7
24.2
17.5
VIOLEET22 (Fu et 20443
4.3
1.3
25.
23.3
23.6
12.8
SoViLA C
21.7
16.6
LangRepo Khdya al 2024
8x
12.2
18.5
20.8
27
14.9
VideoStrearnng
8.3B
13.3
19.4
22.0
30.8
18.1
LLoVi Zhung el al. 202.g
1.8T
15.3
20.0
36.6
37.3
24.3
24.3
HawkEye Waing
20246
7B
37.0
19.5
25.7
44.7
31.6
33.4
23,5
Grounded- VI0 25 Wilt [t
2024b
4B
30.2
18.0
21.1
34.4
26.7
VideoCChtt-PPe TPD Yan cl
7B
41.2
23.4
27.7
47.5
32,8
35.6
25,5
VideoMind
45.2
28.6
25,2
VidcoMind [Lu et
2025
7B
50.2
25.8
31.4
56.0
35.3
39.0
28.2
Divid (Ours)
47.5
32.9
26.4
Divid (Ours)
7B
51.3
27.5
34.5
54.6
38.2
40.8
29.2
seconds and 8. seconds, respectively. Following the zero-shot setting, we evaluate the model on the standard 3.7K test queries. For evaluation, report Recall @ under different Intersection-ove Union (IoU) thresholds, including R1 @0.5 and R. @0.7, along with the mean IoU (mloU).
Grounded Video Question Answering. This task requires the model to not only answer questions about video content but also to provide grounded evidence by localizing the relevant temporal seg- ments. We evaluate our approach three representative datasets: CG-Bench (Chen et al.) (024a), NEXT-GQA (Xiao et al. (024)) and ReXTime (Chen et al.) 2024b). For CG-Bench, we report three metrics. Mean loU (mloU) measures the average overlap between predicted and ground-truth time intervals. Rec. @IoU (R@IoU) computes the average recall at IoU thresholds from 0.1 to 0.5, re- flecting how well the model retrieves relevant segments. Acc, @IoU (A@IoU) combines answer correctness with grounding quality: a prediction is correct only if the answer is right and the pre- dicted interval has tloU > 0 with the ground truth. For NEXT-GQA, evaluation includes two parts. IoU and loP are used to assess whether the predicted temporal segments align with the annotated evidence. Ace@ GQA further requires the answer to be correct and the predicted interval to have ToP 2 0.5 with the ground truth, reflecting grounding consistency. ReXTime evaluates models using mean IoU (mloU) and answer accuracy (Aco@ @oU under the constraint of IoU 2 0.5.
4.3 MAIN RESULTS
Performance on Temporal Grounding, As shown in Table [ our proposed method Divid achieves strong performance on the Charades-STA benchmark under the zero-shot setting. The rows labeled with (FT) indicate models that are fine-tuned on the downstream training set. We first examine the performance of the 1.5B version. Despite being lightweight model, Divid (1.5B) achieves 51.4 in R@0.5 and 47.3 in mloU, outperforming most zero-shot baselines of similar or even larger size. Notably, Divid (1.5B) exceeds the performance of many larger 7B models. For example, it surpasses Momentor (7B), HawkEye (7B), TimeChat (7B), and Chat (7B) by margins ranging from 12.6 to 18.8 in mloU. In particular, Divid (1.5B) outperforms Momentor (7B) by 22.9 in mloU (47.3
7

---

Published as a conference paper at ICLR 2026
Table Grounded VideoQA on ReXTime Chen et al.
2024b).
Method
L.M Size
R@0.3
R@0.5
mloU
Accolotl
L. [lany
2024a
43.69
29.92
17.13
TimeChat (RoB et E924
7B
40.13
21.42
26.29
10.92
VTimeLLM Huane et
3024a
7B
28.84
17.41
20.14
TimeChat (Re) 10/4
7B
14.42
7.61
11.65
eHunhy 217440
13B
29.49
VideoKlind Lau el 2025
1.5B
34.31
22.69
24.83
17.26
VideoMind (Lt
38.22
27.61
20.20
Divid (Ours)
1.5B
40,50
26.82
29.71
18.57
(Ours)
42.56
31.05
35.78
22.26
Table Comparison with existing frame-
Table 6: The impact of temporal guidance.
works.
Charades-STA RI
ReXTime
Method
Method
LLM TFLOPs
RIG005
Charddes-TTA
mloU
mloU
ReXTime
Acc@
Uniform
49.70
mIoU
46.27
mIoU
27.61
Acc@IoU
17.32
Full
28.2
51.55
47.76
30.16
19.11
Weighted
48.62
45.43
26.44
16.78
Slow-Fast
16.2
50.68
46.71
29.49
18:45
Segment
50.96
47.20
28.82
17.59
Divid
10.5
51.37
47.33
29.71
18.57
Importance
50.86
47.10
28.83
17.92
Top-K
51.37
47.33
29.71
18.57
vs. 24.4), while also achieving higher R@0.5 (51.4 vs. 26.6). These comparisons emphasize the efficiency and effectiveness of our model design, showing that careful architectural choices, together with high-quality timestamp-guitt -undded supervision provided by the TempGCap dataset, can compen- sate for and even surpass the benefit of larger model size. The 7B version of Divid further improves performance, achieving 59.5 in R @.05, 31.3 in R @.0.7, and 51.3 in mloU. Compared to TimeMarker (8B), which is trained on 45 million video-text samples, Divid (7B) achieves higher performance by 0.6 in @@0.5, 0.4 in R@0.7, and 2.9 in mloU. This demonstrates not only the strength of our model architecture, but also the effectiveness of the TempGCap dataset in providing fine-graine grainndd,,, instruction instruction-aligned temporal supervision that supports precise grounding. Furthermore, Divid (7B) outperforms Qwen2.5-VL (72B) in mloU by 0.4, despite having only one-tenth of the parameters.
Performance on Grounded VA.eoQA. As shown in Table 2 our proposed method Divid achieves state-of-the-ar performance among open-source models on the CG-Bench benchmark under the zero-shot setting. Our 1.5B model already demonstrates competitive results, achieving 4.77 in mloU, 6.40 in IIUU, and 2.37 in AoIo.. These results surpass several larger models, includ- ing Video-LLaVA (7B), ST-LLM (7B), and VideoLLaMA (7B), demonstrating the effectiveness of our architecture in handling grounded video question answering with limited model capacity. With parameter scaling, the performance of Divid further improves. Our 7B model achieves 5.74 in mIoU, 8.36 in R@IOU, and 4.11 in A@loU, setting a new state-of-the-ar art among all open-source models. Compared to Qwen2-VL. (72B), which is much larger model, our 7B version still achieves higher mloU, higher @@ IoU, and higher A@ IoU. When compared with commercial models, Divid (7B) achieves comparable or even superior performance across most metrics. surpasses Gemini- Pro, Claude-3.5-Sonnee
and GPT-4o-mini on all three metrics. While GPT-4o achieves slightly higher A@ IoU, Divid (7B) matches or exceeds GPT-4o in mloU and R@IoU. These results high- light the strong temporal grounding capability of our framework, demonstrating that with effective spatio-tempora modeling and the proposed instruction-tuminn compete with and even outperform larger-scale or commercial derstanding. As shown in Table[3aadd T Divid achieves state-of-the-art zero-shot performance on both NExT-GQA and ReXTime. The 1.5B model already outperforms most baselines, while the 7B version further improves to 34.5 mloU and 29.2 Acc @GQA on NEXT-GQA, and 35.78 mloU and 22.26 AoU@IoU on ReXTime, surpassing VideoMind (7B) by large margin. These results confirm Divid's effectiveness in fine-grani -eeaiindddd evidence localization and temporal question answering.
dataset TempGCap. Divid is able to models in grounded video un-
4.4 ABLATION STUDIES
To better understand the contribution of each component in our Divid framework, we evaluate on Charades-STA (Gao et al. 2017) for temporal grounding and ReXTime (Chen et al. 2024b) for grounded VideoQA. Unless otherwise specified, we use ViT-g/14 from EVA-CLIP (Sun et al. 2023 the vision encoder and Qwen2-1.55 from (Wang al. 2024d) the large language model.
8

---

Published as a conference paper at ICLR 2026
Table 7: Ablation studies of ST-Router.
Table 8: Ablation studies of datasetss.
Method
Churades- G1.0
mloUi
mloU
ReXTime Aoc
Method
Charades-STi R1600.5
mloU
mloU
ReXTime
Accolol
Add
50.66
46.81
29.11
18:23
Momentor- IM Quan 20244
48.96
44.90
26.77
15.43
Weight
Momentor Quan et al 41240
50.47
46.42
28.75
16.78
Soft-Router
51.37
47.33
29.31
18.57
TempCCap
51.37
47.33
29.71
18:57
Ablation studies of the framework. We compare three architectural variants: Full, Slow-Fast, and our proposed Divid, as shown in T The Full variant adopts the standard LLaVA-style pipeline, where all video frames are retained at high spatial resolution and directly fed into the L.M,, resulting in the highest computational cost. The Slow-Fast variant encodes video into separate temporal and spatial token streams, which are concatenated before being input into the LLM. This design reduces computational cost by limiting the number of high-resolution frames while preserving temporal di- versity. In contrast, our proposed Divid disentangles temporal and spatial modeling inside the LLM. Instead of direct concatenation, Divid leverages temporal attention to guide keyframe selection and fuses the two branches through lightweight soft-router. This structured design improves mIoU from 46.71 to 47.33 on Charades-STA and from 29.49 to 29.71 on ReXTime. Moreover, while achieving comparable performance to the Full model, Divid operates at significantly lower compu- tational cost (10.5 TFLOPs compared to 28.2 TFLOPs), demonstrating an effective balance between accuracy and efficiency.
Ablation studies of the temporal guidance. We compare five strategies for temporally guided keyframe selection, as reported in Table 6 The Uniform method samples frames at equal inter- vals without exploiting temporal attention, which leads to limited temporal awareness and may miss task-relevant moments. The Weighted method divides the video into K segments and performs iirentioo-based pooling within each segment, but the averaging operation tends to blur fine- grained cues, resulting in suboptimal performance. The Segment method improves upon this by explic- itly selecting representative frames from different video chunks, thereby ensuring broader temporal coverage and achieving better results than Uniform and Weighted. The Importance method fur- ther incorporates attention-based probabilistic sampling emphasize highly attended frames, which provides slight gains but still lacks robustness due to stochasticity in selection. Finally, the Top-K strategy directly chooses the K frames with the highest attention scores, explicitly preserving the most salient frames for spatial reasoning. This approach achieves the best overall performance on both Charades-STA and ReXTime, improving mIoU from 46.27 to 47.33 and from 27.61 to 29.71 compared to Uniform. These results demonstrate that selective and query-aware keyframe selection crucial for enhancing temporal grounding.
Ablation studies of the Soft-Router. We evaluate three strategies for fusing temporal and spatial features on the text side. The Add variant directly sums the outputs from the temporal and spatial attention branches. The Weight variant introduces a learnable scalar to balance the contribution of each branch. Soft-Router uses the original pre-attention textual features as conditioning input to dynamically compute token-wist routing weights, allowing each token to adaptively determine the proportion of temporal and spatial information. As shown in Table 7 the Soft-Router achieves the best results on both Charades-STA and ReXTime. Compared to Add, it improves mIoU by 0.52 and 0.60 respectively. The learnable Weight variant provides a slight improvement over Add, but it applies a global weighting across all tokens, which limits its flexibility. In contrast, the Soft-Router enables fine-grained control over spatio-temporal fusion at the token level, leading to more accurate reasoning and better performance. These results highlight the benefit of conditional, tosen-wise fusion for complex video-language alignment.
Ablation studies of the datasets. We study the impact of different instruction-tuning datasets on model performance by replacing TempGCap with subsets or alternatives from the Momentor dataset. As shown in Table R using million segment-level captioning samples from Momentor (denoted as Momentor- 1M) leads to a notable performance drop, with mloU decreasing to 44.90 on Charades- STA and 26.77 on ReXTime. Expanding to the full Momentor- ment, but still lags behind our TempGCap by 0.91 in mloU on Charades-STT and 0.96 on ReXTime. We attribute the superiority of TempGCap to its higher annotation quality and broader video cover- age. While Momentor primarily consists of auto-generated captions from a limited set of YouTube videos, TempGCap combines manually aligned boundaries, line-grained action descriptions, and curated long-form content from diverse sources. This results in more precise temporal supervision
dataset brings some improve-

---

Published as a conference paper at ICLR 2026
and better generalization across temporal grounding tasks. These results highlight the importance of both data scale and quality in training temporally grounded video- languge models.
Additional results and analyses provided in the Appendix.
CONCLUSION
In this paper, we introduce Divid, dual-branch video-language framework that disentangles spatial and temporal modeling within a large language model. By integrating temporal perception, query- aware keyframe selection, and token-level spatio-temporal fusion, Divid enables more accurate and efficient temporally grounded video understanding. We further construct TempGCap, a large-scale instruction-tuning dataset with timestamp-grounded captions that provides fine-grained temporal su- pervision across diverse video domains. Extensive experiments on multiple benchmarks, including Charades-STie STA, NEXT-GQA, ReXTime, and CG-Bench, demonstrate that Divid achieves state-of- the-art performance.
LIMITATIONS AND FUTURE WORK
Our framework relies on temporal attention to guide spatial selection, which may be influenced by biases inherited from the pretrained language model. To address this, future work will explore debi- asing strategies for temporal attention and expand the instruction-tumn -uninn dataset with more diverse and temporally aligned annotations to further improve model generalization.
7 ACKNOWLEDGEMENTS
This research is supported by Artificial Intellieence-Nationa Science and Technology Major Project (20232D0121200), the National Natural Science Foundation of China (62476021, 62531026, 62437001), the Strategic Priority Research Program of Chinese Academy of Sciences under Grant XDB1350103, and the Fundamental Research Funds for the Central Universities (2025JBZX062).
REFERENCES
Lisa Anne Hendricks, Oliver Wang, Eli Shechtman, Josef Sivic, Trevor Darrell, and Bryan Russell. Localizing moments video with natural language. Pruceedings of the IEEE international conference on computer vision, pp. 5803-5812, 2017.
George Awad, Keith Curtis, Asad Butt, Jonathan Fiscus, Afzal Godil, Yooyoung Lee, Andrew Del- gado, Eliot Godard, Lukas Diduch, Jeffrey Liu, et al. An overview on the evaluated video retrieval tasks at trecvid 2022. arXiv preprint arXiv:2306. 131 2023.
Kyungho Bae, Jinhyung Kim, Sihaeng Lee, Soonyoung Lee, Gunhee Lee, and Jinwoo Choi. Mash- vlm: Mitigating action-scene scene hallucination in iideo-Ilms through disentangled spatial-tempors representations. arXiv preprini arXiv:2503. 15871, 2025.
Shuai Bai, Keqin Chen. Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint PrXi:25502.1322. 2025.
Max Bain, Arsha Nagrani, Gil Varol, and Andrew Zisserman. Frozen in time: A joint video and image encoder for end-to-end retrieval. In Proceedings the IEEE/CVF international conference on computer vision, pp. 1728-1736 2021.
the
Daniel Bolya, Cheng- Yang Fu, Xiaoliang Dai, Peizhao Zhang, Christoph Feichtenhofer, and Judy Hoffman. Token merging: Your vit but faster. arXiv preprini arXiv:2210.09461, 2022.
Joao Carreira and Andrew Zisserman. Quo vadis, action recognition? new model and the kinetics dataset. In proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6299-6308, 2017.
10

---

Published as a conference paper at ICLR 2026
Guo Chen, Yicheng Liu, Yifei Huang, Yuping He, Baoqi Pei, Jilan Xu, Yali Wang, Tong Lu, and Limin Wang. Cg-bench: Clue-grounded question answering benchmark for long video under- standing. arXiv preprint arXiv:2412.12073. 2024a.
Jr-Jen Chen, Yu-Chien Liao, Hsi-Che Lin, Yu-Chu Yu, Yen-Chun Chen, and Frank Wang. Rex- time: A benchmark suite for reasoning-across across-rooiintt time in videos. Advances in Neural Information Processing Systems, 37:28662-286777 2024b.
Lin Chen, Xilin Wei, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang. Zehui Chen, Haodong Duan, Bin Lin, Zhenyu Tang, et al. Sharegpt4video: Improving video understanding and gener- ation with better captions. arXiv preprint arXiv:2406.04325. 2024c. URL httptt/s///axi org/ btg/aba/20 2406. 04325
Shimin Chen, Xiaohan Lan, Yitian Yuan, Zequn Jie, and Lin Ma. Timemarker: versatile video-llm for long and short video understanding with superior temporal localization ability. arXiv preprint arXiv:2411.18211 2024d.
Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al. How far are we to gpt-4v? ? closing the gap to commercial multimodal models with upen-source suites. Science China Information Sciences, 67 (12):220101, 2024e.
Wei-Lin Chiang, Zhuohan Li, Lin, Harrison Wu, Gautier Izacard, Arjun Kirubarajan, Shang-War -We Du, Luke Zettlemoyer, Wen-tau Weng, Ion Stoica, Caiming Xiong, and Matei Zaharia. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, 2023. URL https://
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume (long and short papers). pp. 4171-4186, 2019.
Dave Epstein, Boyuan Chen, and Carl Vondrick. Oops! predicting unintentional action in video. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 919-929, 2020.
Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He. Slowfast networks for video recognition. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 6202-6211, 2019.
Tsu-Jui Fu, Linjie Li, Zhe Gan, Kevin Lin, William Yang Wang, Lijuan Wang, and Zicheng Liu. An empirical study of end-to-end video-language transformers with masked visual modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp-
22898 22909, 2023.
Difci Gao, Luowei Zhou, Lei Ji, Linchao Zhu, Yi Yang, and Mike Zheng Shou. Mist: Multi-modal iterative seatial-temporal transformer for long-form video question answering. In Proceedings of the JEEE/CVF conference on computer vision and pattern recognition, pp. 14773-14783, 2023.
Jiyang Gao, Chen Sun, Zhenheng Yang, and Ram Nevatia. Tall: Temporal activity localization via
language query. In Proceedings of the IEEE international conference on computer vision, pp. 5267-5275, 2017.
Siavash Golkar, Mariel Pettee, Michael Eickenberg, Alberto Bietti, Miles Cranmer, Geraud Krawezik, Francois Lanusse, Michael McCabe, Ruben Ohana, Liam Parker, et xval: con- tinuous number encoding for large language models. arXiv preprini arXiv:2310.02988 2023.
Raghav Goyal, Samira Ebrahimi Kahou, Vincent Michalski, Joanna Materzynska, Susanne West- phal, Heuna Kim, Valentin Haenel, Ingo Fruend, Peter Yianilos, Moritz Mueller-Freet -reitar, et al. The" something something" video database for learning and evaluating visual common sense. In Proceedings of the IEEE international conference on computer vision, pp. 5842-5850, 2017.
11

---

Published as a conference paper at ICLR 2026
Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Gird- har, Jackson Hamburger, Hao Jiang. Miao Liu, Xingyu Liu, et al. Ego4d: Around the world in 3,000 hours of egocentric video. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 18995-19012, 2022.
Ayush Gupta, Anirban Roy, Rama Chellappa, Nathaniel D Bastian, Alvaro Velasquez, and Susmit Jha. Toga: Temporally grounded open-ended video qa with weak supervision. arXiv preprint arXiv:2506.09445, 2025.
Bin Huang, Xin Wang, Hong Chen, Zihan Song, and Wenwu Zhu. Vtimellm: Empower Ilm to grasp video moments. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14271-14280, 2024a.
De-An Huang, Shijia Liao, Subhashree Radhakrishnan, Hongxu Yin, Pavlo Molchanov, Zhiding Yu, and Jan Kautz. Lita: Language instructed temporal- -ocalizilooal assistant. In European Confer- ence on Computer Vision, pp. 202-218. Springer, 2024b.
Xilin Jin, Xilin Wei, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Bin Lin, Zhenyu Tang, et al. Chat-univi: Unified visual representation empowers large language models with image and video understanding. arXiv preprint arXiv:2311.08046, 2023.
URL httpthhtts/aattt http:///wwwxiivooo////////////// wttp///rg///aao.oo
08046
Minjoon Jung, Junbin Xiao. Byoung- Zhang, and Angela Yao. On the consistency of video large language models in temporal comprehension. arXiv preprint arXiv:2411.1295/ 2024.
Kumara Kahatapitiya, Kanchana Ranasinghe, Jongwoo Park, and Michael S Ryoo. Language repos- itory for long video understanding. arXiv preprint arXiv:2403. 14622, 2024.
Ranjay Krishna, Kenji Hata, Frederic Ren, Li Fei-Fei, and Juan Carlos Niebles. Dense-captioning events in videos. In Proceedings of the IEEE international conference on computer vision, pp.
706-715, 2017.
Jie Lei. Tamara L Berg, and Mohit Bansal. Detecting moments and highlights in videos via natural language queries. Advances in Neural Information Processing Systems, 34:1 1 1846-1111 1466-18858, 2021.
Hongyu Li, Jinyu Chen, Ziyu Wei, Shaofei Huang, Tianrui Hui, Jialin Gao, Xiaoming Wei, and Si Liu. Llava-st: A multimodal large language model for fine -re-graaddd ppatial-tempore under- standing. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 8592- 8603, 2025a.
KunChang Li, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang, Limin Wang, and Yu Qiao. Videochat: Chat-centric video understanding. arXiv preprint arXiv:2305.06355, 2023a.
Kunchang Li, Yali Wang, Yizhuo Li, Yi Wang, Yinan He, Limin Wang, and Yu Qiao. Unmasked teacher: Towards training-efffcient video foundation models. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 19948-19964 2023b.
Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, et al. Mybench: A comprehensive multi-modal video understanding benchmark. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 2024a.
Xinhao Li, Yi Wang. Jiashuo Yu, Xiangyu Zeng. Yuhan Zhu, Haian Huang, Jianfei Gao, Kunchang Li, Yinan He, Chenting Wang, et al. Videochat-flaske flash: Hicrarchical compression for long-context video modeling, arXiv preprint XxXx:2500.000744 2024b.
Xinhao Li. Ziang Yan, Desen Meng, Lu Dong, Xiangyu Zeng, Yinan He, Yali Wang. Yu Qiao, Yi Wang, and Limin Wang. Videochat-rl: Enhancing spatio-temporal perception via reinforce- ment fine-tuning. arXiv preprint 2wXii:2504.0695. 2025b.
Yicong Li, Junbin Xiao, Chun Feng, Xiang Wang, and Tat-Seng Chua. Discovering petio-temppora rationales for video question answering. In Proceedings of the IEEE/CVF International Confer- ence on Computer Vision pp. 13869-13878. 2023c.
12

---

Published as a conference paper at ICLR 2026
Yuncheng Li, Yale Song, Liangliang Cao, Joel Tetreault, Larry Goldberg, Alejandro Jaimes, and Jiebo Luo. Tgif: A new dataset and benchmark on animated gif description. In Proceedings of the IEEE Conference Computer Vision and Pattemn Recognition, pp. 4641-46511 2016.
Bin Lin. Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, and Li Yuan. Video-llava; Learning united visual representation by alignment before projection. arXiv preprint arXiv:2311.0012,, 2023a. URL http::ttts///t.. wttp:v//aaivvvv/aa///ii org/ abs/ 2311 10122
Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, and Li Yuan. Vila: On pre-training for visual language models. arXiv preprint arXiv:2312.07533, 2023b. URL https://arxiv: org abs//bb///// 2312. 02111.00733 07533
Kevin Qinghong Lin, Pengchuan Zhang, Joya Chen, Shraman Pramanick, Difei Gao, Alex Jin- peng Wang, Rui Yan, and Mike Zheng Shou. Univtg: Towards unified video-language temporal grounding. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp- 2794 2804, 2023c.
Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang. Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, ocr, and world knowledge, January 2024a. URL https://
Plavavvl.githon httbll/l//////oeoooooo h//blo//////44/////////// 01 - 30- llava- n
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, Advances in neural information processing systems, 36, 2024b.
Jiajun Liu, Yibing Wang, Hanghang Ma, Xiaoping Wu, Xiaoqi Ma, Xiaoming Wei, Jianbin Jiao, Enhua Wu, and Jie Hu. Kangaroo: A powerful video-language model supporting long-context video imput. arXiv preprint wwXv:2240.115542 2024c.
Ruyang Liu, Chen Li, Haoran Tang, Yixiao Ge, Ying Shan, and Ge Li. St-llm: Large language models are effective temporal learners. arXiv preprint arXiv:2404.00333 2024d. URL https:
//axxiv.org/abs// 2404. 00308
Ye Liu, Jixuan He, Wanhua Li, Junsik Kim, Donglai Wei, Hanspeter Pfister, and Chang Wen Chen. R 2-tuning:: Efficient image-to-video transfer learning for video temporal grounding. In European Conference Computer Vision, pp. 421-438. Springer, 2024e.
Ye Liu, Zongyang Ma, Zhongang Qi, Yang Wu, Ying Shan, and Chang W Chen. Et bench: To- wards open-ended event-level video-language understanding. Advances in Neural Information Processing Systems, 37:32076-321 2024f.
Ye Liu, Kevin Qinghong Lin, Chang Wen Chen, and Mike Zheng Shou. Videomind: chain-of-lora agent for long video reasoning. arXiv preprint 2rXhv:2503.13444 2025.
Yifan Liu, Yuxuan Zhang, Shikun Wang, Yujia Liu, Yujie Zhang, Qun Liu, Zhoujun Lin, and Houfeng Wang. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326, 2024g.
OpenAI. Gpt-4o technical report. htttpss//ogeaat.co//reeeeeeee cessed: 2025-05-13.
2024a. Ac-
OpenAl. Gpt-4o mini: Advancing hntee//eooccccccciiiiiiiiinnnnee agg.-oomm--cccctiiocoofffffeeeee 2025-05-15.
2024b. Accessed:
OBe Minicpm-v 2.6: A gpt-4y level multimodal lims for single image, multi image, and video your phone. Accessed: 2025-05-15.
wttpe/eeoooppp/ttaeeeeeeeee co/oppnbmb/ wwt./MiiiiiMP 1ii---cM-V-2- 6 2024.
Junwen Pan, Rui Zhang, Xin Wan, Yuan Zhang, Ming Lu, and Qi She. Timesearch: Hierarchical video search with spotlight and reflection for human- long video understanding, arXiv preprint 0rXiv:2504.01407, 2025. URL patttps://ariitt http://///wwwwoooo//a///////////
01407
13

---

Published as a conference paper at ICLR 2026
Long Qian, Juncheng Li, Yu Wu, Yaobo Ye, Hao Fei, Tat-Seng Chua, Yueting Zhuang, and Siliang Tang. Momentor: Advancing video large language model with fine- one-grained temporal reasoning. arXiv preprint arXiv:2402.11433 2024a.
Rui Qian, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Shuangrui Ding, Dahua Lin, and Jiaqi Wang- Streaming long video understanding with large language models. Advances in Neural Information Processing Systems, 37:1 19336-119360 2024b.
Hangyu Qin, Junbin Xiao, and Angela Yao. Question-answering dense video events. arXiv preprint arXiv:2409,04333 2024.
Mengxue Qu, Xiaodong Chen, Wu Liu, Alicia Li, and Yao Zhao. Chatvig: Video temporal ground- ing via chat with video dialogue large language models. Proceedings of the IEEE/CVF Con- ference Computer Vision and Pattern Recognition, pp. 1847-1856, 2024.
Shuhuai Ren, Linli Yao, Shicheng Li, Xu Sun, and Lu Hou. Timechat: A time-sensitive multimodal large language model for long video understanding. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 14313-14323, 2024.
Anna Rohrbach, Atousa Torabi, Marcus Rohrbach, Niket Tandon, Christopher Pal, Hugo Larochelle, Aaron Courville, and Bernt Schiele. Movic description. International Journal of Computer Vision, 123:94-120, 2017.
Xiaoqian Shen, Yunyang Xiong, Changsheng Zhao, Lemeng Wu, Jun Chen, Chenchen Zhu, Zechun Liu, Fanyi Xiao, Balakrishnan Varadarajan, Florian Bordes, et al. Longvu: Spatiotemporal adap- tive compression for long video-language understanding. arXiv preprint arXiv:2410.17455 2024.
Gunnar A Sigurdsson, Abhinav Gupta, Cordelia Schmid, Ali Farhadi, and Karteck Alahari. Charades-ego: A large-scale dataset of paired third and first person videos. arXiv preprint 190.:18804.06626.. 2018.
Enxin Song, Wenhao Chai, Guanhong Wang, Yucheng Zhang. Haoyang Zhou, Feiyang Wu, Haozhe Chi, Xun Guo, Tian Ye, Yanting Zhang, et al. Moviechat: From dense token to sparse memory for long video understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18221-18232 2024.
Quan Sun, Yuxin Fang, Ledell Wu, Xinlong Wang, and Yue Cao. Eva-clip: Improved training techniques for clip at scale. arXiv preprint MxXIv:2303313366 2023.
Xi Tang, Jihao Qiu, Lingxi Xie, Yunjie Tian, Jianbin Jiao, and Qixiang Ye. Adaptive keyframe sam- pling for long video understanding. In Proceedings of the Computer Vision and Pattern Recogni- tion Conference, pp. 291 18-29128, 2025.
Gemini Team. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of con- text. arXiv preprint arXiv:2403.05530, 2024a. URL http:/tpee//aaxiiooor//////////a
05530
QQ Multimedia Research Team. Video-ccam: Enhancing video-language understanding with causal cross modal attention. arXiv preprint arXiv:2408. 14023, 2024b. UtL.ttts:://axxi
abs/ / 2408 14023
Qwen Team et al. Qwen2 technical report. arXiv preprint arXiv:2407.1067/ 2024.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Nicolas Bailly, Albert Sablayrolles, -han-Philippe Vert, and Guillaume Lample. Llama: Open and efficient foundation language models, 2023.
Haibo Wang, Chenghang Lai, Yixuan Sun, and Weifeng Ge. Weakly supervised gaussian contrastive grounding with large multimodal models for video question answering. In Proceedings of the 32nd ACM International Conference Multimedia, pp. 5289-52899999 2024a.
Haibo Wang, Zhiyang Xu, Yu Cheng, Shizhe Diao, Yufan Zhou, Yixin Cao, Qifan Wang, Weifeng Ge, and Lifu Huang. Grounded-videollie Sharpening fine- araigrained temporal grounding in video large language models. arXiv preprint arXiv:2410003229, 2024b.
14

---

Published as a conference paper at ICLR 2026
Jiawei Wang, Liping Yuan, Yuchen Zhang, and Haomiao Sun. Tarsier: Recipes for training and evaluating large video description models. arXiv preprint vrXiv22407.00634 2024c.
Peng Wang. Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen-vl: A versatile vision-language model for understanding text, image, and bounding box inputs. arXiv preprint arXiv:2308. 0att22308..2966 12966, 2023. URL https:
faaxiivorg abs/ 2308 12966
Peng Wang. Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution. arXiv preprint arXiv:2409. 12191, 2024d.
Xin Wang, Jiawei Wu, Junkun Chen, Lei Li, Yuan-Fang Wang, and William Yang Wang. Vatex: A large-scale, high-quality multilingual dataset for video-and-language research. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 4581-4591, 2019.
Ye Wang, Boshen Xu, Zihao Yue, Zihan Xiao, Ziheng Wang, Liang Zhang, Dingyi Yang, Wenxuan Wang, and Qin Jin. Timezero: Temporal video grounding with reasoning gennded Ivlm. arXiv e-prints, pp. arXiv-2503, 2025.
Yueqian Wang, Xiaojun Meng, Jianxin Liang, Yuxuan Wang, Qun Liu, and Dongyan Zhao. Hawk- eye: Training video-text iims for grounding text in videos. arXiv preprint arXiv:2403. 2024e.
Junbin Xiao, Angela Yao, Yicong Li, and Tat-Seng Chua. Can i trust your answer? visually grounded video question answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13204-13211 13244,324,, 2024.
Mingze Xu, Mingfei Gao, Shiyu Li, Jiasen Lu, Zhe Gan, Zhengfeng Lai, Meng Cao, Kai Kang, Yinfei Yang, and Afshin Dehghan. Slowfast-llava-1.5 family of token-efficient video large language models for formffrmm&o video understanding. arXiv preprint arXiv:2503.1840 2025.
Ziang Yan, Zhilin Li, Yinan He, Chenting Wang, Kunchang Li, Xinhao Li, Xiangyu Zeng, Zilei Wang, Yali Wang, Yu Qiao, et al. Task preference optimization: Improving multimodal large language models with vision task alignment. arXiv preprint arXiv:2412. 19326 2024.
Antoine Yang, Antoine Miech, Josef Sivic, Ivan Laptev, and Cordelia Schmid. Zero-shot video question answering via frozen bidirectional language models. Advances in Neural Information Processing Systems, 35:124-141, 2022.
Shoubin Yu, Jaemin Cho, Prateek Yadav, and Mohit Bansal. Self-chained image-language model for video localization and question answering. Advances in Neural Information Processing Systems, 17776799-76771.. 2023.
Liping Yuan, Jiawei Wang, Haomiao Sun, Yuchen Zhang, and Yuan Lin. Tarsier2: Advancing large vision-language models from detailed video description to comprehensive video understanding. 2025. (RRLittts://arxx
wtgg/ba..a.......... hbtp:/abb/2211.... 07888
Xiangyu Zeng, Kunchang Li. Chenting Wang, Xinhao Li, Tianxiang Jiang, Ziang Yan, Songze Li, Yansong Shi, Zhengrong Yue, Wang, et al. Timesuite: Improving milms for long video understanding via grounded tuning. In The Thirteenth International Conference Learning Representations.
Ce Zhang, Taixi Lu, Md Mohaiminul Islam, Ziyang Wang, Shoubin Yu, Mohit Bansal, and Gedas Bertasius. A simple Ilm framework for long-range video question-answerian arXiv preprint arXiv:2312.17235. 2023a.
Hang Zhang, Xin Li, and Lidong Bing. Video-Ilama: An instruction-tunet audio-visual language model for video understanding. arXiv preprint arXiv: 02858, 2023b.
Peiyuan Zhang, Kaichen Zhang, Bo Li, Guangtao Zeng, Jingkang Yang. Yuanhan Zhang, Ziyue Wang, Haoran Tan, Chunyuan Li, and Ziwei Liu. Long context transfer from language to vision. arXiv preprint arXiv: /6855, 2024a. URL htttts:///aaatttte www.iiv.rrgggaaee abs////////// 2406. 16852
15

---

Published as a conference paper at ICLR 2026
Shaolei Zhang, Qingkai Fang, Zhe Yang, and Yang Feng. Llava-mini: Efficient image and video large multimodal models with one vision token. arXiv preprint arXiv:2501.03895, 2025.
Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Ziwei Liu, and Chunyuan Li. Video instruc- tion tuning with synthetic data, 2024b. htthttps::/
http::/:///ax /ttppp:://axxvvoo/o http:///wrr////////a//a/oo////// 2410 02713
Yuxuan Zhang, Yujia Liu, Shikun Wang. Zihang Wang. Yujie Zhang. Yifan Liu, Xiang Li, Qun Liu, Zhoujun Lin, and Houfeng Wang. Video-llama: An instruction-tuned audio-visual language model for video understanding. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 467-475, 2023c. URL https: //aclanthology.oor wtgpb/y...g/2233 emnlp e demo.49
Yuxuan Zhang, Yujia Liu, Shikun Wang, Zihang Wang, Yujie Zhang, Yifan Liu, Xiang Li, Qun Liu, Zhoujun Lin, and Houfeng Wang. Vita: Towards open-source interactive omni multimodal Ilm. arXiv preprint arXiv:2408.0522 012111, 2024c. /t//ttts://arr//orrrg////// abspg///a///240/ 05211
Hang Zhao, Antonio Torralba, Lorenzo Torresani, and Zhicheng Yan. Hacs: Human action clips and segments dataset for recognition and temporal localization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8668 8678, 2019.
Henghao Zhao, Ge-Peng Ji, Rui Yan, Huan Xiong, and Zechao Li. Videocxpert: Augmented Ilm for temporal-sensitive video understanding. arXiv preprint arXiv:2504.07519, 2025.
Yiwu Zhong, Zhuoming Liu, Yin Li, and Liwei Wang. Aim: Adaptive inference of multi-modal Ilms via token merging and pruning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 20180-20192, 2025.
16

---

Published as a conference paper at ICLR 2026
APPENDIX
A
MORE RESULTS AND ANALYSIS
Table 9: Ablation on Attention Parameter Sharing.
Charades-STA mloU
ReXTime
Method
R1605
mloU
Acc@loU
share
51.37
47.33
29.71
18.57
separate
51.18
47.38
29.24
18.89
Ablation on Attention Parameter Sharing. We share the Q/K/V/O projection matrices between the temporal and spatial attention branches to ensure representation consistency and reduce param- eter overhead. As both branches operate on joint video-text sequences, sharing attention weights promotes consistent cross-modal alignment patterns and simplifies model design. This also benefits the token-level fusion performed by the soft-router. To validate this design choice, we conducted an ablation study where we introduced LoRA-adapted, branch-specific Q/K/V/O parameters for each attention branch. This setup allows us to isolate the effect of parameter sharing while keeping the architecture and training cost nearly unchanged. As shown Table 9 the variant with separate parameters leads to only marginal differences in performance across benchmarks. These results suggest that parameter sharing strikes a favorable balance between efficiency, representational co- herence, and empirical performance.
Table 10: Ablation study on temporal and spatial branches.
(a) Temporal resolution ablation.
(b) Spatial resolution in temporal branch.
Charades-STAA
ReXTime
Charades-SU STA
ReXTime
Temp. Res.
R1@0.5
mloU
mloU
Acc@loU
Spatial Res.
R1@0.5
mloU
mloU
Acc IoU
64
49.53
46.01
28.02
17.11
3x3
50,26
46.71
28.88
17.48
4x4
51.37
29.71
256
51.57
47.52
30.35
19.12
5x5
$1.68
47.76
30,51
19.47
(c) Number of selected keyframes.
(d) Spatial resolution of keyframes.
Keyframes
RI@0.5
Charades STA
mloU
mloU
ReXTime
Acc@IoU
Spatial Res.
Charales STA R1
mloU
mloU
ReXTime
Acc
16
49.83
46.46
28.47
17.22
8x8
$1.37
47,33
29.71
18.57
32
51.37
47.35
29.71
18.57
12x12
51.39
47.55
29.90
18.79
64
51.42
47.49
29.97
18.81
Temporal and Spatial branch setting. We conduct comprehensive ablation studies to investigate the influence of temporal and spatial resolutions as well as the number of selected keyframes. As shown in Table [0] (a), increasing the temporal input length from 64 to 128 frames leads to signif- icant gains on both Charades-STA and ReXTime, while further scaling to 256 frames brings only marginal improvements with considerable computational overhead. TO) evaluates the effect of spatial resolution in the temporal branch, where 4 x 4 resolution provides a favorable balance between accuracy and efficiency, and higher resolutions offer limited additional benefit. Regarding the number of selected keyframes, Table 10 (c) shows that using 32 frames is sufficient capture representative visual context, with more frames yielding diminishing returns. Finally. Table 10 (d) reports results on the spatial resolution of keyframes, where increasing from 8 x 8 12 12 slightly improves performance but at the cost of higher computation. Based on these studies, we adopt 128 frames at x 4 resolution for the temporal branch, and keyframes at x 8 resolution for the spatial branch our default configuration. achieving strong performance while maintaining efficiency.
Ablation on Visual Encoders. We evaluate the impact of different visual encoders and their fine- tuning strategies on temporal grounding performance. In this experiment, frozen indicates that the visual encoder remains fixed during training, whereas FT denotes that the encoder fine-tuned dur- ing Stage II. Comparing the first two rows of Table 11 observe that ViT-g/14 14 and SigLIP achieve highly similar performance when both are frozen. Although ViT-g/14 contains a substantially larger number of parameters (approximately 1B), it is pretrained lower input resolution (224 224),
17

---

Published as a conference paper at ICLR 2026
Table 11: Ablation of visual encoder. FT denotes fine-tuning.
Method
Charades-STA R1600.5
mloUi
miooli
ReXTime AccélaaU
ViT-p/Id (frowe))
51.37
47.33
29.71
18.57
Sigl IP (frozen)
51.08
47.09
29.45
18.32
Sigl IP (PT)
52.16
47.77
29.98
18,79
whereas SigLIP is pretrained at 384 x 384. As result, the two encoders perform comparably on Charades-ST:, and ReXTime. Next, comparing the frozen and fine-tuned SigLIP settings, we find that enabling fine-tuning during Stage II leads to consistent improvements across all metries: +1.08 RI @0.5 and +0.68 mloU on Charades-STr, STA, as well as +0.53 mIoU and +0.47 Acc loU on ReXTime. Overall, these results show that while a frozen encoder already offers strong perfor- mance, fine-tuning the visual encoder can further enhance fine-grained temporal grounding when computational resources permit.
Table 12: Ablation of fusion methods and placements.
Method
Charades-STA RI
mloU
mloU
ReXTime
Acc Rlo
Ousput-lovel
26.44
16.63
Block-level (4 layers)
49.56
45.98
28.23
17.24
Block-level (2 liyers)
50.37
47.04
29.63
18,20
Head-fusion
29. 29.34
FFN-fusion
50.77
47.12
29.58
18.30
Ours
51.37
47.33
29.71
18.57
Ablation of Fusion Methods and Placements. Table 12 reports the ablation results different fu- sion strategies and their placements within the decoder. Output-level performs fusion only at the final decoder output, which leads to the weakest results since spatial and temporal information interact only once at the end. Block-level (2 layers) and Block-level (4 layers) perform fusion every 2 or 4 de- coder layers, respectively; reducing the fusion frequency clearly harms performance, demonstrating that frequent interaction between spatial and temporal cues is important. We further compare finer- grained fusion placements. Head-fusion fuses tempprlltempora features after the QKV projections but before the output projection of attention, which results in noticeable performance sttop-lieey because disrupts the attention head structure and misaligns Q/K/V distributions. FFN-fusion ap- plies fusion after the inferior to our method. Overall, our approach, which performs fusion after attention and before the FFN at every decoder layer, achieves the best performance across all metrics. This design allows spatial and temporal information to interact fully within each layer while preserving the stability and structure of the attention mechanism, leading to consistently superior temporal grounding accuracy.
rerports
network, yielding better results than Head-fuste -fadin but still slightly
Table 13: Performance comparison on general VideoQA benchmarks.
Method
Venue
MSVD-QA
MSRVTT-QA
ActivityNet-QA
Acc
Score
Acc
Score
Acc
Score
LLaVA-Mini Zhang et al 2025
ICLR25
70.9
4.0
59.5
53,5
MASH-VLM Bag cl 2015
CVPR25
74.4
4.0
61.9
3.6
49.3
3.4
TOGA Gupta
ICCV25
73.8
3.9
$2.0
Ours
75.5
4.1
61.2
3.6
51.8
3.5
Generalization to VideoQA Benchmarks. Our work is primarily motivated by advancing tempo- rally grounded video understanding, which poses unique challenges that are not fully addressed by existing video- video-language models. To this end, we design both our architecture (Divid) and our instruction-tuning dataset (TempGiCap) with a focus on spatio-temporal disentanglement and fine-grained timestamp supervision. Consequently, our main evaluation is conducted on tempo- rally grounded benchmarks such as Charades-STA, ReXTime, CG-Bench, and NEXT-GQA, which directly reflect our target problem setting. To further assess generalization beyond grounding- oriented tasks, we also evaluate Divid on widely used VideoQA benchmarks including MSVD-QA, MSRVTT-QA, and ActivityNet- QA. As shown in Table [3] Divid achieves competitive performance compared with recent state-of of-te-of-the--ee ahe-art VideoLLMs such as LLaVM-Mini, MASH- VLM, and TOGA. This behavior is consistent with our design choice. Divid's dual-branch decoder, query-guided
18

---

Published as a conference paper at ICLR 2026
keyframe selection, and token-level spatio-temporal routing are optimized for long- range tempo- ral reasoning rather than short-video appearance recognition. In contrast, MSVD-QA, MSRVTT- QA, and ActivityNet-QA primarily evaluate object presence, scene attributes, and short-range ac- tions, with limited emphasis on multi-second temporal dependencies. Therefore, the strengths of our disentangled temporal modeling do not directly translate into large gains on these appearance- centric datasets, a trend also observed in grounding-oriented VideoLLMs such as LLaVA-ST (Li (ct al. (025a) and TOGA (Gupta et al., 2025). Furthermore, TempGCap is constructed vide timestamp- -oollamddgdooeeddd supervision that benefits temporal grounding tasks but is not specifically aligned with the objectives of generic VideoQA benchmarks. Despite this specialization, Divid still achieves competitive performance these VideoQA datasets, demonstrating that its targeted de- sign for temporally grounded long-video reasoning maintains solid generalization to broader video understanding scenarios.
pro-
Table 14: Performance comparison on Starades-STA, finetuning on the training set.
Method
LLM Size
R@0.5
R@0.7
mloU'
VideoChat-Rl et al. 2025b
7B
71.7
50.2
60.8
TimeZero Wang
7B
72.5
47.9
Ours
7B
72.6
50.8
61.3
Comparison with more advanced methods. To further validate the effectiveness of our model architecture and the TempGCap dataset, we compare our 7B model with two advanced temporal grounding systems, VideoChat-R1 (Li et al. (2255) and TimeZero (Wang et al. 2025). both of which are fine-tuned on the Charades-STA training set. For fairness, we accordingly fine-tune our 7B model on the same training set, and the results are reported in Table Notably, both VideoChat- RI and TimeZero are trained with GRPO-based reinforcement learning, incorporating task-specific temporal rewards enhance video reasoning. Moreover, their backbone LLMs are based on the stronger Qwen2.5 family, whereas our model relies on the earlier Qwen2 backbone. Despite these advantages, our model achieves comparable or even higher performance across key metrics. This demonstrates the effectiveness of our design and the strong data efficiency of TempGiapp.
Table 15: Comparison with Attention- based Filtering Methods on NEXT-QA.
Method
(%)
MIST Gao al. 2023
57.2
STR L Cieta al 20250
70.0
GCG Wang 2024a
74.6
Ours
79.4
Comparison with cioontion-bsedd Filtering Methods. As shown in Table 15 we compare approach with three representative attention-based filtering methods on NEXT-QAA MIST (Gao et al.) 2023). STR (Li et al. 2023c), and GCG (Wang et al.] 2024a). Among them, MIST achieves an accuracy of 57.2%. STR improves the performance to 70.0%, and GCG further increases it to 74.6%. In contrast, our method attains the highest accuracy of 79.4% %, outperformig GCG by 4.8% These results highlight the effectiveness of our method.
Comparison with Frame Sampling and Token Pruning methods. As shown in Table 16 first compare our method with the frame sampling approach AKS (Tang et al. (025). Although AKS reduces the number of processed frames, its frame selection is performed before feeding frames into the LLM and is therefore not guided by the language query within LLM. This unguided strategy often fails to preserve query-relevant visual content and can lead to noticeable information loss. Consequently, AKS achieves only 46.91 1 RI @0.5 and 43.52 mIoU on Charades- STA, and 26.87 mloU and 16.57 Ace@ IoU on ReXTime. mloU on Charades-STA. as well as 29.71 mloU and 18.57 Acc @oc@lo IoU on ReXTime. These im- provements demonstrate that disentangled temporal modeling and LLM-internal guided keyframe selection provide more reliable temporal localization than pre-LLM sampling strategies. We further compare our approach with two token pruning methods, ToMe (Bolya et al. 2022) and AIM (Zhong et al., 2025). While these methods effectively reduce the number of visual tokens passed to the LLM. they cannot guarantee the preservation of essential spatio-temporal cues. As a result, ToMe yields only 40.13 R1 @0.5 and 39.31 mloU on Charades-STA. and 25. 15 mloU and 14.55 Acc loU
contrast, our method attains 51.37 RI @0.5 and 47.33
19

---

Published as a conference paper at ICLR 2026
Table 16: Comparison with Frame Sampling and Token Pruning methods.
Charades-STX
ReXTime
Method
LLM TFLOPs
@105
mloU
mloU
AcceloL
AKS Tang et al. 2025
7.9
46.91
43.52
26.87
16.57
ToMe
Bo
10.2
40.13
39.31
25.15
14.55
AIM
Zhong ot 2023
9.9
49.29
45.61
27.84
17.02
Ours
51.37
47.33
18.57
on ReXTime. AIM improves the results to 49.29 RI RI @0.55 45.61 mloU on Charades-STI and 27.84 mloU, 17.02 Acc@ IoU on ReXTimme Despite operating under similar even tighter token bud- gets, our method consistently achieves the highest scores across all benchmarks, highlighting that global token pruning is insufficient for long-form temporal reasoning. Instead, performing query- aware temporal guidance and spatial selection within the LLM preserves critical information while maintaining efficiency.
Table 17: Comparison Computation and Latency.
Method
TFLOPs
Lateney
Full
28.2
1642ms
Slow-Fast
16.2
131
Ours
10.5
1397ms
Computation and Latency Analysis. Divid achieves substantial reduction in computational cost while maintaining competitive inference speed. Table I7 summarrizes the comparison. Before en- gineering optimization, the total latency of our method is 1530 ms, which slightly higher than the Slow-Fast variant at 1311 ms but remains lower than the full model at 1642 ms. After applying several engineering optimizations, such as parallclizing computations during keyframe selection, the latency is reduced from 1530 ms to 1397 ms. This improvement narrows the gap between the theoretical reduction in FLOPs and the practical inference speed. All latency measurements were obtained under the same conditions an NVIDIA A100 GPU. Divid requires only 10.5 TFLOPs, corresponding to a 63 percent reduction compared with the full model that requires 28.2 TFLOPs and a 35.2 percent reduction compared with the Slow-Fast baseline. Despite reducing FLOPS by more than one third relative to Slow-Fast, Divid increases latency by only 6.5 percent and achieves consistently higher performance across all evaluated metrics. These results demonstrate that Divid provides an effective balance between efficiency and accuracy.
Visualization of token-wise fusion weights produced by the spatio-temporal soft-router. To pro- vide deeper understanding of how the proposed soft-router modulates the contributions of temporal and spatial information the level of individual query tokens, we conduct a comprehensive visual- ization and analysis based on aggregated routing behavior across all decoder layers. For every query token the input sequence, we extract the temporal and spatial fusion weights produced at each de- coder layer. These per-layer weights are then averaged to obtain stable and layer-agnostic fusion score that reflects the final routing behavior of the model. This procedure yields two scalar quan- tities per token, corresponding respectively to its temporal preference and spatial preference. After computing these scores for the entire vocabulary of query tokens encountered during evaluation on ReXTime benchmark,
sort the tokens according their temporal and spatial fusion weights and present the Top-100 tokens for each category in Figure 3 This ranking-based approach highlights the tokens for which the routing mechanism expresses the clearest preference, facilitating more in- terpretable inspection of the model's internal fusion dynamics. The results reveal consistent and semantically meaningful routing patterns. Tokens with explicit or implicit temporal semantics, cluding "then", "when", and "start", exhibit markedly higher temporal fusion weights. These tokens frequently appear in contexts that require reasoning about temporal ordering, event progression, or causal relationships, and the router accordingly prioritizes temporal features when processing them. In contrast, tokens that denote concrete objects, visual attributes, or detailed physical interactions are associated with significantly higher spatial fusion weights. Representative examples include nouns such as "lady" and "dog". , as well as action verbs such as "applying" and "holding". These tokens intrinsically depend on fine-grained object appearance or spatial configuration, and the router therefore amplifies the influence of spatial representations to support more precise visual grounding. This visualization provides strong evidence that the soft-router does not operate through uniform or
20

---

Published as a conference paper at ICLR 2026
10
(a) Temporal Fusion Weights (Top- 100 Tokens)
(b) Spatial Fusion Weights (Top-100 Tokens)
Figure 3: Visualization of token-wise fusion weights produced by the spatio-temporal soft-router.
indiscriminate weighting. Instead, it systematically adapts the degree of temporal and spatial fu- sion based on the semantic role of each token, producing token-dependent fusion strategies that are aligned with the underlying linguistic and visual demands. Such behavior supports the objective of disentangled spatial temporall reasoning within the decoder and demonstrates that the router learns structured decision patterns that enhance both interpretability and task-specific temporal grounding performance.
Visualization of failure eases. As shown Figure we visualize two representative failure cases from the ReXTime benchmark and compare our Divid framework with the full-attention baseline. In both examples, two models correctly answer the multiple-choice question, yet their predicted temporal intervals obtain very low tloU scores (below 0.5). This highlights a common challenge in grounded VideoQA: producing the correct answer does not necessarily imply correct temporal grounding. In the upper example, the question asks for the event after the "CrossFit Connect" logo appears. The baseline incorrectly grounds only the moment when the logo is shown, whereas the ground truth corresponds the subsequent content. Our model predicts a longer segment that covers both the query cue and the answer-relevant interval, resulting in a higher tloU, but the overall local- ization remains imperfect. In the lower example, the question asks for the action the man performs before he drops the weights on the floor. The baseline prediction localizes the dropping motion itself, which corresponds only to the question cue rather than the answer. Our model instead pro- duces segment that successfully includes the action occurring prior to the drop, leading to more aligned but still incomplete interval. Such errors likely arise because most existing training samples place the question and the answer within the same temporal region, providing limited supervision for queries that require grounding outside the question segment (e.g., "before" or "after"). Neverthe- less, our model consistently shows better coverage of relevant intervals than the full-at full-attention baseline. In future work, we plan to incorporate more training data with non-overlapping question- answer intervals and improve the model's ability to perform cross-interval temporal grounding.
21

---

Published as a conference paper at ICLR 2026
aupean scrern Cuapee"" "exduraue tatang infrodaction reremas""
guble spitatzing pask-up
1
12.1
D( 0.00 10.45 (1107-0.0335))
OFBESTI
CraSSFR
Baseline
0.0
10.45
D(V)
GIBESTII
CrOSERR
[JD, 14.26 (d100m0.1403)
14.26%
Ground
drogs
floor?
adjaren
User
18.0
15.0
D(V) 22.85, 24.95 (Mol--1
Baseline
22.85
2435
(() 24.99 ((IoU-0.1273)
Figure 4: Visualization of failure cases on ReXTime benchmark.
Visualization of Key Frame Selection. Figure visualizes the key-frame selection behavior across all decoder layers. Dark blue cells indicate frames selected as key frames, while light blue cells denote non-selected frames. We observe that a small portion of selected frames falls outside the ground-truth segment; while this is not idcal, these frames also contain work-related activities, which may attract attention given the query. Nevertheless, the majority of key frames concentrate on the target segment, aligning well with our predicted temporal window. This demonstrates that the temporal branch effectively highlights query-relevant frames and guides the spatial branch toward the correct video region.
B DETAILS OF TEMPGCAP DATASET
Table 18: Comparison of instruction- -unnin datasets for temporally grounded video understanding. "M" denotes manual annotation, "A" denotes automatic annotation.
Dataset
QA-pairs
#Source Videns
Total Duration
Avg. Duration
Annotation
WTimeLLM-Stage (Hoang et al. 2024a]
146K
146K
2,12
52s
A
Intern Vid-G Wang al 20246 Momentor Qian el Ours
715K
83K
10,106h
10.4M
65K
7.260h
403s
A
559K
428K
8,83
73s
M
We present TempGCap, large-scale instruction tuning dataset for timestamp-guide captioning, where models are required to generate natural language descriptions for specific temporal segments within untrimmed videos.
construct TempGCap, we aggregate timestamp-aligned video-text pairs from diverse sources and process them through a unified three-stage pipeline. The final dataset contains 559K samples from 428K videos, totaling over 8K hours of content. As shown in Ta- ble 18 compared with existing instruction-tuuning datasets such VTimeLLM-Stagee (Huang al.) 2024a), Intern Vid-G (Wang et al. (024e). and Momentor (Qian et al. 2024a). TempGCap achieves better balance between scale, diversity, and annotation quality, benefiting from both manual and
22

---

Published as a conference paper at ICLR 2026
How does the woman perform her work tasks, such as video calls, typing, or other compatce-related activities?
A. By stauding at high make msing desitop computee without aenippoones
sitting
tablet and
User
beadphones By using a smariphone to conduct tasks wblk walking in park.
laptop.
[l6d,
Ground Truth
118.0v
136.0s
Prediction
116.1s
144.36
uyeraz
uww
23-
use
Lven
Lowe
taw
Ixer
Figure 5: Visualization of Key Frame Selection.
automatic supervision. This unique combination provides higher temporal precision and broader coverage across video domains, making it particularly suitable for training models on temporally grounded video understanding tasks.
B. DATA COLLECTION AND PROCESSING
The TempGCap dataset is built by aggregating timestamp-aligned video-text pairs from a broad range of publicly available and temporally rich video-language datasets, covering diverse domains such as everyday activities, instructional content, and movie recaps. Representative sources include DiDeMo (Anne Hendricks et al., 2017), ActivityNet Captions (Krishna et al., 2017), HACS (Zhao (Yuan et al.) (025). To construct high-quality timestamp- guided captioning samples from these heterogeneous sources, design a three-stage data process- ing pipeline tailored to the characteristics and limitations of each dataset.
et al. 2019), and
Subl: Reannotating Untrimmed Videos with Manual Temporal Annotations. We begin with datasets that already provide temporal grounding annotations or segment-level boundaries. For the grounding datasets DiDeMo (Anne Hendricks al. 2017) and ActivityNet Captions (Krishna al. 2017]. we directly adapt their manually annotated temporal segments into the cnnestampgguide captioning format. For the temporal action localization dataset HACS (Zhao al. 2019), which in- cludes
temporal boundaries but lacks natural language descriptions, we generate detailed captions for each action segment using the advanced video captioning model Tarsier (Wang al. 2024c). To ensure the quality of the generated captions, we perform two-stage filtering pro- cess combining mule-based checks and LLM- scoring. First, we remove overly long captions, captions containing excessive shot-change descriptions (detected via keyword matching). and cap- tions dominated by static scene descriptions. To quantify static-content dominance, we use Qwen2- 7B (Tem et al. 2024) assign static-content score from 0 to 5 and discard captions with score
23

---

Published as a conference paper at ICLR 2026
higher than 3. Second, we conduct random human audits to further verify the alignment and de- scriptive accuracy of the remaining captions. Since Tarsier tends to produce relatively detailed and lengthy descriptions, the form "Describe... in detail," as illustrated in Figure/6((b) This ensures consistency between the captioning style of the data and the model's 's catructtio--ollowing hehavior.
adapt the instruction format in training samples by using prompts of
To support the construction of Sub2 and Sub3, we introduce the Mansier2-Recap-5888 dataset Yuan ct al. 2025), a large-scale, high-quality corpus of short video clips. It comprises 585K distinct clips spanning 1,972 hours, sourced from open-domain datasets. Each clip (5-20 seconds) is annotated
with detailed caption generated by Tarsier2-7B (Yuan et al., (025). which surpasses GPT-4o (Ope-)
nA 2024a) in generating accurate and descriptive captions for short video segments. To prevent data leakage during zero-shot evaluation and streamline data handling, we filter out subsets such as Charades, Charades-F.go (Sigurdsson et al. 201 8), Ego4D (Grauman et al., 2022], and TGIF (Li jet al.] 2016). The retained subsets, including Activity (Krishna et al. 2017), Kinetics-700 (Car- reira & Zisserman 2017), VATEX (Wang et al. 2019), LSMDC (Rohrbach et al.) 2017). Oops (Ep-
stein et al. 2020), SomethingSomethingll
(Goyal et al.] 2017). TREC-VIT (Awad et al. (2023)
and Web Vid (Bain et al. (021), cover broad spectrum of video domains and provide diverse
pervision for timestamp- guided captioning.
Sub2: Recovering Untrimmed Contexts from Captioned Clips. For several retained subsets such as ActivityNet, Kinetics-700, and VATEX, the original untrimmed videos are publicly acces- sible. We retrieve these full-length videos and reconstruct timestamp-guidee -uudded captioning samples by aligning each captioned clip with its corresponding original timestamps. Importantly, the caption is kept unchanged and strictly corresponds to the original trimmed segment; the additional untrimmed context functions solely as surrounding background, encouraging the model to distinguish relevant content from temporally adjacent but irrelevant regions. To further refine segment boundaries and improve temporal precision, we adopt feature-based boundary adjustment procedure. For each boundary frame, we extract ViT embeddings of the frame and its adjacent neighbor and compute co- sine similarity. If the similarity falls below a threshold of 0.9, we iteratively expand the start or end timestamp outward along the temporal axis until the similarity stabilizes. The maximum expansion range is capped (e.g.. 1.5 seconds) to avoid excessive drift, and segments whose boundary neigh- borhoods exhibit consistently low similarity across multiple consecutive frames are discarded. This refinement removes clips with ambiguous transitions and yields more reliable temporal anchors. We additionally apply several quality-control filters to the recovered untrimmed videos, including discarding corrupted downloads, removing videos with abnormal durations (e.g., shorter than sec- onds), and eliminating duplicates. Finally, we perform random human audits to ensure that the refined segments remain semantically aligned with their captions and that the restored untrimmed contexts do not introduce misleading artifacts.
Sub3: Synthesizing Deeudo-Untrimmed Videos. For subsets where untrimmed videos are unavail- able, we simulate untrimmed contexts by synthesizing for each target clip, we randomly sample 2 to 4 additional clips and concatenate them in shuffled order to form longer video sequence. We also control the distribution of the inserted segments to ensure that target clips appear at different positions within the synthesized video. The timestamp of the target clip is recalculated based on its position in the sequence, and its original caption is retained. captioning instances that emulate
untrimmmd sequences. Concretely,
This synthesis strategy enables us to construct timestamp-guide real-world untrimmed video scenarios with surrounding contextual distractions, helping the model leam to perform temporal grounding under weak and noisy supervision. To avoid distractor clips that are too similar to the target segment, we perform filtering at both textual and visual levels. For textual similarity, we use a BERT (Devlin et al. (019)) model to compute caption similarity between the target clip and each candidate distractor clip, and discard distractors with similarity above 0.5. For visual similarity, we use a UMT (Li et al. 2023b) model to extract features from 8 sampled frames of each clip and remove distractors whose similarity exceeds 0.4. Clips whose captions have been repeatedly used are also filtered out to prevent caption duplication. Finally, we perform manual spot-checking to ensure that the synthesized pseudo untrimmed sequences are reasonable and that distracting segments do not unintentionally overlap in semantics with the target clip.
Data and Domain Leakage Prevention. To prevent data leakage and ensure that our instruction- tuning corpus does not overlap with downstream evaluation benchmarks, we adopt strict filtering at the video identity, subset, and domain levels. First, we remove any video whose name-based identifier (c.g., youtube id or other dataset-specifit video IDs) exactly matches those used in our
24

---

Published as a conference paper at ICLR 2026
Table 19; TempGCap dataset statistics grouped by annotation strategy and domain..
Name
Domain Category
#Videos
#Samples
Avg. Duration
Duration
Caption Len. (w)
Subl: Remnnntating Untrimmee Videos with Mannal Temporal Annotations
DiDeMa
Open Domain
7,678
30,158
28.2
60.2h
7.5
ActivityNet Captions
Human Activities Events
10,009
37,421
s
325.8
13.5
HACS
Human Activities
21,552
68,705
1
834.1
86.3
Sub/ Sum.
39,239
136,284
s
1220.1
48.9
Sub2: Recovering Untrimmed Contexts from Captioned Clips
VATEX
Multilingual Crowd
Human Activities 22,422
22,425
165.7 s
1031.9
82.1
ActivityNett
9,075
32,595
115.7
291.6
97.8
Kinetics 700
49,660
49,673
81.5
Suh2 Sum
81,157
104,693
150.3 s
3387.4 h
86.7
Suk3: Synthesized from Short Clips
without Untrimmed Videns
Accident / Unexpected Events
7,948
7,948
39.1
86.3
80.7
Something Something2
Object centric Human Actions
9,996
9.996
15.0
41.8
67.5
TREC. VTT
Web Video Diverse Topics
14,199
14,199
25.2
81,0
LSMDC
Movic / Narrative Viden
108.271
108,271
16.4
492.8
72.8
Vid
Web scale Open domuin
177,909
177,909
71.0%
3503.6
71.3
Suh3 Suw
318,323
318,323
47.8
4223.9
72.3
Quemiwe: Lleechhe the event occuring from 26.9 second 2412 secone in the video.
Quastion: What huppens scrne
video?
Anewer: Zhe coach beliing eie girl catch rav.
Answer bully the yoar to lop of daggs cround Sace and cips the rest of the ker beitted kitw
Samples converted from dense video caption datasets
Qbeeniow: Describe the coutest of video Bors 5.3 secoods 21.5 secouds detail.
Question: Describe conzent video from 6.7 seconds 27.0 seconds detiil
Answe:: indoor count with markod centrr circle and variaus colored lines, foor soliniasi's ran limards dhowe rad bulls placod on the gronnd. They uash pick ap ball from me ground after poeking up the buils, mhey star wonting souards difieee divections.
Ansser .l man rides beycle through a folld of grast. cerilin as he moves forwand
The scewe tranaiionss the wun risting the biyyle alongside another person both
moving afong a park wash a beer se the kaz-ggoond. As the noe solx the mihovaii of the no andirndhsts are sase risleg thetr bicycles apainer a bacntifll piy The cenera thee.
11.58
(b) Sampled annotated by Tarsier
Figure 6: Visualization of Training samples.
evaluation benchmarks, ensuring that no training sample contains content from test videos. Sec- ond, to avoid domain-level leakage, we manually exclude video sources that closely resemble the target benchmarks. In particular, filter datasets such as Charades, Charades Ego (Sigurds- son et al. 2018), Ego4D (Grauman et al. 2022), and TGIF (Li et al.) (016), whose content or annotation style strongly overlaps with temporal grounding and VideoQA benchmarks. Third, for open-domain datasets that contain officially defined training, validation, and test partitions, such as DiDeMo (Anne Hendricks et al., 2017). HACS (Zhao al. 2019), ActivityNet Captions (Krishnal et al.) 2017), and Kinetics 700 (Carreira & Zisserman, 2017). we explicitly retain only the train- ing split and manually exclude all validation and test subset videos avoid any accidental overlap with downstream evaluation. This three-stage filtering process ensures that TempGCap is clean, non-overlapping, and free of unintentional data or domain leakage under the zero-shot evaluation setting.
B.2 DATASET STATISTICS
Table 19 presents detailed statistics of the TempGCap dataset, categorized according to the un- derlying annotation strategy. The dataset comprises a total 559,300 conestamp-guideee captioning samples derived from untrimmed videos, with cumulative duration of approximately 8,831.4 hours.
Sub1 includes 136,284 samples from 39,239 videos, totaling 1, 1,220.1 hours. The average clip dura- tion is 111.9 seconds, and the average caption length is 48.9 words.
Sub2 includes 104,693 samples from 81,157 videos, totaling 3,387.4 hours. The average clip dura- tion is 150.3 seconds, and the average caption length is 86.7 words.
25

---

Published as a conference paper at ICLR 2026
Sub3 includes 318.323 samples from 318,323 videos, totaling 4,223.9 hours. The average clip duration is 47.8 seconds, and the average caption length is 72.3 words.
Some video sources appear in multiple subsets. After deduplication, the dataset contains approxi- mately 428K unique videos.
B.3 VISUALIZATION
To better illustrate the characteristics of the captions used TempGCap and to analyze the poten- tial noise introduced by the semi-automated annotation process, provide qualitative comparisons Figure [6] The upper examples correspond to samples converted from existing dense-caption datasets, where the descriptions typically summarize coarse, event-level content and often overlook fine-grained or temporally localized actions. In contrast, the lower examples show captions gener- ated by Tarsier, which contain richer action-centric details, clearer temporal progression, and more explicit object-action interactions. Although such fine-grained annotations differ from the style of earlier dense-caption datasets, this shift is not detrimental; on the contrary, the increased granular- ity provides stronger supervision for fine-graine temporal reasoning. Furthermore, by consistently using prompts of the form "Describe to differentiate between coarse -rrained and fine-gra fine-grained description styles. This reduces the poten- tial impact stylistic inconsistencies across data sources and allows the model benefit from the additional temporal specificity provided by Tarsier-generated captions.
detail" during training, the model is explicitly guided
C DETAILS OF TRAINING
Training Loss. All tasks in our framework, including timestamp-guided captioning, temporal grounding, and grounded VideoQA, are formulated as conditional text generation. Following stan- dard practice in large language model (LLM) training. we adopt the conventional autoregressive language modeling objective without introducing task-specific heads or losses.
For each training example indexed by n E {1....,N). let (n)) denote the multimodal input (e.g., video frames and textual prompt), and let
(")) = (v1") (n) .....
(x)
My
denote its target output token sequence of length T("), The decoder models the standard autoregres- sive probability
(n) (n)
Vet x(")).
pe yi
where ver = {vi" ............
...... ..-11 and denotes model parameters.
During training. we apply teacher forcing and compute the next-token prediction loss. Let (a) C (1...., (()) denote the index set prediction tokens, including all answer tokens as well as timestamp tokens. Following common LLM practice, average the negative lilkeleihhodd over all prediction tokens in the batch:
C
log po (v(")
(n) Ueet x(m)).
(9)
(m) n=1 tC)())
This token-wise averaged language modeling loss used for all tasks. Temporal boundaries are represented directly text tokens, no additional regression objectives, bounding-bor losses, or threshold heuristics are required. Similarly, we do not introduce auxiliary losses or class-balancing terms; any imbalance across datasets is handled purely through data sampling strategies (e.g.. up- sampling cnaestamp-grounderr examples), rather than modifying the optimization objective.
Training Setting. Table 20 summarizes the training configurations used in Stage I and Stage II. Stage adopts temporal input of 32 frames with 2 keyframes and a global batch size of 256. Stage II uses 128 frames with 32 keyframes and global batch size of 128. The model is trained
26

---

Published as a conference paper at ICLR 2026
Table 20: Training configuration for Stage and Stage
Config
Stage
Stuge
LLM buckhone
Qwen2-7BB VIT-g/14
VII backbone
Frame resolution
224 x 221
Temporal resolution (7" H² W")
32
Spatial resolution (T x 11 x W)
32 8
128 x
Number key frumes
Global batch size
256
128
Batch size GPU
Accumulated steps
SpepSpeed zero stage
Learning rate
1 x 10-3
2 10"
Leaming schedule
cosine decay
Warmup ratio
0.03
Weight decay
0
Epoch
Optimizer
Adam 8192
Precision
Max length
using the Adam W optimizer with a cosine decay learning rate schedule and a warmup ratio of 0.03. Both stages apply DeepSpeed ZeRO Stage and bf16 precision to improve memory efficiency. The maximum token length is set to 8192, and gradients are accumulated over 4 steps per GPU. For better multimodal understanding, we initialize the LLM weights with those from Qwen2-VL (Wang al. 2024d), while adopting ViT-g (Sun et al. 2023) as the vision encoder. To enhance temporal grounding capability, we perform resampling of grounding data from DiDeMo (Anne Hendricks et al. 2017) and ActivityNet Captions (Krishna et al.) 2017), allowing the model to encounter each sample three times during training. For DeVE-QA QQn et al., 2024), in addition to using it for question-answerigg training, we also repurpose its questions queries to construct grounding tasks. This design enables the model to better adapt to question-form queries in grounding scenarios. The experiments are conducted on 32 A100 GPUs. Stage I training takes approximately 3 hours, and Stage II takes around 80 hours.
D BENCHMARKS
We assess the performance of Divid across multiple benchmarks that target temporal grounding and grounded video question answering (VideoQA). The core datasets used in our evaluation summarized as follows.
Charades-STA (Gao et al. 2017) consists of approximately 10,000 indoor videos, each averaging around 30 seconds. It provides over 16,000 temporal annotations aligned with natural language de- scriptions of everyday activities. This dataset is widely adopted for evaluating temporal localization capabilities in constrained indoor scenarios.
CG-Bench (Chen al. 2024a) focuses on grounded VideoQA in extended videos. It comprises 1,200 curated videos spanning 10 to 80 minutes across diverse topics, accompanied by 12,000 QA pairs. The benchmark classifies questions into three categories- peeeriis-peeeeptionn reasoning, and halluci- introduces evaluation paradigms based on "white box" and "black box" clue verifica- tion, aiming to measure the fidelity of model predictions with respect to visual evidence.
NEXT-GQA Xiao et al. 2024). building on the NEXT-QA dataset, emphasizes reasoning over causal and temporal events. offers 10,500 QA samples with annotated temporal spans, focusing on questions that require causal inference and temporal sequencing. The original descriptive question type from NEXT-QA is excluded to better target event understanding over time.
ReXTime (Chen et al. 2024b) presents benchmark for high-level temporal reasoning in videos, using a semi-automated pipeline to generate QA pairs while ensuring annotation accuracy through manual verification. The dataset includes 921 validation and 2,100 test instances. Notably, there exists a substantial 14.3% performance gap between state-of of-te-of-the-ar models and human annotators, highlighting the challenge of understanding causal structures in complex temporal narratives.
27

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
All datasets used are publicly available and commonly adopted prior research, and no private or sensitive data included. Our model is intended for academic research only, and caution against misuse in applications such as surveillance or disinformation.
REPRODUCIBILITY STATEMENT
We provide detailed descriptions of our model architecture (Section 3), dataset construction (Sec- tion 3.3 and Appendix B), training setups including hyperparameters and hardware (Section 4. and Appendix C). and evaluation protocols for all benchmarks (Section 4.2 and Appendix D). Ablation studies (Section 4.4 and Appendix A) further validate individual design choices. We will release the trained models and the TempGCap dataset to facilitate reproducibility and future research.
USE OF LLMS
We acknowledge that large language model (LLM) is used during the preparation of this manuscript for the purpose of grammar checking and minor language refinement only. The LLM does not contribute to the core methodology, experiments, analyses, or results, All scientific content, claims, and conclusions are the sole responsibility of the authors.
28


Reviews:
[{"review_id": "Sxskb1XvME", "reviewer": "Reviewer_1ujf", "summary": "This paper presents DIVID, a novel dual-branch framework designed to disentangle spatial and temporal modeling within the LLM decoder for temporally grounded video understanding, with key innovations including: (1) a temporal branch that processes low-resolution, densely sampled frames to capture long-range dynamics; (2) a spatial branch that selects high-resolution keyframes via temporal attention for fine-grained visual reasoning; and (3) a token-level spatio-temporal soft-router that adaptively fuses features conditioned on the input query. To support training, the authors introduce TempGCap, a large-scale instruction-tuning dataset containing 559K timestamp-grounded video-text pairs with high temporal precision and diverse video coverage. Experiments on Charades-STA, CG-Bench, NExT-GQA, and ReXTime benchmarks demonstrate that DIVID achieves state-of-the-art performance in both temporal grounding and grounded video QA tasks while significantly reducing computational costs, establishing a new efficiency-accuracy trade-off for video understanding architectures.", "questions": "1. DIVID performs well on grounding tasks but shows limited gains on general VideoQA (e.g., MSRVTT-QA). What do you think limits its generalization to non-grounding tasks?\n2. Should the visual encoder (ViT-G/14) be fine-tuned during training to improve fine-grained temporal grounding?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The idea of disentangling spatial and temporal modeling inside the LLM decoder is novel and well-motivated. The soft-router mechanism is lightweight yet effective, enabling adaptive fusion at the token level.\n2. DIVID outperforms many strong baselines, including larger models (e.g., 72B Qwen2-VL), especially in temporal grounding tasks. The 1.5B model already surpasses several 7B models, showing excellent parameter efficiency.\n3. TempGCap is a valuable contribution to the community. It combines manual and automatic annotations, covers diverse video domains, and provides fine-grained temporal supervision, filling a gap in existing datasets.\n4. The paper includes extensive ablations, efficiency analysis, and generalization studies. The authors carefully analyze the impact of each component (e.g., keyframe selection, soft-router, dataset quality), which strengthens the credibility of the results.", "weaknesses": "1. While the disentanglement is well-executed, the contributions are mostly at the module level, lacking a more fundamental architectural breakthrough.\n2. The soft-router is effective but lacks interpretability. There is no analysis of how the gating weights vary across query types or whether the router learns semantically meaningful fusion strategies.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "E4vmoEJ0ug", "reviewer": "Reviewer_KBdS", "summary": "This paper introduces DIVID, a dual-branch architecture for temporally grounded video understanding using video–language models. The framework explicitly disentangles spatial and temporal modeling within the LLM decoder by employing a temporal branch for dense, low-resolution motion dynamics, and a spatial branch for sparse, high-resolution keyframes guided by temporal attention. The two branches are fused with a query-conditioned spatio-temporal soft-router. In support, the work presents TempGCap, a large-scale dataset of 559k timestamp-grounded video-text pairs. Experiments show strong empirical performance and efficiency gains on several temporal grounding and video QA benchmarks.", "questions": "1. Can the authors provide an explicit composite training loss equation, including all task-specific loss terms and weighting, as actually used in implementation (e.g., for temporal localization vs. QA)? How are thresholds, negative sampling, and class imbalance handled during optimization?\n\n2. What is the rationale for choosing a token-level softmax on projected text features for branch fusion? Did the authors experiment with deeper gating modules, or alternative fusion placements (e.g., block, head, or output level)?\n\n3. Do the temporal and spatial branches demonstrably specialize in the types of reasoning intended (e.g., is the spatial branch consistently responsible for frame-level details, or do branches collapse under certain queries)? Would the authors consider adding qualitative analyses or illustrative attention maps?\n\n4. Given the mixture of manual, pseudo, and synthesized captions in TempGCap, how is data leakage prevented on evaluation sets, and have any systematic biases been observed during annotation or curation phases?\n\n5. The most important is, I noticed that the comparative methods adopted by the authors lag behind the latest state-of-the-art research. Recently proposed models such as Qwen2.5-VL, VideoChat-R1, and TimeZero (introduced about six months ago) have achieved strong performance across various temporal localization and general video question answering benchmarks. What are the advantages of the authors’ proposed method compared with these approaches?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The explicit separation of spatial and temporal processing within the LLM decoder is a clear departure from prior art, which typically performs disentanglement only at the feature extraction or encoder stage. This is well articulated in Figure 1, highlighting the dual-branch structure and adaptive soft-router fusion, which advances the clarity and interpretability of model components.\n\n2. The method's use of temporal attention to guide high-res spatial sampling (rather than uniform or query-agnostic strategies) is empirically backed (Table 6, Figure 1) and brings notable computational and performance improvements, confirmed in ablation studies (Table 5).\n\n3. TempGCap is a substantial new resource, constructed via multiple strategies (manual annotation, recovery, synthesis) for more precise temporally grounded supervisory signals (see Figure 2, Tables 13–14 in the appendix). Dataset quality is assessed in Table 8, showing marked value over prior moment-captioning datasets.", "weaknesses": "1. Empirical claims about \"state-of-the-art\" are largely constrained to the benchmarks provided (Charades-STA, ReXTime, CG-Bench, NExT-GQA). Generalization is briefly touched upon (Table 11), but with limited context; in particular, the extent to which DIVID's improvements generalize to non-temporally grounded or more challenging open-ended video reasoning scenarios is only lightly probed and not explored in depth. Moreover, although some key baselines are included, there are missing baseline comparisons to models adopting more advanced spatio-temporal or cross-modal alignment, as well as absent or only shallow error/mislocalization analysis.\n\n\n2. The design of the dual-branch decoder and the token-level router is presented in a relatively engineering-driven fashion, with little attempt at formalizing why or under what conditions disentangled decoder branches + soft dynamic fusion should yield better temporal localization or reduce hallucination. There is no theoretical analysis of capacity, representation expressivity, or inductive bias gains. The discussion about why fusion at the token level (rather than head-level, block-level, or post-attention fusion) is beneficial is thin and left to ablations.\n\n\n3. The TempGCap construction process involves multiple automated and manual annotation stages (as in Figure 2), but more detail should be provided about inter-annotator agreement, filtering criteria, and possible biases introduced (e.g., via pseudo-untrimmed synthesis). Even though Table 13 breaks out annotation strategies, the risk of subtle domain leakage or noise propagation is underexplored.\n\n\n4. Minor Issues in Exposition:\n- There are a few places where notation is not adequately defined or is overloaded (see Section 3.2, Page 4-5).\n- Typographical issues, e.g., \"the the selected spatial features\" (Page 5).\n- Some results tables (Table 1, Table 2) reproduce highly dense comparisons, but occasionally lack scale clarity (e.g., 1.5B vs 7B), and the presentation would benefit from clearer highlighting to aid comparison.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "l1IYrxSHHs", "reviewer": "Reviewer_mbkF", "summary": "This paper introduces Divid, a dual-branch framework for video understanding that explicitly disentangles spatial and temporal modeling within the Large Language Model (LLM) decoder to improve temporally grounded tasks in long-form videos. The architecture features a temporal branch that processes dense, low-resolution frames to capture global motion dynamics, which in turn guides a spatial branch in selecting a sparse set of high-resolution keyframes relevant to the query. A lightweight spatio-temporal soft-router is proposed to adaptively fuse the information from these two branches at the token level for the final prediction. To support this model, the authors also constructed a large-scale dataset, TempGCap, containing 559K timestamp-grounded video-text pairs to provide rich temporal supervision. Experiments conducted on several temporal grounding and grounded VideoQA benchmarks show the effectiveness of Divid compared to existing methods.", "questions": "Please see Weaknesses", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The core contribution is the dual-branch structure that disentangles spatio-temporal information within the LLM decoder. The idea of using dense, low-resolution frames for temporal modeling to guide the selection of sparse, high-resolution keyframes for spatial modeling is intuitive and sound. Combined with the token-level soft-router for adaptive fusion, this approach effectively addresses the challenges of high computational cost and spatio-temporal entanglement in long video processing.\n\n* The paper identifies the shortcomings of existing instruction-tuning datasets in temporal precision and diversity and builds a high-quality dataset of 559K samples using three complementary strategies (manual annotation, recovering untrimmed contexts, and synthesizing pseudo-long videos). This will be highly valuable for future research in the community.", "weaknesses": "* Although the appendix shows a significant reduction in computational cost (TFLOPs), the reported latency is slightly higher than the slow-fast baseline. The authors acknowledge this is due to the extra keyframe selection step and suggest it can be improved with engineering optimizations, but it remains a minor weakness where the theoretical efficiency gains do not fully translate to practical inference speedup.\n\n* The construction of the TempGCap dataset partially relies on other models (e.g., using the Tarsier model to generate captions), which could introduce model-specific biases or errors into the dataset. While this is a practical approach for large-scale data creation, a more in-depth analysis of the potential noise introduced by this semi-automated process would have further strengthened the claims about dataset quality.\n\n*  The core temporal-guided keyframe selection mechanism (Temporal Attention Scores -> Top-K Selection) is conceptually similar to other attention-based filtering methods [1, 2, 3]. The paper could better contextualize its approach by comparing it more directly with prior work in frame sampling or token pruning to highlight its uniqueness in the context of Video LLMs.\n\n[1] MIST: Multi-modal Iterative Spatial-Temporal Transformer for Long-form Video Question Answering\n\n[2] Discovering Spatio-Temporal Rationales for Video Question Answering\n\n[3] Weakly Supervised Gaussian Contrastive Grounding with Large Multimodal Models for Video Question Answering", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)