A Training-Free Framework for Long Video Understanding via Video-Query-Options Similarity
Authors: Zhirong Wu
Xiaodong Wang
Langling Huang
Teng Xu
Peixi Peng
Venue: ICLR 2026 Poster (2026)
Keywords: long video understanding
multimodal large language model
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We propose a training-free framework for long video understanding that boosts MLLM performance via adaptive frame sampling, dynamic resolution allocation, and video-query-option similarity.

Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable success in image and short video understanding tasks, but their performance on hour-long videos remains limited due to constraint of input token capacity. Existing approaches often require costly training procedures, hindering their adaptability to rapidly evolving MLLM architectures. In this paper, we propose a training-free framework for long video understanding, integrating three key innovations: Adaptive Frame Sampling (AFS), Dynamic Resolution Allocation (DRA), and Video-Query-Options Similarity (VQOS). AFS adaptively increases frame sampling density in highly relevant video segments to preserve critical temporal details, while DRA reduces spatial resolution in less relevant segments to suppress redundant information. VQOS enhances similarity calculation by prompting MLLMs to generate candidate answer options, fusing queries with options to refine relevance estimation. Mirroring human cognitive processes (hypothesis generation → focused verification → irrelevance filtering), our framework effectively improve model accuracy without fine-tuning. The method is implemented on LLaVA-Video and Qwen2.5-VL respectively, and experimental results show our method could achieve state-of-the-art performances over 5 mainstream benchmarks. More visualization results and code are available in the Appendix. Code
is available in https://github.com/wuzhirong520/VTR-VLM.

Published as a conference paper at ICLR 2026
A TRAINING-FREE FRAMEWORK FOR LONG VIDEO UNDERSTANDING VIA VIDEO-QUERY-OPTIONS SIMI- LARITY
Zhirong Wu', Xiaodong Wang Langling Huang', Teng Xu Peixi Peng
School of Electronic and Computer Engineering. Peking University, ?Pengcheng Laboratory. "Douyin Group
(wuzhirong@stu.. ( aatuu.edu.com cn
ABSTRACT
Multimodal Large Language Models (MLLMs) have achieved remarkable success in image and short video understanding tasks, but their performance hour-long videos remains limited due to constraint of input token capacity. Existing ap- proaches often require costly training procedures, hindering their adaptability rapidly evolving MLLM architectures. In this paper, we propose a training-free framework for long video understanding, integrating three key innovations: Adap- tive Frame Sampling (AFS), Dynamic Resolution Allocation (DRA), and Video- Query-Optione Similarity (VQOS). AFS adaptively increases frame sampling density in highly relevant video segments to preserve critical temporal details, while DRA reduces spatial resolution in less relevant segments to suppress redun- dant information. VQOS enhances similarity calculation by prompting MILMs to generate candidate answer options, fusing queries with options to refine rele- vance estimation. Mirroring human cognitive processes (hypothesis generation + focused verification - irrelevance filtering), our framework effectively improve model accuracy without fine-tuning. The method is implemented on LLaVA- Video and Qwen2.5-VL. respectively, and experimental results show our method could achieve state-of-the-art performances over mainstream benchmarks. Code is available in httpt::t///// htttps://githbb. com/ www/wwzhtwwwwccccccco....ww... wuzhi rong520/VT--
INTRODUCTION
In recent years, Multimodal Large Language Models (MLLMs) have made rapid progress (Zhagg
let al.) 2024b Wang et al. 2024a Liu et al. 2025b Zhang al. 2025a Bai al. 2025b Zhu
et al. 2025), with various post-trainigg (Wang et al. 2023c 2026 Bai et al. 2025a) and reasoning
methods (Guo et al. 2025, Wang & Peng. [025 Wang et al. 2025a) being developed enhance video understanding. However, for long eeseos--speciallly hour-long videos- existing MLLMs generally exhibit limited capabilities, leaving substantial room for improvement.
The fundamental challenge in long video understanding stems from the inherent contradiction be- tween limited model context windows and the vast spatiotemporal extent of video content. To mit- igate this, some methods (Zhang et al. 2024a Chen et al. 2024 Shen et al. 2025) extend the maximum token capacity through techniques such as parallel processing or multi-stage training-
Others (Shen et al. 2024 Qin et al.] 2025 Wang et al.) 2025d Li et al. 2024) exploit the inher- ent temporal and spatial redundancy in videos, aiming to compress the input by retaining only the most informative tokens. However, these methods typically require extensive and costly training procedures. Recently, training-free approaches (Ma et al. 2025b] Wang et al.) 2025bb TTag et al., 2025) have demonstrated great potential by extracting meaningful and representative information from long videos without requiring model fine-tuning. In the field of training-fres free long video un- derstanding, retrieval-based strategies offer viable solution. Leveraging the strong performance of existing vasion-language models of existing vision-language models (Radford et al. 2021 Zhai
al. 2023) in image-text retrieval, some methods (Tang al.) 2025
Liu et al. 2025a effectively
"Corresponding author.

---

Published as a conference paper at ICLR 2026
Question: What color the gloves when cleaning parts?
Video: 50 minutes
Casel Action-Related Question
Uniform Sampling:
Answer: Bluc
O. Ia noseedance with vidao Boriage, who protects tie mennaid
A: The shark.
Key Video Segment:
Image Similarity:
Video-Tasi Sanilusity
Rondeedaa:
Case2: : Temporal Dependency Question
Guess: Whitee Blue? Green?
Answer: Green
Q:
roil Dominiqge the man red shir black She hugs him.
lhe stagel
Key Video Segment:
Trage-Test Similurity:
Video Text Samilarity:
Rondedio::
418555
4883253
(a) Unifom Sumpling and Our Methods
(a) Image-Tex: and Video-Ted
Figure 1: Comparisons with existing methods. (a) Most existing MLLMs rely on uniform frame sampling, which- to context length limits-often misses critical information in long videos. Our approach enhances MLLMs' question-answering capabilities by retrieving question/options- relevant frames, densely sampling highly pertinent segments, and employing elevated resolution settings. (b) While some approaches leverage image-fext retrieval, they are inadequate for action- related or temporally dependent questions, as static frames cannot reliably capture dynamic cues or temporal structure. Video-text retrieval is therefore essential in our method.
retrieve key frames that are semantically relevant to the user's query, thereby enabling MLLMS to focus on critical content. However, these approaches exhibit two key shortcomings: (I) they pre- dominantly focus on static image, neglecting important information in videos such as actions, causal relationships and temporal dynamies, shown in Fig. (b), and (2) while effective at selecting key frames, they lack mechanisms to deeply exploit or refine the information contained in those frames.
In this paper, ods for long video understanding. Building upon the state-of-the-art (SOTA) video-text retrieval model (Bolya et al. 2023), which computes similarity scores between video segments and textual queries, we introduce two key components: Adaptive Frame Sampling (AFS) and Dynamic Res- olution Allocation (DRA). The former adaptively samples a greater number of frames from video segments with higher similarity scores, thereby enhancing the representational richness of relevant content. The latter reduces the spatial resolution of less relevant segments, optimizing both com- putational efficiency and focus on salient regions. To further refine the similarity computation, we propose dileo-Query-Optiinn
conduct in-depth exploration of such training-free retrieval-bact -based meth-
Similarity (VQOS), novel strategy wherein original MILM is prompted to generate plausible answer options based on the user query. The similarity between the video and each generated option then computed and fused with the original query for more ro- bust relevance estimation. Our approach closely emulates the cognitive process by which humans comprehend and answer questions about long videos: when faced with question, humans typically formulate several hypotheses, selectively review the vidco verify these hypotheses, and naturally filter out irrelevant information during the process. By mimicking this behavior, our method effec- tively improves accuracy in long video understanding without requiring any model fine-tuning. A typical example is shown in Fig. (a) to illustrate our method.
We integrate method into MLLMs including LLaVA- Video (Zhang et al. 2024b) and Qwen2.5-
VL (Bai et al. 20255), across both 7B and 72B parameter scales. Experimental results on 5 long video understanding benchmarks demonstrate size and 3.6% and 3.2% in 72B size compared with LL LLaVA-Vin Videeo and (ween2.5-VL. Especially, on hour-long video benchmarks LVBench (Wang et al. 2024b) and VideoEval-Pro (Ma et al.) (202aa), our 7B-scale model achieves substantial improvements, outperforming LLaVA- Vi-iodeo and Qwen2.5- VL by an average 8.5% and 8.3%, respectively. Our contributions are summarized as follows:
average performance gain of 5.3% and 5.0% in 7B
We propose AFS and DRA to adaptively optimize frame selection and resolution resizing based on relevance scores for long video understanding.
We develop VQOS mechanism, which leverages the capabilities of MLLMs to generate candidate answers and enhance similarity estimation through multi-hypothesis fusion.
2

---

Published as a conference paper at ICLR 2026
We integrate method into LLaVA- and Qwen2.5-VL across 7B and 72B scales, achieving significant performance gains on 5 long video benchmarks demonstrating its effectiveness and scalability in long video understanding.
RELATED WORK
2.1 MLLMS FOR LONG VIDEO
Existing approaches address the long video understanding challenge through two primary strategies: context extension (Zhang et al.,
2024a Chen et al.,
2024; Shen et al. 2025) and token compres-
sion (Shen et al.; 2024 Qin et al.
2025
Wang al. 2025d Li et al. 2024). The context extension
strategy focuses on increasing the maximum sequence length that models can process during train- ing. For example, Long (Chen et al. 2024) employs multi-stage training pipelines and novel parallelism techniques to expand contextual capacity. In contrast, token compression methods aim to preserve more informative content within reduced number of tokens. Intern Video2.5 (Wang et al. 2025d) adopts hierarchical token compression combined with task-preference optimization to improve representation efficiency, while VideoXI--2 (Qin et al., 2025) introduces task-aware key- value (KV) sparsification to enhance memory utilization. However, these methods require expensive training, limiting their adaptability to rapidly evolving MLLM architectures.
2.2 TRAINING-FREE LONG VIDEO UNDERSTANDING
Training- free approaches for long video understanding aim to extract meaningful and representative information without requiring model fine-tuning. These methods can be broadly categorized into three types: (1) Agent-based approaches (Zhang et al. 2023; Wang et al., 2024c Luo et al., 2024] Ma et al. 2025b Pang & Wang 2025 Zhang al. 2025b involves dividing long video into shorter clips, where agents generate descriptive captions for each clip and subsequently use these textual summaries to answer questions. For instance, Deep Video Discovery (Zhang al., 2025b) utilizes L.-M-based agents to autonomously explore and reason over segmented clips. In contrast our method, these approaches are fundamentally dependent on video captioning, a process that is computationally intensive and prone to loss of fine-grained visual details due to the abstraction of rich visual content into text. Furthermore, they often rely on proprietary models such as GPT- 4o (Hurst et al.) 2024) as the underlying agent, making direct comparison unfair and impractical. (2) Compression- based approaches (Wang al., 2025b Luo et al.. 2025 Gao al.) (023) fo- cus on reducing redundancy in the visual token stream, enabling more efficient processing of long video sequences. These methods typically operate on the internal token representations within the
MLLM, compressing pruning less informative visual tokens. For instance, AdaReTake (Wang) ctt al. 2025b) adaptively removes redundant information the key-value (KV) cache across both temporal and layer dimensions, allowing MILMs to process up 2048 frames efficiently. These techniques are complementary to our approach: while they operate at the token level during infer- ence, our method targets the input frame-sampling stage prior to model ingestion. (3) Retrieval- based approaches (Park et al. 2024 Tang et al. 2025 Liu et al. 2025a) employ lightweight expert models to identify and retrieve keyframes that most relevant to the given question. For in- stance, AKS (Tang et al. 2025) computes frame-question similarity using CLIP (Radford et al.) 2021) embeddings to select semantically aligned frames. Our method is closely aligned with this paradigm; however, we introduce a more refined similarity estimation strategy by leveraging a ded- icated video-text retrieval model to jointly compute the similarity among the video, the question, and the candidate options. Furthermore, we utilize the resulting similarity scores not only for adap- tive frame sampling but also for dynamic frame resizing, thereby enhancing the quality of the input representation.
2.3 VIDEO-TEXT RETRIEVAL
Video-Text Retrieval (VTR) is cross-modal task that aims to measure the semantic similarity be- tween video content and textual queries. Vision language models like CLIP (Radford et al. (021)) and SigLIP (Zhai et al. 2023) excell image-text retrieval, and can also zero-shot to video-text trieval by pooling image embdeddings. CLIP4CLIP (Luo et al. 2022) fine-tunes the CLIP model for video-text retrieval leavaging contrastive learning. Recently, PerceptionEncoder (Bolya al. 2025)
3

---

Published as a conference paper at ICLR 2026
Question: What does Dominique do after the mun with shirt and black and red pants comes to the stgg??
MLLM
She claps him luuge Bun.
What
staee
slpps him Bugs
Anrwer
Option Generation Prompt: Given the vidao and the queston, generate some possible answers.
She bugs himz.
Zoom
Ensodder
Spucse
Out
Multimodal Lange Language Model
Dynumic Resolut lution Allocation
Viden- Text Cosine Similarity
196x112
Adaptive Frame Sample
Zoom
Enooder
Vidto
T1:
Denso
Video Segments
Similarities
2526140
2006766
Overall Framework. We first generate plausible answer options using the original MLLM, concatenate them with the question, and compute similarity scores between the resulting queries and video segments using pre-trained video-text retrieval model. Based on these similarity scores, Adaptive Frame Sampling increases frame density in high-similarity regions, while Dynamic Resolution Allocation increases resolution in more relevant segments.
Figure 2:
pretrained visual encoder with extensive video data, which makes it have excellent performance in video text retrieval task.
3 METHODOLOGY
The overall framework of our approach is illustrated in Fig. 2 By generating options and leveraging a video-text retrieval model to compute their similarity with video segments (Sec. (3.I), we guide
both adaptive frame sampling (Sec. 3.2) and dynamic resolution allocation (Sec. (33).. Specifically, regions with higher similarity receive denser frame sampling and higher spatial resolution. For clarity, we provide pseudocode in Sec. 3.4
3.1
SIMILARITY
Given long video, we uniformly divide into m equal-length segments, denoted as
(Vi, V2...., V..).
For each segment Vi and query Q, employ video-text retrieval (VTR) model to extract the video and text features, denoted as fer, fa E IRd"" The initial similarity score S for the i-th video segment is then computed as the cosine similarity between the video and text features:
fu, fo
S
(2)
Then we prompt the MLLM generate candidate options, where the prompt is shown in Fig. Subsequently, the question is concatenated with each option to form distinct statements:
T = [Ti, T.)..T.).
(3)
These statements are encoded into text features using the same VTR model respectively, and com- bined as F. The final similarity score S, to for the i-th video segment is then computed as the maximum cosine similarity between the video feature and all text features in F::
fr.
S. = max
[f [I
4

---

Published as a conference paper at ICLR 2026
Note that S. is utilized for selecting video segments based on user queries without options, and S; is employed when the options are available. Both undergo the same processing pipeline described in Sec. 3.22 and Sec. 3.3] To enhance diversity in option generation, options can be generated multiple rounds by splitring the whole video into several parts and generating options separately. Specifi- cally, for multiple-choice questions, the option generation be bypassed, candidate answers are provided.
S.c.B.2.ar
3.2 ADAPTIVE FRAME SAMPLING
To effectively represent long video with a limited number of frames, we adopt a adaptive sam- pling strategy that emphasizes semantically relevant content. Instead of treating all video segments equally, we leverage their similarity the input query to guide the sampling process.
To sample N frames from video, we first select the top-k segments based on their video-query query- options similarity scores S1, S2....... Sk. From each selected segment V,, we uniformly sample ps frames such that the following constraints are satisfied:
=N S + Ps-
(5)
These conditions ensure that the total number of sampled frames remains fixed N, and segments with higher similarity scores are assigned more frames, while segments with lower scores are as- signed fewer.
To simplify the allocation process while adhering to the desired priority, we first sort the top-k segments descending order of their similarity scores. These sorted segments are then partitioned into L1 sampling levels, where the 1-th level contains my segments, each segment samples frames satisfying that:
Σ = K. Σ m4 N,
(6)
where > -2 > CL,- For simplicity, we assume roughly equal segment distribution across levels, utilizing Pulp (Mitchell al. 2011) to find feasible {ml) with predefined {et].
This sampling strategy allows more relevant segments to contribute a greater number of frames, thereby enhancing the semantic coverage and representativeness of the selected frame set while maintaining computational simplicity.
3.3 DYNAMIC RESOLUTION ALLOCATION
For long video sequences, existing visual encoders often encounter a fundamental trade-off between the number of processed frames and spatial resolution: given a fixed total input token budget (deter- mined by the model's sequence length limit), increasing the number of frames necessitates down- scaling the spatial resolution of frames, which may result in the loss of critical spatial information in high-importance frames. To mitigate this issue, we propose an Dynamic Resolution Allocation strategy that allocates higher resolution to key frames (frames of high task relevance) and lower resolution to non-key frames.
For given video with resolution H W. define L2 resolution levels as:
0 H
a: W
H:
W.
I,
where 01 > 02 > aL, E (0, 1] are scaling factors that preserve the aspect ratio, and / = 28 is stride constraint imposed by the visual encoder architecture due to patch size and downsampling requirements. Each resolution (H, W,) is further constrained to lie within a valid range Cmin < H, W. S Craz-
A feasible resolution allocation strategy {nL, 122, n2 must satisfy two constraints: (1) All frames are allocated to J. resolution levels, (2) The sum of tokens across all frames equals the budget P:
L
N.
(m H, Wt) P.
(8)
1=1
5

---

Published as a conference paper at ICLR 2026
Frames with higher similarity scores assigned to higher-resolution levels, for example n1 frames with largest similarity scores are allocated resolution (H1, W1).
To simplify computation while maintaining a balanced distribution of tokens across resolution levels, we assume a uniform per-level token budget - that is, each resolution level receives an equal share of the total token budget P Under this assumption, the number of frames allocated to resolution level i can be approximated by:
P
f4
L.
3.4 PSEUDOCODE OF OUR METHOD
As outlined in Algorithm | given set of pre-segmented video clips V and a question our method first encodes the question and all video segments using VTR model to obtain embeddings. Initial video-question similarity scores (S") are computed via cosine similarity, and the top- N most relevant segments are selected to form a refined candidate pool V" Over R rounds of option generation, the algorithm samples N segments from V per round, applies Adaptive Frame Sampling (AFS) guided by S", and enhances frame quality via Dynamic Resolution Allocation (DRA). These processed frames fed into MILM generate diverse candidate answer options. Each option then combined with the original question, re-encoded by the VTR model, and scored against all segment embeddings; the maximum similarity across options yields refined relevance score. Finally, the top-N segments are retrieved based on this similarity score, resampled with AFS and DRA, and passed the MLLM together with the question to produce the final answer. Note that the R rounds of option generation are not temporally dependent and can be parallelized for acceleration.
Algorithm Our Method
Require:
Video segments (V), question (g), option generation round (R),, sampled frame num (N), video-text retrieval model (VTR), multimodal large language model (MLLM), Adaptive frame sampling (AFS), Dynamic Resolution Allocation (DRA).
Ensure: Answer the question according to the video.
1: begrnent_embeeddings
0
2: simiarities + 0
generated options 4: q embedding +- VTR(g)
5: for each V, do
6:
7: end for
8: S + cosine_simin www.iiirrttiiaeemmmdddnnnnnnnee
9: + TopK(V) (V,S0, R N)
10: for t- 1o R. do
N-1 11: V, < (
N-1
VEW + DRA(AFS(S)), V,))
12: campled.frame
13: options + MLLM('Please generate some options...!' sampled
14: generated -otiionss eeteiiinnseeeteeddee (options)
end for
16: for each o E generaaeed_options do
o_embedding +- VTR(g +
18: S+ possinessimiaritjjoommmeee
segment.embeddinng) similarities +- max max(simiarities,
20: end for
Viinal <- Topo ( simiarities, N).
22: sampled.frande
+- DRA(ATS(simiaritte) Vfinat)
23: answer <- MLLM(g, sampled_frumes)
24: return answer

---

Published as a conference paper at ICLR 2026
Table Comparisons on widely used benchmarks. LongVB and VMME refer Long VideoBench and VideoMMEF respectively.
LVBehh (67min)
MLVU (13in)
LongVB (8min)
VMME (17min)
VideoEval-Pro
(38min)
Model
Size
Average
Overall
M-Avg
Overall
Overall
Open
MCQ
Proprietary MLLMs
Gemini-I.5-Pro
33.1
64.0
75.0
39.3
63.4
GPT-4o
48.9
64.6
66.7
71.9
34.2
59.5
57.6
Seed1.5-Vi
200B
64.6
82.1
74.0
77.9
40.7
66.6
67.7
Open-Source MIL.Ms
Qwen2-VI.
7B
44.2
69.8
55.6
63.3
26.5
48.2
51.3
NVILA
42.6
70.6
57.7
64.2
Vide3LLaMA3
7B
45.3
73.0
59.8
66.2
In3ernVL3
8B
71.4
58.8
66.3
24.7
48.4
Training-Based MLLMs For Long Video
LongVA
7B
56.3
52.6
16.5
38.0
VideoChat-Flash
7B
48.2
74.7
64.7
65.3
27.0
51.2
55.2
Intern InternVideo22 Video2.5
74.9
60.6
65.1
27.2
53.2
54.6
Video-XLL XL-2
8B
48.4
74.8
61.0
66.6
28.6
53.0
55.4
Training-Free MILMs For Long Video
LLaVA-Vidco
7B
42.0
69.3
57.4
63.2
24.2
47.6
50.6
+ AKS
47.0
69.1
62.9
65.3
28.9
51.3
54.1
Take
49.6
70.6
59.6
64.0
27.7
53.5
54.2
+ Ours-GO
70.3
32.7
55.3
55.9
+ Ours-PO
54.2
73.4
61.0
65.5
56.9
57.3
Qwen2.5-VL
7B
45.5
69.4
61.0
66.4
27.7
46.6
52.8
Take
51.0
72.9
61.9
67.4
30.8
53.7
56.3
+ Ours-GO
52.7
72.3
63.4
66.7
35.0
$6.9
57.8
+ Ours-GO + AdaReTake
55.5
74.3
35.4
57.3
59.1
+ Ours-PO
55.5
74.1
63.3
67.9
57.6
58.9
+ Ours-PO PO AdaReTake
57.5
74.7
64.2
69.4
58.2
59.9
LLaVA-Video
72B
46.1
71.3
62.4
70.3
26.7
50.1
54.5
+ Ours-GO
51.7
71.7
63.6
70.3
33.1
58.3
58.1
+ Ours-PO
54,8
74.7
64.0
70.3
60.1
59.5
Qwen2.5-VV
75.3
29.9
55.9
58.2
+ Durs-GO
54.0
76.9
66.3
72.7
36.5
61.9
61.4
+ Ours-PO
56.9
77.7
66.3
73.1
64.2
62.5
Table 2: Ablation for components. For Long VideoMME, and VideoEval-Pro,
evaluate on representative and cost-efficient subsets: LVB-L and VMME-L (the long-video sub-
sets of Long VideoBench and VideoMME, respectively) and VEP-M (the multiple-choice subset of VideoBval-Pro).
Method
L.Bench
MLVU
LVB-L
VMME-L
VEP-M
Aavg
LLa VA- VA-Video-7B
42.0
69.3
48.2
51.4
47.6
+ top-N frames retrieval
49,6
70.0
55.3
51.8
53.5
+4.3
+ top-k segments uniform sampling
48.9
70.3
55.7
53.2
53.1
+0.2
+ adaptive frame sampling
50.2
70.4
56.6
53.2
54.2
+0.7
generated options
51.3
70.3
$7.1
54.0
55.3
+0.7
provided options
54.2
73.4
55.9
55.0
56.9
+1.5
Qwen2.5-VL-7B
45.5
69.4
53.7
55.6
46.6
top-N frames retrieval
50.3
70.0
$3.5
55.4
51.4
+2.0
+ top-k segments uniform sampling
50.1
70.3
$3.9
55.1
51.1
+0.0
+ adaptive frame sampling
51.1
70.0
55.3
55.7
52.7
+0.8
+ dynamic resolution allocation
51.9
72.1
59,0
56.0
55,9
+2.1
generated options
52.7
72.3
58.5
56.4
56.9
+0,4
+ provided options
55.5
74.1
56.9
57.6
+1.2

---

Published as a conference paper at ICLR 2026
4 EXPERIMENTS
4.1 IMPLEMENTATION DETAILS
We divide videos into 16-second segments and employ PE-G/14 (Bolya et al. 2025) for video- text retrieval with fps 1. Then integrate our method into both the 7B and 72B variants of LLaVA- Video (Zhang et a 2024b) and Qwen2.5-VL. (Bai et al.) 2025b). For LLaVA- Video, we limit the input maximum of 64 frames, select top-16 segments, and set the frame sampling level to (2, 4, 8) without employing DRA, as it is designed to accept fixed-resolution inputs of 384 x 384. For Qwen2.5-VL,, we the maximum number of frames to 768, select top-48 segments, set the frame sampling level to {8, 16, 32). and constrain the resolution budget to 20480 x 28 x 28 with resolution levels ranging from 84 to 644. Furthermore. we rerun two typical training- free methods AKS [Tang et al. 2025) and AdaRe (Wang et al. 2025b) on the datasets which are not reported in the original papers, and integrate AdaRe'Take into our method using 2048 frames as input.
Two versions of our method are implemented: (1) Ours-GO which means the candidate options are generated by original MLLM without additional prior knowledge, and (2) Ours-PO which indicates the candidate options are given by the dataset. Empirically, we generate options three iterations for LLa VA-Video and once for Qwen2.5-VL. Ours-GO is more general and the comparisons with other methods are fair. In contrast, Ours-PO only adapts to the multiple-choice questions, and the results are just for reference.
4.2 BENCHMARKS
Five widely used datasets are used, including: (1) LVBench (Wang et al. 2024b) a benchmark designed for evaluating extreme long video understanding. (2) MLVU (Zhou et al. 2025 is a multi- task benchmark for long video understanding. We report the M-Avg metric on the dev set. (3) LongVideofe VideoBench (Wu et al. 2024) is benchmark for both short and long video understanding. We report the overall accuracy on its val set without interleaved subtitles. (4) VideoMME (Fu al. 2025) the first-ever comprehensive video understanding benchmark. We report the accuracy results without using subtitles. (5) VideoEval-Pro (Ma et al. (225a)) is benchmark designed for more robust evaluation of long video understanding capabilities. It consists of 465 videos, with each video exceeding 10 minutes, selected from the four aforementioned benchmarks. Unlike previous benchmarks that primarily rely on multiple-choice questions - which may allow models to exploit answer options through guessing - VideoEval-Pro adopts open-ended questions, offering a more robust, comprehensive, and realistic assessment of models' long video comprehension abilities. The open-ended and multiple-choice metrics are reported. Overall, the average video lengths of these benchmarks are about 67 minutes, 13 minutes, minutes, 17 minutes and 38 minutes, respectively. More evalution details could be found in Appendix D
4.3
MAIN RESULTS
The compared results are shown in Table 1 It is evident that:
(1) Compared with LLaVA-Video and Qwen2.5-VL VI. across two model scales (7B and 72B), Ours- Go demonstrates substantial performance gains on five prominent long-form video benchmarks. Specifically, the 7B-scale models exhibit performance improvements of 5.3% and 5.0% respectively, and the 72B-scale models could achieve gains of 3.6% and 3.2% respectively. It shows our method could improve the baseline significantly. Ours-PO outperforms Ours-GO in most cases due to the guaranteed presence of correct answers provided options. It indicates the method could leverage the prior knowledge of the options.
(2) Ours-Gr Go outperforms the SOTA training-free approaches AKS and AdaReTake overall. Ours- Go achieves comparable performances in datasets with relatively shorter videos including MLVU, Long VideoBench and VideOMME, while shows clear advantage in longer video datasets LVBench and VideoEval-Pro (more detailed results these two benchmarks are in Appendix B). Specifically, based on LLaVA- Video-7B, Ours-Go outperforms AdaReTake by an average of 0.6% on the first three datasets and by 2.8% on the latter two. Compared with AKS, Ours-Go lags slightly by an average of 0.4% on the former three datasets but surpasses it by an average of 3.0% on the latter two. It shows our potential for long video understanding.
8

---

Published as a conference paper at ICLR 2026
(3) Since AdkReTake operates via token-level compression, could further integrate the two ap- proaches, yielding improved results 1.3% and 1.0% as shown in "Ours-Go + AdaReTake" and "Ours-PO + AdaReTake".
(4) Overall, our method achieves SOTA average performance except proprictary models on these benchmarks without any training. This underscores the effectiveness and practicality of our ap- proach in real-world long video understanding scenarios.
Since AKS doesn't provide code for Qwen2.5-VL, here compare it only LLaVA- Video for rigorous. We don't I compare with other training-free methods due to differing used models and miss- ing results on certain benchmarks, making direct comparison infeasible. More detailed comparisons with AKS and results of other training-free methods are included in Appendix A for reference.
4.4 ABLATION STUDIES
Effect of components. We demonstrate perfor- mance improvements after integrating partial com- ponents of our method into LLaVA- VA-Video-7B and We could ob-
Table 3: Ablation for VTR models LVBench. Res. refers to video resolution. Time refers to average inference time (sec- ond) per minute video.
Qwen2.5-VL-7B, as shown in Table [ tain the follow key findings: (1) Image-text retrieval for top-N frames yields significant gains of 4.3% and 2.0% respectively. Retrieving top-k video seg- ments with uniform sampling within each segment provides only marginal gains of 0.2% and 0.0%. indicates that direct retrieval on video segments plays similar role with frame retrieval. (2) In trast, incorporating our AFS based on video-text re- trieval provides additional improvements of 0.7% and 0.8%, and our DRA in Qwen2.5-VI--7B con- tributes substantial 2. 1% gain. The reason that the proposed AFS and DRA could focus on more important cues adaptively. (3) Furthermore, utiliz-
Model
Size
Res.
Ace
Time
CLIP-B/32
0.2B
224
49.8
0.6
CLIP-B/16
0.2B
224
50.5
0.7
CLIP-L/14
0.4B
224
50.8
0.9
CLIP-L/14-336
0.4B
336
51.2
1.7
PE-L/14
0.6B
336
52,0
1.8
PE-G/14
2.4B
448
54.2
15.5
SigLIP-LLaVA
0.8B
384
52.0
ing our generated options for similarity computation yields 0.7% and 0.4% improvements, while leveraging provided multiple-choie options achieves further gains of 1.5% and 1.2%.
Ablation for VTR models. Table 3 presents zero-shot video-text retrieval results using image- text retrieval models (CLIP) and pretrained video-text retrieval models (PerceptionEnnoode)) Re- sults demonstrate that models with larger parameters, higher resolution, and video-specific pre- training could enhance performance. For computation constrained scenarios, smaller mod- els remain viable alternatives (Appendix G.T.. We also introduce "SigLIP-LLaVA", which directly reuses LLaVA-Video's visual encoder encoder-requiriing requiring no additional VTR model (see Ap- pendix G.2). Moreover, in streaming scenarios, VTR inference time be effectively hid- den by overlapping with video ingestion latency, rendering it negligible (see Appendix G.3
Ablation for segment length and sample fps. shows results for LLa VA-VVieo-- and Qwen2.5-VL-7B on LVBench. When segment length is 0, video segments will degenerate to single images. Results indicate that seg- ment lengths of 8s or 16s yield superior per- formance, with VTR model sampling fps of 1.0 being optimal. When the segment length too long (c.g., 32s 64s), performance drops due to loss of fine-grained event details. Lower fps and longer segments reduce compu- tational cost, making this configuration suitable for cesourree-oonsrrinne scenarios.
Table 4: Ablation for segment length and fps.
Segment Length
Model
FPS
16s
32s
64s
0.5
51.8
53.4
52.7
51.5
48.7
LLaVA
51.4
54.2
52.0
50.8
47.4
2.0
51.0
53.6
53.7
50.9
48.6
0.5
48.9
54.3
54.5
54.0
50.3
Qwen
1.0
49.8
54.0
55.2
52.8
50.0
2.0
49.2
54.8
54.3
52.9
49.8
Ablation for video segment retrieval. Table 5] shows results for Video-- and Qwen2.5-
VL-7B LVBench with varying top-k selections and different total frame counts. The

---

Published as a conference paper at ICLR 2026
sults indicate that retrieving video segments may degrade performance, due to the inclu- sion irrelevant content and reduction of information per segment. Additionally, to comple- ment local retrieval, also explore how global information can be integrated in Appendix F
Ablation for similarity computation. presens results on VideoEval-Pre and LVBench using LLaVA-Video-17 comparing performance with question- only imputs, generated options, and pro- vided options. We also report the up- per bound of our method using correct answers for similarity computation. Re- sults shows that provided options signifi- cantly enhance performance, while gen- crated options are essential for open- ended questions lacking predefined op- tions.
Table 5: Ablation for video segment retrieval.
Top-K
Model
Frames
16
32
64
16
48.6
47.7
46.0
LLaVA
32
50.0
51.0
49.3
47.5
64
51.9
53.3
54.2
48.5
46.7
64
50.4
50.2
49.4
46.5
45.3
256
51.6
51.5
52.2
52.6
49.8
Qwen
512
52.8
53.4
53.5
53.9
1024
54.6
53.7
53.8
4.5 OPTION GENERATION QUALITY
To systematically evaluate the quality of the generated options, we introduce two complementary metrics: (1) Option Coverage Accuracy (OCA) the proportion of questions for which at least one generated option semantically equivalent to the ground-truth answer. high OCA indicates that the generator reliably includes the correct answer within its output for most questions. (2) Mean Proportion of Correct Options (MPCO) the average fraction semantically correct options among all generated options. This metric penalizes models that generate many incorrect or irrelevant options alongside the correct one. Details of these metrics could found in Appendix E.T
Fig. illustrates how the number of option generation rounds affects LLa VA-Videod performance on VideoEval-Pro in terms of answer accuracy, OCA, and MPCO. As the number of rounds increases from 4, OCA steadily rises (54.6% - 63.8%). indicating improved coverage of semantically correct options. In contrast, MPCO consistently declines (26.8% - 20.0%), reflecting the dilution of correct options in an expanding pool of distractors. Notably, model accuracy initially benefits from improved OCA (rising from 54.2% to 55.3% within 3 rounds), but begins to slightly decline thereafter (55.3% + 55.1%) suggesting that beyond certain point, the inclusion of additional distractors outweighs the gains from better option coverage, ultimately harming answer accuracy. More experiments about option generation could found Appendix E.2 and Appendix E.
Table 6: Ablation for similairty computation.
VideoEval-PPro
55.25
Accurat OCA
Method
LVBench
61.8
126
Open
MCQ
55.00-
MPCDON)
only question
31.0
54.2
50.2
54.75
generate round
31.2
54.3
54.50
generate rounds
32.1
54.3
50.8
54
56
20.9
generate 3 rounds
32.7
55.3
51.3
54.25
54.2
11.0
20,0
generate 4 rounds
31.9
55.1
50.4
provided options
56.9
54.2
Option-Gisneration Rounds
correct answer
36.4
60.9
58.6
Figure 3: Impact of option generation rounds
CONCLUSION
We propose a training-free framework for long video understanding through dynamic adjustment of frame resolution and sampling density, leveraging similarities between video, query, and options. Evaluated on five benchmarks with multiple MLLMs, our approach achieves significant performance gains, demonstrating scalable and effective paradigm for long-form video comprehension.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
The study was funded by the Shenzhen Science and Technology Program (KQTD200240729100011666),, the National Natural Science Foundation of China under con- tracts No. 62422602, No. 62372010, No. 62425101, No. 62332002, No. 62372010, No. 62206281, and the major key project of the Peng Cheng Laboratory (PCL2021A13 and PCL2025A02). Computing support was provided by Pengcheng Cloudbrain.
REFERENCES
Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang. Zhaohai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Junyang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Yang Liu, Dayiheng Liu, Shixuan Liu, Dunjie Lu, Ruilin Luo, Chenxu Lv, Rui Men, Lingchen Meng. Xuancheng Ren, Xingzhang Ren, Sibo Song, Yuchong Sun, Jun Tang, Jianhong Tu, Jianqiang Wan, Peng Wang, Pengfei Wang, Qi- uyue Wang, Yuxuan Wang, Tianbao Xie, Yiheng Xu, Haiyang Xu, Jin Xu, Zhibo Yang. Mingkun Yang, Jianxin Yang, An Yang, Bowen Yu, Fei Zhang, Hang Zhang, Xi Zhang, Bo Zheng, Hu- men Zhong, Jingren Zhou, Fan Zhou, Jing Zhou, Yuanzhi Zhu, and Ke Zhu. Qwen3-vl technical report. arXiv preprint arXiv: 1,,1 2025a.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint vxXi:2550.11922) 2025b.
Daniel Bolya, Po- Yao Huang, Peize Sun, Jang Hyun Cho, Andrea Madotto, Chen Wei, Tengyu Ma, Jiale Zhi, Jathushan Rajasegaran, Hanoona Rasheed, al. Perception encoder: The best visual embeddings are not at the output of the network. arXiv preprint arXiv:2504. 13181, 2025.
Yukang Chen, Fuzhao Xue, Dacheng Li, Qinghao Hu, Ligeng Zhu, Xiuyu Li, Yunhao Fang, Haotian Tang, Shang Yang, Zhijian Liu, et al. Longvila: Scaling long-context visual language models for long videos. arXiv preprint arXiv:2408. 2000:::08.0...8888.... 2024.
Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Video-mme: The first-ever comprehensive evalu- ation benchmark of multi-modal Ilms video analysis. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 24 20108-22 11812411811 2025.
Hong Gao, Yiming Bao, Xuezhen Tu, Bin Zhong, and Minling Zhang. Apvr: Hour-level long video understanding with adaptive pivot visual information retrieval. arXiv preprint N.Xx::2500.04953, 2025.
Daya Guo, Dejian Yang, Haowei Zhang. Junxiao Song. Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501. 12948, 2025.
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276 2024.
Xinhao Li, Yi Wang, Jiashuo Yu, Xiangyu Zeng. Yuhan Zhu, Haian Huang, Jianfei Gao, Kunchang Li, Yinan He, Chenting Wang, et al. sideochat-flash, Hierarchical compression for long-context video modeling. arXiv preprint arXiv:2501. 0.05544, 2024.
Shuming Liu, Chen Zhao, Tianqi Xu, and Bernard Ghanem. Bolt: Boost large vision-language model without training for long-form video understanding. Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 3318-3327, 2025a.
Zhijian Liu, Ligeng Zhu, Baifeng Shi, Zhuoyang Zhang, Yuming Lou, Shang Yang, Haocheng Xi, Shiyi Cao, Yuxian Gu, Dacheng Li, et al. Nvila: Efficient frontier visual language models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 1112--1134 2025b.

---

Published as a conference paper at ICLR 2026
Huaishao Luo, Lei Ji, Ming Zhong, Yang Chen, Wen Lei, Nan Duan, and Tianrui Li. Clip4clip: An empirical study of clip for end to end video clip retrieval and captioning. Neurocomputing, 508: 293-304, 2022.
Yongdong Luo, Xiawu Zheng, Xiao Yang. Guilin Li, Haojia Lin, Jinfa Huang, Jiayi Ji, Fei Chao, Jiebo Luo, and Rongrong Ji. Video-rag: Visually-alignes retrieval- -aggeente long video com- prehension. arXiv preprint arXiv:2411 2.00033,, 2024.
Yongdong Luo, Wang Chen, Xiawu Zheng, Weizhong Huang, Shukang Yin. Haojia Lin, Chaoyou Fu, Jinfa Huang, Jiayi Ji, Jiebo Luo, et al. Quota: Query-oriente token assignment via cot query decouple for long video comprehension. arXiv preprint arXiv:2503.0860 2025.
Wentao Ma, Weiming Ren, Yiming Jia, Zhuofeng Li, Ping Nie, Ge Zhang, and Wenhu Chen. Videoeval-pro: Robust and realistic long video understanding evaluation. arXiv preprint arXiv:2505. 14640, 2025a.
Ziyu Ma. Chenhui Gou, Hengcan Shi, Bin Sun, Shutao Li, Hamid Rezatofighi, and Jianfei Cai. Drvideo: Document retrieval based long video understanding. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 18936-189447 2025b.
Stuart Mitchell, Michael OSullivan, and lain Dunning. Pulp: a linear programming toolkit for python. The University of Auckland, Auckland, New Zealand, 65:25, 2011.
Ziqi Pang and Yu- Xu-Xiong Wang. Mr. video:" mapreduce" is the principle for long video understand- ing. arXiv preprint (rXx::2504.16055 2025.
Jongwoo Park, Kanchana Ranasinghe, Kumara Kahatapitiya, Wonjcong Ryu, Donghyun Kim, and Michael S Ryoo. Too many frames, not all useful: Efficient strategies for long- long-form video qa. arXiv preprint S4V1:22006009996, 2024.
Minghao Qin, Xiangrui Liu, Zhengyang Liang, Yan Shu, Huaying Yuan, Juenjie Zhou, Shitao Xiao, Bo Zhao, and Zheng Liu. Video-xl-2: Towards very long-videe understanding through task-aware kv sparsification. arXiv preprint arXiv:2506. /9225, 2025.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp- 8748 8763.773..00 PmLR, 2021.
Xiaoqian Shen, Yunyang Xiong, Changsheng Zhao, Lemeng Wu, Jun Chen, Chenchen Zhu, Zechun Liu, Fanyi Xiao, Balakrishnan Varadarajan, Florian Bordes, al. Longvu: Spatiotemporal adap- tive compression for long video-language understanding. arXiv preprint 2rX1v:2410.17434 2024.
Yunhang Shen, Chaoyou Fu, Shaoqi Dong, Xiong Wang, Peixian Chen, Mengdan Zhang, Haoyu Cao, Ke Li, Xiawu Zheng, Yan Zhang, et al. Long-vita: Scaling large modti models million tokens with leading short-context accuray. arXiv preprint 2502v:2502.05177. 2025.
Xi Tang, Jihao Qiu, Lingxi Xie, Yunjie Tian, Jianbin Jiao, and Qixiang Ye. Adaptive keyframe sam- pling for long video understanding. In Proceedings of the Computer Vision and Pattern Recogni- tion Conference, pp. 291 18-29188. 2025.
Peng Wang. Shuai Bai, Sinan Tan, Shijie Wang. Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang. Wenbin Ge, et al. Qwen2-vl: Enhancing rision-languse model's 's perception of the world at any resolution. arXiv preprint arXiv:2409. 12191, 2024a.
Weihan Wang, Zehai He, Wenyi Hong. Yean Cheng, Xiaohan Zhang, Ji Qi, Xiaotao Gu, Shiyu Huang, Bin Xu, Yuxiao Dong, al. Lvbench: An extreme long video understanding benchmark. arXiv preprint 20Xiv:2406.08033, 08035, 2024b.
Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei. Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, et al. Internv13.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv preprint 2508. / 18265, 2025a.
12

---

Published as a conference paper at ICLR 2026
Xiao Wang, Qingyi Si, Jianlong Wu, Shiyu Zhu, Li Cao, and Liqiang Nie. Adaretake: Adap- tive redundancy reduction to perceive longer for video-language understanding. arXiv preprint arXiv:2503.12559, 2025b. 20255..
Xiaodong Wang and Peixi Peng. Open-rl-video.
com/
Wanq www.iiandoongiooo waw.nngg1999PPP 89//0pen RI-RRiiieee 2025.
Xiaodong Wang, Jinfa Huang, Li Yuan, and Peixi Peng. Leanpo: Lean preference optimization for likelihood alignment in video-Ilms. arXiv preprint arXiv:2506.0526. 2025c.
Xiaodong Wang, Langling Huang. Zhirong Wu, Xu Zhao, Teng Xu, Xuhong Xia, and Peixi Peng- Livibench: An omnimodal benchmark for interactive livestream video understanding. arXiv preprint arXiv:2601.15016, 2026.
Xiaohan Wang, Yuhui Zhang, Orr Zohar, and Serena Yeung-Levy. Videoagent: Long-form video understanding with large language model as agent. In European Conference on Computer Vision, pp. 58-76. Springer, 2024c.
Yi Wang, Xinhao Li, Ziang Yan, Yinan He, Jiashuo Yu, Xiangyu Zeng, Chenting Wang, Changlian Ma, Haian Huang, Jianfei Gao, et al. Internvide02. 5: Empowering video milms with long and rich context modeling. arXiv preprint arXiv:2501.12386 2025d.
Haoning Wu, Dongxu Li, Bei Chen, and Junnan Li. Longvideobench: A benchmark for long-conter interleaved lideo-language understanding. Advances in Neural Information Processing Systems, 37:28828 28857, 2024.
Zeyu Xu, Junkang Zhang, Qiang Wang, and Yi Liu. E-vrag: Enhancing long video understanding with resource-efficient retrieval augmented generation. arXiv preprint arXiv:2508. 01546, 2025.
Huaying Yuan, Zheng Liu, Minghao Qin, Hongjin Qian, Yan Shu, Zhicheng Dou, Ji-Rong Wen, and Nicu Sebe. Memory-enhanced retrieval augmentation for long video understanding. arXiv preprint (Sti::5003.094433 2025.
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training In Proceedings of the IEEE/CVF international conference computer vision, pp. 11975-11986, 2023.
Boqiang Zhang, Kehan Li, Zesen Cheng, Zhiqiang Hu, Yuqian Yuan, Guanzheng Chen, Sicong Leng, Yuming Jiang, Hang Zhang, Xin Li, et al. Videollama 3: Frontier multimodal foundation models for image and video understanding. arXiv preprint arXiv:2501.13106 2025a.
Ce Zhang, Taixi Lu, Md Mohaiminul Islam, Ziyang Wang, Shoubin Yu, Mohit Bansal, and Gedas Bertasius. A simple IIm framework for long-range video question-answering. arXiv preprint 1722 2023.
Peiyuan Zhang. Kaichen Zhang, Bo Li, Guangtao Zeng, Jingkang Yang, Yuanhan Zhang, Ziyue Wang, Haoran Tan, Chunyuan Li, and Ziwei Liu. Long context transfer from language vision. arXiv preprint arXiv:2406. /6852, 2024a.
Xiaoyi Zhang, Zhaoyang Jia, Zongyu Guo, Jiahao Li, Bin Li, Houqiang Li, and Yan Lu. Deep video discovery: Agentic search with tool use for long-form video understanding. arXiv preprint arXiv:2505. 18079, 2025b.
Yuanhan Zhang, Jinming WU, Wei Li, Bo Li, Zejun Ma, Ziwei Liu, and Chunyuan Li. Video instruction tuning with synthetic data. arXiv preprint arXiv:2410.02713. 2024b.
Junjie Zhou, Yan Shu, Bo Zhao, Boya Wu, Zhengyang Liang. Shitao Xiao, Minghao Qin, Xi Yang. Yongping Xiong, Bo Zhang, et al. Mlvu: Benchmarking multi-task long video understanding. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 13691-13701, 2025.
Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Intermv13: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.10477 2025.
13

---

Published as a conference paper at ICLR 2026
Table Comparisons on widely used benchmarks. LongVB and VMME refer Long VideoBench and VideoMME,, respectively.
LVBench
MLVU
LongVB
VMME
VileoEval-Prro
Model
Average
Overall
M-Avg
Overall
Overall
Open
MCQ
LLaVA-Video-71
42.0
69.3
57.4
63.2
24.2
47.6
50.6
+ AKS (CLIP)
47.0
62.9
65.3
54.1
+ Ours-GO (CLIP)
49.1
71.5
60.2
64.1
29.9
54.8
54.9
+ AKS (PE)
46.4
69.1
60.7
64.1
30.0
52.9
53.9
+ Ours-Go (PE)
51.3
70.3
61.0
64.8
32.7
55.2
55.9
Qwen2.5-VL-77
45.5
69.4
61.0
66.4
27.7
46.6
52.8
+ AKS (CLIP)
45.0
65.9
58.6
63.2
25.4
46.2
50.7
+ Ours-GO (CLIP)
50.9
60.9
66.7
55.7
+ AKS (PE)
46.1
67.1
58.8
62.4
26.0
47.6
51.3
+ Ours-GO (PE)
52.7
72.3
63.4
66.7
35.0
56.9
57.8
A MORE QUANTITATIVE COMPARISON
A. EXTRA COMPARISON WITH AKS
To fairly compare with AKS (Tang et al. 2025) under same retrieval model, two additional experi- ments are conducted: (1) Firstly, we replace the vision-language model in AKS with PE-G/14 (Bolya al., 2025), used our method, while keeping all hyperparameters identical those the orig- inal AKS. This variant is denoted "AKS (PE)". We compare it with the original AKS and "Ours- GO (PE)" (our method with PE-G/14); (2) The CLIP-B/32 (Radford et al. 2021] is used our method ("Ours-Go (CLIP)"), which is consistent with the original AKS. We conduct experiments under both LLaVA-Video-27 Video-7B (Zhang et al. 2024b) and Qwen2.5-VL. VL-BBB (Bai et al. 2025b) back- bones. Since the AKS does not support Qwen2.5-VL-77 we implement it with 768 frames to align with our method, while keeping all other hyperparameters unchanged. The results are presented in Table
As shown in Table when paired with LLaVA-Video-Te AKS (PE) exhibits inferior performance compared the original AKS using CLIP-B/32. Even though AKS (PE) outperforms AKS in both fall short of the baseline Qwen2.5-VL-7B. Since AKS
combination with Qwen2.5-VL doesn't 't provide source code on PE-G/14 and Qwen2.5-VL-7B. these comparisons are not rigorous and just for reference. It indicates that the native employment of better retrieval model and backbone could not improve performances directly, which demonstrates our advantages.
Except for the generalization of hyperparameters, the reason may come from the method mecha- nism. AKS aims at maintaining temporal coverage across the video. This design is beneficial for relatively short videos (e.g.. in Long VideeBBneeecc (Wu et all] (024) and VideoMME (Fu et al.) 2025)). becomes less effective-ane theoretically less meaningful for hour-long videos, where the sheer duration and sparse distribution of critical events make coverage less important than precise, query- focused retrieval. In such scenarios, overemphasis on coverage may dilute the selection of highly relevant segments, ultimately harming performance. In contrast, full-video coverage and instead focus on retrieval accuracy, leading to superior performance espe- cially longer video benchmarks (e.g., LVBench (Wang al.] 2024b) and VideoEval-Pro (Ma et al. 2025a)).
approach does not prioritize
A.2 OTHER TRAINING-FREE METHODS
More training-free methods are listed in Table 8 MRVideo (Pang & Wang 2025) DeepDiscov- ery (Zhang et al. 2025b). Men Vid Yuuan et al. 2025). QuoTA (Luo et al. 2025). E-VRAG (Xu al. 2025) and APVR (Go et al. 2025). However, due the of different (and in some cases unspecified) base models, as well as missing evaluation results on certain benchmarks, a direct com- parison is not feasible; thus, the results are provided for reference only.
14

---

Published as a conference paper at ICLR 2026
Table 8: More training-free free methods. VideoMME, respectively.
LongVB and
VMME refer Long and
LVBench
MLVU
Long
VMME
Method
Used Models
M-Avg
Overall
Overall
Overall
MRVideo
Gemini-2.0- Flash/GPT40
60.8
DeepDiscovery
GPT-4.1+0penA1
74.2
MemVid
44.4
58.1
63.7
QuoTA
Qven2-VV-2B++LLLAAAWveee-7
71.9
65.9
E-VRAG
?
70.2
63.1
65.4
APVR
69.4
68.4
Ours-GO
PercppiionEicceeervvveee25551...
52.7
72.3
63.4
66.7
Ours-PO
55.5
74.1
63.3
67.9
Table 9: Detailed results on Video
Method
LP
LR
HP
HR
Overall
LLaVA-7B
28.5
13.6
20.7
19.3
24.2
+Ours-GO
40.8
19.7
20.7
22.3
32.7
Qwen-7B
33,9
15.6
24.8
17.8
27.7
+Ours-Go
43.3
18.4
30.6
22.3
35.0
LLaVA-72B
31.3
17.7
24.8
19.3
26.7
+Ours-Go
39.2
23.8
20.7
26.5
33.1
Qwen-72B
35.0
22.4
25.6
21.6
29.9
+Our--GO
44.5
17.7
35.5
24.2
36.5
B DETAILED RESULTS ON VIDEOEVAL-PRO AND LVBENCH
B. ADDITIONAL RESULTS ON VIDEOEVAL-PRO
We additionally report accuracy of 4 sub-tasks both Open and MCQ subset: (1) Local Perception
(LP): Identify and retrieve visual elements or actions within short clips from long videos, covering object, action, attribute, and entity recognition, well as segment and needle-in-a-haystacc QA. (2) Local Reasoning (LR): Perform reasoning over short event sequences, including temporal, causal, and object-action relationships within localized time windows.
(3) Holistic Perception (HP): Un- derstand global visual patterns through aggregated spatial or structural information, primarily in- volving visual counting. (4) Holistic Reasoning (HR): Achieve high-level understanding of long videos by reasoning about events, narrative structure, and underlying intent.
As shown in Table, and Table, 10 Ours-GO achieves relatively higher accuracy gains on the local perception task. This improvement can be attributed to its retrieval- architecture, which explicitly focuses on retrieving video segments most relevant to the question, thereby enhancing the model's 's sensitivity to local visual details.
Results in Table. show that Ours-PO achieves greater performance improvements on holistic reasoning task compared to Ours-GO. This is primarily because the option generation process in our framework is guided by local, question-relevant segments, which makes it challenging to in- fer answers that depend global video semantics require complex, multi-step reasoning long-range dependencies. However, since Ours-PO leverages provided options which ensure the inclusion of the correct answer-ii mitigates this limitation and improving performance holistic reasoning task.
B.2 ADDITIONAL RESULTS ON LVBENCH
We additionally report accuracy on six sub-tasks: (1) Entity Recognition (ER): Identify and track entities, their relations, actions, and associations over time. Event Understanding (EU): Rec-
15

---

Published as a conference paper at ICLR 2026
Table 10: Detailed results on VideoEvalPro-MCQ
Method
LP
LR
HP
HR
Overall
LLaVA-7B
53.2
46.9
39.7
35.2
47.6
+Ours-GO
63.4
$1.7
34.7
43.2
55.2
+Ours-PO
63.4
55.1
34.7
49.2
56,9
Qwen-7B
50.9
49.0
33,9
39.0
46.6
+Ours-GO
52.4
43.8
47.3
56,9
+Ours-PO
63.4
50.3
42.1
52.3
57.6
LLaVA-72B
57.1
32.2
41.7
50.1
+Ours-GO
63.4
62.6
33.1
53.0
58.3
+Ours-PO
64.5
61.9
35.5
58.0
60.1
Qwen-72B
60.5
59.9
38.0
48.9
55.9
+Ours-GO
67.8
59.2
47.9
53,0
61.9
+Ours-PO
70.1
60.5
46.3
57.2
64.2
Table 11: Detailed results on LVBench
Method
ER
EU
KIR
TG
Rea
Sum
Overall
LLVVA-Vides Vidco-7B
43.6
40.0
39.5
34.5
46.8
32,8
42.0
+Ours-GO
54.8
49.1
55.7
35.0
44.8
32.8
51.3
+Ours-PO
57.8
51.3
59.5
47.3
48.8
36.2
54.2
Qwen2.5-VL-7B
44.3
43.9
50.5
40.9
50.2
36.2
45.5
+Ours-GO
53.5
49.8
62.2
41.8
54.2
36.2
52.7
+Ours-PO
57.3
52.9
62.5
41.4
53.2
34.5
55.5
44.6
45.0
48.8
39.5
50.7
37.9
46.1
+Ours-GO
55.5
49.0
58.4
37.7
49.3
32.8
51.7
+Ours-PO
59.2
52.7
60.5
45.5
52.2
34.5
54.8
Qwen2.5-VL-72B
49.2
49.1
54.0
36.8
56.7
34.5
49.6
+Ours-GO
55.4
49.9
63.6
37.3
54.7
41.4
54.0
+Ours-PO
58.6
54.1
65.6
42.7
55.7
31.0
56.9
ognize video-level semantics including genre, events, and scene changes. (3) Key Information Retrieval (KIR): Extract precise factual details, such as on-screen text. (4) Temporal Grounding (TG): Locate and describe events at specific timestamps. (5) Reasoning (Rea): Perform causal, emotional, intentional, and prospective reasoning about video content. (6) Summarization (Sum): Generate abstractive summaries capturing the full video narrative.
Consistent with the findings on VideoEval-Proo the results in Table II demonstrate that Ours-GO achieves notable improvements on tasks that primarily require understanding of local visual con- such as entity recognition, event understanding, and key information retrieval. On more chal- lenging, holistic tasks that demand comprehensive understanding and synthesis-such reasoning and summarization, Ours-PO outperforms Ours-GO.
C QUALITATIVE COMPARISON
We provide several videos in the Supplementary Materials to visually compare our method with the baseline using the Sove2.55VLL-75 backbone. The top of each video displays the given question and options. Below that, on the left side are the frames sampled by our method, and on the right side are the frames uniformly sampled by the baseline method. At the bottom, the similarity scores over video time in method are visualized, such as Fig 4 Specifically, the horizontal axis denotes the video timeline (in seconds), where the blue curve represents our video-query-option similarity, the orange area indicates sampled frames (with darker shading corresponding to higher sampling density). The green line depicts Limilarity-basee resolution allocation, while the red line represents uniform sampling resolution. The green scanline corresponds method, and the red scanline to
16

---

Published as a conference paper at ICLR 2026
Figure 4: The horizontal axis denotes the video timeline (in seconds), where the blue curve repre- sents video-query-opiicn similarity, the orange area indicates sampled frames (with darker shading corresponding to higher sampling density). The green line depicts similarity- resolution allo- cation, while the red line represents uniform sampling resolution.
Table 12: Benchmarks infomation
Benchmark
Videos
QAs
Avg Duration
LVBench
103
1549
67.3 min
VideoEvalPro
465
1289
38.2 min
VidcoMME
900
2700
17.0 min
MLVU
1122
2174
12.6 min
Long VideoBench
735
1337
7.9 min
uniform sampling. Along the right side of each scanline, the associated frame resolution displayed. We provide two video examples:
(1) In the video "01262.mp4", to answer "What's the weakness of the villain?", our method-unlin uniform sampling-focusee sampling on the critical sequence where Tom and Jerry fetch water to confront the villain, guided by vidco-query-option similarity scores. At 2:42, the woman defeats the villain with bucket of water, clearly revealing his weakness-water. at this moment, the similar- ity score peaks, sampling is densest, and resolution is highest (560x308), enabling Qwen2.5-VL-7B to correctly answer the question. Conversely, uniform sampling assigns all frames small resolu- tion (252x 140) and includes many frames that are irrelevant to the question. As a result, uniform sampling approach leads an incorrect answer.
(2) In the video 201254.mp4". when tasked with answering the question "How often do the people take water breaks?", our method strategically samples five key moments of people taking water breaks at a high resolution. At each of these moments, the time is clearly visible at the bottom of the screen. Leveraging this visible time information, the Qwen2.5-VL-77 can accurately infer that people take water breaks every 5 minutes. On the contrary, due to the relatively short duration of these five moments, uniform sampling fails to capture all of them. Additionally, because uniform sampling uses a low resolution, the Qwen2.5-Ve 5-VL-7B model is unable to recognize the time displayed on the screen. These factors ultimately lead an incorrect answer when using uniform sampling.
D MORE EVALUTATION DETAILS
D. BENCHMARKS DETAILS
we present the number of videos, the number of question answer (QA) pairs, and the average video duration across the five long video benchmarks we utilized. Additionally, in Fig. 5] illustrate the duration distribution of these five benchmarks. These demonstrate that LVBench and VideoEval-Pro contain a higher proportion of long videos compared to the other benchmarks. This characteristic enables method to achieve better performance these two benchmarks.
In Table 12
17

---

Published as a conference paper at ICLR 2026
(1002000
27
Weddo
VileV
MIAT
Figure 5: Benchmark duration distribution.
D.2 EVALUATION PROMPTS
We provide the prompts used in our evalutaion on 5 long video benchmarks LVBench
(question)
(A) (optionA]
(B) (option8)
Answer with the option's 's letter from the given choices directly.
VideoEvalPro-MCQ
Select the best answer to the following multiple-choice question based the video. Respond with only the letter (A, C, D) of the correct option, with no text around (question) A. (optienA) B. (optionB) ......
VideoEvvlProoOOen
[question)
Keep the answer short and concise.
VideoMME
Select the best on the video and the subtitles. Respond with only the letter (A, B, C, or D) of the correct option. lquestion]
the following multiple-choice question based
A. {optionA)
B. (optionB)
Answer with the option's letter from the given choices directly.
MLVU
Carefully watch this video and pay attenticn to every detail. Based your observations, select the best optien that accurately addresses the question.
Question: (questien)
Options:
A. (optionA)
B. (option8)
Only give the best option, Best Option:
18

---

Published as a conference paper at ICLR 2026
Long BongVideoBench
(questian)
A. (optionA)
В. (optionB)
Answer with the option's letter from the given choices directly.
DETAILS OF OPTIONS GENERATION
E.1 QUALITY METRICS DEFINITION
Formally, let denote the total number of evaluation questions. For the i-th question, let Qi be the question text, A, the ground-truth answer, and O, the set of generated options. We define SCOC (Semantic Correct Option Count) as the number of options in O, that are judged to be semantically equivalent to A,, where semantic equivalence is determined via GPT-based judgment. Specifically, we employ simple prompt to instruct the large language model gpt-4. 1-2025-044-4 4 to count the number of semantically correct options for given question, along with its corresponding answer and candidate options. The prompt used is as follows:
You are intelligent chatbot designed to evaluate the correctness of given options.
You will be provided with question, reference answer, and options.
Without relying on external knowledge, determine whether the correct answer is included in the provided optiens, and count the namber of correct answers in the options.
Question: (questien)
Reference Answer: (target.)
Options: (gen_opts)
Please directly return the number of correct options, without any additional text. If no option is correct, return D.
The metrics OCA and MPOC are then computed as follows:
OCA Σ Lscoc. > 0).
(10)
|SCO,, I|scoc, > (ooc.,,))) >a).
MPOC
(11)
En I|SCOC > 0)
10:1
where [[:] denotes the indicator function (equals if the condition is true and 0 otherwise).
E.2 DIFFERENT WAYS TO GENERATE OPTIONS
We compared options generated with and without using the video, and we also evaluated the im- pact of using stronger models (Seed1.5VL and GPT-5) for option generation (Table 13p. All periments were evaluated using the LLaVA-Video:. model on the VideoEvalPro-MCC. dataset.
Table 13: Different Ways to Generate Options
The results demonstrate that incor- porating video context during op- tion generation consistently enhances performance. In contrast, with- out video information, even powerful models like GPT-5 and Beed1.5VI yield only marginal gains over the baseline. Notably, when video con- text is used with stronger model
Method
Option Model
With Video
Accuracy
Base
47.6
Ours-PO
56,9
Ours-GO
-LaVA-Video-11
Yes
55.3
Ours-GO
LLaVA-Video-77
No
53.9
Ours-GO
Seed1.5VL
Yes
56.2
Ours-GO
ScL
No
54.9
Ours-GO
No
54.8
19

---

Published as a conference paper at ICLR 2026
such Seed1.5VL, performance im- proves substantially, approaching the accuracy of methods that pro- vided with options.
E.3 CROSS- MODEL OPTION GENERATION
Cross-model option generation yield similar even improved performance. We evalute
different MLLMs to generate options on VideoEvvlPPo-MCC benchmark:
The results in Table 14 suggest that our method is not overly dependent on the alignment between the option- generation and answering models, and cross-model option generation can still deliver strong, consistent im- provements.
Table 14: Cross-model Option Generation
Answering Model
Option Model
Accuracy %
LLaVA-Video-71
47.6
LIAVA-Video-7BB
ILaVA-Video-7B
55.3
LLaVA-Video-17
Qwen2.5-VIL-7B
56.4
Qwen2.5-VL-7B
46.6
Qwen2.5-VL-7B
-LaVA-Video-7B
56.6
F INCORPORATING GLOBAL INFORMATION
Qwen2.5-VL-7B
Qwen2.5-VL-7B
56.9
acknowledged that the proposed retrieval-base based approach may poten- tially overlook certain global contex-
tual cues inherent in the full video. To mitigate this limitation, we conducted extensive experiments explore optimal strategies for integrating global context. Specifically, we investigated two existing strategies: "Retrieval By Global Summary Embedding" and "Context Fusion of Global Summary." Furthermore, we designed a novel approach, "Insert Global Frames," to facilitate the more effective incorporation of global information.
F.I RETRIEVAL BY GLOBAL SUMMARY EMEBDDING
We first generate a global textual summary of the entire video by uniformly sampling frames and prompting the MLLM with: "Please describe the video in detail." We then integrate this summary into the retrieval process in three ways:
(1) APPEND: The global summary is appended
additional "statement" the set in Equation 3 before computing the VQOS score (Equation 4). (2) CONCAT: The global summary is concatenated each individual statement in Equation 3 before comput-
ing the VQOS score (Equation 4). (3) MIX: The global summary encoded into an embedding and weighted-addittvel fused with the original text embeddings in Equation 4.
We evaluated these strategies using LLaVA-
Table 15: Results with global summary emebd- ding
Video-7B on the VideoEvalPro-MMQ bench- mark, the results Table [5 show that: All three strategies resulted in performance degra- dation compared our original method. No- tably, the weight X increases in the MIX variant, accuracy further declines. This sug- gests that the global textual summary often in- troduces irrelevant or misleading information that interferes with the relevance scoring pro- cess particularly since the summary is generic and not quest.
Method
Weight Accuracy %)
Ours-PO
56.9
APPEND
46.0
CONCAT
52.1
MIX
0.1
56,3
MIX
0.3
55.2
MIX
0.5
51.4
20

---

Published as a conference paper at ICLR 2026
Table 16: Context fusion of global summary
Method
Accuracy %) w/o global
Accuracy($0) 96) w/ global
Baseline
47.6
43.6
Ours-PO 56.9
52.0
F.2 CONTEXT FUSION OF GLOBAL SUMMARY
We also tested directly fusing the global summary into the input context by concatenating with the original question (e.g., "[Question] [Global summaryJ"). Results using LLaVA-Video-71 on VideoEvalPro-MCQ:
The results in Table [6 show that incorporating the generated global description consistently led to performance drop in both settings. This indicates that the generated summary may introduce noisy or redundant information that interferes with downstream reasoning, rather than providing useful global context. One plausible explanation is that textual summaries inherently lossy representations of the rich, multimodal in- formation present in videos. Consequently, directly adding global textual summary may be less effective than enhancing visual coverage through frame selection.
Table 17: Insert Global Frames
T
Accuracy( %))
0
56,9
4
56.9
8
56.8
16
57.4
32
55.9
F.3 INSERT GLOBAL FRAMES
Motivated by these findings, we propose visual-centric alternative: instead of relying on textual summaries, we enhance temporal coverage by explicitly inserting representative frames from under- represented segments of the video.
Specifically, after applying our original frame sampling method, we divide the long video into T uniform temporal segments. For any segment that contains sampled frames, we add the frame at its temporal center. To maintain a fixed total number of frames (ensuring a fair comparison), we simultancously remove the frame with the lowest VQOS score.
We evaluated this refined sampling strategy on LLaVA-Video-DD with varying values of T. The re- sults in Table 17 show that moderate temporal segmentation (e.g., T=16 yields slight but consis- tent performance gain, suggesting that strategically augmenting underrepresented temporal regions with representative frames can better capture global dynamics than relying on textual summaries. However, excessive segmentation (e.g., T=32) appears to disrupt the original sampling distribution, leading to a performance decline.
G RUNTIME AND ACCELERATION
G.1 ACCURACY AND RUNTIME TRADE-OFF
The main computational overhead our method stems from computing the VQOS, which involves sampling frames from long videos at specified FPS and extracting their embeddings. As discussed Sec 4.4 this cost can be effectively reduced under limited computational budgets by either using smaller VTR model, lowering the sampling FPS, or both.
To quantify this trade-off, we evaluate accuracy on LVBench and measure total runtime on the same single GPU setup, comparing approach against baselines well as other training-free free methods: AKS and AdaReTake.
The results in Table 18 show that method achieves favorable balance between accuracy and ef- ficiency. Even with a lightweight VTR model (e.g., PE-L/14) at a low sampling rate (0.5 FPS) yields significant accuracy gains over other methods while incurring only modest runtime overhead. In trast, higher-fidelity configurations (e.g.. PE-G/14 at 1.0 FPS) deliver peak computation.
55.5% accuracy with Qwen2.5-VI.-7B but require substantially
21

---

Published as a conference paper at ICLR 2026
Table 18: Accuracy and runtime trade-off
Model
Method
VTR-Model
Retrieval FPS
Accuracy(%)
Runtime(hour)
42.0
2.4
AKS Take
1.0
47.0
3.6
49.6
53,4
LLa
0.5
52.7
3.9
PE-L/14
Ours-PO
1.0
52.0
5.5
PE-G/14
1.0
54.2
29.0
45.5
3.6
AKS
1.0
45.0
4.8
AaaeeTTake
51.0
35.9
Qwen2.5-VL
0.5
54.5
5.2
PE-L/14
Ours-PO
1.0
55.2
6.7
PE-G/14
1.0
55.5
30.2
In practice, we can select the configuration that best aligns with their accuracy requirements and computational constraints. This flexibility makes our framework adaptable to diverse real-world deployment scenarios.
G.2 ACCELERATION WITHOUT EXTRA MODEL
Notably, both the VTR model and the MLLM's visual encoder perform similar operations: extract- ing visual tokens and (in VTR's case) pooling them into video embeddings. This raises an important opportunity for optimization: What if we eliminate redundant computation by reusing the visual to- kens already extracted by the MI.LM's visual encoder? Specifically, instead of having VTR model computes frame embeddings and then recomputing visual tokens by MLLM, the MLLM's native visual encoder to extract both the visual tokens and derive the pooled video segment embeddings. This would effectively eliminate time since token extraction and embed- ding aggregation would occur in a single unified pass. Therefore, we could revise Algorithm into Algorithm by eliminating the additional VTR model.
could directly use
For LLAVA-Video, the visual encoder fine-tuned from SigI.I Zhhii et al. 2023). a model origi- nally pretrained on large-scale image-text retrieval tasks making it inherently well-suited for use as a VTR model. Notably, Vi-iiooo removes SigLIP's original vision pooling head (approx- imately 15M parameters), tation, we reintroduce the pooling head by directly copying it from the original SigLIP checkpoint, without any additional fine-tuning or parameter updates. Additionally, we directly leverage SigLIP's pretrained text encoder to compute text embeddings. Since the text encoder of SigLIP is lightweight and each question embedding computed only once per query, the associated computational cost negligible. While one could alternatively use the MI.LM's native text encoder and attach trainable pooling head to generate text embeddings, this would require additional fine-tuning violating our design principle of training-frr free adaptation.
unnecessary within the MLLM's architecture. In our implemen-
The resulting model, which we refer to as "SigLIP-LLaVA", evaluated in Table 3 Remarkably, without task-specific adaptation of the pooling head, SigLIP-LLaVA achieves strong perfor- mance on LVBench - outperforming the baseline by approximately 10%, and falling only 2.2% behind our best pert-rerforming model. This finding further supports the availability of Algorithm 2 In contrast, for Qwen2.5-VL. the visual encoder does not include a native pooling head suitable for gen- erating video embeddings meaning that introducing such component would require additional training. Since the primary goal of this work is to explore training-free architectural optimizations, do not implement this extension in our current pipeline. However, this remains a compelling avenue for future research.
G.3 STREAMING LONG VIDEO UNDERSTANDING
Our method is primarily designed to enhance offline long video understanding capabilities without requiring any training, and as such, computational efficiency inference latency is not the main
22

---

Published as a conference paper at ICLR 2026
Algorithm 2 Our Method Without Extra Model
Require:
Video segments (V), question (g), option generation round (R), sampled frame num (N), multimodal large language model (MLLM), Adaptive frame sampling (AFS), Dynamic Resolu- tion Allocation (DRA).
Ensure: Answer the question according to the video.
1: segment embeddings 0
2: visual lokens cache t- 0
3: simiarities 0
4: ception t
5: g.embedding MLLM MLLMeexx.encoddr()))
6: for each V; E V do
7: visual tokens, biisull_emeeddin < MLLM MLLLMMiiuull....oomc.oom encooder V/))
8: segment hwwettteeeeeaappppeeeee
9: visual lokens httpeeeeett((iisullloooooe
10: end for
11: S + siniiessimiarittyg.eemmeee
12: V' TooK(V, S0 R N)
13: for t- to do
14: V, < ( VEW
15: sampled.framee
DRA(AFS(S)) V,))
16: visual tokens + tisuul. www.ae.ggaaccaaeaeeeeeeeeeeee set(ssaaeeeeefammees
options MILM(" 18: generated MILLM(Pleasse Please. generale
oplions...!' visual tokens)
19: end for
20: for each o generated_options do
o.embedding MLLM text encoder(g
22: S + cosine-simiarrtty( wwwpiiddiioommmdiiii..oo....g segment.embeddien
similarities +- max(simiarities, S) 24: end for
Viinal ( (V, simiarities, N)
26: sampled_frames +- DRA(AFS DRA(AFS((imiarilies (simiarilies, V(inat)
27: visual Lookens +- visual wwaheesssccchhgeee sramplee...rrm)))
28: answer <- MLLM(g, visual tokens)
29: return onswer
23

---

Published as a conference paper at ICLR 2026
focus. Nevertheless, our approach can be easily adapted for streaming long-vides understanding scenarios with minimal modifications.
As outlined in Algorithm 3] at certain sampling fps, the system checks whether the user has asked question. no question is detected, frame is sampled from the video stream. The VTR model is used to compute the frame embedding, which is then cached. Once T frames have been accumulated, the system computes a video segment embedding from these T frame embeddings and stores it. If a question is posed, the VTR model used encode the question into an embedding. This question embedding is then compared with historical video segment embeddings to compute similarity scores (VQOS). Then, AFS and DRA are applied to sample N relevant frames from the historical cache. These frames, along with the question, are fed into a Multimodal Large Language Model (MLLM) to generate an answer.
Algorithm 3 Streaming Video Processing with Our Method
Require: Video stream, sampling fps (fps), video segment length (T), multimodal large language model (MLLM), video-query-option similairity (VQOS), adaptive frame sampling (AFS), dy- namic resolution allocation (DRA).
Ensure: Answers to user questions on the streaming video.
1: visual Lommalltkeen_caere cachee +- 0
2: segment .enrentiembddiings + 0
frame embeds frame_emeeds buf
0
4: while video stream is active do
5: if video_time mod /ps == 0 then
6:
if user has question y then geembedding, teæt_tokene
7:
MLLMLiex.encoddrgg)
8:
similarities + sampled vishall tokens DRA(AFS( similarities, visual tokenss.acher
q.embedding)
9:
10:
answer + (taMM(tettttokemm sampled wisual .tokens)
11:
Print answer
12:
else
Jrame cample_framed (riamevvide__sreeee
14:
frame.embeddingg visual tokens + MLLM.iisuull.ooodeeffffrrrr visual tokens cache.append( vviuull token)
16:
frome embeds fer.ferppppdd ramme embedding) if len( beffebuufeer == then
18:
segment segment__mmeeddnng t segment hetneeettemmbedddigg.ppppddddede frame bu end
buf fer)
19:
20:
21:
22:
end if end if
23:
24:
end while
Notably, our method performs well under the configuration fps = 1 and T' 16, as demonstrated in Section 4 In streaming video scenarios, the time required for frame embedding extraction is negligible: even when using the largest video-text retrieval (VTR) model (PE-G/14)) the frame embedding computation takes only 15.5/60 & 0.26 seconds (Table 3 which is well below the inter-frame interval of I = second. Thus, embedding extraction introduces no bottleneck in 7ps
streaming video processing.
24


Reviews:
[{"review_id": "RcZ5ynZIkr", "reviewer": "Reviewer_9bhz", "summary": "The paper proposes three techniques to improve video prefilling without fine-tuning the MLLM. Inspired by human cognition, the framework first generates multiple candidate answers and uses their text embeddings (via a PE-G encoder) to retrieve relevant video clips. Given a computational budget, it then applies adaptive frame and spatial-resolution sampling to prefill the selected clips into the MLLM. Experiments on five benchmarks demonstrate consistent improvements over the baseline by a clear margin.", "questions": "It would be interesting to understand how the quality of the generated answer options affects final performance. For example, what happens if the options are produced by an MLLM without seeing the video (i.e., based purely on a textual question)? In addition, would using a stronger model (e.g., GPT-5) to generate the options lead to further gains?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. The idea of generating candidate answers to guide retrieval is intuitive yet underexplored. Using answer-level text embeddings instead of query embeddings aligns well with how humans reason backward from possible outcomes, and the paper provides clear experimental validation of its effectiveness.\n\n\n2. The ablations are well-structured and provide good insight into how each component contributes to the overall improvement.", "weaknesses": "1. The proposed two-pass pipeline introduces additional computation compared to single-pass or efficient prefilling methods. It would be helpful if the paper discussed the runtime overhead and its trade-off with accuracy.\n\n2. Since only the retrieved clips are passed to the MLLM, global contextual information might be lost. Have the authors considered combining local retrieval with a global summary embedding or a context fusion?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZsPwxiuElt", "reviewer": "Reviewer_1kjF", "summary": "This paper presents a training-free framework for long video understanding that integrates three components—Adaptive Frame Sampling (AFS), Dynamic Resolution Allocation (DRA), and Video-Query-Options Similarity (VQOS)—to efficiently manage the token budget by focusing computational resources on relevant video segments, reportedly achieving state-of-the-art results on several benchmarks.", "questions": "See Weaknesses", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The proposed framework is training-free and model-agnostic, making it a highly practical solution that can be readily applied to various off-the-shelf MLLMs (as demonstrated on LLaVA-Video and Qwen2.5-VL), enhancing its accessibility and potential for broad adoption.", "weaknesses": "1. I observed that the model's performance initially improves but then declines as the number of candidate options increases. Could the authors explain this non-monotonic behavior? Is this phenomenon related to the concept of pass@k?\n\n​2. Could segmenting a long video into multiple short clips risk scattering information relevant to a long-term temporal question across different segments? If so, might this lead to incorrect segment localization when answering questions that require integrating evidence from across the entire video timeline?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ysQ3cq3DQw", "reviewer": "Reviewer_AQQ4", "summary": "This paper presents a training-free framework aimed at improving long video understanding in MLLMs by integrating Adaptive Frame Sampling (AFS), Dynamic Resolution Allocation (DRA), and Video-Query-Options Similarity (VQOS). The approach leverages a video-text retrieval backbone to guide query-conditioned frame sampling and resolution strategies. VQOS uses the MLLM to generate candidate answer options and fuses them with queries to perform more targeted retrieval, akin to a hypothesis-verification cognitive process. The system is implemented on LLaVA-Video and Qwen2.5-VL (both 7B and 72B sizes) and tested across five mainstream long video benchmarks, showing measurable improvements over SOTA training-free and training-based methods. Extensive ablation and component analyses are provided.", "questions": "1. How do you ensure that this “option generation” does not leak answer information or overfit to the model’s own prior? \n2. Would cross-model option generation (e.g., generate options with LLaVA, use Qwen2.5-VL for retrieval) yield similar improvements?\n\nMore questions please see the Weaknesses.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "**1. Empirical Evidence:** \nTable 1 (Page 6) and the associated ablations (Table 2, Table 3, Table 4, Table 6) demonstrate consistent and sizable gains on several long video understanding tasks, especially on LVBench and VideoEval-Pro. Improvements are especially marked for models applied to hour-long videos. Ablations quantify the effect of each component (AFS, DRA, option generation), while Figure 3 and Figure 4 provide insight into the trade-offs and behavior of the system—e.g., the diminishing returns in OCA versus MPCO as more option rounds are added (Page 9), as well as the visualization of similarity-guided sampling over time across the video timeline (Figure 4, Page 15).\n\n**2. Generalization:**\n The model's robust generalization is further highlighted as the results show that method can be seamlessly integrated with various mainstream MLLM backbones (such as LLaVA-Video and Qwen2.5-VL), yielding consistent performance improvements across the board. This adaptability underscores its powerful generalization and transfer capabilities.", "weaknesses": "**1. Marginal originality:**\nThe core idea, using a retrieval model to select relevant video segments and adjusting sampling density, is essentially a combination of AKS (adaptive keyframe sampling) and AdaReTake (token-level redundancy reduction). The “Video-Query-Options Similarity” (VQOS) module is a minor twist: prompting the base MLLM to generate options and then fusing similarity across them. This is not a fundamental innovation but a straightforward heuristic that adds inference-time complexity.\n\n**2. Option generation bias:**\nThe method lets the same MLLM that will later answer the question generate “candidate options.” This risks information leakage and circular reasoning—if the model already encodes the answer distribution, the retrieval pipeline simply re-weights what the model already “believes.” The ablations (Table 2) show small gains from generated options (+0.7%, +0.4%), suggesting this stage adds little actual reasoning power.\n\n**3. VQOS dependence on retrieval quality.**\nPerformance gains (3–5%) largely stem from better retrieval models (PE-G/14). When using weaker CLIP backbones, the improvement vanishes. Thus, the method’s robustness across retrieval encoders or unseen domains is unproven.\n\n**4. Figures are cluttered:**\nFigure 2’s diagram is overly dense and still fails to clearly show the interaction between VQOS, AFS, and DRA. The legends mix fonts and arrows in ways that reduce readability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)