FOCUS: Efficient Keyframe Selection for Long Video Understanding
Authors: Zirui Zhu
Hailun Xu
Yang Luo
Yong Liu
Kanchan Sarkar
Zhenheng Yang
Yang You
Venue: ICLR 2026 Poster (2026)
Keywords: Keyframe Selection
Multimodal large language models
Long Video Understanding
Combinatorial Pure-exploration
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Multimodal large language models (MLLMs) represent images and video frames as visual tokens. Scaling from single images to hour-long videos, however, inflates the token budget far beyond practical limits. Popular pipelines therefore either uniformly subsample or apply keyframe selection with retrieval-style scoring using smaller vision-language models. However, these keyframe selection methods still rely on pre-filtering before selection to reduce the inference cost and can miss the most informative moments.

We propose FOCUS, Frame-Optimistic Confidence Upper-bound Selection, a training-free, model-agnostic keyframe selection module that selects query-relevant frames under a strict token budget. FOCUS formulates keyframe selection as a combinatorial pure-exploration (CPE) problem in multi-armed bandits: it treats short temporal clips as arms, and uses empirical means and Bernstein confidence radius to identify informative regions while preserving exploration of uncertain areas. The resulting two-stage exploration-exploitation procedure reduces from a sequential policy with theoretical guarantees, first identifying high-value temporal regions, then selecting top-scoring frames within each region. Extensive experiments across four long-video question-answering benchmarks and four popular MLLMs demonstrate that FOCUS delivers substantial accuracy improvements while processing less than 2% of video frames. For videos longer than 20 minutes, it achieves an 11.9% gain in accuracy on LongVideoBench, demonstrating its effectiveness as a keyframe selection method and providing a simple and general solution for scalable long-video understanding with MLLMs.

Published as a conference paper at ICLR 2026
FOCUS: EFFICIENT KEYFRAME SELECTION FOR LONG VIDEO UNDERSTANDING
Zirui Zhu 1:2 Hailun Xu² Yang Luo'
Yong Liu' Yang You
Kanchan 2 Zhenheng Yang 2
NNational University of Singapore "TikTok
Project Lead *Corresponding Author
(zirui, youy) @comp. nus mus.eddu.s.
(kanchan, sarkar, vangzhenheng)@@ikkeeee com
ABSTRACT
Multimodal large language models (MLLMs) represent images and video frames as visual tokens. Scaling from single images to hour-long videos, however, inflates the token budget far beyond practical limits. Popular pipelines therefore either uniformly subsample or apply keyframe selection with retrieval-style scoring using smaller language models. However, these keyframe selection methods still rely pre-filtering before selection to reduce the inference cost and can miss the most informative moments.
We propose Focus, Prame-Optinistte Confidence Upper-bound Selection, a training-free, model-agnostic keyframe selection module that selects query-relevant frames under a strict token budget. FOCUS formulates keyframe selection as com- binatorial pure-exploration (CPE) problem multi-armed bandits: it treats short temporal clips as arms, and uses empirical means and Bernstein confidence radius identify informative regions while preserving exploration of uncertain areas. The resulting two-stage pploration-exploiaatiin procedure reduces from a sequential policy with theoretical guarantees, first identifying high-value temporal regions, then selecting top-scoring frames within each region. Extensive experiments across four long-video question-answering benchmarks and four popular MLLMs demon- strate that FocUS delivers substantial accuracy improvements while processing less than 2% of video frames. For videos longer than 20 minutes, it achieves an 11.9% gain in accuracy Long VideoBench, demonstrating its effectiveness keyframe selection method and providing simple and general solution for scalable long-video understanding with MLLMS.
INTRODUCTION
"The art of being wise the art of knowing what to overlook." T William James
Recent advances large language models (LLMs) and multimodal large language models (MLLMs) have significantly improved visual understanding and reasoning. In current frameworks, images are encoded into visual tokens aligned with text and jointly processed by the L.LM. Extending this paradigm to videos especially long, untrimmed ones inssoouuuss key challenge: the sheer number of frames leads to an overwhelming number of visual tokens, making inference computationally prohibitive.
common solution is aggressive downsampling (Wang et al., 2022b; Lin al., 2023; Maaz et al., 2024; Zhang et al., 2025c), but uniformly sampling handful of frames (c.g., 64 from a one-hour video) often misses critical content (Tang et al., 2025; Zhang et al., 2025b). Increasing the frame rate, on the other hand, causes token explosion (Wang al., 2024c). This offe-off motivates the need for keyframe selection: choosing a small set of informative frames that preserve semantics while staying within token limits.
Recent methods address this by scoring frame relevance with pre-trained vision-language encoders (e.g., CLIP (Radford al., 2021) BLIP (Li al., 2022)) and then pick the highest-relevance

---

Published as a conference paper at ICLR 2026
frames (Tang al., 2025; Zhang et al., 2025b). These text-image matching approaches typically training-free and plug in easily before the visual encoder in MLLM stacks, retrieving frames with higher relevance other than uniform sampling. Despite their success, current keyframe selection methods still face scalability and efficiency limitations. For a one-hour video at 30 fps (over 10 frames), exhaustively scoring all frames entails the order of 1011-1012 FLOPPs with a vision- language encoder like BLIP (Li et al., 2022), This scaling pressure forces existing methods to uniformly sample the video lower frame rate before the scoring process. This pre-filtering process before keyframe selection undermines the goal of identifying most informative keyframes from all frames (Zhang et al., 2025b; Tang et al., 2025).
In this work, we propose FOCUS, Prame-Opting Con- fidence Upper-Bound Selection, a training-free plug- and-play keyframe selection method designed to pro- cess extremely long videos with minimal computational overhead. FOCUS is easy to implement in practice while offering an elegant theoretical foundation.
LongVideeeenhh (necke)
The key insight behind FOCUS is grounded in the ob- servation that natural videos exhibit strong temporal locality: adjacent frames highly correlated in ap- pearance and motion ( (Wiegand et al., 2003; Wang et al., 2016; 2022b). This local smoothness naturally extends to frame-query relevance scores.
0.2
Daayy (seconds)
Temporal autocorrelation (ACF) of per-frame query relevance on Long VideoBench and Video-MME. We compute frame-level relevance per video and take the ACF over time lags (seconds); solid lines show the median across videos and shaded bands the interquartile range. The dashed line marks the correlation half- life level (p(5) 0.5).
Figure 1:
Concretely, for each video-query pair compute frame-level relevance sequence }. where rr is the co- sine similarity between the visual embedding of frame t and the text embedding of the query produced by BLIP. We then measure temporal dependence via the autocorrelation function (ACF) p(o) corr(r;, r++&) lag 5 (in seconds), and aggregate p()) across videos. As illustrated in Figure 1, both Long VideoBench and Video-MME exhibit strong short-range correlation: the median ACF remains above 0.5 for roughly the first seconds.
This observation implies that exhaustive scoring of all frames is unnecessary. Instead, we can formulate keyframe selection as a bandit problem to adaptively allocate computation: quickly filtering out irrelevant temporal regions, concentrating scoring on promising segments, and ultimately prioritizing the most informative keyframes.
FOCUS first partitions the video into short temporal clips, each treated arm in multi-armed bandit. The clip selection is then framed as a Combinatorial Pure-Exploration (CPE) problem: the goal is to identify subset of arms that maximizes expected cumulative relevance under limited budget. Each arm maintains an empirical mean relevance and a Bernstein-stykke confidence radius. Computation is adaptively allocated to clips that are either promising (high mean) or uncertain (large confidence radius), following an triimism-i-thh--fccee -ft--uurraactiittt principle. This iterative process enjoys theoretical convergence guarantees. To leverage parallel computation, we reduce the iterative strategy to coarse-to-fine schedule: optimistic means guide exploration, while unbiased empirical means inform final arm selection. Within each selected arm, we extract the top-relevance frames to construct the final keyframe set.
We validate the effectiveness of our approach on two video understanding benchmarks, including Long VideoBench (Wu et al., 2024) and Video-MME (Fu et al., 2025). The proposed Focus is tested as an off-the-shelf module on with four popular MLLMs. FOCUS improves answer accuracy over state-of-the-an -fr-t---tttttttt keyframe selection baselines across benchmarks while maintaining lower inference cost. The gains are especially pronounced on long-form videos: for videos longer than 20 minutes on Bongo,ee,,,,,,, FOcus delivers 11.9% accuracy improvement while still cutting inference cost.
In summary, our main contributions are three-fold: (1) We formulate qwery-aware keyframe selection as budgeted combinatorial pure-exploration (CPE) problem in multi-armed bandit setting; (2) We introduce FOCUS, a training-free model-agnostic keyframe selection module that selects query-
2

---

Published as a conference paper at ICLR 2026
Text Query
Text Token
Multimodal Large language Model
Response
Frame- Cemmm-Opimmisic Confidence
Upper-bound Selection
Keyframes
Visual Token
Video
Video
Coanse
F
Fine
A
Exploration
Exploitation
P4
A
Am
Au
Optimistic
Unbiased
Selected
Selected Keyfhames
Clip-Level Selection
Multi-Armed Bandit
Frame Selection within Arms
Figure Overview of Focus. Focus partitions videos into fixed-length clips as bandit arms, applies optimistic confidence upper-bound arm selection and selects final keyframes within each promising arms.
relevant frames under strict token budget; (3) We validate the effectiveness of Focus on two long-video understanding benchmarks, achieving consistent gains across four popular MILMs.
METHOD
2.1
PROBLEM FORMULATION
Keyframe Selection Setup. Let a video be V = (æI, ET) and denote the corresponding text query as q. Let the frame index be subset of frames indexed by K S with [K| and produces an answer & (q. (a))e)). Let Rs (K V.g) denote the task-level utility of the selected frames (e.g., quality of generated answer, relevance to query, or other performance metrics).
(1....,T), A downstream MILM consumes
Oracle and Surrogate Objective.
The oracle objective chooses to maximize expected utility:
K (V.9)
arg EE E. V.q)].
(I)
KCT, =
Direct optimization to equation is infeasible due the combinatorial search space and the high cost of black-box evaluations of 4. We further expand the task-level utility R4 (K V. q) to summation of frame-level utility Yr 10:
arg max KCT, =k
E
Σ y1]-
tEK
However, estimating the contribution each frame the task-level utility also intractable. We therefore posit that Ye is indirectly observable via vision-language encoder &e that outputs relevance score r; = (0 qi 0) = yt + Ew, where Ey denotes encoder-induced noise. We assume es follows some distribution that are supported on [0, 1] and with zero mean and o variance. Under this assumption, the relevance score is unbiased estimator of y, which is also commonly used in many works (Tang et al., 2025; Yu et al., 2024) implicitly.
Exhaustively scoring all T frames to get {r:} } is computationally prohibitive, especially for hourly long videos which contains over 100 frames. This computational constraint motivates us to model keyframe selection under budget constraints, where we strategically allocate limited sampling budget to identify the most promising temporal segments before producing the final set keyframes. Instead of directly optimizing equation 2 the frame level, will approximate it through combinatorial pure-exploration multi-armed bandit formulation at the clip level, which significantly reduces exploration cost.

---

Published as a conference paper at ICLR 2026
Algorithm Iterative Optimistic Confidence Upper-bound Arm Selection
Require:
Maximization oracle TopM({1a) m) - A C A
1: Initialize: Empirical means Ao < 0 and No(a) t 0 for all
2: Pull each arm a E A for q times and observe the rewards.
3: mq and Na (n) q for all
4: Update empirical means f.a (n) for all a.
5: for mig, m.gg,,.+......
6: An + To
7: Compute confidence radius Bu(n) for all A for - to M do
D (n) defined in equation
8:
9:
if E An then
10:
[a(n) + (n) (n)
else
12:
(n) t (a) + (T)
end
14:
end for
15:
An + TopM(ā, m)
16:
if A, An then
return An
18:
end
Pn:
arg max (n) AE( (An )..))
break ties arbitrarily
20:
Pull arm Ps. and observe the reward
21: Update empirical means (Pp).) with the observed reward
22:
Np.. (n 1) +- Np. (n) +1
23:
end for
2.2
CLIP-LEVEL SELECTION AS MULTI-ARMED BANDIT
For video V (æI..... (21,..,, TT). we partition the timeline into M non-overlapping fixed-length clips == [Au)ar where each clip A, C 'T spans frames [s., ea] and is treated as a bandit arm. We define pulling arm a as uniformly sampling a frame E Ag and observing its query relevance score as the reward. The unseen frame-level utility of the sampled frame is modeled as yr ~ Vao where Va has mean μa and variance o²-
Intuitively, our goal is to focus on the most promising clips which means we have to identify the optimal subset 5* C A. Formally, we define the decision class 24 subset of the power set of A. The optimal member S* of decision class S is defined as
S arg max SEB
Σ Pa.
aES
Under the classic CPE framework, the learner's objective is to identify S* after interacting with the arms over sequence of rounds. In the keyframe selection setting, our final goal is to further select keyframes from the selected arms. Denote (k, | the number of keyframes allocated the a-th selected arm. We further define the frame-level optimal key frame subset IK; as
K
arg [K, kkk tEK.
Σ yt-
K.C...
The final keyframe subset K* is then defined K* Uaes- K. Empirically, assume the
decision class S is all size-m subsets of A and keyframes are equally distributed across the promising arms. This setting gives an elegant theoretical guarantee of regret bound as shown in section C and is also proved to be effective in experiments.
2.3 OPTIMISTIC CONFIDENCE UPPER-BOUNE ARM SELECTION
2.3.1 OPTIMAL. ARM SELECTION.
Generally, we play exploration game by pulling an arm a and observing the reward r, at each round We maintain two empirical statistics for each arm during this process: empirical mean
4

---

Published as a conference paper at ICLR 2026
Algorithm 2 FOCUS:
Opame-Optimistic Confidence Upper-bound Selection
Require: Maximization oracle TopM(1/a) m) + A C
1: Initialize: Empirical means Ao (a) <- 0 and Na + 0 for all
// Stage I: Coarse exploration
2: Pull each arm E A for q times and observe the rewards.
3: n +- mq and Na (n) f- q for a.
4: Update empirical means ji for all
5: Compute confidence radius (n) for all a E A
6: (n) Pag (n) + (n) for all A
7: Accarse & (p,
> Optimistic Means UCB
// Stage II: Fine- treeegg exploitation
8: Pull each arm E Aquarse for times and observe the rewards.
9: Update empirical means P. (n) for a E Accars
10: Asie <- TopM( m)
Unbiased Empirical Means
11: return Ane
(n) and an empirical Bernstein confidence radius (variance-adaptivv)) (a(n,, following the UCV-V style bound (Audibert et al., 2009):
26 lnn
n
(a(n) = V max (a)))
(5)
Here a² is the empirical variance of arm Na is the number of pulls for arm at round n and n = LocA (n) is the total number of pulls. The confidence radius ensures that the empirical mean is within the confidence radius of the true mean with high probability, i.e.
P (n) pal s [ 1 6
(6)
Please refer Appendix B for the detailed proof.
As shown in Algorithm 1, the optimistic confidence upper-bound arm selection starts with an initialization phase where pull each arm for times and observe the relevance scores rewards. We then update the empirical means P. and compute the confidence radius 3g (n) for each arm a. Note the relevance score r' an unbiased estimator of y/ so we have B[A.] = Pa. Then we choose the best m arms using the empirical means P.a (nt, i.e., A. = TopM m). where pa is the vector of all arms' empirical means and TopM[-, m) returns a set of the arms with the largest empirical means.
We further refine the arm selection by evaluating the "potential" of each arm. To be specific, for arm E An, we compute the lower confidence bound of the empirical mean, i.e. LCB.(() = (a(n)) (n); for arm a & A.,, we compute the upper confidence bound the empirical mean, iee. UCB. (n) (n) (n). If
max UCB.(n)
2 min LCB, (nt), oEA.
agAn
this indicates that some arms outside the current top-m set still potential to be included in the top-m set. Thus, we choose the arm a that we are most uncertain about, i.e..
arg max a aE(A,, VA. (A,, \..))
(n).
(8)
We then pull this arm for times and repeat the process until the top-m set is unchanged, i.e. An+1 = An. We then return the top-m set Ann.
It is easy to see Algorithm I is guaranteed to return the optimal top-m set A. with high probability (see Section C for the detailed proof). However, the iterative process is empirically inefficient (or intractable) the sequential arm-pulls and updating can not be parallelizable. We have pull the arms one-by-oreee by-ee which means forward the langune model with batch size sequentially. This costs significant waste of GPU utilization.
5

---

Published as a conference paper at ICLR 2026
2.3.2 TWO-STAGE ARM SELECTION.
To make the procedure practical and easy parallelize, we specialize Algorithm into the two-stage. batch variant in Algorithm 2. The overall framework is shown in Figure 2.
Stage Coarse initialization. We pull each arm q times parallel and update the empirical means Ha and confidence radii Sa (n) for all a E A. This stage coincides with the initialization phase of Algorithm and serves as a coarse exploration pass that produces reliable per-arm statistics low coordination cost.
Stage II: Fine-grained exploration (batched), Using the optimistic scores (a = (a(m)) + (n), we select the top am arms, Awuars TopM [i, cem), and allocate additional pulls to each a E Accure (performed in a single batch). Here, a is hyperparameter that controls the ratio of the coarse exploration budget the fine-grained exploration budget. This stage is batched counterpart of the iterative loop in Algorithm I: it implements the "optimism in the face of uncertainty" principle by concentrating samples on arms with the largest UCB values, while avoiding per-step scheduling overhead.
Final Arm Selection. After the fine exploitation, we form the final set by selecting the best m arms according to the unbiased empirical means, Aiie Tf, m). This choice mirrors 8-PAC identification routines, where optimistic scores guide exploration but the recommendation itself is based on the empirical means Pa (11) rather than the optimistic means (n).
2.4 FRAME SELECTION WITHIN SELECTED ARMS
Given the selected arm set Aine and a total budget of K frames, we sample hi frames per arm Agiin with equal allocation (i.e., Ka = round (iee)) adjusted to sum to K). For each arm a with index set T, and observed rewards {Ta.a at sampled indices Te C Ta. we simply interpolate all rewards fa.: within the arm using the nearest-neighbor assignment. We then form per-arm sampling distribution according to the interpolated rewards and draw Ka frames without replacement from Pa- The final keyframe set is AC UseAum Ka
EXPERIMENTS
3.1
EXPERIMENTAL. SETUP
Benchmarks We follow the LMMs-Eval framework Zhang et al. (2024a) and adopt the open-source evaluation protocol from AKS for benchmarks, prompts, and scoring. Our experiments focus on two long-video multiple-choice QA benchmarks: Long VideoBench Wu et al. (2024) and VideoMME Fu al. (2025). These datasets feature videos lasting up to an hour, where effective keyframe selection becomes crucial for performance. To ensure fair comparison (Tang al., 2025), we disable subtitles, perform zero-shot evaluation, and keep model parameters frozen- woozen-vayying only the frame selection strategy (our method versus uniform sampling). We also evaluate MLVU (Zhou et al., 2025) and VSI-Bench (Yang et al., 2025) to assess generalization; detailed results on MLVU and VSI- -SI-Bench are provided in Section F.
Implementation Details We test both open-source video MLLMs (Qwen2VL (Wang et al., 2024a), LLaVA-OV (Li et al., 2025), LLaVA-Video (Zhang et al., 2025c) and Qwen2-7B (Yang al., 2024) language model) and the commercial GPT-4o (0513). For frame relevance scoring, we use BLIP ITM (Li et al., 2022) to compute rt e' qi 9), where r; estimates the latent frame-level utility as described in Section 2. which is justified as promising choice by Tang et al. (2025). This also ensure fair comparison setting as the frame-level utility estimated using the same model.
3.2 PERFORMANCE ANALYSIS
We evaluate FOCUS by using it to select keyframes as the visual input for the four aforementioned MLLMs, and compare against the commonly used uniform sampling strategy. The results on Bede and Video-MME summarized in Table 1.

---

Published as a conference paper at ICLR 2026
Model
#Frame
LLM
LongVideoBench
Video-MME
GPT-4V
256
61.3
59.9
Gemini-1.5-Flan
256
61.6
70.3
Gemini-1.5-Pr
256
64.0
75.0
VideoLLaVA
7B
39.1
39.9
MiniCPM-V 2.6
64
8B
54.9
60.9
VL2-440
40B
59.7
61.2
64
72B
63.9
70,6
GPT-4o
32
51.6
61.8
GPT-4o w/ Ours
32
54.8 f 3.2
62.5 1 0.7
Qwen2-VL VL-7B
32
7B
55.6
57.4
Qwen2-VL w/ Ours
32
7B
62.3 1 6.7
59.7 1 2.3
LLaVA-OV-7B
32
7B
54.8
56.5
LLaVA-OV-7B w/ Ours
32
7B
60.7
1 5.9
58.3 1 1.8
LLaVA-Video-7B
64
7B
58.9
64.4
LLaVA-Video-7B w/ Ours
64
7B
63.5 1 4.6
65.4 1 1.0
Table Video-question answering accuracy (%) of various MLLMs on LongVideoBeni and Video- MME. FocUS is integrated into GPT-4o, Qwen2-VLL VL., LLaVA-OV, and ALaVA-Video, The suffix "w/ Ours" denotes models using keyframes selected by our method; otherwise, frames are uniformly sampled. #Frame indicates the number of frames provided to the MLLM, and LLM denotes the language model size. We also include performance of additional popular MLLMs for reference.
Improved Performance via Frame Selection. As shown in Table Focus consistently outperforms uniform sampling across both open-source and closed-source MLLMs on both LongVideoBench and Video-MME.
Specifically, on Long VideoBench, FOCUS improves accuracy by 3.2% on GPT-40, 6.7% on Qwen2- VL-7B, 5.9% VLaVA-OV-B,, and 4.6% on LLaVA- VLVVA-Vde---7B. On Video-MME, the gains 0.7%, 2.1%, 1.8%, and 1.0% on the same models, respectively.
We observe clear trend that larger MILMs with mance. However, FOCUS significantly narrows this gap by identifying the most informative frames, thereby boosting the performance of smaller MLLMs. For instance, Qwen2-VL-7B with FOCUS outperforms Gemini-1.5-PP on Long VideoBench, despite using 8x fewer input frames. This highlights the effectiveness of FOCUS as a plug-and-pla keyframe selection module for a wide range of MILMs.
frame inputs tend to achieve better perfor-
Interpretability through Visualizations. We visualize the frames selected by FOCUS alongside uniformly sampled frames for two examples from Long VideoBench and Video-MME in Figure 3.
Note that Vonggideeennch and Video-MME differ substantially in how their video-question pairs are constructed. general, Bong features more detailed and specific questions, while Video-MME focuses concise, high-level queries. Moreover, Long VideoBench tends to ask about specific scenes or events, whereas Video-MME emphasizes global understanding the video content.
To highlight this distinction, manually mark the most informative frames relative to the query using yellow stars. These frames are more temporally concentrated in Long VideoBench (around specific events) and more uniformly distributed across the timeline in Video-MME.
This difference helps explain why FOCUS achieves greater performance gains on Long VideoBench: method assumes that frame-level relevance scores are i..dd. common setting in multi-armed bandit formulations. This assumption neglects temporal dependencies between video segments. Consequently, retrieval-basee methods for keyframe selection typically require regularization (Tang et al., 2025; Yu et al., 2024) to promote diversity and ensure coverage.
If temporal dependencies between segments (arms) taken into account, the problem setting shifts toward Lipschitz or metric bandits (Kleinberg et al., 2008; Bubeck et al., 201 1), and contextual bandits (Chu et al., 201 1; Agarwal et al., 2014). We leave such extensions future work.
7

---

Published as a conference paper at ICLR 2026
In the zcene woman insnalin in fnfornt a cask erginer, and theee are bio other people fehina the counter The forr ie the somme are clearly suidle. cluck ciathes person
Question:
Question: What magic does the mugission firs perform on the suage?
The shr repider
weaving othiegroos
sights howel seiliy??
Answer: He cats poper the avater celetrity the juige is thinking of
Answer: The black haired woman in the olive greon jacker.
Answer Ours: He paper make the matar the celebrity the judge thinktng
Amswer Ours:
The woman the olive-grenn nemch cost.
Quentio: the video?
the following patterns appear
Question: lorg marbie sable, there piece baked food. person weiring ring using chopsticky to apply sance. What kind of ring this perjon wearing?
Answer: Pizza partors, dice, the United Nation ombion.
Answer: Diamond ring on the midile finger
Dice, pazza parfors, United
Annrer Owrs: Gold ring on the midiile finger.
Comparison between uniformly sampled frames and those selected by FOCUs. The left column shows two examples from Long BineeeBeB:,::; the right column shows two from Video-MME. Yellow stars indicate manually annotated frames that are most informative the query, many of which are successfully captured by FOcUs.
Figure 3:
3.3 COMPARISON WITH STATE-OF-THE-ARI
LongVideo VideoBench
Video-MME
Method
Short
Medium
Long
Overall
Short
Medium
Long
Overall
Uniform
67.5
57.4
51.8
58.9
76.4
62.6
54.3
64.4
Top-K
72.3
58.0
60.5
62.3
75.4
60.4
53.0
62.9
AKS
72.3
59.2
56.1
62.1
76.3
62.8
54.7
64.6
FOCUS (ours)
72.3
59.0
63.7
63.5
76.5
63.5
56.1
65.4
Table Comparison between our method and state-of-the-un keyframe selection baselines under matched keyframe count. Results are reported by video length buckets: Short, Medium, and Long. For Video-MME, we adopt its original categorization: Short (<2 min), Medium (4-15 14-5 min), and Long (30-60 min). For Long VideoBench, we define Short as videos shorter than 3 minutes, Medium as 3-20 minutes, and Long as over 20 minutes to ensure a balanced distribution.
To further validate the effectiveness of FOCUS, we compare it against state-of-the-art training-free keyframe selection methods both Long VideoBench and Vidco-MME. Specifically, consider recent approaches based on vision- Lasoon-aagguae similarity:
Top-K:
Computes relevance scores between each frame and the query, then selects the top-K scoring frames. Due computational constraints, we apply a pre-filtering step by downsampling videos to frame per second.
AKS (Tang et al., 2025): A recent method that adaptively balances frame relevance and temporal coverage. It considered the current fate-of-theeane and also incorporates pre-filtering via downsampling to frame per second (Tang et al., 2025).
We also compare against Q-Frame (Zhang al., 2025b), another recent training free method that uses resolution adaptation. Detailed comparisons across multiple MLLMs are provided in Section E, where FOCUS consistently outperforms both AKS and Q-Frame.
8

---

Published as a conference paper at ICLR 2026
Method
Filtering-free
Frames Seen (%)
GPU hours
AKS w/o pre-fitteing
X
100
255
AKS pre-filtering
X
3.7
9.3
FOCUS (Ours)
1.6
5.5
Table 3: Efficiency comparison of keyframe selection methods Long Vong..dbBen "Pre-filtering" refers to downsampling videos to 1 fps prior to selection. Note that the official AKS pipeline includes this pre-filterigg step by default. "Frames Seen (%)" counts the proportion of frame-level BLIP forward passes relative to scoring all frames; GPU hours are measured on a single H100 (80GB).
Fair comparison protocol. We ensure a fair comparison by: (i) evaluating all methods using LLa Video-7B, the best-performing MLLM in our setup; (ii) fixing the number of selected keyframes k 64: (iii) using the same vision- languaee encoder (e.g., BLIP) for frame scoring whenever possible. Results are summarized in Table 2.
Consistency across different lengths. FocUs achieves consistent performance gains across all video length categories, with particularly strong improvements on long videos. On Long VideoBench. FocUS outperforms uniform sampling by 11.9% and Top-K by 7.6% on videos longer than 20 minutes. On Video-MME, the respective improvements are 1.8% and 1.4%.
We also observe that on short videos, all keyframe selection methods perform similarly and con- sistently outperform uniform sampling. We attribute this to a possible saturation the reasoning capabilities of the underlying MLLM (LLaVA-Video- where input selection plays less critical role.
Efficiency comparison. We report the efficiency of each method in Table 3, measuring both the number of frames "seen" (i.e., scored by vision-language model) and the total GPU hours required to perform keyframe selection. All GPU hours are measured using a single NVIDIA H100 (80GB) GPU the Bong dataset.
As shown, AKS without pre-filtering is computationally infeasible in practice, as it requires scoring all video frames-amoune amounting to over 255 GPU hours by the optimistic estimation. With pre-filtering the cost drops significantly to 9.3 GPU hours. In contrast, FOCUS the most efficient: it requires only 1.6% of the BLIP forward passes and just 5.5 GPU hours, while simultaneously achieving the best overall performance.
Ablation Studies. We conduct comprehensive ablation studies to validate key design choices, including the two-stage ppplorationeexploitaaioon procedure, Bernstein confidence radius, clip length, and vision-language encoder selection. Results and detailed analysis are provided in Section G.
3.4 EFFICIENCY-ACCURAN TRADE-OFF
FocUS exposes natural trade-off between accuracy and computational cost through single hyperparameter a, which controls the fraction of arms selected for fine- ene-grained exploration. We report accuracy and efficiency under different a settings in Table 4.
Accuracy (%)
Frames Seen (%)
GPU hours
0.1
62.9
1.1
3.5
a 0.25
63.5
1.6
5.5
a 0.5
63.6
2.5
Table 4: Effect of a on the performance and efficiency of Focus. "Frames Seen (%)" counts the proportion of frame-level BLIP forward passes relative to scoring all frames; GPU hours are measured on single H100 (80GB).
We observe that choice of a has significant impact on the efficiency while remain stable on the performance. With 0.1, FOcUS evaluates 1.1% of frames and finishes in 3.5 GPU hours. At

---

Published as a conference paper at ICLR 2026
0.25, the fraction rises 1.6% with of 5.5 GPU hours, yielding 63.5% accuracy. Setting a = 0.5 achieves the highest accuracy (63.6%) but requires evaluating 2.5% of frames and 9.2 GPU hours-only -only negligible gain over 0.25 for substantially higher cost, indicating diminishing returns from exploring more arms.
4 CONCLUSION
We addressed the core bottleneck of long-video understanding in MLLMs-the explosion of visual tokens-by by introducing FocUs, training-free, plug-and-play keyframe selection method that allocates computation under strict budget. FOCUS first partitions the video into temporal clips, treats each as an arm in bandit problem, and then identifies query-relevant regions via combinatorial pure- exploration strategy using empirical means and Bernstein confidence bounds. To improve efficiency. we reduce the iterative bandit process coarse-to- to-fine two-stage procedure that preserves optimism while enabling parallel inference.
Experiments on two challenging long-video QA benchmarks demonstrate that FOCuS consistently improves accuracy across four MILMs while processing fewer than 2% of video frames. Our results show that lightweight, training-free keyframe selection seeeiono---hennn significantly enhance the scalability and practicality of MLLMs for long-video understanding.
-when guided by statistical principles can
5 REPRODUCIBILITY STATEMENT
We provide comprehensive theoretical analysis of our method in Appendix B and Appendix C. All models and datasets used in our study are publicly accessible.
REFERENCES
Alekh Agarwal, Daniel Hsu, Satyen Kale, John Langford, Lihong Li, and Robert Schapire. Taming the monster: A fast and simple algorithm for contextual bandits. In Eric P. Xing and Tony Jebara
(eds.), Proceedings of the 31 International Conference on Machine Learning, volume 32 of Proceedings of Machine Learning Research, pp. 1638-1646 Bejing, China, 22-24 Jun 2014. PMLR. URL https: ppwceedinceeggseeee mlr prwps//ee/eeee......e....eee setess/v2e v22 2322//gaawaabllll html.
Shipra Agrawal and Navin Goyal. Analysis of thompson sampling for the multi aulti-armed bandit problem. In Conference on learning theory, pp. 39-1. JMLR Workshop and Conference Proceedings, 2012.
Bean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lene, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: visual language model for sew-shot learning, Advances in neural information processing systems, 35:23716-2311 23736, 2022.
Eves Audibert and Sébastien Bubeck. Best arm identification in multi-armed bandits. In COLT-23th Conference on learning theory-2010, pp. 13-p. 2010.
Jean-Ya Yves Audibert, Rémi Munos, and Csaba Szepesvári. iesoraatoo-expploitattion tradeoff using variance estimates in multi-armed bandits. Theoretical Computer Science, 110(19):1866-1902 2009.
Peter Auer, Nicolo Cesa-Bianchi, and Paul Fischer. Finite-time analysis of the multiarmed bandit problem. Machine learning, 47(2):335225 256, 2002.
Sébastien Bubeck, Rémi Munos, and Gilles Stoltz. Pure exploration in multi-armed bandits problems. In International conference on Algorithmic learning theory, pp. 23-37. Springer, 2009.
Sébastien Bubeck, Rémi Munos, Gilles Stoltz, and Csaba Szepesvári. <i>x</is-armed bandits. Journal of Machine Learning Research, 16555) 2011. URL http:://tm//.... org/ papers/v12 wttprr//v1//bubeeckk buwbckkla.akaaa html.
Wei Cao, Jian Li, Yufei Tao, and Zhize Li. On top-k selection in multi-armed bandits and hidden bipartite graphs. Advances in Neural Information Processing Systems, 28, 2015.
10

---

Published as a conference paper at ICLR 2026
Lin Chen, Xilin Wei, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Zhenyu Tang, Li Yuan, et al. Sharegpt4video: Improving video understanding and generation with better captions. Advances in Neural Information Processing Systems, 37:19472- 19495, 2024a.
Wei Chen, Yajun Wang, Yang Yuan, and Qinshi Wang. Combinatorial multi-armed bandit and its extension to probabilistically triggered arms. Journal of Machine Learning Research, 17(50):1-33, 2016.
Xi Chen, Xiao Wang, Soravit Changpinyo, AJ Piergiovanni, Piotr Padlewski, Daniel Salz, Se- bastian Goodman, Adam Grycner, Basil Mustafa, Lucas Beyer, Alexander Kolesnikov, Joan Puigcerver, Nan Ding, Keran Rong, Hassan Akbari, Gaurav Mishra, Linting Xue, Ashish V Thapliyal, James Bradbury, Weicheng Kuo, Mojtaba Seyedhosseini, Chao Jia, Burcu Karagol Ayan, Carlos Riquelme Ruiz, Andreas Peter Steiner, Anelia Angelova, Xiaohua Zhai, Neil Houlsby, and Radu Soricut. PaLI: A jointly-scaled multilingual language- image model. In The Eleventh International Conference Learning Representations, 2023. URL https: //openreve wwweeviwwww..... net/ forum? o
Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, and Jingjing Liu. Uniter: Universal image-text representation learning. In European conference computer vision, pp. 104-120. Springer, 2020.
Yukang Chen, Fuzhao Xue, Dacheng Li, Qinghao Hu, Ligeng Zhu, Xiuyu Li, Yunhao Fang, Haotian Tang, Shang Yang. Zhijian Liu, Yihui He, Hongxu Yin, Pavlo Molchanov, Jan Kautz, Linxi Fan, Yuke Zhu, Yao Lu, and Song Han. Longvila: Scaling long-context visual language models for long videos. 2024b.
Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries open-source multimodal models with model, data, and test-time scaling. arXiv preprint XXiv:2412.002271 2024c.
Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, al. How far are we gpt-4v? closing the gap to commercial multimodal models with open-source suites. Science China Information Sciences, 67(12)220010, 2024d.
Zhe Chen, Jiannan Wu, Wenhai Wang. Weijie Su, Guo Chen. Sen Xing. Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistie tasks. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 24185-24198, 2024e.
Chuanqi Cheng, Jian Guan, Wei Wu, and Rui Yan. Scaling video-language models to 10k frames via hierarchical differential distillation. arXiv preprint arXiv:2504.022. 02448, 2025.
Wei Chu, Lihong Li, Lev Reyzin, and Robert Schapire. Contextual bandits with linear payoff functions. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pp. 208-214. JMLR Workshop and Conference Proceedings, 2011.
Danny Driess, Fei Xia, Mehdi S. M. Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, Yevgen Chebotar, Pierre Sermanet, Daniel Duckworth, Sergey Levine, Vincent Vanhoucke, Karol Hausman, Marc Toussaint, Klaus Greff, Andy Zeng. Igor Mordatch, and Pete Florence. Palm-e: An embodied multimodal language model. In arXiv preprint arXiv:2303.03378. 2023.
Eyal Even-Dar, Shie Mannor, and Yishay Mansour. Pac bounds for multi-armed bandit and markov decision processes. In International Conference on Computational Learning Theory, pp. 255-270. Springer, 2002.
Eyal Even-Dar Shie Mannor, Yishay Mansour, and Sridhar Mahadevan. Action elimination and stopping conditions for the multi-an armed bandit and reinforcement learning problems. Journal of machine learning research, 7(6), 2006.
11

---

Published as a conference paper at ICLR 2026
Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang. et al. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal lims in video analysis. In Proceedings of the Computer Vision and Puttern Recognition Conference, pp. 24108-244 24118, 2025.
Zijun Gao, Yanjun Han, Zhimei Ren, and Zhengqing Zhou, Batched multi-armed bandits problem. Advances in Neural Information Processing Systems, 32, 2019.
Weiyu Guo, Ziyang Chen, Shaoguang Wang. Jianxiang He, Yijie Xu, Jinhui Ye, Ying Sun, and Hui Xiong. Logic-in-frames: Dynamic keyframe search via visual semantic-logicht verification for long video understanding. arXiv preprint arXiv:2503. 13139, 2025.
Kai Hu, Feng Gao, Xiaohan Nie, Peng Zhou, Son Tran, Tal Neiman, Lingyun Wang, Mubarak Shah, Raffay Hamid, Bing Yin, et al. M-lim based video frame selection for efficient video understanding. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 13702-13712, 2025.
Kevin Jamieson, Matthew Malloy, Robert Nowak, and Sébastien Bubeck. lil'ucb: An optimal exploration algorithm for multi-armed bandits. In Conference on Learning Theory, pp. 423-439. PMLR, 2014.
Chao Jia, Yinfei Yang. Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoe Le, Yun-Hsuan Sung. Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In International conference on machine leurning, pp. 4904-4916. PMLR, 2021.
Tianyuan Jin, Yu Yang, Jing Tang. Xiaokui Xiao, and Pan Xu. Optimal batched best arm identification. Advances in Neural Information Processing Systems, 37:134947-134980 2024.
Kwang-Sung Jun, Kevin Jamieson, Robert Nowak, and Xiaojin Zhu. Top arm identification in multi-armed bandits with batch arm pulls. In Artificial Intelligence and Statistics, pp. 139-148. PMLR, 2016.
Shivaram Kalyanakrishnan and Peter Stone. Efficient selection of multiple bandit arms: Theory and practice. In ICML, volume 10. pp. 511-518, 2010.
Shivaram Kalyanakrishnan, Ambuj Tewari, Peter Auer, and Peter Stone. Pac subset selection in stochastic multi-armed bandits. In ICML, volume 12, pp. 655-662, 2012.
Zohar Karnin, Tomer Koren, and Oren Somekh. Almost optimal exploration in multi-armed bandits. In International conference on machine learning, pp. 1238-12-4 1246. PMLR, 2013.
Emilie Kaufmann, Olivier Cappé, and Aurélien Garivier. On the complexity of best-arm identification in multi-armed bandit models. The Journal of Machine Learning Research, 17(1):1-42, 2016.
Robert Kleinberg, Aleksandrs Slivkins, and Eli Upfal. Multi-armed bandits in metric spaces. In Proceedings of the fortieth annual ACM symposium on Theory of computing, pp. 681-690, 2008
Tze Leung Lai and Herbert Robbins. Asymptotically efficient adaptive allocation rules. Advances in applied mathematics, 6(1):4222, 1985.
Tor Lattimore and Csaba Szepesvári. Bandil algorithms. Cambridge University Press, 2020.
Jie Lei, Linjie Li, Luowei Zhou, Zhe Gan, Tamara I. Berg, Mohit Bansal, and Jingjing Liu. Less is more: Clipbert for iide-aand-language learning via sparse sampling. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 7331-7341, 2021.
Bo Li, Yuanhan Zhang. Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. LLLVV--oevisiion: Easy visual task trans- fer. Transactions on Machine Learning Research, 2025. ISSN 2835-8856. URL https: //openreview. net./ forum? d==zvVqQULv6..
Dongxu Li, Yudong Liu, Haoning Wu, Yue Wang, Zhiqi Shen, Bowen Qu, Xinyao Niu, Fan Zhou, Chengen Huang, Yanpeng Li, et al. Aria: An open multimodal native mixture-od mixture-of-experte of-eeeesss model. arXiv preprint arXiv:2410.05993 2024a.
12

---

Published as a conference paper at ICLR 2026
Junnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, and Steven Chu Hong Hoi. Align before fuse: Vision and language representation learning with momentum distillation. Advances in neural information processing systems. 34:9694 97054 2021.
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-ina -mmaee pre- training for unified vision- language understanding and generation. In International conference on machine learning, pp. 28888-12908 PMLR, 2022.
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. B Bootstrapping language-iman -mmaee pre- -re-traiiing with frozen image encoders and large language models. In International conference on machine learning, pp. 19730-19742. PMLR, 2023.
KunChang Li, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang, Limin Wang, and Yu Qiao. Videochat: Chat- video understanding, 2024b. URL httpttts//xxxi....ttt abs/ / 2205. 06355.
Linjie Li, Yen-Chun Chen, Yu Cheng, Zhe Gan, Licheng Yu, and Jingjing Liu. HERO: Hier- archical encoder for Video+Language onn-reepresnn pre-training. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.), Proceedings of the 2020 Conference on Empir- ical Methods in Natural Language Processing (EMNLP), pp. 2046-2065, Online, November 2020. Association for Computational Linguistics. doi: 10. 10.11853/v https: / aa///clannnn htt///aaaaaatttooaa/aaaa org/ 2020. http:/200..mmmmmmma.m.. emnlp-main 161//
1/t0://12222022m//// 1mt.200.mmmmmmmi.11.....1 161. URL
Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh, and Kai-Wei Chang. Visualbert: A simple and performant baseline for vision and language. arXiv preprint arXiv:1908.03557, 2019.
Yanwei Li, Chengyao Wang, and Jiaya Jia. Llama-vid: An image is worth 2 tokens in large language models. In European Conference on Computer Vision, pp. 323-340, Springer, 2024c.
Hao Liang, Jiapeng Li, Tianyi Bai, Xijie Huang, Linzhuang Sun, Zhengren Wang, Conghui He, Bin Cui, Chong Chen, and Wentao Zhang. Keyvideollm: Towards large-scale video keyframe selection. CoRR, 2024.
Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, and Li Yuan. Video-llava: Learning united visual representation by alignment before projection. arXiv preprint arXiv:2311.10122. 2023.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023, URL https: / /arxivv.... org/ abs 2304. 08485.
Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jac Lec. Llava-next: Improved reasoning, ocr, and world knowledge, January 2024a. URL https: ///lavaavvaaa www.vvl.iiiit.. www.ggttuub..... io ho/blogg/022--- 01 20-30--30 1lava- next/.
Jiajun Liu, Yibing Wang, Hanghang Ma. Xiaoping Wu, Xiaoqi Ma, Xiaoming Wei, Jianbin Jiao, Enhua Wu, and Jie Hu. Kangaroo: A powerful languale model supporting long-context video input, 2024b. URL https: aarxivii org/ abs/ /2///4400 2408 15542.
Shuming Liu. Chen Zhao, Tianqi Xu, and Bernard Ghanem. Bolt: Boost large vision- language model without training for long-form video understanding, In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 3318-3327, 2025.
Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and and-language tasks. Advances in neural information processing systems, 32, 2019.
Huaishao Luo, Lei Ji. Ming Zhong, Yang Chen, Wen Lei, Nan Duan, and Tianrui Li. Clip4clip: An empirical study of clip for end end video clip retrieval and captioning. Neurocomputing, 508: 293-304, 2022.
Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Shahbaz Khan. Video-chatyne Towards detailed video understanding via large vision and language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL. 2024), 2024.
13

---

Published as a conference paper at ICLR 2026
Behrooz Mahasseni, Michael Lam, and Sinisa Todorovic. Unsupervised video summarization with adversarial Istm networks. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 202-211, 2017.
Vianney Perchet, Philippe Rigollet, Sylvain Chassang, and Erik Snowberg. Batched bandit problems. 2016.
Rui Qian, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Shuangrui Ding, Dahua Lin, and Jiaqi Wang. Streaming long video understanding with large language models. Advances in Neural Information Processing Systems, 37:1 19316-13399199 119360, 2024.
Minghao Qin, Xiangrui Liu, Zhengyang Liang, Yan Shu, Huaying Yuan, Juenjie Zhou, Shitao Xiao, Bo Zhao, and Zheng Liu. Video-xl-2: Towards very long-video understanding through task-aware kv sparsification. arXiv preprint arXiv:2506. 19225, 2025.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. $748-8763. PmLR, 2021.
Xiaoqian Shen, Yunyang Xiong, Changsheng Zhao, Lemeng Wu, Jun Chen, Chenchen Zhu, Zechun Liu, Fanyi Xiao, Balakrishnan Varadarajan, Florian Bordes, et al. Longvu: Spatiotemporal adaptive compression for long video-language understanding, arXiv preprint arXiv:2410.17454 2024.
Yunhang Shen, Chaoyou Fu, Shaoqi Dong, Xiong Wang, Yi-Fan Zhang, Peixian Chen, Mengdan Zhang, Haoyu Cao, Ke Li, Xiawu Zheng, et al. Long-vita: Scaling large multi-modal models to 1 million tokens with leading short-context accuracy. arXiv preprint arXiv:2502.05177, 2025.
Enxin Song. Wenhao Chai, Guanhong Wang, Yucheng Zhang, Haoyang Zhou, Feiyang Wu, Haozhe Chi, Xun Guo, Tian Ye, Yanting Zhang, et al. Moviechat: From dense token to sparse memory for long video understanding. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18221 18221-18221 18232, 2024.
the
Weijie Su, Xizhou Zhu, Yue Cao, Bin Li, Lewei Lu, Furu Wei, and Jifeng Dai. VI-bert: Pre-training of generic vingal-linguistic representations. In International Conference on Learning Representations, 2020. URL https: hwtp://wwwwweeeeeeeeeeeee net/ forum? wtwpummmdddsyyypxxeeeeee
Chen Sun, Austin Myers, Carl Vondrick, Kevin Murphy, and Cordelia Schmid. Videobert: A joint model for video and language representation learning. In Proceedings of the IEEF/CVF international conference on computer vision, pp- 7464-7473. 2019.
Hao Tan and Mohit Bansal. Lxmert: Learning cross-modality encoder representations from trans- formers. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, 2019.
Xi Tang, Jihao Qiu, Lingxi Xie, Yunjie Tian, Jianbin Jiao, and Qixiang Ye. Adaptive keyframe sampling for long video understanding. In Proceedings of the Computer Vision and Puttern Recognition Conference, pp. 29118-229999 29128, 2025.
Adrienne Tuymman and Rémy Degenne. The batch complexity of bandit pure exploration. arXiv preprint arXiv:2502.01425, 2025.
Limin Wang, Yuanjun Xiong, Zhe Wang, Yu Qiao, Dahua Lin, Xiaoou Tang, and Lue Van Gool. Temporal segment networks: Towards good practices for deep action recognition. In European conference on computer vision, pp. 20-36. Springer, 2016.
Peng Wang, An Yang, Rui Men, Junyang Lin, Shuai Bai, Zhikang Li, Jianxin Ma, Chang Zhou, Jingren Zhou, and Hongxia Yang. Ofa: Unifying architectures, tasks, and modalities through a simple seguence-to-sqqeeme learning framework. In International conference on machine learning, pp. 33331-233342 PMLR, 2022a.
Peng Wang. Shuai Bai. Sinan Tan, Shijie Wang. Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang. Wenbin Ge, et al. Qwen2-vl: Enhancing vision- language model's 's perception of the world at any resolution. arXiv preprint 2409 /2/9/, 2024a.
14

---

Published as a conference paper at ICLR 2026
Xiaohan Wang, Yuhui Zhang, Orr Zohar, and Serena Yeung-Levy. Videoagent: Long-form video understanding with large language model as agent. In European Conference on Computer Vision, pp. 58-76. Springer, 2024b.
Yi Wang, Kunchang Li, Yizhuo Li, Yinan He, Bingkun Huang, Zhiyu Zhao, Hongjie Zhang, Jilan Xu, Yi Liu, Zun Wang, et al. Intervvideo: General video foundation models generative and discriminative learning. arXiv preprint arXiv:2212.0311 2022b.
Yi Wang, Kunchang Li, Xinhao Li, Jiashuo Yu, Yinan He, Guo Chen, Baoqi Pei, Rongkun Zheng, Zun Wang. Yansong Shi, et al. Internvide02: Scaling foundation models for multimodal video understanding. In European Conference on Computer Vision, pp. 396-416. Springer, 2024c.
Ziyang Wang, Shoubin Yu, Elias Stengel-Eskin, Jaehong Yoon, Feng Cheng, Gedas Bertasius, and Mohit Bansal. Videotree: Adaptive tree-based video representation for 1lm reasoning on long videos. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp.
3272-3283, 2025.
Yuetian Weng, Mingfei Han, Haoyu He, Xiaojun Chang, and Bohan Zhuang. Longylm: Efficient long video understanding via large language models. In European Conference Computer Vision, pp. 453-470. Springer, 2024.
Thomas Wiegand, Gary J Sullivan, Gisle Bjontegaard, and Ajay Luthra. Overview of the h. 264/ave video coding standard, IEEE Transactions on circuits and systems for video technology, 13(7): 560-576, 2003.
Haoning Wu, Dongxu Li, Bei Chen, and Jumnan Li. Longvideobench: A benchmark for long-contex. interleaved video-language understanding. Advances Neural Information Processing Systems, 37:28828-22 28857, 2024.
Lin Xu, Yilin Zhao, Daquan Zhou, Zhijie Lin, See Kiong Ng. and Jiashi Feng. Pllava Parameter-fiee llava extension from images to videos for video dense captioning. 2024.
An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, al. Qwen2 technical report, 2024. URL https://arxiv. org/abs/2407.10677 7:8, 2024.
Antoine Yang, Antoine Miech, Josef Sivic, Ivan Laptev, and Cordelia Schmid. Zero-shot video question answering via frozen bidirectional language models. Advances in Neural Information Processing Systems, 35: 141, 2022.
Jihan Yang, Shusheng Yang, Anjali W Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in space: How multimodal large language models see, remember, and recall spaces. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 10632-10644 10643,, 2025.
Lewei Yao, Runhui Huang, Lu Hou, Guansong Lu, Minzhe Niu, Hang Xu, Xiaodan Liang, Zhenguo Li, Xin Jiang, and Chunjing Xu. FILIP: Fine-grained interactive language-image pre-training. In International Conference on Learning Representations, 2022. URL https: rpenrevie.. net/ / Eorum? id CpDhssEDDD
Yuan Yao, Tianyu Yu, Ao Zhang. Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al. Minicpm-v: A gpt-4v level milm on your phone. arXiv preprint a4Xiv:2408. 01 81800 2024.
Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung, Mojtaba Seyedhosseini, and Yonghui Wu. Coca: Contrastive captioners are image-text foundation models. Transactions on Machine Learn- ing Research, 2022. ISSN 2835-8856. URL https: hwtp///pweee.eeee//////ee net /tt//fouuu forum? iid= Ee277P3AYC.
Sicheng Yu, Chengkai Jin, Huanyu Wang, Zhenghao Chen, Sheng Jin, Zhongrong Zuo, Xiaolei Xu, Zhenbang Sun, Bingni Zhang, Jiawei Wu, et al. Frame-voyager: Learning to query frames for video large language models. arXiv preprint arXiv:2410.032266, 2024.
15

---

Published as a conference paper at ICLR 2026
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 11975-11986, 2023.
Boqiang Zhang, Kehan Li, Zesen Cheng, Zhiqiang Hu, Yuqian Yuan, Guanzheng Chen, Sicong Leng, Yuming Jiang, Hang Zhang, Xin Li, et al. Videollama 3: Frontier multimodal foundation models for image and video understanding. arXiv preprint arXiv:2501, 100.:20001..16,, 2025a.
Chaoying Zhang, Yaping Dai, Yanyan Cheng, Zhiyang Jia, and Kaoru Hirota. Recurrent attention 1stm model for image chinese caption generation. In 2018 Joint 10th International Conference on Soft Computing and Intelligent Systems (SCIS) and 19th International Symposium on Advanced Intelligent Systems (ISIS), pp. 808-813. IEFE, 2018.
Hang Zhang, Xin Li, and L Lidong Bing. Video-llama: An instruction-tuned audio-visual language model for video understanding, arXiv preprint MxXiv:2306002856 2023.
Kaichen Zhang, Bo Li. Peiyuan Zhang, Fanyi Pu, Joshua Adrian Cahyono, Kairui Hu, Shuai Liu, Yuanhan Zhang, Jingkang Yang. Chunyuan Li, al. Lmms-eval: Reality check on the evaluation of large multimodal models. arXiv preprint arXiv:2407. 12772, 2024a.
Ke Zhang, Wei-Lun Chao, Fei Sha, and Kristen Grauman. Video summarization with long short-term memory. European conference on computer vision, pp. 766-782. Springer, 2016.
Peiyuan Zhang, Kaichen Zhang, Bo Li, Guangtao Zeng, Jingkang Yang, Yuanhan Zhang, Ziyue Wang, Haoran Tan, Chunyuan Li, and Ziwei Liu. Long context transfer from language to vision. arXiv preprint arXiv:2406. /6852, 2024b. URL https::tt//.o.....t. //tp:i/aavvvvv.///a///// org/ abs /200 1882.
Shaojie Zhang, Jiahui Yang, Jianqin Yin, Zhenbo Luo, and Jian Luan. Q-frame: Query-aware frame selection and multi-resolution adaptation for video-Ilms. In Proceedings of the IEEF/CVF International Conference on Computer Vision (ICCV), pp. 22056-22065, October 2025b.
Yuanhan Zhang, Bo Li, haotian Liu, Yong jae Lee, Liangke Gui, Di Fu, Jiashi Feng, Ziwei Liu, and Chunyuan Li. Llava-next: A strong zero-shot video understanding model, April 2024c. URL dittps://laat gi thub. 1 otoiio//llllo www.oo///2//2 201//0224-0000 -000022-0--00 30-3000-1000000 llava -ewtavv-et-vieeeeeeee
Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Ziwei Liu, and Chunyuan Li. Video instruction tuning with synthetic data. Transactions Machine Learning Research, 2025c.
Bin Zhao, Xuelong Li, and Xiaoqiang Lu. Hierarchical recurrent neural network for video summa- rization. In Proceedings of the 25th ACM international conference on Multimedia, pp. 863-871, 2017.
Junjie Zhou, Yan Shu, Bo Zhao, Boya Wu, Zhengyang Liang, Shitao Xiao, Minghao Qin, Xi Yang, Yongping Xiong, Bo Zhang, et al. Mlvu: Benchmarking multi-task long video understanding. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 13691-13701, 2025.
Kaiyang Zhou, Yu Qiao, and Tao Xiang. Deep reinforcement learning for unsupervised video reward. In Proceedings of the AAAI conference
summarization with on artificial intelligence, volume 32, 2018.
Bin Zhu, Bin Lin, Munan Ning, Yang Yan, Jiaxi Cui, WANG HongFa, Yatian Pang, Wenhao Jiang, Junwu Zhang. Zongwei Li, Cai Wan Zhang, Zhifeng Li, Wei Liu, and Li Yuan. Languagebind: Extending video-language pretraining to n-modality by language-based semantic alignment. In The Twelfth International Conference on Learning Representations, 2024. URL https:// openreview net fort//ooumm20 -i-.iidddu202.y.....
16

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1
RELATED WORK
A.1.1
MULTIMODAL LARGE LANGUAGE MODELS (MLLMS) FOR VIDEO UNDERSTANDING
Recent MILMs extend large language models with visual encoders, encoding images or frames into visual tokens that are fused with text to support open-ended video understanding. Most follow an encode-project-fuus pipeline with instruction tuning, as exemplified by the LLa VA family, Video- DANVAVidee-LLMMM/iieeee-
and LLaMA-Vid/VidooChe (Liu et al., 2023; Lin et al., 2023; Zhang et al., 2023; Maaz et al., 2024: Li et al., 2024c:b). Progress has largely come from scaling data/backbones and strengthening cross- alignment (MiniCPM-V, V, Intern /ieetV// VL2. Qwen2- advances via ShareGPT4Video and LanguageBind) (Yao et al., 2024; Chen et al., 2024e;d;e; Wang et al., 2024a; Chen et al., 2024a; Zhu et al., 2024), together with architectural refinements that unify multi uuati-granullrity visual inputs and tighten temporal adapters, and that improve projector efficiency curricula VA-One Vi-OneVision, Video, Aria, PLLaVA, Kangaroo) (Li et al., 2025; Liu et al., 2024a; Zhang et al., 2024c; Li et al., 2024a; Xu et al., 2024; Liu et al., 2024b). Finally, several models explicitly target extended context and hierarchical summarization for long-form understanding (Long LongVA VA. LongVLM, VU) (Chen al., 2024b; Zhang al., 2024b; Weng et al., 2024; Shen al., 2024).
VI.; data-centric and modality-
However, this tokenization-firs: paradigm encounters token explosion on long videos, where dense sampling yields prohibitive sequences. Recent efforts reduce the budget by compressing restructur- ing tokens: MovieChat (Song et al., 2024) compacts frames into sparse memory, Video-XL-2 (Qin al., 2025) synthesizes condensed tokens, and VideoStreaming (Qian et al., 2024) processes streams incrementally to cap tokens. maaninngttollaugmeent
agents (e.g., VideoAgentt (Wang et al., 2024b)) curb perception via selective analysis, while hierarchical controllers (Video'Tree (Wang et al., 2025)) and scaling recipes (VideoLLaMA 3 (Zhang et al., 2025a)) aid long- long-horizon reasoning. Beyond compression, VIL.AMP (Cheng et al., 2025) uses mixed-precision tokenization to emphasize dif- ferential frames/patches and allocate capacity adaptively; long-context instruction-tuning such as Long-VITA (Shen et al., 2025) complements these strategies for long videos.
A.1.2 VISION-LANGUAGE PRETRAINED MODELS
Cross-modal vision-language pretraining spans two-stream fusion, single-stream fusion, dual-encoder contrastive learning, and encoder-decoder hybrids. Two-stream models such as ViLBERT (Lu et al., 2019) and LXMERT (Tan & Bansal, 2019) encode vision and text separately and fuse via cross- attention, while single-stream
-isuuaaartssisa (Li et al., 2019). VL-BERT (Su et al., 2020), UNITER (Chen et al., 2020) concatenate region features with text in unified Transformer using MLM and alignment losses. Large-scale dual encoders like CLIP (Radford et al., 2021) 1) and ALIGN (Jia et al., 2021) learn contrastive embeddings for zero-shot transfer, with FILIP (Yao et al., 2022) improving fine-grained patch-token alignment. Hybrid objectives combine contrastive and generative training (Li et al., 2021; Yu et al., 2022; Wang al., 2022a; Chen et al., 2023) unify captioning and VQA. The BLIP family integrates vision encoders with language modeling-Bli et al., 2022) and BI.I-2 (I.i et al., 2023) (via a lightweight Q-Former) wwhle Flamingo (Alayrac et al., 2022) and M-LM-E (Driess et al., 2023) inject visual inputs into large LMs for few-shot multimodal reasoning.
(Li
Extending to video, early pretraining models learned joint teriiio-emmoralllangggee representations with lightweight fusion and sparse sampling. VideoBERT (Sun et al., 2019) pairs frame sequences with transcripts in a BERT-style objective for retrieval and script generation, while HERO (Li et al., 2020) and ClipBERT (Lei et al., 2021) improve efficiency via hicrarchical encoding and key-frame sampling for vidco-text retrieval and QA. Building directly on large image-text models, Clip4Clip (Luo et al., 2022) reuses CLIP encoders and matches videos to text via contrastive similarity, and FrozenBiLM (Yang et al., 2022) freezes bi-directional LM while aligning video encoder for zero-shot VQA.
1.3 KEYFRAME SELECTION
In video representation learning, keyframe selection spans two major paradigms.
17

---

Published as a conference paper at ICLR 2026
Training-free keyframe selection. Recent training-free methods leverage pretrained vision- language models and lightweight heuristics to pick informative, query-relevant frames. Adaptive Keyframe Sampling (AKS) maximizes trompt-frame similarity while enforcing temporal coverage via a split-and-judge policy (Tang et al., 2025); Q-Frame ranks frames by query- meeyy-oodditioned impor- tance and preserves few at higher resolution for detail (Zhang et al., 2025b). Text-frame alignment with frozen models further enables plug-and-play selectors (Key ViMeOLLM. BOLT) that boost Video- LLM performance without fine-tuning (Liang al., 2024; Liu al., 2025). To avoid redundancy and preserve structure under token budget, Logic-in- in-amee--rees performs dynamic, logic-verified search (Guo et al., 2025), while VideoTree builds hierarchical, query-adaptive frame pyramid that expands salient scenes (Wang et al., 2025).
Instruction-aligned and learned selectors. Instruction-guided approaches train selectors with LLM/MLLM feedback: Voyager learns to query frame combinations by ranking sets with a pretrained Video-LLM (Yu et al., 2024), and Hu al. (2025) supervise lightweight selector using MLLM-derived single-frame relevance and multi-frame complementarity Classical summarization remains relevant: supervised LSTTM-based models (vsL.STM, dppL.STM; STMM hierarchical RNNs) learn importance/diveesity from human summaries (Zhang et al., 2016; ;; 2018; Zhao et al., 2017), while unsupervised RL/adversarial methods (DR-DSN, SUM-GAN) optimize diversily-representttvvees or realism without labels (Zhou et al., 2018; 8; Mahasseni et al., 2017); however, these are typically ausk-agnostic and may miss frames critical for query-driven VQA.
A.1.4 MULTI-ARMED BANDITS AND BATCHED EXPLORATION
Multi-armec bandits (MAB) encompass both regret minimization and pure exploration. Regret- oriented methods such as UCB variants and Thompson Sampling establish logarithmic-regret founda- tions for sequential decision-making making (Auer et al., 2002; Lai & Robbins, 1985; Agrawal & Goyal, 2012). Pure exploration instead targets high-confidence identification with minimal samples, formalized as best-arm (and top-k) identification (Even-Dar et al., 2006; Bubeck et al., 2009; Kalyanakrishnan & Stone, 2010; Cao et al., 2015). Early elimination schemes (Successive/Mediaa Elimination) provide PAC guarantees (Even- -ar et al., 2006; 2002), while manfidence-bound and racing Comilies-LUCH UCB-E, and near-optimal lil UCB UCB-sharpen sample complexity and approach known lower bounds (Kalyanakrishnan et al., 2012; Audibert & Bubeck, 2010; Karnin et al., 2013; Jamieson et al., 2014; Kaufmann et al., 2016). Beyond single arms, combinatorial pure exploration (CPE) seeks an optimal subset under structural constraints, combining bandit confidence bounds with combinatorial oracles to search exponentially large spaces efficiently (Chen et al., 2016; Lattimore & Szepesvári, 2020),
Fully sequential adaptivity can be impractical when decisions must be made in few rounds in parallel. Batched (parallel) bandits address this by operating over small number of adaptivity rounds, yet retain near-sequential sample efficiency for pure exploration in theory and practice (Perchet et al., 2016; Jun et al., 2016; Gao et al., 2019). Batch-Climination/LLCCB USCt-syll -sylee procedures match sequential complexity up to constants with only handful of updates (Jun et al., 2016), and bound trade-offs between batches and samples are well understood with matching algorithms (Perchet et al., 2016; Kaufmann et al., 2016; Tuynman & Degenne, 2025). Recent designs such as Tri-BBAI attain asymptotically optimal fixed-confidence BAI with just three batches, underscoring the feasibility of resource-constraineet exploration (Jin et al., 2024).
B BERNSTEIN CONFIDENCE RADIUS
Theorem B.1. Let Na be the number of pulls for arm round n and = EoEA (a(n) is the total number of pulls. Let (n)n) be the empirical mean of arm round n and 0% (n) be the empirical variance of arm at round n. We define the empirical Bernstein Confidence Radius Ba (n) as
262 Inn
3 Inn max max(1, (n)))
Ba(n) = V max(1,
Then we have the following bound holds with probability at least - 9.
[pa(n) ptal= S (n.
18

---

6(M-m) Published as a conference paper at ICLR 2026
frame-query query relevance setting, the reward ry and latent frame reward ya is naturally bounded [0, 1]. Therefore, according to Bernstein inequality, for any E (0, 1), we have
Proof. Under the setting of frame-
2 In a +
31n 8 Na(n) 2 21-6.
p S (a(n) +
(n)
And symmetrically, we have
20% In (n)
31n (a(n)
p 2 Pa(n)
-1
> 1 11-6 8.
Therefore, we have
283 In 3 Na
3 à Na(n)
[a(n)) - Hal $
2 1 26.
Choose 8 3 then we have
20% In 3 (n)
31m Na(n)
|Ma - (aa(1))
8
holds with probability at least
When (n) = O, the statement is trivially true. Thus, we have the following bound holds with probability least 1 e:
IMa - A (a(n)) S (m))
C
REGRET BOUND
Arm-level Regret Bound
Theorem C.1. Algorithm 2 returns the oracle top-s S* with probability least 1 6M when lerminated.
Proof. When Algorithm terminates, the following condition holds:
max (a. (a(n) S min (an(a) (n)..
agš
a
According to Theorem B.I, with probability at least 1 arms a. Therefore, for any a & S,
6 we have IMa -a(n)) S Ba(n) for all
6
P e <
Thus, the probability that there does not exist such an arm least of the $ set. And this completes the proof.
where size
n
Frame-level Regret Bound define the frame-level regret as the difference between the optimal frame-level reward and the reward of the selected frames.
frame
Σ
Σ
N
V1
yv-
tEK.
As long obtain the oracle top-s set S", the frame-level regret is also guaranteed to be small. As Frame-level sampling is actually finite so we can always find the top-k frames with the highest rewards.
E
frame
Σ
Σ -
Σ IEK:
26 = 0.
IEK
WESt
For tighter bound, leave this future work.
19

---

Published as a conference paper at ICLR 2026
Question: When chart representing the Carch Eshnicity uppeary in the video, with Alte ocmspyingg the lorgest portion, red being the second, and light groen least, of followiin souronees displeyaad
Ansserr 60% "deciarea" Czock.
Ground Truch: 25% "Unspeeified"
Question: In yome with green wall riles, there womsn with long hatt wearing white dress. In the lower of the soneo near her head, white tez appears says started playing dlnins change happons
appears in resorsmm?
Answer: bag appoars har shoulder.
Ground Trach: A black bag appears on her thoulder
Figure 4: Two representative failure modes of LLaVA-Video-1 when using FOCUs to select keyframes. Yellow stars mark manually annotated framee that are most informative for the query. In the first case, FOCUS correctly selects these frames, but the MILM still fails to answer due to its limited ability reason over the relatively complex chart. In the second case, FOCUS fails to capture the critical frames during a compact, rapid scene transition: the relevant segment lasts only 1-2 seconds within 10-minute video, making the keyframes difficult identify even for human experts.
D VISUALIZATIONS OF FAILURE CASES
To provide more comprehensive understanding of the proposed Focus, we analyze two typical failure patterns of LLAVA-Video-1 when using FOcUs to select keyframes in Figure 4, which most failure cases fall into.
In the first case, the query asks: "When pie chart representing the Czech ethnicity appears in the video, with blue occupying the largest portion, red the second, and light green the least, which of the following sentences is displayed on the screen?" Across the entire video, this pie chart appears multiple times and is interleaved with other background content. Consequently, even though Focus correctly selects the most informative frames, the MILM is confused by the subtle differences between multiple similar pic charts. This failure pattern is mainly attributable to the limited reasoning and perception capabilities of the MLLM itself, rather than to the keyframe selection method.
In the second case, the video a 10-minute vlog with frequent scene transitions. The query asks: "In room with green wall tiles, there is woman with long hair wearing white dress. In the lower part of the screen near her head, white text appears that says 'someone started playing drums the back.' What change happens her when she appears in the restroom?" The relevant segment lasts only 1-2 seconds within the 10-minute video, making the keyframes difficult to identify even for human experts. As shown in Figure 4, FOCUS successfully selects frames where the correct text appears, but still fails to capture the most critical frames. This pattern reveals that, in some intrinsically challenging cases, the adaptive sampling strategy of Focus may risk missing crucial information.
E COMPARISON WITH STATB-OF-THE-ART
Here compare proposed FOCUS against state-of-the-art training-free keyframe selection methods on both Long Video and Video-MME. Specifically, we consider two recent approaches based vision vasion-language similarity:
AKS (Tang et al., 2025): A plug-and-ple adaptive keyframe sampling module that recursively balances query-frame relevance and temporal coverage under fixed frame budget. By first downsampling the video to I frame per second, scoring each frame with prompt-frame matching model, and then applying audge-and-splic procedure to allocate keyframe slots across segments, AKS maximizes informative coverage and serves as strong state-of-the-an baseline for long- video QA.
Q-Frame (Zhang al., 2025b): A training-free, query- aware frame selection and multi-resolution adaptation framework that can be plugged in front of diverse Video-LLMs. uses a text-image matching network (e.g., CLIP) to compute query-frame similarity scores, samples compact set
20

---

Published as a conference paper at ICLR 2026
Model
#Frame
LLM
LongVideeBench
Video-MME
Owen2-VL-7B
32
53.6
57.4
Qwen2-VL.-7B w/ AKS
32
7B
57.8
59.7
Dwen2-VL.77B w/ Q-Frame
32
7B
57.4
56.5
Qwen2-VL-7B w/ Ours
32
7B
62.3 6.7
59.7 f 2.3
LLaVA-OV-7B
32
7B
54.8
56.5
LLaVA-OV-7B w/ AKS
32
57.4
57.7
LLaVA-OV-7B w/ Q-Frame
32
7B
54.8
56,8
LLaVA-OV-7B Ours
32
60.7 5.9
58.3 f 1.8
LLAVA-Video-7B
64
7B
58.9
64.4
LLaVA-Video-7B w/ AKS
64
62.1
64.6
LLaVA-Video-7B w/ Q-Frame
64
7B
59.9
64.5
LLVA--iiee-7B w/ Ours
64
7B
63.5 1 4.6
65.4 f 1.0
Table 5: Video question-answering accuracy (%) of different MILMS on Long BineoBeenn and Video-MME. We compare FOCUS with AKS and Q-Frame on Qwen2-VI., LLaVA-OV, and LLaVA- Video. The suffix "w/ Ours" denotes models using keyframes selected by Focus; likewise, "w/ AKS" and "w/ Q-Frame" indicate using keyframes from the corresponding baselines. #Frame is the number frames fed into the MLLM, and LLM denotes the language model size.
of highly relevant frames via stochastic selection, and assigns them heterogencous resolutions so that crucial frames are preserved high fidelity under fixed token budget.
We report the results in Table 5. Across all three backbones and both benchmarks, FOCUS con- sistently outperforms both AKS and Q-Frame under the same frame budget. particular, FOCUS improves the plain Qwen2-VL VL-7B, LLaVA-OV-7B, and LLaVA-Video-7B models by 4.6-6.7% on Long Becc and up 2.3% on Video-MME indicating that our keyframe selection strategy transfers robustly across different MILMs.
For the two compared baselines, AKS consistently outperforms Q-Frame on both Long Beng and Video-MME whenever Q-Frame is evaluated. We attribute this to the more sophisticated and adaptive sampling scheme of AKS, which explicitly balances query-frame relevance and temporal coverage instead of relying solely on similarity scores.
By contrast, Q-Frame behaves more like cokee-compression mechanism: maps fixed frame budget to a fixed number of visual tokens so that the MILM can "see" more frames than it is originally designed for. However, the lack of an explicit temporal sampling or search design means that Q-Frame does not actively reason about where informative moments occur in long videos, which limits its performance in the long-form setting.
F EXPERIMENTS ON MORE BENCHMARKS
Model
#Frame
LLM
MLVU
VSI-Bench
Twen2-VL-7h
32
7B
59.7
36.5
Owen2-VL-7B w/ AKS
32
7B
64.3
36.9
Qwen2-VL-7B w/ Ours
32
7B
67.0 1 6.7
39.0 2.5
LLaVA-Video-7B
64
7B
68.2
41.7
LLaVA-Video-7B w/ AKS
64
7B
71.2
42.2
LLaVA-Video-7B w/ Ours
64
7B
72.7 1 4.5
42.4 1 0.7
Table Video question-ansween accuracy (%) of different MLLMS on MLVU and VSI-Bench. We compare FOCUS with AKS on Qwen2-VL. and LLaVA-Vided The suffix "w/ Ours" denotes models using keyframes selected by FOcus; likewise, "w/ * indicates using keyframes from the corresponding baselines. #Frame is the number of frames fed into the MLLM, and LLM denotes the language model size.
21

---

Published as a conference paper at ICLR 2026
To further investigate the generalization ability of Focus beyond long-form QA benchmarks, conduct experiments on two additional datasets:
MLVU (Zhou et al., 2025): A comprehensive multi-task long-video understanding benchmark constructed from 1,730 long videos (3 minutes to 2 hours) spanning movies, surveillance, egocen- tric recordings, cartoons, and game videos. It defines nine evaluation tasks that jointly probe both global and local reasoning abilities of MLLMs, and reveals substantial performance degradation as video length grows.
VSI-Bench (Yang et al., 2025): A video-based visual-spatial intelligence benchmark built from 288 egocentric indoor videos (ScanNet, ScanNet++, ARKitSeenes) with over 5,000 ques- tion-answer pairs. It focuses 3D spatial understanding and memory from first-person streams, evaluating MLLMs on tasks such as spatial layout reasoning, navigation, and distance estimation.
We summarize the results in Table 6. On MLVU, method improves Qwen2-VL-7B from 59.7% to 67. 57.0% (+7.3%) and LLLVA-Video- from 68. 2% to 72.7% (44.5%), while also outperforming AKS by +2.7% and +1.5% points, respectively. On VSI-Bench, which emphasizes fine-grained spatial reasoning over relatively short egocentric clips. our method still yields consistent gains: for Qwen2- VL-7B, accuracy increases from 36.5% 39.0% (+2.5%), and for LLaVA LLaVA-Video-P2 Video-7B from 41.7% to 42.4% (+0.7%), respectively. These results indicate that our temporal search mechanism generalizes well across different backbones and tasks, with particularly pronounced benefits on long and heterogeneous videos.
At the same time, the improvements on VSI-Bench are understandably smaller than on long-video benchmarks, When videos are short and informative content is more uniformly distributed, uniform sampling already captures many salient frames, leaving less headroom for sophisticated temporal search. We explicitly regard this limitation and promising direction for future work spatially-aware frame selection in low-redundancy settings.
G ABLATION STUDIES
G.1 TWO-STAGE EXPLORATION TTT.ORAATON-EXPPLOTTTTTO
One of the core designs of FOcUS is the two-stage pelloration-epplottttiie procedure. To better understand the contribution of each stage, we introduce two variants of Focus:
This variant only performs the coarse exploration stage to identify promising temporal arms. In the final keyframe selection step, it randomly samples frames from all frames within the selected arms without any further refinement.
Focus-C:
This variant only performs the fine-grained exploration stage by uniformly sampling frames over the whole video and interpolating the rewards via nearest- assignment. The final keyframes are then drawn directly from the resulting video-level sampling distribution, without the arm-level pre-selection.
FoCUs-F:
Uniform
FOCUS-C
FOCUS-F
FOCUS
Qwen2-VI 55.6
61.7
61.5
62.3
LLaVA-OV
54.8
58.4
57.7
60.7
LLaVA-Videe 58.9 62.3
62.5
63.5
Table 7: Ablation of the two-stage exploration-exploitttion procedure Long VongVideoBench. Uniform denotes naive uniform frame sampling. Focus-C uses only the coarse exploration stage to select promising temporal arms, and then randomly samples frames within them. FOcUS-F uses only the fine-gre fine-grained exploration stage over the entire video. The full FOCUS combines both stages and consistently achieves the best performance across all MLLMs, indicating that coarse arm selection and fine- ine-grained refinement are complementary.
We conduct experiments Long VideeBenc with Qwen2-VL-7B. LLaVA-OV-7B, and LLaVA- Video-7B, and summarize the ablation results in Table 7. Both Focus-C and FOcus-F provide substantial improvements uniform sampling across all three backbones, demonstrating that
22

---

Published as a conference paper at ICLR 2026
selection and fine-grate fine-grained exploration each effective their own. The full two-stage variant further yields the best performance in all cases, achieving an additional gain of up to 2.3% the single-stage variants, which confirms that coarse localization of promising regions and subsequent fine-grained exploitation are complementary rather than interchangeable.
G.2 BERNSTEIN CONFIDENCE RADIUS
Compared with the classical UCB algorithm, the Bernstein confidence radius more robust to high-variance rewards. To better understand its contribution, we introduce variant of FocUS that relies on the empirical mean without variance-aware exploration bonus when selecting top-relevance frames:
FoCUS-M: This variant uses the empirical mean reward to rank arms and select top-relevance frames, instead of the Bernstein confidence radius.
Uniform
FOCUS-M
FOCUS
Qwen2- VL
55.6
61.7
62.3
LLaVA-OV
54.8
58.1
60.7
LLaVA- Video
58.9
63.0
63.5
Table 8: Ablation of the Bernstein confidence radius LongVideoBench. Uniform denotes naive uniform frame sampling. FOCUS-M uses the empirical mean to rank arms and select top-relevance frames. The full FOCUS leverages the Bernstein confidence radius to form variance-aware upper confidence bounds.
We summarize the results Table 8. The empirical-mer variant (FOCUS-M) already yields large gains over uniform sampling across all three backbones, showing that even simple bandit-style selection is beneficial. However, the full method with the Bernstein confidence radius consistently achieves the best performance, providing up 2.6% improvement over uniform and up 2.6% improvement over the base models. This confirms that a variance-aware confidence radius is more effective than the empirical mean alone for selecting top-relevance frames, as it encourages additional exploration of -igh-unnertaintt clips, especially when a clip contains diverse or rapidly changing scenes.
G.3 EFFECT OF CLIP LENGTH
In the formulation of Focus, each video is partitioned into fixed-length clips that serve as bandit arms. The clip length is a crucial hyper-paramett that controls the granularity of exploration and exploitation. To better understand its effect, conduct experiments Long Vene with LLaVA-Video-7B and summarize the results in Table 9.
Uniform
8s
16s
32s
ACC GPU hours
58.9
63.7
63.5
62.3
8.1
5.5
4.1
Table 9: Ablation of the clip length on Long Bide with LLaVA- LLLVAA-Vide-- Video-7B. Uniform denotes naive uniform frame sampling (thus additional GPU hours for keyframe selection are reported). For Focus, we vary the clip length from 8s to 32s and report both QA accuracy and the GPU hours required for keyframe selection. Note that the GPU hours measured single NVIDIA H100 (80GB) GPU.
As shown in Table all clip-length settings of FOCUS significantly outperform uniform sampling (58.9% vs. 62.3-63.7%) indicating that our bandit-basse selection is robust to the choice of over a reasonably wide range. Shorter clips (e.g.. 8s) provide slightly better accuracy by enabling more fine-gne fine-grained exploration, but they also incur higher computational cost, while longer clips (e.g., 32s) reduce GPU hours at the price of a modest performance drop. practice, we find = 16 seconds to offer good trade-off between accuracy and efficiency.
23

---

Published as a conference paper at ICLR 2026
G.4 EFFECT OF VISION-LANGUAGE ENCODER
Our method can be seamlessly integrated with different vision-language encoders to estimate frame- query relevance scores. In the main experiments, adopt BLIP align with our primary baseline AKS for a fair comparison, and also because prior work has shown BLIP to be a robust and effective choice for frame-level relevance estimation. To provide a more comprehensive evaluation, we further conduct experiments with three encoders: CLIP (Radford et al., 2021), SigLIP (Zhai et al., 2023), and BLIP (Li al., 2022).
Uniform
CLIP
SigLIP
BLIP
ACC
58.9
60.2
60.9
63.5
Table 10: Ablation of the vision-language encoder on Long VideoBench with LLaVA-Video-T7 Uniform denotes naive uniform frame sampling. For our method, we instantiate the frame-query scoring module with CLIP, SigLIP, and BLIP.
As summarized in Table 10, all three encoders yield clear improvements over uniform sampling, confirming that our bandit-basee selection is compatible with different rision-language backbones. Among them, BLIP achieves the strongest performance, while CLIP and SigLIP still provide 1.3% and 2.0% gains, respectively. These results suggest that our framework robust to the choice of encoder, but can further benefit from stronger frame-query relevance models, and that future advances in vision-language pretraining are likely to directly translate into better keyframe selection performance.
H LIMITATIONS
In this work, assume the frame-query relevance scores are i.i.d. and the temporal dependencies between frames are not considered. However, practice, the frame-query relevance scores are dependent the temporal dependencies between frames. As different parts may have strong correlations, this assumption may not hold. this setting, we can use the Lipschitz/metric bandit problem (Kleinberg al., 2008; Bubeck et al., 201 1) contextual bandit problem (Chu et al., 201 Agarwal et al., 2014) to model the problem. We leave this as future work.
I THE USE OF LARGE LANGUAGE MODELS (LLMS)
We used GPT-5 and Claude 4 solely for proofreading and light copy-editing (typos, grammar, and minor phrasing). All technical content, scientific claims, mathematical proofs, algorithms, experiment design and execution, dataset handling, figures, and evaluations were authored and verified by the human authors. LLMs were not used generate ideas, code, data, results, or reviews; they did not contribute content at the level of a co-author. All suggested edits were manually inspected and accepted or rejected by the authors.
24


Reviews:
[{"review_id": "hV6E2QZ0Gv", "reviewer": "Reviewer_Ccxf", "summary": "The paper presents FOCUS (Frame-Optimistic Confidence Upper-bound Selection), a training-free and model-agnostic keyframe selection approach designed to improve long-video understanding in multimodal large language models (MLLMs). The authors formulate keyframe selection as a combinatorial pure-exploration problem within a multi-armed bandit framework, leveraging empirical means and Bernstein-style confidence bounds to balance exploration and exploitation. A two-stage coarse-to-fine procedure is further proposed to enable efficient and parallel computation. Experiments on LongVideoBench and Video-MME demonstrate consistent accuracy gains across multiple MLLMs. Overall, the paper provides an efficient and theoretically grounded solution to long-video understanding under tight token budget constraints.", "questions": "As mentioned in the Weaknesses section, I believe additional ablation studies would greatly help clarify the method’s behavior and limitations. In particular, it would be valuable to see how different choices of M (number of clips) or other hyperparameters affect both performance and efficiency.\nAdditionally, it would be interesting to explore how the proposed method performs on spatially focused video understanding tasks, where temporal redundancy is less dominant. Such experiments could provide insights into the generality and potential boundaries of the FOCUS framework.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is original in formulating keyframe selection for long-video understanding as a combinatorial pure-exploration multi-armed bandit problem. This is a novel and reasonable perspective that provides new theoretical and algorithmic insights for researchers working on efficient video representation and token budgeting.\n2. The proposed two-stage coarse-to-fine procedure effectively addresses the non-parallelizable nature of sequential arm-pulling and updating, providing a practical solution that improves efficiency with minimal performance loss. This design is both elegant and empirically effective.\n3. The paper also shows strong theoretical grounding and empirical validation. The theoretical analysis is clear and complete, and the experiments are comprehensive, covering multiple benchmarks and models. The results align well with the theoretical claims, reinforcing the soundness and significance of the contribution.", "weaknesses": "1. In Section 2.2, the paper assumes that “frame-level utility within the same arm share the same distribution.” It is unclear how this assumption is ensured in practice, especially regarding how the M non-overlapping fixed-length clips are partitioned. For instance, when the video contains shot changes or scene transitions, it is not clear how these are handled or whether the authors explored alternative segmentation strategies.\n2. The experiments are limited to LongVideoBench and Video-MME, both of which focus on long-form video QA. Evaluating the method on datasets with different characteristics—such as spatial reasoning benchmarks (e.g., VSI-Bench) or shorter video datasets—would provide a better understanding of the method’s generalizability and potential limitations.\n3. I am curious about how the number of clips (M) affects performance and efficiency. Since the method’s core formulation relies on partitioning videos into M fixed-length clips, an ablation study on M (and possibly related hyperparameters) would make the analysis more complete.\n4. There are a few minor typos (e.g., a missing period around line 025). The authors are encouraged to carefully proofread the paper to minimize such small errors.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jfNJSLOFyv", "reviewer": "Reviewer_9vQp", "summary": "This paper introduces a novel, training-free, and \"\"model-agnostic\"\" algorithm for efficient video processing by sampling the most relevant frames. Addressing the limitations of long videos and the short context windows of many Multimodal Large Language Models (MLLMs), the method leverages principles from Bandits research to select frames without relying on external models, unlike approaches such as AKS. The algorithm employs a coarse-to-fine-grained exploitation strategy, demonstrating improved performance over uniform sampling, top-k selection, and the model-based AKS method across various models (Llava, Qwen, GPT-4o) and benchmarks (Video-MME, LongVideoBench). A key advantage is its significantly higher efficiency and reduced GPU hours, as it avoids additional model inference. The method also includes an adjustable hyper-parameter, alpha, to manage the trade-off between accuracy and computational cost (number of frames processed).", "questions": "Can you clarify the BLIP usage in your method? If that's the case, why BLIP and not another text/vision encoder such as Siglip? Do you have any ablations?", "limitations": "", "rating": 4, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The method seems quiet effective in selecting the frames.\n- Seem to work well across different MLLMs\n- Better accuracy over the AKS method while improving efficiency.", "weaknesses": "- The authors present the method as model-agnostic; however, they appear to leverage BLIP for frame relevance scoring to compute their latent frame-level utility. Even if only 1% of the frames are processed through BLIP, it still relies on a model, making the claim of model-agnosticism questionable. This point should have been better explained in the paper. \n- Lack of comparison with training-based method.\n- Could have added more benchmarks such as MLVU, NextQA. MVBench\n- Typos, abstract \"within each region On two long-video\"", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bKe9xtskRS", "reviewer": "Reviewer_YVCf", "summary": "This paper introduces FOCUS, a training-free and model-agnostic keyframe selection method for long-video understanding with multimodal large language models (MLLMs). FOCUS frames keyframe selection as a Combinatorial Pure Exploration (CPE) problem under a multi-armed bandit (MAB) formulation, treating short temporal clips as “arms.” By applying a Bernstein-style upper confidence bound (UCB-V) and a two-stage coarse-to-fine exploration scheme, the method efficiently identifies query-relevant temporal regions before selecting keyframes within each. Experiments on LongVideoBench and Video-MME, across four MLLMs (GPT-4o, Qwen2-VL, LLaVA-OV, and LLaVA-Video), show that FOCUS achieves 3–6% accuracy improvements over uniform sampling and comparable or slightly higher performance than recent training-free methods (AKS, Top-K), while processing only ~2% of frames.", "questions": "- Include ablation studies disentangling the effect of each module (coarse/fine, α, confidence radius).\n- Compare against stronger or more diverse baselines (Q-Frame, Frame-Voyager, Logic-in-Frames, VSLS, $T^{*}$, Nar-KFC, TimeSearch).\n- Evaluate robustness on longer and more diverse benchmarks, such as MLVU and LVBench (general), Ego4D/ EgoSchema (egocentric reasoning), and OVO-Bench / HLV-1K (reasoning).\n- Analyze failure cases and provide qualitative insights on when FOCUS fails (e.g., multi-event reasoning).\n- Consider extending the method with lightweight learnable scoring or adaptive thresholding to move beyond static heuristics.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- **Conceptually elegant and efficient.** The formulation connects keyframe selection with variance-adaptive exploration in MABs, offering a lightweight theoretical lens for efficient inference. The two-stage batched procedure is practical and easily parallelizable, reducing GPU cost by 40–60% compared to AKS.\n\n- **Training-free and modular.** The pipeline is plug-and-play, requires no fine-tuning, and integrates smoothly into existing LVLM inference workflows.\n\n- **Empirically consistent.** Evaluations span multiple datasets, video lengths, and MLLM architectures, with consistent accuracy and efficiency trade-offs (Table 1–4).\n\n- **Clear presentation.** The paper is well-written with informative figures and pseudocode; Algorithm 2 is easy to follow.", "weaknesses": "- **Incremental novelty.** The “combinatorial pure-exploration bandit” framing is conceptually sound but reuses standard UCB-V principles with minimal adaptation to video reasoning. Similar adaptive sampling ideas have appeared in AKS, Q-Frame, T* and Frame-Voyager.\n\n- **Weak theoretical substance & Limited methodological depth.** The regret bound assumes i.i.d. frame rewards and bounded noise, which do not hold in temporally correlated videos. The theoretical claim does not extend to frame-level optimality, limiting its practical significance. Despite the theoretical motivation, the final algorithm reduces to a fixed two-stage heuristic controlled by α and lacks adaptive or learnable exploration scheduling.\n\n- **Missing core ablations.** The paper does not isolate contributions from key components—coarse vs. fine stages, confidence bounds vs. mean-based selection—so the source of improvement remains unclear.\n\n- **Incomplete experimental comparison.** Evaluation omits recent reasoning-driven or event-centric baselines (e.g., Q-Frame, Logic-in-Frames ...), providing only partial evidence of superiority.\n\n- **Lack of robustness and qualitative analysis.** No systematic study of failure cases, sensitivity to α or clip length, or behavior on complex multi-event queries is provided, leaving generalization uncertain.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BqRvEKngel", "reviewer": "Reviewer_aTVY", "summary": "The paper proposes FOCUS (Frame-Optimistic Confidence Upper-bound Selection), a training-free, model-agnostic keyframe selection method for long video understanding with multimodal LLMs (MLLMs). To address the prohibitive token cost of processing all frames, FOCUS formulates keyframe selection as a combinatorial pure-exploration (CPE) problem in a multi-armed bandit setting, where short temporal clips are treated as arms. It employs a two-stage exploration–exploitation strategy: first using optimistic confidence upper bounds (UCB) to identify promising clips, then selecting top-scoring frames within them. Evaluated on LongVideoBench and Video-MME, FOCUS processes <2% of frames yet achieves consistent gains over uniform sampling and SOTA retrieval-based methods, e.g., +11.9% accuracy on videos >20 minutes. The method is simple, theoretically grounded, and plug-and-play compatible with existing MLLMs", "questions": "- FOCUS first partition the timeline into $M$ non-overlapping fixed-length clips. Does this destroy the spatiotemporal consistency of the video? Which makes it difficult to capture continuous segments with high information density?\n\n- As discussed in the Limitation section, FOCUS assumes that each frame of the video is independent, which is completely contrary to the nature of the video. Does this limit FOCUS’s applicability to short videos?\n\n- Table 1 lacks the experimental results of other frame selection methods, such as AKS and Q-Frame, based on the same baseline.\n\n- The paper provides a visualization of FOCUS's superiority over uniform sampling in Figure 3. It is meaningful to include the negative aspects of FOCUS, which helps readers better understand its limitations.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The paper introduces FOCUS, a training-free keyframe selection method that formulates the task as a combinatorial pure-exploration bandit problem. It partitions the video into clips (arms), uses optimistic confidence upper bounds to identify informative regions with minimal sampling, and then selects top frames within those clips. This two-stage exploration-exploitation strategy enables MLLMs to achieve strong performance on long-video QA benchmarks while processing fewer than 2% of frames. It significantly outperforms uniform sampling and existing retrieval-based methods, especially on videos over 20 minutes.", "weaknesses": "- The concept of ACF and the calculation of $r_t$ in Figure 1 require further explanation, which will help readers better understand the motivation of the paper. \n\n- This pre-filtering process before keyframe selection undermines the goal of identifying the most informative keyframes from all frames. The findings are interesting, but it is not certain whether the two-stage ARM selection proposed in the article will fall into the same limitations.\n\n- Lack of experimental results on MLVU, a commonly used long video understanding benchmark.\n\n- Minor Weaknesses\n  - Line 36: multimodal LLMs (MLLMs) -> multimodal large language models (MLLMs)\n  - Line 107: multimodal LLMs -> MLLMs\n\n[1] Zhou J, Shu Y, Zhao B, et al. Mlvu: A comprehensive benchmark for multi-task long video understanding[J]. arXiv e-prints, 2024: arXiv: 2406.04264.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)