A.I.R.: Enabling Adaptive, Iterative, and Reasoning-based Frame Selection For Video Question Answering
Authors: Yuanhao Zou
Shengji Jin
Andong Deng
Youpeng Zhao
Jun Wang
Chen Chen
Venue: ICLR 2026 Poster (2026)
Keywords: Video Frame Selection
Vision Language Model
Training-Free
Video understanding
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Effectively applying Vision-Language Models (VLMs) to Video Question Answering (VideoQA) hinges on selecting a concise yet comprehensive set of frames, as processing entire videos is computationally infeasible. However, current frame selection methods face a critical trade-off: approaches relying on lightweight similarity models, such as CLIP, often fail to capture the nuances of complex queries, resulting in inaccurate similarity scores that cannot reflect the authentic query-frame relevance, which further undermines frame selection. Meanwhile, methods that leverage a VLM for deeper analysis achieve higher accuracy but incur prohibitive computational costs. To address these limitations, we propose A.I.R., a training-free approach for Adaptive, Iterative, and Reasoning-based frame selection. We leverage a powerful VLM to perform deep, semantic analysis on complex queries, and this analysis is deployed within a cost-effective iterative loop that processes only a small batch of the most high-potential frames at a time. Extensive experiments on various VideoQA benchmarks demonstrate that our approach outperforms existing frame selection methods, significantly boosts the performance of the foundation VLM, and achieves substantial gains in computational efficiency over other VLM-based techniques.

Published as a conference paper at ICLR 2026
A.I.R.: ADAPTIVE, ITERATIVE, AND REASONING- BASED FRAME SELECTION FOR VIDEO QUESTION ANSWERING
Yuanhao Zou'; Shengji Jin"; Andong Deng', Youpeng Zhao', Jun Wang', Chen Chen University of Central Florida, Weill Cornell Medicine
ABSTRACT
Effectively applying Vision-Language Models (VLMs) to Video Question An- swering (VideoQA) hinges selecting concise yet comprehensive set of frames, as processing entire videos computationally infeasible. However, current frame selection methods face critical trade-off:: approaches relying on lightweight similarity models, such as CLIP, often fail to capture the nuances of complex queries, resulting in inaccurate similarity scores that cannot reflect the authentic query-frame relevance, which further undermines frame selection. Meanwhile, methods that leverage a VLM for deeper analysis achieve higher accuracy but incur prohibitive computational costs. To address these limitations, we propose A.I.R., a training-free approach for Adaptive, Iterative, and Reasoning-basd based frame selection. We leverage powerful VLM to perform deep. semantic anal- ysis on complex queries, and this analysis is deployed within a cost-effective it- erative loop that processes only small batch of the most high-potential frames at a time. Extensive experiments on various VideoQA benchmarks demonstrate that our approach outperforms existing frame selection methods, significantly boosts the performance of the foundation VLM, and achieves substantial gains in computational efficiency over other VLM-based techniques. Project Page: https::/ucf-aat wtt.eeffairrgghhaaaae. github.iol
INTRODUCTION
Recent advancements in large Vision-Language Models (VLMs) have revolutionized tasks at the intersection of vision and text (Li et al. 2023 Hurst et al. 2024 Alayrac et al. 2022). Building on their success with static images, the focus has shifted to the more challenging domain of video understanding, which demands model's ability to reason visual narratives. Powerful foundation VL are now being adapted to tackle video understanding
complex temporal dynamics and
tasks like Video Question Answering (VideoQA) (Bai et. 2025 Zhu et al. 2025 [i. (022a))
One critical challenge video understanding is the perception bottleneck. The extensive number of frames in long videos makes processing the entire video computationally infeasible and exceeds the limited context windows of VLMs. To this end, foundation VLMs typically resort to the frame sampling strategy. The most common one uniform sampling, which selects frames at a fixed interval. Although uniform sampling maximizes the temporal coverage of the video, its content- agnostic nature usually selects redundant frames while omitting crucial, query-relevant moments.
To alleviate the issue above, prominent line of research focuses on query-related frame selection. Many of these approaches (Zhang al., 2025 Sun et al.) 22255) Liu al. 2025) Tang et al. 2025) leverage lightweight multi-modal models like CLIP (Radford al.) (021) compute query- frame similarity, which then guides their proposed algorithm for effective selection. However, to answer complex query like 'After introducing Tofu muking, what kind of traditional technique scenic spot did the youtuber introduce according to what is shown in the video? ?' (Fig. [ (a)), which requires temporal reasoning (c.g., 'After') and holistic semantic understanding, a lightweight model lacks
"Equal contribution
correspoondng author, chen. www.wchww..c....c ucw..du.uu..

---

Published as a conference paper at ICLR 2026
(b)
Similarity Score
Query
Query-related
Query
o Relevance
After Intraducing Tata making.
Frame Selection
Zhrenhatd
technique scenic
what kind of traditionaa did the
Lightweight Model
QQuery
CLIP Model
youtuber intraduce sccortling
Top-k
Chonde
what is shown in the video?
CLIP
Query-Framm Frame
Frames
JA, Nakii Kariocut jt Synka Mexomn
Model
Similarity Score
antioleg.
1eg. Casine denderte))
(A +5..
Anowering
Similority:
Tofu
Temple 0.4 Wo
C Video
VLM
Computation Cost Explosion
Large Vision Language Model
Top-
t21 frame:
VLM Inference
Analysic VLM
Relevance
e. Intert/---40
Analysis
-622s
Score
VLM
Uniformll Sampling, From N Rotall Frames
A Answer
?
Query
(a)
125
#hames
Figure I:
An illustration of the general pipeline of query-related frame selection and the key chal- lenges in query-related frame selection. (a) The pipeline features two branches for query-reland frame selection: using a lightweight model (e.g.. CLIP) to produce a similarity score between the query and each frame, or using large Analysis VL.M to generate relevance score. (b) Lightweight models suffer from ambiguous similarity, failing on complex queries. (c) Large VLMs lead to a computation cost explosion scaled with frame number.
these capabilities and instead treats the query as bag of keywords Yuksekgonul et al. 2022 Xie al.] (2025). Consequently, as shown in Fig. [ (b), it incorrectly assigns high similarity scores (score: 0.7) to frame related to the query's s context 'Tofu' or visually similar frame with 'Marker' (score: 0.8). Crucially, the frame containing the correct option 'A Buddhist Temple', is ranked lower (score: 0.4) because it has weaker surface-level association with the query's keywords. Therefore, the lightweight model produces an inaccurate similarity score that fails to reflect the true query- frame relevance, leading to the failure of the selection algorithm.
Taking human perception as an analogy, using a lightweight model (e.g., CLIP) resembles quickly glancing over the video, which may lack sufficient attention and comprehension for complex situ- ations. Other query-related frame selection methods (Hu et al. 20255 Yu et al., 2025] Wang et al. 2025 Ranasinghe al., 2025 Yu et 2023), which employ VLMs as insightful analyzers for each query-frame pair, are more akin to carefully examining each frame. As shown in Fig. aa), given a query-frame pair, an analysis VLM generates relevance score using its strong comprehension of complex queries. An answering VLM is then applied to produce the final answer based on the se- lected frames. However, this deep analysis comes a staggering computational cost. As illustrated in Fig. (c), using Intern VL-3-8B (Zhu et al. 2025) as an Analysis VLM can take more than one second to process a single frame. For methods that require analyzing an initial set of 128 frames (Hu et e 2025 Wang et al. 2025), ) this leads to prohibitively computational cost explosion (i.e., 162 seconds). Therefore, while VL.--asedd analysis methods are highly effective, their computational cost renders them impractical for many real-world applications.
To address these limitations (poor performance of lightweight models on complex queries and high computational cost of VLM-based analysis), we introduce A.I.R., training-free framework for Adaptive, Iterative, and Reasoning- based frame selection that operates in two stages. In the first stage, it adaptively identifies query-relevant events (i.e., temporal regions) based on the unique dis- tribution of query-frame similarity scores of each video. From these events, it samples an adaptive number of frames per event duration, yielding a high-relevance initial frame set. In the second stage, A.I.R. executes an efficient iterative loop that adaptively allocates VLM computation: in each iteration, it ranks all candidates and forwards only a small batch of the high-potential frames for deep. reasoning- based analysis. Frames validated by the VLM then trigger a localized search for additional frames in their temporal neighborhood, uncovering key frames initially assigned with low similarity scores. This synergistic feedback loop, coupled with an Early Stop mechanism, en- ables R. to converge on the most relevant video segments while requiring only fraction of the computational cost of conventional VLM-based approaches. Our contributions are threefold:
We introduce Adaptive Initial Sampling that moves beyond the uniform sampling. It dynamically identifies and samples candidate frames around potential events based on query-frame similarity and controls the output frames via an adaptive budget, robustly handling videos of varied length.
2

---

Published as a conference paper at ICLR 2026
We propose novel Iterative Frame Selection algorithm that makes deep VLM analysis computa- tionally tractable, distinguishing itself from prior methods that rely on a computationally expen- sive, single-pass analysis over large, fixed frame sets.
Experiments prove that our method can be huug-nnd-play with diverse foundation VLMs while achieving substantially higher efficiency and accuracy than existing VLM analysis-based methods.
RELATED WORKS
Vision Language Models for Video Question Answering. The paradigm for Video Question An- swering (VideoQA) has shifted from early CNN-RNN architectures (Donahue et al. 2015 Venu- gopalan et al., 2015) to modern Vision-Language Models (VLMs) (Alayrac et al., 2022 Li et al. 2021 Liu et al., 2023b a). Leveraging the extensive knowledge and reasoning skills acquired during pre- pre-training, these models have spurred two main research directions. The first focuses on models specifically adapted for the video modality, such Video-LLaVa (Lin et al. 2023) and Intern- Video (Wang et al., 2022). A second, more recent trend emphasizes general-purpose VLMs that can process both images and videos, demonstrating strong zero-shot and adaptive capabilities across var- ious tasks. Aligning with this latter direction, our work utilizes a single, powerful VLM-selected
from VILA (Lin et al., ) QwenVL (Bai al. 2025 Wang al. 2024a). LLaVA-One (Li et al. 2024a), and Intern 3L33 (Zhu et 2025 to perform both une-grained analysis of individual
frames and high-level video question answering.
Query-Related Frame Selection via Lightweight Models. A prominent line of research leverages lightweight models like CLIP (Radford et al.) 2021) to efficiently compute query-
similarity scores. Building on this foundation, various methods have been proposed: BOLT (Liu et al.) 2025) applies inverse transform sampling for diversity. MDP3 (Sun al., (025) models the task as a Markov Decision Process, Q-Frame (Zhang et al. 2025) processes frames with dynamic resolutions, and AKS (Tang et al. 2025) introduces an adaptive "Split & Judge" strategy. T (Ye et al., (025) applies an iterative temporal search strategy for frame sampling, which first analyzes the objects in the query and then employs an object detector to find frames with relevant grounded objects in each iteration. However, all these methods depend solely models for their selection logic. In contrast, R. adopts more sophisticated, two-stage approach. We strategically utilize CLIP only for an initial, coarse gaasee-rrained sampling, while reserving powerful VLM for fine-grained, reasoming- reasoning-based analysis of complex queries.
the superficial output scores of lightweight
Query-Related Frame Selection via VLMs. A second branch of methods utilizes powerful VLMs for a deep. semantic analysis of the query-frame relationship. Within this branch, training-free methods like VideoTree (Wang et al. 2025) use online proprictary models (e.g.. GPT-4) to gen- erate rich captions for guidance, while others like MVU (Ranasinghe et al., 2025). LLoVi (Zhang al. 2023), and VideoAgent (Wang et al. 2024b) employ various agents for a comprehensive and reflective analysis. Alternatively, training- traiedng-based methods such as Frame-Vor Voyager (Yu et al., (025) fine-tune a VLM to select frames, while SeVi (Yu et al. (003) jointly trains a dedicated localizer and answerer. While effective, these VLM-based methods often perform computationally ex- pensive, single-pass analysis over a large set of frames. A.L.R. overcomes this limitation through a novel iterative loop that makes deep VLM analysis tractable, adaptively allocating computation by analyzing only small, nigh-potentian batches of frames in each step.
METHOD
3.1 OVERVIEW
As illustrated in Fig. 2 our proposed approach, A.L.R., performs frame selection in three stages: Adaptive Initial Sampling, Iterative Frame Selection, and QA Stage. The process begins by sampling n frames from the video (containing N total frames) at fixed frame rate. As a pre-processing step, these frames are passed to a CLIP model (Radford et al. (02I) compute query-frame similarity scores, which is stored as a sparse vector S E I This similarity signal S is the input to the Adaptive Initial Sampling stage (Sec. 3.2 which identifies initial set of K high-potential frame
'The vector S E R- is sparse only the sampled entries have computed values; all other entries with no values are marked with NaN and ignored in subsequent operations. Values of S will be updated (Alg. D)
3

---

Published as a conference paper at ICLR 2026
Serforr Frorree
Initia Sampled
C
Iterative Frame Selection($3.3)
Answer
Fere
=lle
1. Interval Potential Ranking
R
Video
C Adaptive Initial
Inas
Loog
Reasoning- Bosse VLM Analysis
Arswering
Sampling ($3.2)
3. Early Stop Mechanism
VLM
Query
Query
Localized Density Sampling
QA Stage
Figure 2: General pipeline of A.I.R. with three stages: (1) Adaptive Initial Sampling that identifies potential 'events' based on query similarity and dynamically samples frames around them using an
adaptive budget; (2) Iterative Frame Selection that progressively refines the frame selection via four steps; and (3) QA Stage that feeds the final selected frames into Answering VLM.
indices, Fiitial. Subsequently, the Iterative Frame Selection stage (Sec. 3.3
Alg. T progressively
refines this initial set through a four-step loop. This process yields a final, optimized set of frames, Final- The last one, QA Stage, utilizes an Answering VLM and these selected frames for a one-time inference. Notably, the final number of selected frames, | B, is not fixed but is determined by an adaptive budget that scales with the video's 's length (see A.2.1) Finally, we provide theoretical analysis of our method's ss efficiency in Sec. 3.4
3.2 ADAPTIVE INITIAL SAMPLING
As shown in Fig. 3 (a), with the initial n. frames and the query, unlike other frame selection methods (Liu et al. 2025| Sun et al. 2025 Tang et al., 2025) that directly work on uniformly sampled frames with their proposed approaches, we perform query-related frames in advance (K < n)), which not only provides prior guidance, but also reduces the computation cost for subsequent Iterative Frame Selection in Sec. 333 To achieve this goal, we must first adaptively separate high-relevance frames from low-relevance ones. Hence, we propose an adaptive threshold T, inspired by Gaussian Mixture Models (GMMs) (Huang & Chau 2008 Zhao et al., 2019). Specifically, we hypothesize that the similarity scores S are drawn from a mixture of two underlying distributions: a high-relevance cluster and low-relevance one. We fit a GMM with two components to model these two clusters. Let the means and standard deviations of them be (11, 01) and (12, 02). The threshold
Adaptive Initial Sampling. This process selects K
calculated as:
T = max (141, (2) - 7 max(o1.
where y is a hyperparameter that controls the stringency of the threshold. This formulation ensures the threshold T is adaptive as it is dynamically computed for each video based on that video's s unique distribution of similarity scores. Using the adaptive threshold T, we identify an initial set of candidate events g', An event defined as any maximal, contiguous temporal region in the video where all similarity scores are at or above the threshold T'. The events set E' is formalized as:
='= 11stan pond Vi E
where E represents the j-th event via its start and end frame index. This initial segmentation can be noisy, sometimes splitting a single action into multiple events or creating very short segments. Therefore, we refine E' with two heuristic-based steps to produce final set of validated events E: (1) Merging: Any two consecutive events separated by duration less than a minimum distance gend" < dmii) are merged into a single, more coherent event; and (2) Pruning: Any event with total duration less than a minimum length (i.e., 1gnd
(i.e., [+11
1gtart < (minn then removed.
Finally, with a clean set of refined events ε, we perform Event-Wise Sampling to select the K initial candidate frames. Our sampling strategy is guided by two key principles, as shown in Fig. 3 (a): (1) Comprehensive coverage: every identified event is represented by at least one frame; and (2) Proportional allocation: longer, more sustained events are allocated larger portion of the sampling budget. To satisfy these principles, we first calculate the number of frames (kz)) to sample from each event based on its relative duration, ensuring that k, 2 1. We then select ky peak frames with the highest pre-computed similarity scores from within that event's boundaries. This proportional strategy ensures that the most significant events are more thoroughly represented. The detailed formulation for computing ky can be found in A.2.2 This process yields initial sampling set Finitial = (fi..... fx }, where ff, E Fiitial represents frame index and K < n. This initial frame set is the input for the subsequent lterative Frame Selection.
4

---

Published as a conference paper at ICLR 2026
(a) Adaptive Initial Sampling
(b) Iterative Frame Selection
/ /
Similarity S 'computed CUP)
1. Interval Potential Ranking
US UN
i CRIN Cuotetts
238
Gouesian
Miciure
let
(c0mponent-22 201
Adartive Thenndd le To
Mf Choter
Q
Reasoning based VLM Analysis
Top C Cand Simppies Fravey F Horked trjevvnn))
Gurvetariion
Early Stop Mechanism
Sinilariyy Soann
Dvens Segraemaiion
Analysn VLM
Instruction
Agenggied
VLM
saag sheut persor
wha.aaaooooon ao tomgle
cade qatk. Reppinoeee the Aloe bo..
Fando wor Swepling
making rigit colan
Fruume
Sm
D
Lacallood Danuitt
0g 4. Localized Density Sampling
wandd Ff
At Leart Thame per Enert More Framnss longer Everts
Loop back
Sumpled Framees-
Figure 3: Two main stages in our A.I.R.. (a) Adaptive Initial Sampling: A GMM-based adaptive threshold is applied the query-frame similarity S to identify potential events, and then event-wise sampling is conducted on the refined events to obtain K frames (Fimitaa). (b) Iterative Frame Se- lection: In each iteration, 1) High-potential candidates are selected via Interval Potential Ranking; 2) A VI.M performs caasoning-bssed analysis to validate the best frames; 3) An Early Stop mecha- nism checks if the frame budget is met; And 4) if not met, the Localized Density Sampling (LDS) discovers more frames around the validated frames and feed them into the next iteration. Notably, LDS is performed on the original video (N frames) instead of the uniformly sampled n frames.
3.3 ITERATIVE FRAME SELECTION
While Adaptive Initial Sampling provides a strong set of K candidate frames, analyzing all of them with a powerful VLM remains computationally expensive, as K often scales with the initial sample size n. Therefore, we introduce the core of our framework: Iterative Frame Selection. The goal of this stage is to progressively refine the candidate set using an Analysis VLM in a cost-effective manner. This is achieved through synergistic four-step loop (see Fig. (b) and Alg. [ In each iteration, (1) Interval Potential Ranking first identifies the high-potential candidate frames. (2) The candidate frames are then evaluated by the Reasoning-Bass Based VLM Analysis, which generates a relevance score and a textual justification for each, allowing us to retain a validated set of truly relevant frames. (3) The Early Stop Mechanism checks if the adaptive sampling budget 4A.2.1
has been met; if so, the process terminates efficiently. (4) If the budget is not yet met, Localized Density Sampling discovers new, fine-grained frames in the vicinity of the validated frames from (2), which are then fed back into the candidate pool for the next iteration. This cycle of prioritizing, analyzing, and exploring continues until reaching a maximum number of iterations or an early stop is triggered.
Step 1: Interval Potential Ranking. Given a set of sampled frames F. (initially, J= Finitall (FI) from Adaptive Initial Sampling, the primary role of Interval Potential Ranking is select a small batch of C' high-potential candidates for the Analysis VLM in each iteration. Instead of ranking individual frames by their raw similarity scores, we propose ranking the temporal intervals between these frames. This interval-based approach is more robust as it considers the collective evidence within temporal region, providing a more comprehensive signal of potential event than a single frame's s score. As illustrated in the first step of Fig. (b), the process begins by partitioning the entire original video into a set of disjoint temporal intervals. These intervals are defined by the indices of the current sampled frames in F. A given interval I, is formally defined as:
= [fc, f+1l = (j N f. S S f411.
(3)
To ensure the entire video is covered, this partitioning also includes the segments from the start of the video the first selected frame (i.e.. [1, fi], and from the last selected frame to the video's 's end (i.c., [fx, N))
5

---

Published as a conference paper at ICLR 2026
For each interval, we calculate its potential a score indicating its importance to the query based on its corresponding slice of the pre- creppoepteed similarity signal, S Seff-- Inspired by signal process- ing principles (Boreczky & Rowe 1996, Wolf [996] Liu et al., 2003), this potential is computed as a product of three factors: Relevance, Complexity and Length (see A.2.4 for details). For a discrete
similarity signal S, the potential of the interval is calculated as:
J= 1S SS S411
fi+
(o)) = Mean(S
1+
(1 Geen lg(fi+1 - fill, (4)
Si
Relevance
Longth
Complexity
where Glen is hyperparameter that balances the influence of the Length. After ranking all intervals by their potential scores, we select the C' candidate frames from the highest-ranked intervals (e.g., selected fi, fit1 from interval I,) as candidate set Feaud for the subsequent VLM analysis.
Step 2: Reasoning-Basee VLM Analysis. Following the Potential Interval Ranking, the C selected frames Feand are analyzed by Analysis VLM for focused, reasoning- based evaluation. We leverage the zero-shot, instluctioo--ollovving capabilities of foundation VLMS to assess the relevance of each frame quantitatively. Guided by detailed prompt (see Fig. 3 (b) and A.2.5), the VLM is instructed to reason step-by-step, providing both a textual justification and a relevance score (c.g., an integer from 1 5) for each candidate frame. Based on the relationship to a predefined threshold 0, these scores are classified as 'Positive' (> f). 'Neutral' (= f), or 'Negative' (< 0) and collected into a vector R E NC. We retain the "Positive" frames to form validated frame set F as:
F {fa E Feand| R, > 0}.
(S)
A fallback mechanism is implemented to handle the case where not enough frames are positively validated (i.e., IF* B/Imax]] Imax is the maximum iterations). In this scenario, instead select [B| [B/mmax] - IF* more frames from the candidate frame set Feand- The selection is based on two-tiered priority system: candidates are first grouped by their VLM rating, with frames rated as 'Neutral' given the highest priority, followed by the 'Negative' frames. Within each group, the pre-computed similarity score of each frame is then used to determine the final selection order. We then aggregate the validated frames to a cumulative final selection set, F
Step 3: Early Stop Mechanism. After each round of VLM analysis, we update the cumulative final selection set Final To maximize efficiency and prevent unnecessary computation, an Early Stop Mechanism is then immediately triggered. We check if the total number of selected frames has met or exceeded the adaptive sampling budget ((Finall 2 B). As illustrated in Fig. (b), if the budget is fulfilled, the iterative loop halts, saving all subsequent VL.M analysis costs. If the budget not yet met, the process continues to the Localized Density Sampling to sample additional frames.
Step 4: Localized Density Sampling (LDS). As illustrated in Fig. 3 (b), if the iterative process has not yet met its sampling budget after the Early Stop Mechanism, proceeds to this final step to discover more candidate frames. Our strategy is motivated by the principle of temporal coherence: the most valuable, undiscovered information is concentrated in the temporal vicinity of the frames VLM has just validated (i.c., F*). We therefore propose LDS, a search strategy that samples new frames from the original high-frame rate vidco (from the total N frames instead of n2), allowing it to capture more fine-graime une-grained moments missed by the Adaptive Initial Sampling. LDS employs an exponentially growing sampling stride. This design balances two objectives: it performs dense, high-resolution search immediately around a validated frame to find precise details, while efficiently exploring the broader context with increasingly sparse samples. For each validated frame fü F new sampled frames generated by LDS are formalized as:
LDS(F.) { roo a 3(m=1)) | 10mm=1 = 1,2, 1.,,..., ........
(6)
where a is the initial stride, / > 1 controls the stride's exponential growth, and D determines the number of frames sampled, which is proportional to the remaining budget, B |Fimall- Crucially, these newly discovered frames are not added directly to the final selection. Instead, they are fed back into the start of the loop. Their query-frame similarity scores are computed to update the signal S, and they are added to the candidate pool F (now K is set to (FD) for the next iteration's Interval Potential Ranking. For instance, exemplified by the process in Fig. (b), frame (with "Tofu') rated as 'Positive" in one iteration can trigger a localized search that uncovers the definitive, answer-providing frame (i.e., the frame with Buddhist Temple') ') via LDS.

---

Published as a conference paper at ICLR 2026
3.4 EFFICIENCY ANALYSIS
We analyze the core computational efficiency of A.I.R. by focusing on the primary bottleneck: the number of frames processed by Analysis VLMs (not Answering VLMs). Conventional VLM-based analysis methods (Hu et al. 2025 Yu et al. 2025 Wang et al. 2025) uniformly sample a large, fixed set of base frames (c.g.. 128 frames) and perform VLM inference on all of them. The total VLM workload for such a method is therefore Taase frames. In contrast, A.I.R. uses a targeted, iterative strategy. each iteration, the VLM analyzes only a small batch of C' candidate frames. Due to our Early Stop Mechanism, the total number of frames that undergo VLM analysis, AAA.RR.. is bounded. In the best-case scenario, the process stops after one iteration, analyzing only C' frames, while in the worst-case, it runs for the maximum of Imax iterations. Thus, the VLM workload R...R. is strictly constrained by the best workload Wbest and the worst workload Wworst as:
Wbest S D.A.RRR. I Wwarst:
where Whest = C. and Wworst: = -mmx- Our hyperparameter setting (e.g., C = 12, Imax = 6) ensures our worst-case VIM workload is significantly smaller than conventional methods that analyze a large, fixed number of frames (i.e.. Wworst < Tibase = 128). Furthermore, A.I.R. offers a more intelligent trade-off 3 than methods that achieve efficiency with small, fixed budget (e.g., Aaase is 16 or 32, Yu et (l. 2023);
et al. (2025); Wang et al. (2024b)). The key advantage of our framework its adaptivity: the VLM workload only increases as demanded by the video's length (see Tab. 13) and is bounded by Whest and Wworst- This adaptivity allows our method to be more computationally efficien than these fixed-budget approaches (Tab. 7 and Tab. 6
4 EXPERIMENTS
4.1 IMPLEMENTATIONS
We use 4 widely-used foundation VLMs as our backbones: VILA-L.5-B Lin et al. 2024). WW-255-22557BB (Bai et al., 2025), InternVL-3-8B (Zhu et al. 2025), and LnnaVA--OeVisioon (Li et al., 2024a). We use the same VLM for analysis and answering. Additionally, we employ EVA-CLiP-L (Sun et al., ) to compute the query-frame similarity score S. We evaluate A.L.R.
on various long video benchmarks, i.e. Video-MME (Fu al. 2025). MIVU (Zhou et al., 2025)
and Long VideoBench (LVB, Wu et al. (2024). Besides, we also employ short video benchmarks EgoSchema (Mangalam et al. (023) and NextQA (Xiao et al.] (02T). More details are in A.3] To ensure a fair comparison with other competing frame selection methods, in Tab. D and we re- evaluated methods with available code (f) in our controlled environment (Imms-eval, Zhang et al. (024a)), while for others, we compare against their reported metrics (*) in similar settings.
4.2 COMPARISON WITH THE OFATE-OF-THE-ART
We conduct a comprehensive evaluation to validate the effectiveness of A.I.R. across diverse sce- narios, with results presented for long-vide video benchmarks in Tab. [ and viirt benchmarks in Tab. 2 To demonstrate the superiority of our approach, we benchmark it against two key categories of methods: foundation VLM baselines, where powerful VLMs use the uniform sampling strategy, and competing state-of-the-art (SoTTA) frame selection methods. Our extensive experiments lead to the following key findings: (1) Powerful Plug-and-Play Enhancement: As a model-agnostic, training-free ffre module, A.I.R.'s 's benefits generalize across diverse VLMs. For example, on NexIQA, applying our method to QwenVL-22.5 results in a massive +7.0 accuracy boost. Similarly, it also pro- vides substantial gains for VILA-1155 LLaVA-One
and InternVL3, confirming its versatility. (2) So Accuracy with Superior Efficiency. A.I.R. elevates foundation VLMs to new SOTA levels while being more frame-efficient. For instance, when paired with VL-3 on LVB bench- mark, our method achieves a +4.5% absolute gain over the baseline, while analyzing fewer frames on average than the fixed budgets of competitors (i.e., < 32 vs. 32). (3) Robust Performance Across Diverse Benchmarks: While the impact of intelligent frame selection is most critical for
3 We validate this via hyperparameter ablations (i.c., candidate frames and max iterations Imaxx in A.4.5 's shown in Tab. 7 the time cost of other operations, such as initial sampling and frame selection of our method, minor compared to VL. inference time, we ignore their theoretical efficiency analysis.
7

---

Published as a conference paper at ICLR 2026
Table 1: Comparison of VLMs and various frame selection methods on Video-MME, MLVU, and LongVideo Bench. denotes reported results, while i means reproduced ones (see Sec.
Model
L.M Size
#Frames
Video-MME
MLVU..v
LVB
wlo sub.
Troinine-Based Foundation
LLaVA-One [L 2024n
7B
32"
58.2
61.5
64.7
56.4
QwenV1.22.5 (Rai el al 2025
7B
max: 768
65.1
71.6
70.2
45.3
InternVL.3'
Zhn
8B
66.3
68.9
71.4
58.8
VILA-115'
47.1
Fair Comparison with Frame Selection Methods
VILA-115
8B
8
48.9
54.2
44.7
47.9
+Frame- *Frame-Vovager" Yu al 2025
8B
8
50.5
53.6
49.8
+MDP3' et
+Q-Frame" (7hang et al. 2025
8B
8
50.7
55.0
54,4
51.6
+Ours
53.7
58.6
54.2
52.9
QwenVL-2.5'
7B
32
60.8
62.7
59.3
58.1
+MDP3
7B
32
63.8
65.7
66.2
60.0
+Ours
65.0
67.5
InternVL-3!
8B
32
65.6
67.3
68.4
58.3
+MDP3
32
74.0
+Ours
8B
32
68.2
69.2
74,5
62.8
LLLV/ OneVision
7B
32
$8.5
61.7
62.4
56.6
*AKS' (Tnng al 2025
7B
32
58.4
59.3
MMP3
7B
32
60.5
64.0
68.3
59.0
+Ours
78
32
61.4
65.1
69.3
60.7
Table 2: Comparison of VLMs and frame selection methods on Egoschema and NextQA.
Model
VLM Size
#Frames
Egoschema
NexIQA
Full
Subset
VIM anatyyit-husedd Frome Selection Methods
LLoVi (Zhang
GPT3355 GPT4
0.5FPS IFPS 32
52.2 54.1
VideoAgent (Wang el 2024b
60.2
71.3
Hu et al. 2024
8.5B
65.9
78.4
SeVILA Yu 2023
4.1B
VideoTres Wan' ar
GPT4
61.1
MVU* Raniisnche ccal 2025
13B
16
37.6
60.3
$5.2
DrVideo Iit ai. 20233
GPT4
61.0
66.4
T" et 2025
7B
66.6
80.4
Fair Comparison with Frame Selection Methods
VILA-1.5
8B
49.5
52.8
+Frame-Voryy Voyager"
8B
53.6
67.3
+MDP3'
8B
48.5
$1.0
66.1
+Ours
8B
50.7
53.6
70.3
QwenVL-2255
B
32
37.6
59.4
74.3
+MDP3'
7B
32
56.8
61.6
74.4
+Oury
7B
<.32
58.8
62.4
81.3
In-ernVL-33
8B
62.5
71.6
82.3
+MDP3'
8B
32
61.6
70.0
82.3
+Ours
8B
<.32
63.3
72.2
82.6
LLaVA-OmeVisionn
7B
+MDP3'
7B
32
60.3
60.8
78.9
*BOLT"
7B
32
60.7
64.0
79.5
7B
long videos (Tab. .. A.I.R. also proves effective on shorter video datasets., while also outperforms
competing methods on short-video benchmarks (Tab. 2 More analysis is provided in A.4.1
4.3 ABLATION STUDY
Influence of Sampled Frame Number: We investigate how the performance of our approach scales with an increased maximum sampling budget (the upper bound of B, see A.2.IT. To this end, we adjust the adaptive sampling budget by modifying the maximum frame limit. As illustrated in Fig our approach consistently outperforms the uniform sampling method on VLMs such as Qwen 2.5 and LLaVA-One Vision. Furthermore, when compared to CLIP-based query-awarg sampling methods like BOLT (Liu et al. 2025), our approach shows an average performance advantage of approximately 1.5%. More detailed ablation study is in Tab. T and A.4.3
Influence of Different VLM Scales. As shown in Tab. 3 the results of various scales of VLMs reveal two key findings: (1) A.I.R. provides consistent performance uplift across all tested scales of the QWenVL-2.5 models (7B, 32B, and 72B), confirming the robustness of our method; and (2) We observe that the accuracy gain is most pronounced on the smaller 7B model (+4.2%), while the
8

---

Published as a conference paper at ICLR 2026
Table 3: Comparison of different VLM scales with A.I.R. on Video-MME (w/o subtitle, 32 frames).
QwenNV-2.5-7B
-30
Protien (1o
(527 Cones
+12
694%
62
Model
Acc.
b4
250
750
Llava- Oniionnne
QwenVL-2.5-7B'
60.8
+Ours
65.0
TARN (330
Preperr
OUR
satne Sampling
QwenVL-2.5-32B
63.7
56
+18
+Ours
66.2
QwenVL-2.5-728
67.0
Cantl
Figure 4: Accuracy comparison on 6 question types of MME (w/o
Figure 5: Accuracy vs. varying sampled frame numbers Video-MME (w/o
+Ours
68.2
GPT-4u
61.8
sub., 32 frames) using InternVI VL.3-8B.
subtile) using various foundation VL.Ms.
+Ours
65.1
Table 4: Ablations of A.I.R.' 's components on Video- MME using Intern VL3-8B. We compare on average
Table 5: Comparison of various CLIP models with A.I.R. (using InternVL-3-SB
frames for answering VI.MS and accuracy (Ace.).
on Video-MME (w/o subtitle) with max sampling budget of 32 frames).
#
Method
Avg. Frames
Acc.
Uniffrmm Sampling
12.0
65.6
2 A.Z.R. (Our method) methoe))
24.8
68.2
Acc.
Fixed 32
Budget)
Uniform Sampling (32 frames)
65.6
4 + w/o Adaptive Similarity Thresholding
25.5
67.3
2
A.I.R. + CLIP-VIT-B
66.8
Adaptive Initial Sampling
25.1
66.9
A.L.R. EVA-CLIP-L
68.2
> w/o Iterative Frume Selection (32 Frames)
32.0
65.2
A.I.R. + LongCI.PP-..
67.4
7 < w/o Interval Potential Ranking
26.2
66.7
A.I.R VIT--
67.8
Reasoning based
Analysis
32.0
66.0
A.I.R. SigLIP-large
67.1
9 -+ w/o Lucalized Density Sampling
24.5
67.2
margin narrows as the base VLM becomes more powerful. This suggests that our frame selection
is most critical for smaller models with a weaker intrinsic ability to discern relevance. We fur- ther validate the scalability of our plug-and-play framework on state-of-the-arn models like GPT-4o, demonstrating consistent performance gains even when scaling to extremely large VLMs.
Ablation of Components in A. (More Analysis in A.4.4). We showcase the effectiveness of the individual components of A.I.R. in Tab. 4 and Tab. 5 The key findings are: (1) Our method signif- icantly outperforms the uniform sampling baseline, while achieving trade-off between efficiency and accuracy compared to our fixed 32 frames version (Tab. 4. (2) Ablating core components, es- pecially the Reasoning-basee VLM Analysis, consistently degrades performance, confirming their synergistic contribution (Tab. 4.. (3) Our framework is robust to the choice of vision encoder, outperforming the baseline across all five CLIP variants that other methods used, and we selected EVA-CLIP-L (uun al.
2023 our default due its superior performance (Tab. 5
Experimental Efficiency Analysis (More Analysis in A.4.5).
The results in Tab. 6 highlight A.I.R''s 's superior efficiency-perfommene performance trade-off against compet- ing methods on NexiQA. When we set the worst workload Wworst to 72, which controls the max frames analyzed by VLM, our method not only achieves SoTA accuracy of 82.6% but does so by adaptively analyzing only 32.2 frames on average. Even with a low worst workload (1worst=16), A.L.R. still attains an impressive 81.7% accuracy with just 12.4 frames, again outperforming other methods. Tab. 7 further demonstrates A.I.R.'s 's computational efficiency on Video-MME. Tab, 7 illustrates the controlled time comparison of A.I.R. against our own baseline, Direct VLM Anal- ysis. which mimics conventional fixed-budge manner that other methods apply using the same VLM VinterVV3-383
For the intermediate frame analysis (VLM Analysis Time), the baseline re- quires 162.03s to process 128 frames, while A.I.R. takes just 42.31s by adaptively processing only 36.5 frames via Iterative Frame Selection. Beyond this VIM Analysis Time comparison, the com- putational overhead of our frame Adaptive Initial Sampling and Iterative Frame Selection stage is minimal, adding up to 0.21s compared to the uniform sampling baseline.
Generalization to Grounding Task Analysis. To demonstrate that A.I.R.'s adaptive frame lection generalizes beyond VideoQA, we evaluate its performance on Charades-STT (Gao et al.) 2017). challenging temporal grounding benchmark where models must localize specific moments in videos based on natural language queries. As shown in Tab. [8 our training- free method achieves strong performance across all metrics: 59.5% R1 @.3, 39.5% R1 @0.5. 18.0% R1 @0.7, and 38.8%

---

Published as a conference paper at ICLR 2026
Table 6: Comparison VLM analysis-
Table 7: Efficiency comparison on Video-MME. En-
based frame selection methods Nex-
tries with stage names like '(QA Stage)' denote the time cost for these stages, regardless of any additional
IQA. #Analyzed means the number of
frames through VLM Analysis. "TF" de-
VLM analysis time. VLM Analysis Time the time
notes whether Sraining-free or not. We use
used for analyzing all frames intermediately.
Intern BeernVL3--B. The same applies to Tab. 7
Method
#Analyzed
Time(s)
Method
#Analyzed
TF Acc.
Baseline (Uniform Sampling 32 frames)
0.87
A.L.R. (QA Stage)
0.81
Frume- Vovager
Yu et al.
2025
128
X
67.3
A.I.R. (Adaplive Initial Sampling)
0.03
Hu et
2025
128
78.4
A.I.R. (Iterative Frame Sclection)
0.18
Video Froe Wang ct al
2025
128
75.6
VLM Analusts Time
VideaAAeen [ang
Analysis
A.I.R. (Mlaouee
32,2
82.6
A.R.R (wivora 72)
36,5
42.31
/
73.8
A.L.R. Ulisoneer 32)
Direct VLM Analysis
20.3
42.47
21.92
MVU (Ransssinghe
2025
16
55.2
Direct VLM Analysis
A.I.R. UNiwore
16J
12.4
81.7
A.I.R. (Miworrs
16)
14.1
14.61
Table 8: Generalization results temporal grounding benchmark Charades-STA (Gao et al. 2017).
Model
0.33
RI@0.5
RI @1007.7
mloU
Truined Temporal Grounding VideoLLMs
VTimeLLMM (Huang et ul. 2024
$1.0
27.5
11.4
31.2
HawkEye (Wang kly 20240
50.6
31.4
14.5
33.7
ThmeChat (Ren èl ma 22224
32.2
13.4
30.6
TumeSuite Zeng
20223
69.9
48.7
24.0
TRACE (Gho
2024
40.3
19.4
General VideoLLMs
GPT-4o (Hurst al. 2024
55.0
32.0
11.5
35.4
Qwen2. V-V 7B fhater er 2025
44.5
30.3
15.2
30.1
VA--BB DB-DOO (Zhang ala 2024b
22.6
10.1
2.2
14.6
Aria (Li et al. (20246)
39.0
18.6
6.6
26.7
Trained Frame Selection Methods GenS Yao et 2002)]
62.9
38.7
15.2
38.0
Training-Free Free Frame Selection Methods
A.I.R. (Qwen2.5-VL.-7)
59.5
39.5
18.0
38.8
mloU. Notably, A.I.R. substantially outperforms general-purposs VideoLLMs like Qwen (44.5% 59.5% RI @0.3) and even surpasses GPT-4o (32.0% vs. 39.5% RI @0.5). More impres- sively, our method achieves comparable or superior results to GenS (Yao et (025), a trained frame selection method specifically designed for grounding tasks, while remaining completely training- free. While specialized temporal grounding models like TimeSuite (Zeng et al. 2024) achieve higher performance through task-specific training, our results demonstrate that A.I.R.'s adaptive sampling and iterative refinement effectively identify temporally relevant segments without requir- ing any grounding-spectio supervision. This validates the broad applicability of our approach across diverse video understanding tasks.
CONCLUSIONS
In this paper, we introduced A.I.R., a training-free frame selection approach, which addresses the challenge of efficient and accurate frame selection for Video Question Answering. By employing an Adaptive Initial Sampling stage followed by an iterative, reasoning-driven VLM analysis, A.L.R. intelligently focuses computational resources on the most salient temporal regions. Our extensive experiments demonstrate that this approach not only significantly enhances the performance off- the-shelf VLMs on complex benchmarks but also achieves this with substantially less computational cost than conventional VLM-based analysis methods. By balancing high accuracy with practical efficiency, A.I.R. offers promising path for deploying powerful VLMs in real-world, long-video understanding applications.
REPRODUCIBILITY STATEMENT
We provide the full details for reproducing our method in the main paper. All datasets used for experiments publicly available. The partial code of method, including the code for two core stages (i.c., Adaptive Initial Sampling and Iterative Frame Selection), will be provided in the supplementary materials.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work used Delta at UIUC NCSA through allocation CIS250367 and 250473 from the Advanced Cyberinfrastructure Coordination Ecosystem: Services & Support (ACCESS) program, which is supported by U.S. NSF grants 2138259, 2138286, 21 2138307, 2137603, and 2138296.
REFERENCES
Cean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, lain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: visual language model for few-shot learning. Advances in neural information processing systems, 35:23716-
23736, 2022.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv: 13923, 2025.
John S Boreczky and Lawrence A Rowe. Comparison of video shot boundary detection techniques. Journal of Electronic Imaging, 5(2): 1996.
Jeffrey Donahue, Lisa Anne Hendricks, Sergio Guadarrama, Marcus Rohrbach, Subhashini Venu- gopalan, Kate Saenko, and Trevor Darrell. Long-term recurrent convolutional networks for visual recognition and description. In Proceedings of the IEEE conference on computer vision and pat- tern recognition, pp. 2625-26344 2015.
Sunqi Fan, Hao Guo, and Shuojin Yang. Agentic keyframe search for video question answer- ing. arXiv preprint 250. 2025.
Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Video-mme: The first-ever comprehensive evalu- ation benchmark of multi-modal Ilms video analysis. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp.
2025.
Jiyang Gao, Chen Sun, Zhenheng Yang, and Ram Nevatia. Tall: Temporal activity localization via language query. In Proceedings of the IEEE international conference on computer vision, pp- 5267-5275, 2017.
Yongxin Guo, Jingyu Liu, Mingda Li, Qingbin Liu, Xi Chen, and Xiaoying Tang. Trace: Temporal grounding video IIm via causal event modeling. arXiv preprint 1rXiv:2410.05643, 2024.
Kai Hu, Feng Gao. Xiaohan Nie, Peng Zhou. Son Tran, Tal Neiman, Lingyun Wang, Mubarak Shah, Raffay Hamid, Bing Yin, et al. M-llm based video frame selection for efficient video understanding. Proceedings of the Computer Vision and Pattern Recognition Conference, pp- 13702-13712, 2025.
Bin Huang, Xin Wang, Hong Chen, Zihan Song. and Wenwu Zhu. Vtimellm: Empower Ilm to grasp video moments. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14271-14280. 2024.
Zai-Kai Huang and Kwok-Wing Chau. A new image thresholding method based on gaussian mixture model. Applied mathematics and computation, 205(2):899-900 2008.
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card, arXiv preprint arXiv:2410.21276 2024.
Bo Li, Yuanhan Zhang. Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.00 033326 2024a.
Dongxu Li, Yudong Liu, Haoning Wu, Yue Wang, Zhiqi Shen, Bowen Qu, Xinyao Niu, Fan Zhou, Chengen Huang, Yanpeng Li, et al. Aria: An open multimodal native mixture-ou mixture-of-expert of-exes model. arXiv preprint arXiv:2410.05993 2024b.
11

---

Published as a conference paper at ICLR 2026
Junnan Li. Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, and Steven Chu Hong Hoi. Align before fuse: Vision and language representation learning with momentum distillation. Advances in neural information processing systems. 34:969449999 9705, 2021.
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference machine learning, pp. 19730-19742. PMLR, 2023.
Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, and Li Yuan. Video-llava: Learning united visual representation by alignment before projection. arXiv preprint arXiv:2311.00122 2023.
Ji Lin, Hongxu Yin, Wei Ping, Pavlo Molchanov, Mohammad Shoeybi, and Song Han. Vila: On pre-training for visual language models. In Proceedings the IEEF/CVF conference on computer vision and pattern recognition, pp.
2024.
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee, Improved baselines with visual instruction tuning, 2024. URL http:://rxxi. org/abs/2310.033744 3(4):5, 2023a.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892-34916 2023b.
Shuming Liu, Chen Zhao, Tianqi Xu, and Bernard Ghanem. Bolt: Boost large vision-language model without training for long-form video understanding. Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 38133318333 1-237,, 2025.
Tianming Liu, Hong-Jiang Zhang, and Feihu Qi. A novel video exxttramee algorithm based perceived motion energy model. IEEE transactions on circuits and systems for video technology, 13(10): 1006-1013, 2003.
Yongdong Luo, Xiawu Zheng, Xiao Yang. Guilin Li, Haojia Lin, Jinfa Huang, Jiayi Ji, Fei Chao, Jiebo Luo, and Rongrong Ji. Video-rag: Visually-aligned strieval-augmente long video com- prehension. arXiv preprint arXiv:2411.
2024.
Ziyu Ma, Chenhui Gou, Hengcan Shi, Bin Sun, Shutao Li, Hamid Rezatofighi, and Jianfei Cai. Drvideo: Document retrieval based long video understanding. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 18936-18946, 2025.
Karttikeya Mangalam, Raiymbek Akshulakov, and Jitendra Malik. Egoschema: A diagnostic bench- mark for very form video language understanding. Advances in Neural Information Process- ing Systems, 0446212--62444 2023.
Alan Oppenheim. Discrete-time signal processing. Pearson Education India, 1999.
Peter Pirolli and Stuart Card. Information foraging. Psychological review, 106(4):643, 1999.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8763. PMLR, 2021.
Kanchana Ranasinghe, Xiang Li, Kumara Kahatapitiya, and Michael Ryoo. Understanding long videos in one multimodal language model pass. In International Conference on Learning Repre- sentations, 2025.
Shuhuai Ren, Linli Yao, Shicheng Li, Xu Sun, and Lu Hou, Timechat: A time-sensitive multimodal large language model for long video understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14313-14323, 2024.
Leonid Rudin, Stanley Osher, and Emad Fatemi. Nonlinear total variation based noise removal algorithms. Physica D: nonlinear phenomena, 60(1-4):259-22 1992.
Hui Sun, Shiyin Lu, Huanyu Wang, Qing-Guo Buo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang, and Ming Li. Mdp3: A training-free free approach for list frame selection in iidoo-Ilms arXiv preprint arXiv:2501.c 02885, 2025.
12

---

Published as a conference paper at ICLR 2026
Quan Sun, Yuxin Fang, Ledelll Wu, Xinlong Wang, and Yue Cao. Eva-clip: Improved training techniques for clip at scale. arXiv preprint arXiv:2303.15388 2023.
Xi Tang, Jihao Qiu, Lingxi Xie, Yunjie Tian, Jianbin Jiao, and Qixiang Ye. Adaptive keyframe sam- pling for long video understanding. In Proceedings of the Computer Vision and Pattern Recogni- tion Conference, pp. 29 29122,,,, 2025.
Subhashini Venugopalan, Marcus Rohrbach, Jeffrey Donahue, Raymond Mooney, Trevor Darrell, and Kate Saenko. Sequence to sequence-video to text. In Proceedings of the IEEE international conference on computer vision, pp. 4534-4542, 2015.
Peng Wang. Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model's 's perception of the world ar any resolution. arXiv preprint arXiv:2409. 12191, 2024a.
Xiaohan Wang, Yuhui Zhang, Orr Zohar, and Serena Yeung-Levy. Videoagent: Long-form video understanding with large language model as agent. In European Conference Computer Vision, pp. 58-76. Springer, 2024b.
Yi Wang, Kunchang Li, Yizhuo Li, Yinan He, Bingkun Huang, Zhiyu Zhao, Hongjie Zhang, Jilan Xu, Yi Liu, Zun Wang, et al. Internvideo: General video foundation models via generative and discriminative learning. arXiv preprint arXiv:2212.03 2022.
Yueqian Wang, Xiaojun Meng, Jianxin Liang, Yuxuan Wang, Qun Liu, and Dongyan Zhao. Hawk- eye: Training video-text lims for grounding text in videos. arXiv preprint arXiv:2403.10228, 2024c.
Ziyang Wang, Shoubin Yu, Elias Stengel-Eskin, Jachong Yoon, Feng Cheng, Gedas Bertasius, and Mohit Bansal. Videotree: Adaptive tree-based video representation for Ilm reasoning on long videos. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 3272- 3283, 2025.
2025.
Wayne Wolf. Key frame selection by motion analysis. In 1996 IEEE international conference on acousties, speech, and signal processing conference proceedings, volume 2, pp. 1228-1234 IEEE, 1996.
Haoning Wu, Dongxu Li, Bei Chen, and Junnan Li. Longvideobench: A benchmark for long-context interleaved vineo-language understanding. Advances in Neural Information Processing Systems, 37:28828-2588 288857. 2024.
Junbin Xiao, Xindi Shang, Angela Yao, and Tat-Seng Chua. Next-qa: Next phase of question- answering to explaining temporal actions. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition, pp. 9777-9786, 2021.
Chunyu Xie, Bin Wang, Fanjing Kong, Jincheng Li, Dawei Liang, Gengshen Zhang, Dawei Leng, and Yuhui Yin. Fg-clip: Fine-grained visual and textual alignment. arXiv preprint arXiv:2505.05071, 2025.
Linli Yao, Haoning Wu, Kun Ouyang, Yuanxing Zhang, Caiming Xiong, Bei Chen, Xu Sun, and Jun- nan Li. Generative frame sampler for long video understanding. arXiv preprint 299:: 2025.
Jinhui Ye, Zihan Wang, Haosen Sun, Keshigeyan Chandrasegaran, Zane Durante, Cristobal Eyza- guirre, Yonatan Bisk, Juan Carlos Niebles, Ehsan Adeli, Li Fei-Fei, et al. Re-thinking temporal search for long-form video understanding. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 8579-8591, 2025.
Shoubin Yu. Jaemin Cho, Prateek Yadav, and Mohit Bansal. Self-chained image- language model for video localization and question answering. Advances in Neural Information Processing Systems, 76777, 2023.
13

---

Published as a conference paper at ICLR 2026
Sicheng Yu, Chengkai JIN, Huanyu WANG, Zhenghao CHEN, Sheng JIN, Zhongrong ZUO, Xiaolei XU, Zhenbang SUN, Bingni ZHANG, Jiawei WU, et al. Frame-voyager: Learning to query frames for video large language (205).. In Proceedings of the Thirteenth International Conference on Learning Representations, ICLR, pp. 24-28, 2025.
Huaying Yuan, Zheng Liu, Minghao Qin, Hongjin Qian, Yan Shu, Zhicheng Dou, Ji-Rong Wen, and Nicu Sebe. Memory-enhanced retrieval augmentation for long video understanding. arXiv preprint arXiv:2503.09149, 2025.
Mert Yuksekgonul, Federico Bianchi, Pratyusha Kalluri, Dan Jurafsky, and James Zou. When and why aision-language models behave like bags-of-words, and what to do about it? arXiv preprint arXiv:2210.019360 2022.
Xiangyu Zeng, Kunchang Li, Chenting Wang, Xinhao Li, Tianxiang Jiang. Ziang Yan, Songze Li, Yansong Shi, Zhengrong Yue, Yi Wang, et al. Timesuite: Improving milms for long video understanding via grounded tuning. arXiv preprint urXiv:2410.19722 2024.
Ce Zhang. Taixi Lu, Md Mohaiminul Islam, Ziyang Wang, Shoubin Yu, Mohit Bansal, and Gedas Bertasius. A simple Ilm framework for Long-range video question-answering arXiv preprint arXiv:2312.17255, 2023.
Kaichen Zhang, Bo Li, Peiyuan Zhang, Fanyi Pu, Joshua Adrian Cahyono, Kairui Hu, Shuai Liu, Yuanhan Zhang, Jingkang Yang. Chunyuan Li, and Ziwei Liu. Lmms-eval: Reality check on the evaluation of large multimodal models, 2024a. UB..atpp::/axxx..oorr//b/s/
12772
Peiyuan Zhang, Kaichen Zhang, Bo Li, Guangtao Zeng, Jingkang Yang, Yuanhan Zhang, Ziyue Wang, Haoran Tan, Chunyuan Li, and Ziwei Liu. Long context transfer from language to vision. arXiv preprint arXiv:2406. 16852, 2024b.
Shaojie Zhang, Jiahui Yang. Jianqin Yin, Zhenbo Luo, and Jian Luan. Q-frame: Query-aware frame selection and multi-resolution adaptation for video-llms. arXiv preprint arXiv:2506.22138 2025.
Like Zhao, Shunyi Zheng, Wenjing Yang, Haitao Wei, and Xia Huang. An image thresholding approach based on gaussian mixture model. Pattern Analysis and Applications, 22(1):75-88, 2019.
Zhuo Zhi, Qiangqiang Wu, Wenbo Li, Yinchuan Li, Kun Shao, Kaiwen Zhou, al. Videoagent2: Enhancing the llm-based agent system for long-form video understanding by uncertainty-aware cot. arXiv preprint arXiv:2504 04471, 2025.
Junjie Zhou, Yan Shu, Bo Zhao, Boya Wu, Zhengyang Liang, Shitao Xiao, Minghao Qin, Xi Yang, Yongping Xiong, Bo Zhang, al. Mlvu: Benchmarking multi-task long video understanding. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 13691-13701, 2025.
Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Internv13: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.1047, 2025.
Jialong Zuo, Yongtai Deng, Lingdong Kong, Jingkang Yang. Rui Jin, Yiwei Zhang, Nong Sang, Liang Pan, Ziwei Liu, and Changxin Gao. Videolucy: Deep memory backtracking for long video understanding. In The Thith Annual Conference on Neural Information Processing Sys- tems.
14

---

Published as a conference paper at ICLR 2026
A APPENDIX
Further details are provided in the appendix. We include our LLM usage statement in A.1 elaborate on our proposed A.I.R. framework A.2 and provide full implementation details in We present additional quantitative results analysis in A.4, qualitative results analysis in A.5 and limitations in
A.6
A.1
USE OF LLMS
The LLM is auxiliary tool for research. Specifically, we Gemini 2.5 Pro proofread and polish the paper. No other usages LLM are precluded from this statement.
A.2 MORE DETAILS ON OUR METHOD
A.2.1 ADAPTIVE SAMPLING BUDGET
To ensure our frame selection is efficient for videos of varying lengths, we introduce an adaptive sampling budget B (i.e., the number of frames we need to sample). The budget scales proportionally with the video's duration, allocating fewer frames to short videos (We define the short video as one less than 5 minutes) while respecting the limit of the VLM. Given video with a total of N frames and sampling frame rate of FPS, we define the budget as:
B max (min ([iiax
FPS N
[Vmax)
Vinin
300
rate
where is the number of sampled frames, Vinax is the maximum allowed number of sampling budget (i.e., max sampling budget), which constrained by VLM's context length, and Vmin is predefined minimum number of frames required for robust analysis. rate the frame storage rate per second, and for most video files, it is set to 24. This formulation ensures that the budget scales linearly, while being strictly clamped between the Vmin and V max.
A.2.2 EVENT-WISE SAMPLING
To ensure that longer, more sustained events receive greater attention while guaranteeing all events are represented, we allocate our total sampling budget K proportionally based on the duration of each refined event. Let L, 27t0 pstart be the duration of event E E E. The number of frames to sample from this event, kj. is calculated using a aooorand-add--aa scheme:
L L
k- = K
1,
k = K.
j=1
This formulation ensures both of our key criteria are met, as illustrated in Fig. (a): the proportional term allocates larger budget to longer events, while the 1+1' guarantees that even the shortest events are sampled at least once. The set of per-event budgets ky is then normalized to ensure that the total number of sampled frames is exactly K. After determining the budget k, for each event, we perform the final selection by identifying and choosing the k, frames that have the highest pre-computed similarity scores from within that event's boundaries. We denote this as the function Sclect(E,, fort(Se,), kj), which means select ky frames from & based on the sorted Se,- Finally, we form the initially sampled frame set F as:
Fiitiil n
Sort Sortoet
& E
(10)
where n ensures all selected frames within events are gathered together as an initial frame set Finitial- This frame set will then be used for the Iterative Frame Selection stage.
A.2.3 ITERATIVE FRAME SELECTION ALGGORITHM
We propose an iterative loop with four synergistic steps, detailed in Alg. In each iteration, the process begins with (1) Interval Potential Ranking. This step identifies temporal intervals between the currently sampled frames, calculates Potential score. P for each interval using Eq. 4 and
15

---

Published as a conference paper at ICLR 2026
Algorithm 1: Pseudo Code for Iterative Frame Selection
Input: Analysis VLM, query, sampling budget B. query- -eerr-frraee similarity S, initial sampled frames Finitial; [ = K, number of candidate frames C, max iterations Imax. threshold posttiive Output: Finalized Sampled Frames final.
Functions (Select (F, Sori(S), C): selects C frames from F based on the sorted order of S)
F ===Fao Finitial Fūnal 0:
for iteration
Imax do
/- 1. Interval Fotential Ranking
Initialize Intervals N(K+1)x2;
/+ Using Eq
+/
Initialize IR KK for potential scores; for - to K +
| P. Potential(L):
/+ Using Eq
+/
Feand (orect(
/* Select top-C candidate frames
*/
/+ 2. Reasoning Beasoning-Based VLM Analysis
+/
Initialize R Nc for VLM scores;
R Analysis VLM(qqeryy Feand);
Batch Analysis */
11
F (J; Feand R; 0):
/* Frame Set zefinement using Eq.
12
if | = 0 and Faand] > 0 then
F Select (Fandd, (url(S), [B/Ima)]];
/* Handle edge case
+/
14
Final Final UF*:
/+
3.
Early Stop Mechanism
15
if
[Finall 2 B then
Frial Select(Fimat. Sort(S), B); Break;
/* Select top-B frames */
17
/-
4. Localized Density Sampling (LDS)
for + to F do 18
19
8 LDS(FT); J F
/+ Sample more frames (o) using Eq.
+/
21
update Ss
Compute and update query-frane similarity */
22
K = FF;;
return Frinll
selects small pool of C' candidate frames (Feand) from the highest-potential regions for analysis. These candidates are then evaluated by (2) Reasoning- Based VLM Analysis. The VLM provides relevance score R for each frame, and only those exceeding predefined threshold are retained as set of validated frames, FF. To ensure robustness, if frames are validated by the VLM, fallback mechanism selects a small number based on their initial similarity scores. The validated frames are then added to our final selection pool, Final The loop governed by (3) an Early Stop Mechanism, which terminates the process once the total number of selected frames meets the required budget B. the loop continues, the newly validated frames in F serve as anchors for (4) Localized Density set of fine-grained frames
Sampling (LDS). For each validated frame, this step generates from its immediate temporal vicinity, denoted as in Alg. [ and elaborated in Eq. 6 These newly discovered frames 5 are then added to the main pool of candidates, enriching the search space for the subsequent iteration. This cycle of prioritizing, validating, and exploring allows our method to efficiently converge the most critical evidence.
A.2.4 INTERVAL POTENTIAL RANKING
Inspired by the research the Signal Processing domain, we define the three factors to rank an interval as: (1) Relevance: The average signal magnitude, analogous the DC component (Oppen-
heim 1999). Higher values suggest greater pertinence; (2) Complexity: The signal's 's Total Variation,
which quantifies volatility and often highlights significant events like scene changes (Rudin et al.) 1992]; (3) Length: The logarithmic duration of the interval, which prioritizes larger unexplored seg-
ments while modeling diminishing returns (Pirolli & Card 1999). Thus, the potential of interval I,
16

---

Published as a conference paper at ICLR 2026
is calculated as:
J.E St dt fi+1-1 fi
(1=
fitm S: dt Je fi+1 - Ji
Potential(L.)
Ig(f++1 -f)-
(11)
Because similarity signal is discrete, apply the discrete version (Eq. potential scores, which serve as a crucial indicator to select high-potential candidates for next-step VLM analysis.
compute the
A.2.5 PROMPTS OF OUR METHOD FOR VLMS
Our method employs a structured, two-stage prompting strategy to guide the Vision Language Mod- els (VLMs) in their respective roles, as illustrated in Prompt for Analysis VLM and Prompt for Answering VLM below. The first prompt is designed for the Analysis VLM, which functions relevance-scoring module. It instructs the VLM to act as an "expert visual reasoner" and evaluate how well each sampled frame matches the user's query. To ensure consistent and quantifiable out- put, we provide a detailed 5-point scoring rubric, from "1 Not relevant at all" to "5 Perfectly matches" (the positive threshold 0 is set to 3). The prompt enforces strict output format requiring a brief justification sentence and the numerical score, which allows for systematic filtering of vi- sual information. The second prompt is for the Answering VLM, which is responsible for the final maniin. This prompt is framed as a standard multiple-choice question. directs the model synthesize the information from the most relevant frames (identified by the Analysis VLM) and select the correct option. To facilitate straightforward and automated evaluation, the output is con- strained only the letter (A, B, C, or D) of the chosen answer (for dataset with five options, we add one more 'E' here). This dual-prompt design creates robust pipeline where visual evidence is first critically assessed and filtered before final answer determined.
Prompt for Analysis VLM
You are an expert visual reasoner. Step by step analyze how well this image matches the following query (with options):
{QUERY)
Rate the relevance from to 5, using these exact definitions:
1 Not relevant all: no relation between image content and the query.
2 2- Slightly relevant: only minor contextual hints, but not central to answering.
3 Moderately relevant: contains preparatory or follow-up context (e.g., setup actions) re-
lated to the query.
4 Highly relevant: shows clear evidence to support the query, but still has ambiguity or missing details.
5 Perfectly matches: fully sufficient and unambiguous evidence to answer the query.
Avoid overthinking. Trust your immediate judgment.
Examples:
Image of an empty room (no people or objects) + Reasoning: There is nothing related to the query. Score:
Image showing exactly the queried action (clear, direct match) -> Reasoning: It perfectly
depicts the required event. Score: 5
Respond only in this format:
Score: <1-5>
Reasoning: brief one sentenne justification>
17

---

Published as a conference paper at ICLR 2026
Prompt for Answering VLM
Select the best answer to the following multiple-choice question based on the video and the subtitles. Respond with only the letter (A, B, C, or D) the correct option.
{QUERY)
A.3 ADDITIONAL IMPLEMENTATION DETAILS
A.3.1 HYPERPARAMETER SETTING
For query-related similarity S E RN, set it sparse array with NaN at no-updated entries, so we only focus on the entries with values and skip the computation of the NaN ones. For Adaptive sample frames Viii to 8 in Eq. which corresponds to the maximum number of frames accepted by VILA-1.5, For the GMM threshold in Eq. we set 7 to 0.7. For Event-Wise Sampling, we merge events occurring less than seconds apart (1.e., dmin = 2 * 24) and filter out events lasting less than 3 seconds (i.c., min = -3* 24), since all video files are stored in 24 fps. We
Initial Sampling,
FPS (i.e., FPS 1) and set the minimum frame budget
2 max (C) our Iterative we set the candidate pool size C' = 12 (the maximum number accepted to one batch inference for our machine) and the maximum number iterations Imax 6. The VLM rating scores are from I to 5 (0 3). Finally, for Localized Density Sampling (Eq. ( we set a = max(0.05 [[,], 15). representing 5% of the current interval's 's length with a minimum value of 10, and set 3 to 1.5.
Frame Selection (Alg. [
A.3.2 EXPERIMENTAL ENVIRONMENT
All of our experiments conducted using NVIDIA GH200 GPUs, with Arch64 as the CPU chitecture. Our inference is using Pytorch 2.5 with CUDA-11.6 and GCC-11.4.0. One-time ex- periment of our method is run using aforementioned machines via Imms-eval (Zhang et al. 2024a) inference framework. When using InterVL3 as our foundation VLM, the inference time (Zhu et al., 2025) is around 10 hours. For Video-MME w/o subtitle (Fu et al.] (025). The cost time ranges from 9 hours to 17 hours per different VLMs. Due to the limit of Imms-eval framework, all of our inference is under batch size, while set the sub-batch size 12 (e.g., for VLM analysis on frames). For CLIP cache, because each benchmark has queries us- ing the same video repeatedly, we store the image features for quick cosine-similarity compu- tation. All of our videos are loaded from high-res files (all data are provided by Imms-eval, https::/g com/ Evolvv v AyvolvingIMMS LMg LLL.MMM--Laaa/A lmms- so preprocessing operations like Resizing may increase the time for computation of CLIP features.
Table Comparison with different methods on Qwen2-
Video-MME
Egoschema
Model
#Frames
LVB
NextQA
w/o sub
Full
Subset
Qwen2-VL-7B Wang el al.
2024a
32
57.6
60.9
55.5
61.9
64.0
77.6
+Q-Frame Zhang et all 2025
58.3
61.8
58.4
+AKS (Tang et al 2025
32
59,9
60.5
Hu et al. 2025
58.7
57.0
65.9
78.4
+Ours
32
60.0
63.1
58.9
62.5
66.2
80.1
A.4 ADDITIONAL RESULTS ANALYSIS
A.4.1 ADDITIONAL COMPARISON WITH SOTAS
Comparison with other SOTA Methods on Owen22-V--7B. Besides the main comparison with So- TAs using Qwen VL-2.5, we conducted an additional set of experiments with Qwen2- VL-7B (Wang et al. 2024a) (Tab. 9 to further demonstrate the generalizability and robustness of our A.I.R. frame- work. We compare our performance against a 32-frame uniform sampling baseline as well as several other state-of-the-art (SoTA) frame selection methods. The results presented in Tab. The data
18

---

Published as a conference paper at ICLR 2026
Table 10: Comparison of Agent-based VLM Frame Selection Methods on Video Question Answer- ing Benchmarks.
Method
Venue
Base Model
#Frames
Video-MME
MLVU LVB
EgoSchema NexIQA
w/o sub w/ sub
Full Subset
Agend-bused Frame Selection Methods
VideoLucy (Zao al.
SeurlPS'25
DeepSeck-R1
72.5
76.1
VideoRAGi (ra
2024
NeurlPS
Vide--idd
72.4
To ct al
2025
CVPR'25
LLaVA-OV-7B
8
66.6
80.4
DiVideo NV Maeall 2025
25
GPT-4
61.0
66.4
MemVid Youn st 20245
a5Xiv'25
Qwen2VL-7B
128
63.7
65.7
VideoAgen122 Li 2025
arXiv'25
GPT-4o
80.5
AKEYS (Fan ur 2036
arXiv
GPT-4o
<32
63.6
68.6
78.1
Our Frame Selection Method for Comparison
A.I.R.
GPTT-o
$32
65.1
72.0
82.5
A.I.R. A.I.R.
Qwen2VL-7B
<32
60.0
63.1
58.9
62.5
66.2
80.1
InlemVV13-88
<32
68.2
69.2
74.5
62.8
63.3
72.2
82.6
A.L.R.
LLaVA-OV-7B
332
61.4
65.1
69.3
63.2
81.6
shows that A.I.R. consistently outperforms both the baseline and competing methods on this new VLM backbone. Notably, on the Video-MME (w/o sub) benchmark, our approach achieves the highest accuracy of 60.0%, surpassing strong methods like AKS (59.9%) and Q-Frame (58.3%), and providing a significant +2.4% uplift over the baseline. Our method also achieves the top perfor- mance on Egoschema and NexIQA. While AKS shows a strong result on LVB, our method remains highly competitive with a score of 58.9%, substantial improvement over the baseline's 's 55.5%. This experiment validates that the benefits of our adaptive, iterative, and reasoning- -based approach are not tied to a specific VLM architecture and can be effectively generalized.
Comparison with Agent-based Frame Selection Methods. further validate A.I.R's effec- tiveness against cutting-edge approaches, we compare against seven recent agent- agent-based methods from NeurlPS' 25, CVPR'25 25, and ar5iv'25 in Tab. 10 These methods leverage multi-step reason- ing and planning capabilities of large language models, often using frontier models like GPT-4o or DeepSeek-R1. Our results reveal three key findings: (1) Superior performance with matched backbones and exceptional frame efficiency. When using identical base models, A.I.R. consis- tently outperforms beaee methods while maintaining high frame efficiency. For instance, with GPT-4o and comparable frame budgets ($32 frames), A.I.R. surpasses AKEYS on EgoSchema Sub- set by +3.4% and NextQA by +4.4%. Similarly, with LLa VA- One Vision 7B, A.I.R. exceeds T on NexiQA (+1 (+)).)) Moreover. compared to VideoRAG which uses 64 frames with LLa Video-7B to achieve 58.7% on LVB, A.I.R. with LLaVA-One Visionn-7 reaches 60.7% 7% with only <32 frames, achieving 2x frame efficiency with +2.0% accuracy improvement. (2) Competitive performance with smaller models. Remarkably, A.I.R. with the 8B-scale Intern VL3 achieves performance com- petitive with or superior to based methods using much larger frontier models. On NexIQA, VnterVLL3338 with A.I... reaches 82.6%, surpassing Video/Agent2's GPT-4o result (80.5%) and AKEYS's GPT-4o result (78.1%). On EgoSchema, InternVL33883 achieves 72.2% (Subset) and 63.3% (Full), exceeding DrVideo's GPT-4 performance (66.4% Subset, 61.0% Full). This demon- strates that A.I.R.s 's intelligent frame selection can unlock the full potential of smaller, open-source models to match or exceed the performance of expensive proprietary models. (3) Comprehen- sive and transparent evaluation. Unlike many agent-based methods that report results selectively across 2-3 benchmarks, A.I.R. provides consistent results across all five major benchmarks (Video- MME, MLVU, LVB, EgoSchema, NextQA) for multiple VLM backbones, demonstrating robustness and eliminating concerns about cherry-picked picked results. While agent- methods offer powerful multi-step reasoning capabilities. they face practical challenges including high computational costs from multiple LLM calls and dependency on proprietary models. In contrast, A.I.R. achieves com- petitive performance through streamlined, training-free framework that seamlessly integrates with any VLM backbone.
A.4.2 DETAILED ANALYSIS OF QUESTION TYPE
As illustrated in Fig. we evaluate the performance of our proposed method, A.I.R,, against two baseline approaches: MDP3 (Sun et al.) 2025) and Uniform Sampling, across six distinct problem domains. The results clearly demonstrate that A.I.R. consistently and substantially outperforms both baselines across all evaluated categories. Notably, A.I.R. exhibits exceptional proficiency in tasks requiring textual and holistic understanding, achieving its highest scores in Information Synopsis (81.1%), OCR Problems (79.1 (79, and Perception (78.9%). It also maintains strong lead in Recog-
19

---

Published as a conference paper at ICLR 2026
Table 11: Ablation study on different frame numbers for training-free methods.
Video-MME
Egoschema
Model
#Frames
MLVU
LVB na!
NexIQA
w/o sub
w/ sub
Full
Subset
QwenVI.-2.5
32
60.8
62.7
59.3
58.1
57.6
59.4
74.3
+Ours
S 32
66.3
61.4
58.8
81.3
V1.2.5
64
61.2
65.0
63.8
59.0
58.8
62.0
75.9
+Ours
65.9
67.2
62.5
59.8
82.5
V1.2V
256
63.4
67.3
67.8
60.2
58.8
74.3
+Ours
S 256
66.4
67.9
71.7
62.8
59.9
63.6
83.3
LaVVA-OneVssion
58.9
58.9
54.2
59.2
77.4
+BOLT"
8
56.1
63.4
55.6
59.2
62.2
77.4
+Ours
57.9
60.5
63.8
57.3
59.4
63.4
78.9
LLaVA-One OneVision
16
56.9
60.0
61.2
55.7
59.5
61.4
78.1
+BOLT"
57.0
59.9
78.3
+Ours
< 16
59.1
60.8
65.6
58.9
60.4
62.2
79.4
LLaVA-OneVision
32
58.5
61.7
62.4
56.6
60.2
61.8
79.3
+BOLT
32
59.9
66.8
59.6
60.7
64.0
79.5
+Ours
61.4
60.7
61.4
81.6
nition (69.9%) and Reasoning (62.9%), showcasing its robust capabilities on high volume of test cases. Interestingly, the most challenging domain for all methods, including A.I.R., is the Counting Problem, where A.I.R. scores 49.3%. While this is its lowest score, it still represents significant margin improvement over both MDP3 and Uniform Sampling. In comparison, the MDP3 method offers moderate improvement over the basic Uniform Sampling baseline but is clearly surpassed by the advanced capabilities of A.I.R. in every task. Overall, the analysis highlights the comprehensive strengths and superior performance of the A.I.R. model across diverse set of complex problems.
A.4.3 ADDITIONAL ABLATIONS OF SAMPLED FRAME NUMBER
In this section, we provide detailed ablation study on the effect of the maximum sampling budget on the performance of our A.I.R. framework. We compare our method against uniform sam- pling baseline and other state-of-the-an methods across various benchmarks, using two distinct VLM backbones: QwenVL-225 and LLaVA-One Vision. The comprehensive results are presented in Tab. 11
The analysis on the Qwen VL-2.5 backbone demonstrates that A.I.R. provides a substantial and con- sistent performance improvement over the uniform sampling baseline every tested budget (32, 64, and 256 frames). For instance, with a maximum of 32 frames, our method boosts performance on Video-MME (w/o sub) from 60.8% to 65.0% and on MLVU from 59.3% 67.5%. As the maxi- mum budget increases, the performance of our method continues to scale effectively, consistently maintaining significant advantage over the baseline, which sees only modest gains from having more frames.
The three-way comparison on the LLa Vision backbone further validates our approach. Across nearly all settings, A.I.R. outperforms both the uniform sampling baseline and the strong competitor, BOLT. For example, with 32 32-frame budget, A.I.R. achieves 61.4% on Video-MME, surpassing both the baseline (58.5%) and BOLT (59.9%), and scores 81.6% on NexiQA, again outperforming both the baseline (7933%) and BOLT (79.5%). The effectiveness of our intelligent selection is highlighted by the fact that A.I.R. with maximum of 16 frames (59.1% on Video- MME) often outperforms the uniform sampling baseline that uses fixed 32 frames (58.5%). This comprehensive analysis of two different VLMs confirms that adaptive and iterative framework a robust and highly effective strategy for improving video understanding across a wide range of frame budgets.
A.4.4 ADDITIONAL ANALYSIS OF METHOD ABLATIONS
Influence of Method Components. To validate the contribution of each component within the A.I.R. framework. we conduct a systematic ablation study, with results presented in Tab. 4 Our full method (#2) achieves 68. 28.2% accuracy, significant improvement of +2.6% over the Uniform
20

---

Published as a conference paper at ICLR 2026
Table 12: Ablation study on the hyperparameters C' (candidate pool size) and Iuaa (max iterations). The trade-off between efficiency (Time) and performance (Aceuracy) is evaluated NextQA using Intern VL3-8B. The chosen default configuration is highlighted. Times (in seconds) represent the VLM inference time, while (16.0f) represents the overall frames analyzed by VLM.
# Max Sampling Candidate Pool
Max Iterations (Iuax)
Accuracy (%)
Time (s)
Budget (Vmux)
Size (C)
16
8
2
14.87 (13.6f)
81.61
2
16
4
4
12.62 (12.4f)
81.68
3
16
8
4
18.92 (20.3f)
81.92
4
32
8
4
29.17 (27.
82.20
5
32
12
34.24 (32.20)
82,63
6
32
16
8
36,52 (35.6f)
82.91
Sampling baseline (#1), while also using fewer frames on average. Disabling individual components generally degrades performance, confirming their synergistic importance. The most critical compo- nent is the entire Iterative Frame Selection stage (#6); its removal causes the largest accuracy drop to 65.2%, demonstrating that the iterative refinement process is essential for identifying relevant frames. The Reasoning- ing accuracy to 66.0% and increasing the frame count to 32. The Interval Potential Ranking (#7) and Adaptive Initial Sampling (#5) are similarly important, with their removal reducing accuracy to 66.7% and 66.9%, respectively. Notably, replacing our adaptive budget with a fixed 32-frame bud- get (#3) yields a marginal 0.1 % increase in accuracy to 68.3%. However, selected our adaptive approach as the default because it strikes superior nearly identical top-tier performance while using 22.5% fewer frames on average (24.8 vs. 32.0), making it more practical and scalable solution.
VI.M Analysis (#8) is also highly impactful, with its removal reduc-
perfiennyy-eeooomnccee trade-off: it achieves
Ablation on CLIP Models. We analyze the impact of different CLIP variants on the A.I.R. frame- work in Tab. 5 The results show that all configurations utilizing our method demonstrate a marked improvement over the Uniform Sampling baseline, highlighting the fundamental effectiveness of our framework irrespective of the specific vision backbone employed. Among the tested variants, the combination of A.I.R. with EVA-CLIP-L achieves the highest performance, reaching an accuracy of 68.2%. We also observe that larger models generally provide better results, with CLIP-VIT-L outperforming its base-sized counterpart by 1.0%. Based on this analysis, we selected EVA-CLIP-L as the default vision encoder for our main experiments due to its superior empirical performance.
A.4.5 ADDITIONAL EFFICIENCY AND HYPERPARAMETER ANALYSIS
We analyze the impact of our key hyperparameters- -Ma Sampling Budget (Vmax), Candidate Pool Size (C), and Max Iterations (Imaxx ) the trade-off between performance and efficiency, with sults on NexIQA presented in Tab. I2 The results demonstrate that various A.I.R. configurations can achieve state-of-the-arr accuracy while remaining highly frame-efficient. For instance, our settings with a 16- (6-frame budget (rows #1 -#3) surpass 81.6% accuracy using fewer than 14 frames on aver- age, significantly outperforming fixed-budget baselines that score much lower with similar frame count. Focusing on the configurations with Vmax 32 (rows #4-#6) allows for deeper analysis of this trade-off. The setting row #4 (C 8, Imax 4) establishes a strong baseline at 82.20% ac- curacy with 27.2 analyzed frames. Our chosen default setting (row #5, C = 12, Imax = 6) improves accuracy to 82.63% for a modest increase in VLM workload to 32.2 frames. Further increasing the hyperparameters (row #6) yields the highest accuracy (82.919 1)), but this marginal +0.28% gain quires a larger workload (35.6 frames), indicating a clear point of diminishing returns. We therefore select the configuration from row #5 our default, it strikes optimal balance between high per- formance and computational cost. Overall, these results show that A.I.R. is robust to hyperparameter changes and our chosen default provides the best efficiency-perffieeeee performaace trade-off
In Tab. 13 we provide a granular efficiency analysis by breaking down the performance of A.I.R. across short, medium, and long videos. This detailed view reveals several key advantages of our adaptive framework:
Adaptive Cost vs. Fixed Waste. The results show that A.I..'s computational cost intelligently scales with video length, in stark contrast to conventional fixed fixed-budget methods. For instance, a 128-frame baseline incurs constant, high cost of 162.00 03s regardless of video duration. A.I.R.,
21

---

Published as a conference paper at ICLR 2026
Table 13: Detailed Efficiency Comparison on Video-MME using Intern VL3-8B.
Model
Short Video
Medium Video
Long Video
Overall
6min
5 15min
2 15min
Avg. 17 min
CLIP Inference (IFPS)
21.30
40.32
78.55
46.72
Uniform Sampling (Answering)
0..6
0.98
1.06
0.87
A.I.R. (Sampling, Selection Answering)
0.63
1,10
1.32
1.02
Conventional VL.M inference
162.03 (1280)
162.03 (1281)
162.03 (1280)
162.03 (128h)
A.I.R. (VLM inference, V Viax 32, C mmax
26.34 (25.3f)
46.17 (37.4f)
54.42 (46.71)
42.31 (36.5f)
Conventional VLM inference
42.47 (32.or)
42.47 (32.04)
42.47 (32.01)
42.47 (32.0l)
A.I.R. (VLM inference, Vinax = 16, C Timax 4)
14.45 (12.50)
18,90 (17.4f)
32.41 (30.9f)
21.92 (20.30
Conventional VLM inference
20.39 (16.0f)
20.39 (16.0f)
20.39 (16.0f)
20.39 (16.0f)
A.I.R. (VLM inference. Viax 16, C Imax
9.05 (9.70)
13,47 (12.81)
21.31 (19.80)
14.61 (14.1I
however, is far more efficient on short videos (e.g., 26.34s in our default setting) and, while its cost increases for longer videos (54.42s), it remains nearly three times faster than the fixed- approach, effectively eliminating wasteful computation.
Intelligent, Sub-Linear Scaling. Crucially, the VLM workload for A.I.R. does not scale linearly with the video's duration. In our default setting (Vmax 32), the number of analyzed frames grows from 25.3 for short videos to 46.7 for long videos. This sub-linear growth suggests that A.I.R. effective at pinpointing information-dens. regions, tying its computational cost more closely to the query's 's informational complexity rather than the video's raw length.
Minimal Framework Overhead. The top section of the table highlights the efficiency of the core algorithm itself. The total pipeline time for A.I.R. (1.02s on average) is remarkably close to the fastest Uniform Sampling baseline (0.87s). This demonstrates that the complex logic of adaptive sampling, ranking, and iteration adds minimal computational overhead, making the entire frame- work lightweight and practical.
Taken together, this granular analysis confirms that A.I.R. is a scalable solution that intelligently manages its VLM workload based on the specific characteristics of the video, making it highly practical for real-world scenarios with varying video lengths.
A.4.6 COMPREHENSIVE HYPERPARAMETER ABLATIONS
To validate the robustness of A.I.R. across different hyperparameter configurations, we conduct a systematic ablation study on Long VideoBench (LVB) and NexIQA using Intern As shown in Tab. 14 we independently vary each of the five key hyperparameters while keeping others fixed at their default values: 7 0.7 (GMM coefficient), = 15 (minimum frame distance), 3 = 1.5 (LDS growth factor), Lini = 20 (minimum event length for pruning), and dmin = 2.0s (minimum gap for merging events).
Individual Parameter Analysis. Our results demonstrate strong robustness across all hyperparame- ters. For the GMM coefficient (rows 1-3), we observe that moderate values (0.5-0.7) yield the best performance, with = 0.5 and y = 0.7 both achieving 82.6% on NexIQA. Extreme values (7 = 0.3 7 = 0.9) lead to slightly degraded performance (-0.3% -0.4%), as they either select too many low-relevance frames or miss important events. For minimum frame distance a (rows 4-6), larger values (20-25) prevent redundant sampling but may miss fine-grained details, while smaller values (10) enable denser coverage. The LDS growth factor 3 (rows 7-9) shows consistent performance across the tested range (1.2-2.0), with moderate values (1.5-1.7) performing slightly better. The pruning threshold Imin (rows 10-12) and merging gap dman (rows 13-14) both demonstrate stability, with performance varying by less than 1,0% across different settings.
Strategy-Based Combinations. We further test four strategy combinations (rows 15-18) designed for different video characteristics: (15) Short-Video strategy with dense sampling (7 = 0.5, a = 10, Lmin 15, dmin 1); (16) Long- Video strategy with sparse sampling (a 25, Lmin = 30, dmin 3): (17) Conservative strategy with strict thresholds (7 = 0.3, a = 20); and (18) Aggressive strategy 20). All combinations maintain competitive performance (61.7-62.9% on LVB, 81.8-82.7% on NexIQA), demonstrating that A.L.R. can be ficxibly adapted to different video types while preserving effectiveness.
with loose thresholds (y 0.9,
22

---

Published as a conference paper at ICLR 2026
Table 14: Systematic ablation study of A.I.R. hyperparameters using InternVL3-8B. Default con- figuration (gray) uses parameters from Appendix A3..1 ? 0.7, 15, 3 = 1.5, mmnn=20,, dmin=2.0s. We vary each parameter independently to isolate effects. All configurations maintain performance within #1.1% of baseline, demonstrating strong robustness. Rows 15-18 test strategy combinations: Short- Video (dense sampling), Long- Video (sparse sampling), Conservative (strict), Aggressive (loose). All Vmax
32, 12, Imax 6.
?
a
/
min
dmin
LVB
NexIQA
0.7
15
1.5
20
2
62.8
82.6
Y ablation (GMM coefficient)
0.3
15
1.5
2
62.2
82.3
2
0.5
15
1.5
20
2
62.6
82.6
0.9
15
1.5
20
2
62.0
82.2
a ablation (min-frame distance)
4
0.7
10
1.5
20
2
62.4
82.8
0.7
20
62.8
82.1
6 0.7
25
1.5
20
2
62.9
82.0
B ablation (LDS growth growthfactorr factor)
0.7
15
1.2
20
2
62.4
82.4
8
0.7
15
1.7
20
2
62.6
82.5
0.7
15
2.0
20
62.1
82.1
Imin ablation (Pruning)
10
0.7
15
82.4
11
0.7
15
1.5
25
2
62.8
82.1
12
0.7
15
1.5
30
2
62.7
81.8
dmin ablation (Merging)
13
0.7
15
1.5
20
1
62.3
82.8
14
0.7
15
1.5
20
62.6
82.3
Strategy- based combinations
15
0.5
10
1.5
15
1
61.7
82.7
16
0.7
25
62.9
81.8
17
0.3
20
1.7
20
2
61.8
82.0
18
0.9
20
1.2
20
2
62.1
81.9
Robustness Analysis. Critically, all 18 configurations maintain performance within 11.1% of the default setting on both benchmarks, confirming that A.I.R. is highly robust to hyperparameter choices.
A.5 QUALITATIVE RESTULTS ANALYSIS
As shown in Fig. and Fig. A.I.R''s 's strength lies in its iterative refinement process. The system initially casts a wide net and then uses a Vision-Language Model (VLM) to progressively score and filter video frames. For instance, when searching for character's post-meeting actions, the VLM intelligently discards irrelevant scenes like pre-meeting preparations. A localized sampling mechanism then focuses on temporal regions around these high-scoring frames, creating feedback loop that efficiently uncovers more relevant content. This dynamic process allows A.I.R. to converge on the precise moments that answer query-whetthe whether it's character having lunch or an artist applying llaze-yielding final selection of frames that is both comprehensive and highly relevant.
A.I.R's superiority is further demonstrated in direct comparisons against baseline methods in Fig. and Fig. Baselines like Uniform Sampling are indiscriminate, capturing disconnected or irrelevant moments (e.g., logos, underwater scenes) due to their fixed-interve nature. While an improvement, CLIP Top-K often falls into "relevance traps," leading to high redundancy by over-sampling visually similar frames (like a car scene) or wasting its budget on tangentially related content (people near volcano instead of the lava flow). In stark contrast, A.I.R. constructs coherent narrative. It precisely captures the key chronological events from establishing shots to the core lava formation, Vlogger's complete daily routine-while ensuring temporal diversity. These results confirm
23

---

Published as a conference paper at ICLR 2026
that our adaptive approach overcomes the critical limitations of simpler methods, achieving superior semantic relevance and temporal coherence.
A.6 LIMITATIONS
We acknowledge several limitations of our current framework. First, the performance of A.I.R. is fundamentally bound by the capabilities of its underlying analysis VLM. For instance, as shown benchmarks (Tab. [Tab.[]nnd] [ using more advanced model like OnterVV--3-1 yields sig- nificantly better results than less capable models such as VILA-1.5, even at a comparable parameter size. Additionally, a known shortcoming of most frame selection methods, including ours, their performance on fine-grained tasks like object counting, as highlighted in Fig. 4 Moreover, our ap- proach currently processes only the visual track, omitting crucial information that may present in the audio. Finally, while significantly more efficient than brute-force analysis, the iterative na- ture of our framework introduces computational latency that may not be suitable for real-time applications.
24

---

Published as a conference paper at ICLR 2026
minutes video from Video-MME, https://www. youtube cow.///wwwtti.1--ssccchchhhh Query: How many colors used the videa? A. Five., Four, Three,, Two.
Sampling budget Iteration T
Analysis Answering VLM: InternVL3-88
Adaptive Initial Sampling Fiitial
Iterative Frame Selection Iteration 1/6
Potential Ranked 12 frames:
Resesinn:: The aboms
Positive
Soaee
innas of eokkes of glace usel.
Posttive
Negotive imaze provice any inferatioon about glaze colors used the vidro
Some 3
Resaning: The inage sazes thrse potury Bonlsbeing norther claily delined
Neutrol
mage
Neucrol
puaring yirk place enk sceenn bewl
bonn und valves the
posutlle caken, but la oust tamberr charty detined.
posth Bay draaclile calans
New Samp perpleddinn frames |VL Volidioedd Frames
Localized Density Sampling
Positive-rated Frames Lmaar (S), add from Neutral
F.:
Iterative Frame Selection
Iteration 2/6
Reasonig- Analysis
Localized Density Sampling
Current Sampled Frames F.
itecattin 3/6
Final Sampled Frames (22 frames) F
Until max iteration (6) meet or Early Stop
Taal
Sampled Distribution:
Figure 6: Detailed workflow of A.I.R. 33-minute Video-MME example showing the iterative frame selection process for the query "How many colors of glaze are used in the video?" with VLM reasoning scores and localized density sampling.
25

---

Published as a conference paper at ICLR 2026
minutes video from Video-MME, https://www, youtube Query: What did the male protagonist the video immediately after finishing his personal report at the meeting? A. Play games.,.. Watch friend unbox watch,,C. Have lunch.,D. Chat with friends. Sampling budget B=32
Max Iteration T
VLM-IntemmVL33-8
Adaptive Initial Sampling Fiitial
Iterative Frame Selection Iteration 1/6
Potential Ranked 12
Score:
Negalive
Neutrol las does pronis informaticn
Some
Scoee
Negative
laptep, providl
propering
inage
Resariing The image shows living room setup with indication of penorzal menting, related
pestagmneffs actions
abour what the make protngrsiss dir aumoliatly alle finaling lis penseal rpperr
porile inforration the male finnbing persond upon menting
persoral report meeting.
Niw Sumpled ald fremmss
Valicited
Dositive-rated Frames (0) 4 Zuave (5), add 5 frames from Neutrol
Localized Density Sampling
Current Sampled Frames FF:
Iterative Frame Selection
Iteration 2/6
frames:
Mrasonig-Baseer Analysis
Localized Density Sampling
Current Sampled F.;
Final Sampled Frames (28 frames)
Fassi
Until iteration Early Stop
Figure 7: Detailed workflow of A.I.R. on a 34-minute Video-MMr example showing the iterative frame selection process for the query "What did the male protagonist in the video do immediately after finishing his personal report at the meeting?" with VLM reasoning scores and localized density sampling.
26

---

Published as a conference paper at ICLR 2026
Question: How was Nahuku formed according to the video? A. was created by lava tubes. B. It was created by a river of molten lava.
C. It was formed by the workers in the park. D. It was formed due to the function of wind.
Uniform Sampling
CLIP (Top-K)
A.L.R.
Auprorn U/
Figure 8: Frame selection comparison on volcanie formation video. Uniform Sampling, CLIP (Top-K, K=16), and A.I.R. for the query about Nahuku's formation, with similarity score graphs showing A.I.R's 's focused sampling around relevant lava segments.
27

---

Published as a conference paper at ICLR 2026
Question: Which of the following descriptions of the heroine's daily activities in day correct chronological order?
A. Shopping, doing laundry, making the bed., B. Online shopping, making the bed, doing laundry., C. Buy coffee, shop, make her bed., D. None of the above.
Uniform Sampling
CLIP (Top-K)
A.I.R.
Figure 9: Additional frame selection on daily routine video comparing Uniform Sampling, CLIP (Top-K, K=16), and A.I.R. for chronological activity ordering. A.I.R. effectively excludes numerous irrelevant frames while accurately capturing the complete sequence of key activities (buy coffee, shop, make her bed, do laundry).
28


Reviews:
[{"review_id": "kDJ2uKT1Uz", "reviewer": "Reviewer_73LF", "summary": "The paper tackles a core challenge in Video Question Answering (VideoQA) — how to efficiently select a small set of informative video frames that best correspond to a textual query. To overcome this trade-off, the authors propose A.I.R., a training-free framework for Adaptive, Iterative, and Reasoning-based frame selection. A.I.R. adaptively chooses the most relevant frames by combining efficient CLIP-based similarity analysis with targeted reasoning from a VLM. A.I.R. consists of two main stages, including adaptive initial sampling and iterative frame selection.  The method was extensively tested with various foundation VLMs, including InternVL-3, QwenVL-2.5, LLaVA-OneVision, and VILA-1.5, on both long-video and short-video benchmarks.", "questions": "The paper does not provide sufficient details or rationale regarding the hyperparameter settings used in the A.I.R. framework. How could this issue be addressed to improve the transparency and reproducibility of the proposed method?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The authors provide a clear problem motivation, starting from the inefficiency of uniform sampling and the limitations of existing CLIP-based and VLM-based frame selection methods. The technical pipeline is presented logically, with consistent terminology and step-by-step exposition of each stage (Adaptive Initial Sampling → Iterative Frame Selection → QA). \n2. This paper shows high-quality figures and conceptual illustrations. Figures such as Figure 1 (problem motivation), Figure 2 (overall pipeline), and Figure 3 (detailed visualization of both Adaptive Sampling and Iterative Selection stages) are visually rich and well-annotated, effectively guiding the reader through the method.\n3. The technical part is given with many details. Readers can easily grasp the general idea of this paper.", "weaknesses": "1. Although the proposed A.I.R. framework achieves practical performance improvements, its core design is composed mainly of heuristic strategies rather than a principled methodological innovation. Many components—such as the adaptive thresholding via Gaussian Mixture Models, interval potential ranking, and localized density sampling—are intuitive extensions or empirical engineering choices rather than conceptually new formulations. As a result, the framework feels more like an optimized combination of existing ideas than a grounded advancement. This reliance on heuristics makes it difficult for readers to extract generalizable insights or new theoretical understanding from the paper, which somewhat diminishes its methodological contribution.\n2. The proposed A.I.R. framework involves a large number of hyperparameters, which makes the overall method difficult to interpret and reproduce. This abundance of loosely motivated hyperparameters weakens the methodological transparency of A.I.R. and increases the barrier for readers to understand or extend the framework fully.\n3. The performance advantage of the proposed A.I.R. framework is not particularly significant, and the experimental section lacks comprehensive comparisons with the most recent state-of-the-art methods. Although the paper reports moderate gains (often around 1–2% improvement on several benchmarks such as Video-MME and MLVU), these margins are relatively small given the additional complexity introduced by the multi-stage adaptive and iterative procedures. Moreover, the experiments primarily compare A.I.R. against earlier CLIP-based or training-free frame selection baselines (e.g., MDP3, BOLT, Frame-Voyager), but do not include newer or stronger contemporary methods that could provide a more convincing assessment of its competitiveness. Consequently, the empirical results do not clearly demonstrate that A.I.R. represents a substantial advancement over current techniques.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BtLgwqkfGs", "reviewer": "Reviewer_jkbb", "summary": "The paper proposes A.I.R., which is a multi-stage frame selection heuristics for long video question answering. The pipeline emphasizes efficiency by minimizing the number of frames fed to the heavy-weight reasoning-based VLM for analysis. The pipeline first computes question-frame similarity scores using a light-weight CLIP-style VLM, and then perform multi-stage selection to determine the frames for the expensive MLLM-powered question answering. The selection process include an initial adaptive sampling step, where scores are modeled as a 2-component GMM (relevant and irrelevant), followed by a loop of interval potential ranking, reasoning-based VLM analysis, early stop mechanism and localized density sampling to add or remove from the candidate frame set until a fixed budget is used. The frames are then input to a MLLM with the question to generate the answer. The method is evaluated on a wide range of models (VILA-1.5, QwenVL-2.5, InternVL-3, LLaVA-OneVision) and on multiple datasets (VideoMME, MLVU, LVB, EgoSchema, NextQA), showcasing its effectiveness in a broad range of settings.", "questions": "* Following weakness 1, have the authors encountered any failure cases of the CLIP text encoder, e.g., questions longer than the max number of tokens or CLIP failing to capture the semantic of some complex expressions?\n\n* Following question 1, if systematic failure cases exist, could the paper include a discussion about how they are mitigated by designs in the proposed pipeline, or how they could possibly be mitigated in future works?\n\n* Following weakness 2, if resources permit, could the paper include more results on larger or closed models to better demonstrate its advantages of being training-free and avoiding large numbers of MLLM calls?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "* The paper addresses the topic of long-video understanding, which is of high real-world value and is essential for a broad set of video analysis applications.\n\n* By demonstrating the effectiveness of human-derived heuristics, the paper reveals key limitations of recent LLM-agent-based frame selection pipelines, which often incur much higher costs due to frequent calls to expensive LLMs/MLLMs.\n\n* The paper is very well written, with a clear framework and a good coverage of technical details.\n\n* The proposed method is evaluated against a wide range of recent methods across multiple datasets and shows strong performance, demonstrating its efficiency and effectiveness in a comprehensive way. Ablation experiments also demonstrate the effectiveness of individual components of the proposed method.", "weaknesses": "* The CLIP affinity score is used extensively in multiple stages of the pipeline, which raises questions about how the quality of this score may affect the overall efficiency and accuracy of the pipeline. CLIP models are known to have several limitations, such as a relatively low token number limits of the text encoder (tens to hundreds, versus thousands or more in MLLMs), being less capable at modeling complex relationships in text expressions [1], and being less capable at extracting fine-grained visual information [2]. A low precision or recall may result in excessive frames being sent to reasoning VLM or information loss, potentially impacting the efficiency or quality of the pipeline. A more detailed discussion [e.g., see questions (a, b)] on the issue might clarify the strength and limitations of the proposed approach.\n\n* In Table 2 and 3, it seems that the performance gain of frame selection is closing rapidly as the backbone model quality improves, which raises some concerns about the long-term impact of the work or the frame selection pipelines in general.\n\n\n[1] Yuksekgonul, Mert, et al. \"When and Why Vision-Language Models Behave like Bags-Of-Words, and What to Do About It?.\" The Eleventh International Conference on Learning Representations.\n\n[2] Xie, Chunyu, et al. \"FG-CLIP: Fine-Grained Visual and Textual Alignment.\" Forty-second International Conference on Machine Learning.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8BqwHAlmgl", "reviewer": "Reviewer_oTZs", "summary": "This paper presents a training-free framework to improve accuracy in long-form VQA. The method starts with the adaptive initial sampling by using CLIP-based query-frame similarity and Gaussian mixture modeling, followed by an iterative reasoning loop that selectively applies a foundation VLM to high-potential frames. Experiments on several benchmarks show that the proposed method consistently improves accuracy.", "questions": "In addition to questions in weakness part, the authors are encouraged to address the following issues.\n\nThe iterative refinement uses several manually tuned hyperparameters (e.g., γ, α, β), and it's better to discuss their generalization across datasets.\n\nIt's suggested to discuss the future extensions, especially its combination with agent-based long-form video understanding systems.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The paper is well-written and easy to follow, with clear motivation, framework design, and experimental presentation. \n\nThe three-stage pipeline is also technically coherent and logically structured. The approach is training-free, which is good for integration into various VLMs,\n\nThe experiments and ablation studies demonstrate consistent improvements across multiple benchmarks.", "weaknesses": "While the paper is technically solid, its novelty and conceptual contribution are limited. The proposed method offers an effective improvement by integrating various schemes, but the overall contribution is incremental, because it only refines existing query-based frame-selection paradigms rather than offering a fundamental advance in long-video reasoning.\n\nThe paper does not include comparisons or discussions with agent-based methods, which represent another important line of work in long-video understanding.\n\nThe proposed method remains query-specific and cannot support multi-turn or persistent reasoning across video contexts.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fwfWokNIJB", "reviewer": "Reviewer_iMeg", "summary": "The paper proposes A.I.R., a training-free Adaptive, Iterative, and Reasoning-based framework for frame selection in VideoQA. It first adaptively samples query-relevant events based on similarity distributions, then iteratively refines frame selection using VLM reasoning within a cost-effective loop. Experiments show superior efficiency and accuracy across benchmarks.", "questions": "1. The proposed rule-based frame selection achieves higher recognition accuracy, but the paper does not compare computational efficiency with existing frame selection methods (e.g., frame selection time).\n\n2. The key contribution is the VLM-based iterative frame selection algorithm, yet the correctness of the iterative process requires further clarification. Additionally, it is unclear how performance compares if all frames are selected in a single, non-iterative pass using the same VLM under the maximum budget.\n\n3. Essentially, the paper introduces another key-frame selection method for multimodal (video-language) understanding to improve accuracy. It remains unclear whether this approach generalizes to other video-language tasks, such as video-language retrieval.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "The paper is original in introducing a reasoning-based, training-free iterative framework for frame selection, addressing the cost–accuracy trade-off in VideoQA. The methodology is solid and well-motivated, combining adaptive sampling with iterative refinement. Results show significant efficiency gains, and the paper is clear and impactful, improving VLM applicability to real-world video understanding.", "weaknesses": "The paper lacks runtime comparisons with existing frame selection methods, needs stronger justification of iterative VLM selection, and does not discuss generalization to other video-language tasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)