Threading Keyframe with Narratives: MLLMs as Strong Long Video Comprehenders
Authors: Bo Fang
YuXin Song
Haoyuan Sun
Qiangqiang Wu
Wenhao Wu
Antoni B. Chan
Venue: ICLR 2026 Poster (2026)
Keywords: Multimodal Large Language Models
Long Video Understanding
Keyframe Selection
Keyframe Narratives
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We boost existing MLLMs for training-free long video understanding by introducing a keyframe threading strategy with interleaved narratives.

Abstract: Employing Multimodal Large Language Models (MLLMs) for long video understanding remains a challenging problem due to the dilemma between the substantial number of video frames (i.e., visual tokens) versus the limited context length of language models. Traditional uniform sampling often leads to selection of irrelevant content,  while post-training MLLMs on thousands of frames imposes a substantial computational burden. In this paper, we propose _Narrating KeyFrames Capturing_ (Nar-KFC), a plug-and-play module to facilitate effective and efficient long video perception. Nar-KFC generally involves two collaborative steps. First, we formulate the _keyframe_ selection process as an integer quadratic programming problem, jointly optimizing query-relevance and frame-diversity. To avoid its computational complexity, a customized greedy search strategy is designed as an efficient alternative. Second, to mitigate the temporal discontinuity caused by sparse keyframe sampling, we further introduce interleaved textual _narratives_ generated from non-keyframes using off-the-shelf captioners. These narratives are inserted between keyframes based on their true temporal order, forming a coherent and compact representation. Nar-KFC thus serves as a temporal- and content-aware compression strategy that complements visual and textual modalities. Experimental results on multiple long-video benchmarks demonstrate that Nar-KFC significantly improves the performance of popular MLLMs. Codes are publicly available at https://github.com/bofang98/Nar-KFC.

Published as a conference paper at ICLR 2026
THREADING KEYFRAME WITH NARRATIVES: MLLMS AS STRONG LONG VIDEO COMPREHENDERS
Bo Fang', Yuxin Song", Haoyuan Sun Qiangqiang Wu', Wenhao Wa', Antoni B. Chan City University of Hong Kong, Baidu Inc, Tsinghua University, 'The University of Sydney bofang6 comy. ci cityu. edu. hx, abchanécityu.e edu.
ABSTRACT
Employing Multimodal Large Language Models (MLLMs) for long video under- standing remains a challenging problem due to the dilemma between the substantial number of video frames (i.e., visual tokens) versus the limited context length lan- guage models. Traditional uniform sampling often leads to selection of irrelevant content, while post-training MLLMs on thousands of frames imposes a substantial computational burden. In this paper, we propose Narrating KeyFrames Capturing (Nar-KFC), a plug-and-phe and-play module to facilitate effective and efficient long video understanding. KFC generally involves two collaborative steps. First, we formulate the keyfrume selection process as an integer quadratic programming problem, jointly optimizing query-relevance and frame-diversity. To avoid its computational complexity, a customized greedy search strategy is designed as an efficient alternative. Second, mitigate the temporal discontinuity caused by sparse keyframe sampling, we further introduce interleaved textual narratives generated from non-keyframes using off off-the-shelf captioners. These narratives are inserted between keyframes based on their true temporal order, forming coherent and compact representation. Nar-KFC thus serves as temporal- and content-aware compression strategy that complements visual and textual modalities. Experimental results on multiple long-vides benchmarks demonstrate that Nar-KFC significantly improves the performance of popular MLLMs. Codes are publicly available at https: //g/ thub. com /bo//oof f Fa 98/ Nar-KFC.
INTRODUCTION
Building upon the success of revolutionary Large Language Model (LLMs) (Touvron al., 2023; Team et al., 2024), recent advances in Multimodal Large Language Models (Liu et al., 2023; Li al., 2024b; Wang et al., 2024b; Chen al., 2024c; Tong et al., 2024; Lin al., 2024b) have significantly improved open-world visual understanding. Moving beyond static images, natural extension of MLLMs is their application video understanding. Existing studies have validated their effectiveness in comprehending short videos (~10 s) (Yang et al., 2022; Kim et al., 2024; Yao et al., 2024a). However, when scaling MLLMs to long videos (Fu et al., 2025; Wu al., 2024b; Chandrasegaran et al., 2024; Zhou et al., 2025) (e.g.. hours), several critical challenges emerge.
The primary challenge stems from the inherent context limitation of MLLMs, which cannot accom- modate the vast volume of visual tokens generated from the whole video. A prominent solution is to extend the context window of language models and fine-tune tune them on carefully collected long videos. Current video-oriented LLMs, known VideoLLMs (Lin et al., 2024a; Jin et al., 2024; Song et al., 2024; Xu et al., 2024a; Chen et al., 2024b; Zohar et al., 2024; Shu et al., 2025; Cheng al., 2025a; Wang al., 2025a). typically undergo post-training on existing L.MMs/MLLMs through: 1) employing a relatively large stride uniform sampling scheme, and 2) incorporating token-level merging or compression techniques to enable broader temporal coverage. However, uniform sampling often fails to preserve key moments relevant to specific instructions, while feeding an excessive number of frames as input introduces redundancy, leading to substantial computational overhead. An alternative solution follows a training- free paradigm (Zhang et al., 2024a; Kahatapitiya et al., 2024; Wang et al., 2024d; 2025b; Park et al., 2024; Ma et al., 2025), where raw videos are first converted
"Corresoondigg author.

---

Published as a conference paper at ICLR 2026
into sequential captions, which are subsequently processed using the long-range reasoning abilities of LLMs (Achiam et al., 2023). Compared to direct video frame encoding, textual captions inherently require far fewer tokens, allowing efficient inference in a single forward pass. Nonetheless, the translation from video frame to caption inevitably results in critical information loss (e.g. important visual features), potentially leading to hallucinated answers caused by the L.LM bias.
Regarding the aforementioned paradigms, e.g., training a VideoLLM or reasoning with LLMs on textual captions, are current MLLMs fully equipped comprehend long videos despite their limited context length? Instead of relying on uniform sampling, recent studies have focused on learning select query-relevant keyframes (Yu al., 2023; Hu et al., 2025; Yao et al., 2025) to facilitate inference with MLLMs. Due to the temporal redundancy among adjacent frames, trival similarity-based keyframe selection tends to retrieve frames located within narrow time windows, thereby compromising accuracy. To this end, adaptive keyframe sampling (Tang et al., 2025), inverse transform sampling (Liu et al., 2025b), DPP sampling (Sun et al., 2025) have been proposed to promote content diversity to mitigate the concentration of keyframes. Despite decent boost over existing MLLMs, these methods largely depend on handcrafted or heuristic strategies with limited theoretical formulations, and empirically, the retrieved frames can be temporally distant, especially in long videos. Consequently, the keyframe selection process can introduce temporal discontinuities into the input provided to the MILM, ultimately hindering its holistic understanding of video content.
In this paper, we propose Nar-KFC (Narrating KeyFrames Capturing), a training-free framework for long video understanding with MILMS. Unlike previous approaches, KFC jointly considers query-relevance, frame-diversine and temporal-contiue first stage KFC selects keyframes by considering both query relevance and frame diversity, so as to resolve the issues of critical information loss from uniform sampling and the too-narrow focus using just query- query-relevance. We consider keyframe selection as a graph problem, where each node is frame and the edge weight (score) between nodes combines query- gulevarelevant similarities and frame-to- to-free frame dissimilarities (frame-diversity). The optimal keyframes are obtained by finding the subgraph with largest total edge weight, which can be formulated as an integer quadratic programming (IQP) problem. However, since IQP is NP-hard with exponential complexity, finding exact solutions is infeasible in practice. To overcome this, we introduce robust and efficient greedy search (GS) strategy, which, with proper preprocessing of the score matrix, achieves near-optimal performance with significantly reduced computational complexity.
through two collaborative stages, The
The second stage KFr-KFC addresses the problem of temporal discontinuities caused when selecting keyframes uneven timestamps. Specifically, Nar-KFC works by threading keyframes (visual tokens) with non-keyframe narratives (text tokens), generated by captioning the intermediate, unselected frames in between, aiming reconstruct the video as a continuous and coherent sequence in both textual and visual modalities. A narrative interval is further applied to control the total number of captions and to reduce the similarity between neighboring descriptions. Leveraging only lightweight 2B captioning model, e.g.. Qwen2-VL. VL-2B (Wang et al., 2024b), Nar-KFC demonstrates significant improvements over existing MLLMs. In summary, the contributions of this paper are three-fold:
Jointly considering query-relevance and frame-diversinn we formulate the keyframe captur- ing process (KFC) in long videos as a subgraph selection problem, implemented as an integer quadratic programming problem. We introduce a customized greedy search algorithm to solve this problem with significantly reduced and practical time complexity.
We propose KFC, which threads the optimized keyframes with non-keyframe narratives. By interleaving the two modalities in a temporally continuous manner, KFr-KFC constructs coherent and compact video representations, enabling broader video coverage under the constraint of frame length limitations in current MLLMs.
Our KFC and KFC- are generally compatible with many MLLMs, achieving consistent improvements across four mainstream MILMS on multiple long-video benchmarks.
RELATED WORK
Transformer-basee LLMS have revolutionized the field of natural language processing (Brown et al., 2020; OpenAI, 2023; Grattafiori et al., 2024; Achiam et al., 2023). By incorporating multimodal inputs such as images and videos (Li et al., 2024b; Zhu et al., 2023) with a vision encoder, e.g.. ViT (Dosovitskiy et al., 2020), researchers further extend powerful L.LMs to multimodal large
2

---

or Published as a conference paper at ICLR 2026
language models (MLLMs) for open world visual understanding (Alayrac et al., 2022; Li et al., 2023a; Liu et al., 2023). Despite similar advancements of MILMs on various video understanding tasks including video captioning (Chen et al., 2024a; Yang et al., 2023; Wu et al., 2024a), video question answering (Maaz et al., 2023; Li et al., 2023b; Min et al., 2024), and temporal reasoning (Qian et al., 2024), significant challenges emerge when scaling long videos due to the substantial amount of video frames not fitting in the limited context length of LLMs (Wu et al., 2024b).
Recent studies have explored methods extend the context length of LLMs (Wan et al., 2024; Xiong et al., 2024), or introduced various token-level merging and compression techniques (Song et al., 2024; Shen et al., 2024; Li et al., 2024d; Wang al., 2024c; Shu et al., 2025) to accommodate more frames as input. However, these approaches typically require additional fine-tuning of existing language models, which increases computational complexity and introduces the risk of hallucinations (Liiu et al., 2024c). Given that textual tokens are significantly fewer than visual frames, another line of research first converts all video frames into textual descriptions, which are then used for long video inference, either by summarizing them (Zhang et al., 2024a; Park et al., 2024) or identifying central frames based on textual similarity via agents (Wang et al., 2024d; 2025b; Ma et al., 2025; Ye et al., 2025; Liu et al., 2025a). Nonetheless, the converting process inevitably leads to critical information loss, thereby compromising performance. Other studies, while maintaining the number of input frames, adopt alternative sampling strategies instead of default uniform sampling obtain higher-quality frames for input. In general, query relevance is the primary criterion for selecting frames that are semantically closest to the query (Yu et al., 2023; Lin al., 2024b; Wang al., 2024d;a; Suo et al., 2025). Methods such as AKS (Tang et al., 2025), BOLT (Liu et al., 2025b), Trame-Voyyger (Yu et al., 2025) further propose adaptive sampling, inverse transform sampling, and optimal frame combination sampling to identify keyframes that are both query-relevant and temporally distinctive. Nevertheless, the methods often rely on manually designed heuristics without principled theoretical guidance, and the selected keyframes are often undistributed and distant over long intervals, especially in hours-long videos (e.g., 3600 frames per hour at fps). This temporal sparsity weakens the relationships between frames and can cause confusion in MLLM inference.
In contrast to previous works, we formulate long video keyframe selection as a graph-based op- timization problem with a clearly defined objective, and further leverage the efficiency of textual descriptions. Our approach jointly considers query relevance, content diversity, and temporal continu- ity, aiming to construct optimal combinations of keyframes with interleaved narratives, under the constraints of MLLM context length.
KFC
3 METHOD
Integer Quadratic Programming
Greedy Search
3.1 KFC: KEYFRAME CAPTURING
Uniform sampling commonly used in short video under- standing for consistent temporal structure. However, for long videos, it often misses important information with limited input. While recent works emphasize selecting query-relevant frames for long video QA, they tend to overlook the problem of narrow focus due to the high sim- ilarity between adjacent frames. To address this, we first propose a keyframe capturing method that simultaneously considers query-relevance and frame-diversity. modeling the selection process as subgraph selection problem.
max x s.t. 1 = K X E (0,13
S(i
query- relevance
frame- diversity
SQR
Sp
Preliminaries. General video understanding tasks, e.g., video summarization and grounding (Liu et al., 2024b; Xiao et al., 2024) and video QA, can be similarly formulated as (V,g) v) -> Answer, where V = W represents video with N frames, fi is the i-th frame, and q is the query. Considering an MLLM model as a neural function M(.) with its limited contextual perceiv- ing length, the normal video QA process reasoned by an MLLM model can be formulated as
query frames
Illustration keyframe capturing (KFC). SQR and SED scores are computed via inner dot production.
Figure I:
Answer, 1 < K < N. meaning that only K frames are captured for representing video V. We next consider two criteria for selecting the K frames, querycrelevance and frame-diversity.
3

---

Published as a conference paper at ICLR 2026
Input Video
Q: Why does mather brown bear in make her cubs run to higher ground?
Captioner
Nar-KFC
KFC
standing niver..
nocky.
A photographer holding
341s The brown bears wolking grazt
10r: A Fish jumping out of the water...
Captaring
II5 powerfed waterfall crashing
Arols around
365: seaplane with pentoons londs...
off-the-shelf
MLLM
Becouse she spotted two adult male brown bears.
main stream
Figure 2: Illustration of Nar-KFC. We represent long videos by threading KFC-optimized keyframes with temporally interleaved narratives, where the narratives are generated frame-wise by an off-the-shelf captioner. Nar-KFC constructs a continuous representation to facilitate MLLM inference.
Query-relevance. Since different questions can be asked on the single video, it is crucial to identify frames that correspond to specific query first. Here, standard two-stream rision-language model (VLM), e.g.. CLIP (Radford et al., 2021), is used to extract embeddings [fr)/r and q for the frames and the query, respectively. After standard normalization of all embeddings, the query-relevance score SQR is computed as the cosine similarity between the two, SQR (2) = simmf., q).
Frame-diversity. To avoid retrieving query-relevant only frames that are narrowly located in a small time range, we explicitly encourage diversified content when choosing the K frames. In particular, use the inverse of cosine similarity between every pair of frame embeddings (normalized) to represent the diversity score. The function exp(.) is applied to constrain the score between 0 and 1, sim(f,,f,,)
formulated as SFD(i.
Objective. The final score combines SQR and SED to jointly identify keyframes that are both query-relevant and diversified for KFC,
S(i.g) (i) + (FD(i.)) sim(f., exp((sii({{,.)))
(I)
Next, illustrated in Fig. 1, construct graph where each node is frame, and the edge weight between node pair (i, is Si..j). The selection K keyframes can then be cast as a subgraph selection problem with the original objective as follows: given N nodes (frames), construct sub- graph by selecting K nodes (keyframes) so as to maximize the total edge weight of the subgraph. Mathematically, this objective can be expressed as the optimization problem:
Σ S(i.)),
max YCLL NE.Y-
(i,))Z
where Y (u ,yK] is the index set of the K keyframes and I denotes all pairs (i,j).
3.1.1 THEORETICAL OPTIMUM: INTEGER QUADRATIC PROGRAMMING
Our objective closely resembles the classic Knapsack problem (Salkin & De Kluyver, 1975), which be commonly solved by dynamic programming integer linear programming. The problem in (2) can be rewritten equivalently as an integer quadratic programming (IQP) problem,
max x" Sx s.L. 1f K, x4 (0,
where x; indicates that the i-th frame is selected. x [x] 22, IN and E N the score matrix with S S(i,j) for < j. and S 0 otherwise. Here, only the upper triangle of is considered. A discussion symmetrical S is detailed in Appendix $E.1. The search space C(N, K), and the time complexity of solving IQP is exponential regardless of whether the objective is convex or non-convex, making impractical to get exact solutions in real cases. Modern optimization tools, e.g., CPLEX ( et al., 2014), typically address this by relaxing the binary
4

---

Published as a conference paper at ICLR 2026
constraint and allowing 2; E [0, 1], converting the problem into continuous optimization task. Solutions can then be obtained using methods like interior- pooert or Lagrange multiplier methods, with complexity of ( Subsequently, the Branch & Bound algorithm (Morrison et al., 2016) is applied to prune the search space and retrieve optimal integer solutions of I;, but the worst-case time complexity remains exponential.
3.1.2 PRACTICALLY FEASIBLE APPROACH: GREEDY SEARCH
Solving the IQP optimally is computationally intractable for large N, e.g., long videos with thousands of frames. To search keyframes within practical latency constraints, we propose an efficient greedy search (GS) strategy that yields robust and near-optimal effects the IQP solution. We first pre- process the score matrix to reduce noise across adjacent columns/rows, and shrinks the problem size for greater computational efficiency. Specifically, we apply singular value decomposition (SVD) to the score matrix S, retaining the top singular values to construct a low-rank approximation S, E R N This matrix is then uniformly downsampled to Syd IR x A with downsampling ratio The GS algorithm begins by selecting the most query-relevant frame the starting point. It then iteratively adds the frame with the highest cunulative score relative to the already selected frames. In the final refinement step, the algorithm examines the k-nearest neighbors each selected frame yo, replacing y/ with neighboring frame if yields higher cumulative score based on S,. A summary the algorithm is provided in Alg. I, and its overall time complexity is O(NK).
Algorithm 1: Practically Feasible Approach with Greedy Search
Input: Query-relevant score SQR, score matrix S, number of retained singular values
downsample ratio d, number of frames N, neighbor window k.
Output: Indices of selected K frames set Y = (U, .UK)
S,d < Downsample(S.,, d); // Decompose and downsample $ 31 argmax; SQR (i); Y + {vi} // Initialize with most query-relevant frame for i + to K do
S, LowRank(S))
for j < to N do
= argmax, Eyer Sra(y. // Select frame with highest sum
Y - Yum
for i +- 1 to K do L Y/ Refine yt, k(S,): // Refine selection within k-nearest neighbors
return Y sorted vn "2... Nkk;
3.2 KFC- THREADING KEYFRAME WITH NARRATIVES
Keyframes captured by KFC significantly enhance the performance of MILMs compared to the default uniform inference mechanism. However, overlooks the temporal-continuity in frame sequences. Due the severely uneven distribution of selected frames, temporal relationships become weak, often leading to confusion during inference.
To this end, propose Nar-KFC, which threads keyframes with text narratives construct continuous and coherent input in an interleaved form. Specifically, we first use lightweight off-the- shelf captioner, c.g., Qwen2-VL-2B, generate captions {c for non-keyframes using simple prompt as " "<USER> Describe this video frame in no more than 15 words." Given the unevenly distributed keyframes {fr K from KPC, we insert captions from non-keyframes between the keyframes, arranging them according to their true temporal order. Each denotes the timestamp, and a uniform interval is set between captions to control the total number of inserted narratives. The overall long video inference to MILM model M is formulated as:
M(Ify. Cyr+A. ...
Go Cy ... .Cyx fux - Answer.
Viability of KFC. MILMs are typically trained via instruction tuning on both visual and textual modalities, making them well-suited to process our interleaved inputs of keyframes and narratives. Rationality of KFC. The approach provides a temporally continous input that helps MLLMs "narrate" " the story between keyframes. From another perspective, NFC-KFC can be seen as form of compression, retaining only the most informative keyframes, while representing less critical
5

---

Published as a conference paper at ICLR 2026
segments with brief textual descriptions. This complementary two-stream mechanism is analogous to method like Two-Streem (Simonyan & Zisserman, 2014), which combines RGB frames with optical flow. Also, it shares conceptual similarities with SlowFast (Feichtenhofer et al., 2019) and FlowFast-L (Xu et al., 2024b), where the caption stream serves as a fast branch traversing a broader temporal range (as in the low frame rate of the slow branch in SlowFast). These mechanisms together help explain the effectiveness of Nar-KFC in (long) video understanding.
4 EXPERIMENTS
4.1 EXPERIMENT SETTINGS
Evaluation Benchmarks. We evaluate our methods on several widely-used long-video question-
answering benchmarks: 1) Video-MME (Fu et al., 2025), consisting of 2,700 human-annotated QA pairs, with an average video duration of 17 min: 2) Long VideoBench (Wu et al., 2024b) validation set (denoted as LVB), which contains 1,337 QA pairs with average duration of 12 min: 3) MLVU (Zhou et al., 2025), where we use the multiple-chokee 9 categories, with an average duration of 12 min. Besides, open-ended generation performance on MMBench-Videe evaluated, to verify the fine-grained capabilities of our methods. We provide more results of on relatively short EgoSchema (3 min) (Mangalam et al., 2023) and NEXTQA (44 sec) (Xiao et al., 2021) benchmarks in Appendix $D.3.
task (M-avg), comprising 2,593 questions across
(Fang et al., 2024) and MLVU-OpenEnded (Zhou et al., 2025) (G-avg) are
Evaluation Models. We consider multiple advanced MLLMs, including Intern VL. (Chen et al., 2024c), Qwen2.5-VL (Bai et al., 2025b), LLaVA-One (Li et al., 2024b), VL-VV-iiie (Zhang et al., 2024d), IL3 (Zhu et al., 2025), and Qwen3-VL (Bai et al., 2025a), to verify the effectiveness of our methods. We re-implement baseline results (uniform sampling) of these MLLMs using VLMEvalKit (Duan et al., 2024), which may yield slight differences compared to other public toolkits, e.g., L.MM--vall (Li et al., 2024a).
Implementation Details. We use -IIP-VVT---336px (Radford al., 2021) extract query and video frame embeddings. Candidate frames are sampled from raw videos at 1 fps. For solving the IQP, we limit the maximum search nodes to 40k in CPLEX (Blicklú al., 2014). In our customized greedy search algorithm, we empirically retain the top 4 singular values to form the low-rank approximation of the score matrix S and further downsample following previous work (Yu et al., 2025; Sun et al., 2025). The refinement window size k is set to 2 (see ablations of hyperparameters Appendix ŞE.3). Unless otherwise stated, all ablations are conducted using the Intern VL2 model on Video-MME. Experiments are run on 8 A A100 GPUs.
N
a fixed resolution of 128 128
4.2 BENCHMARK RESULTS
Comparisons with State-of-the-Arts. We conduct comprehensive comparisons between our ap- proach and several recent MLLMs and VideoLLMs in Tab. 1. Earlier works, e.g., Video-LLaVA (Lin et al., 2024a), Chat-UniVi-VI.55 (Jin et al., 2024), VideoLLaMA2 (Cheng et al., 2024), etc, are fully included in Appendix $D.1. Our methods, KFC and Nar-KFC, deliver consistent and significant gain over five baselines across three long-video benchmarks. On Video-MME (no sub.), Nar- KFC outperformsfive MLLM baselines by 4.38% in average. Using the strongest baseline, i.c., In3, Nar-KFC achieves state-of-the-arr performance (63.8%), surpassing previous VideoI.LMs even those using larger LLMS (e.g., VILA--4B, 58.3%) or more frames (e.g., Video-XI. 3501122000mm55555 55.5%) Incorporating larger numbers of frames may introduce noise and irrelevant information, which can be well addressed by our keyframe capturing and narrating strategies. On LVB, our method also achieves notable performance improvements, e.g., 52.3% vs. 53.9% with Intern and 52.7% vs. 55.3% with Qwen2.5- LL, although the overall gain is partly offset by videos shorter than 1 min, demonstrating clear advantages in long video understanding. On MLVU, our KFC-only strategy (without narrations) yields an average improvement of over 6% across five MILMMs. The use of query-relevant and diverse keyframes significantly boosts performance on Needle-in-a-haysee in-a-haystccck (Zhang et al., 2024b) and counting questions. Furthermore, appending narratives provides additional and robust gains by preserving temporal continuity. Detailed analysis are further presented in Appendix $D.4.
Comparisons with varying number of keyframes. In Fig. 3, we compare KFC and KFC against uniform sampling with varying frames across three benchmarks and three models. Due to Qwen2.5- VL's dynamic resolution mechanism (Dehghani et al., 2023), increasing keyframes often leads to memory overflow, so its results are omitted. Notably, Nar-KFC shows substantial gains when the number of keyframes is limited (e.g., 4 or 8), due to its ability provide broad video

---

Published as a conference paper at ICLR 2026
Table Comparisons with previous VideoLLMs Video-MME, LVB, and MLVU. All methods evaluated using frames. For Iieo-MMEE report performance two standard settings: without subtitles (no sub.) and with subtitles (sub.). LVB denotes the Long VideoBench set. Methods that use significantly more frames and larger-sized LLM are marked in gray. The reported results are accuracy percentage.
Vidco-MME ao sub. sub.)
Model
Size
LVB
MLVU
Short
Medium
Long
Overall.
~12m
~12
VILA (Lin et al., 2024b)
8B
37.8761.6
44.3/46.2
40.3 /
50.5
46.3
LLaVA-NeXT-OW2 (Liu 2024a)
58.0/
47.0/
43.4/
49.5/
MiniCPM-V2.6 (Yao 2024h)
61.1 /63.8
50.3/50.2
46.4/45.4
52.6/53.1
51.2
55.4
Lu (Shen al., 2024)
64.7/
58.2/
59.5/
60.6/
65.4
BOLT (Liu al., 2025b)
7B
66.8
54.2/
47.3/
S6.1
55.6
63.4
Prame-Voyagee (Yu al., 2025)
8B
67.3/
56.3/
48.9/
57.5/
65.6
LongVILA 250ffm (Chen ct al., 2024b)
8B
61.8/
49.7/
39.7/
50.5
Video-XL. 2000m Video-XL_ (Shu al., 2025)
7B
67.0/6744
53.2/60.7
49.2/54.9
55.5/610
50.7
64.9
LLaVA-NeXT-Videe (Zhang al., 2024c)
34B
61.7
50.1/5222
44.3/47.
52.0/
50.5
58.8
VILA (Lin et al., 2024b)
34B
/
58.3/62.7
51.2/55.7
58.3/61.6
57.8
IntervL22 (Chen et al., 2024c)
8B 62.1763.9
48.2748.7
45.2744.9
51.972.3
52.3
54.3
KFC
8B 64.5
50.0/52.3
46.5
55.5
53.3
62.2
Nar-KFC
8B
67.2
54.7/57.9
47.1/48.9
56,3/
53.9
64.4
Qwen2.5-VL (Bai et al., 2025b)
7B
65.97664
66.4
54.4754.3
45.8/46.9
55.4
52.7
53.8
KFC
7B
68.8/70.7
52.6/
49.3/514
56.9/
54.3
62.6
Nar-KFC
7B
70.1/710
54.4/55.2
49.0/49.4
58.6
55.3
64.4
LLaVA-One Vision (Li et 2024b)
7B
65.2/67.1
51.7/54.4
45.1/46.1
53.3/55.9
54.5
58.5
KFC
69.4/69.1
52.9/568
4688/48.8
58.2/
55.6
65.0
Nar-KFC
67.2
68.6
57.1/59.8
49.1/510
97.88599.8
56.5
66.2
LLaVA-Video (Zhang et al., 2024d)
67.2
53.2/53.4
47.2/47.3
56.97
54.2
60.5
KFC
7B 68.3
55.1
49.4/516
/9.67
56.5
66.9
Nar-KKC
7B
72.2
62.3
52.0/
61.6 63.0
57.7
67.7
InterVL3 (Zhu ct 2025)
8B
68.7
70.9
50.0 58.3/5.2
50.9
59.0 60.0
53.6
60.9
KFC
8B
70.9/719
60.6
60.1
50.9/5188
60.8 61.4
54.5
67.5
Nar-KFC
8B
72.9/33.
62.9 62.7
/
63.8 64.1
54.8
68.4
VitenVL2
LLeVA-OueVision
VL
Vidco-MME: Acenecy
LongWideoDenchh Accurasy %
36
MLVU Accuracy
700
56
54
54
65
-2
Unifarm
Uniform
Uniform
KFC
Nar KFC
Kar
Number of frames
Number of frames
Number of frames
Figure 3: Accuracies (%) of uniform sampling, KFC, and KFC-KFC versus numbers of keyframes.
coverage via interleaved textual narratives. As the number of keyframes increases, the performance gap between uniform sampling and our methods narrows. This can be attributed to: 1) uniform sampling is more likely capture key moments when more frames are used; and 2) many video QA questions typically only require a few number of frames to accurately answer in current benchmarks. Interestingly, MLVU, KFC alone outperforms Nar-KFC with 32 keyframes, suggesting that when sufficient keyframes are present, the added benefit of narratives diminishes. These results underscore the strength of KFC in selecting informative keyframes while demonstrating that narratives are particularly valuable when MILMss have limited context capacity. We further scale Nar- KFr-KFC to 72B models and compare them with proprietary models and SOTA VideoLLMs in Appendix $D.2.
Improvements on open-ended generation tasks. In Tab. 2). we show NFr-KFC consistently improves performance on open-ended generation tasks that require fine-grained reasoning, demon- strating that even with global-level frame selection and captioning, our methods can still enhance fine-grained tasks. Interestingly, we find that KFC shows decreased performance on the MLVU- OpenEnded summary task, likely because uniformly sampled frames cover the entire video range, whereas KFC-selected frames may be more concentrated. Our Nar-KFC addresses this issue by providing more comprehensive video information.
4.3
ABLATION AND ANALYSIS
KFC and Nar-KFC ablations. We report the ablation results of KFC and Nar-KFC components on the Video-MME (sub.) and MIVU benchmarks in Tab. 3. Simply inserting narratives between
7

---

Published as a conference paper at ICLR 2026
Table 2: Improvements on open-ended generation tasks. All results reported based on 8-frame evaluation. For MMBench- Video. GPT-4-114 used as the judge model, while for MLVU--penEnded, 2- serves as the default judge model, following the official implementations.
MMBench- Video
MLVU-OpenEnded
Model
Perception
Reasoning
Overall
Sub_scene
Summary
G-Avg
3 (Zhu et al., 2025)
1.54
1.61
1.57
5.47
4.40
4.92
+ KFC
1.56
1.58
1.58
5.73
4.23
4.95
Nar-KFC
1.76
1.78
1.78
5.69
4.39
5.02
Qwen3-VI--BB (Bai et al., 2025a)
1.62
1.65
1.64
6.17
6.01
6.09
KFC
1.75
1.69
6.32
6.00
Nar-KFC
1.75
1.76
1.76
6.28
5.97
6.12
Table 3: Main component ablation results in Nar-KFC. "S, M. L" refer to short, medium, and long video categories in the Video-MME (sub.) benchmark.
Table 4: Effects of including pre-processing and refinement stages in the KFC Greedy Scarch (GS) method. V-MME denotes the overall Video- MME (sub). Line (ii') indicates Downsampling without Lawkank The final KFC (GS) strategy integrates all components from (i) to (iv).
Video-MME
Strategy
MLVU
Time
S
M
L
Overall
Uniform
63.9
48.7
44.9
52.5
54.3
((1)
+ Narratives
66.1
54.9
45.2
55.4
59.4
O(N)
Ex# Strategy
V-MME
MLVU
(ICP)
65.9
52,9
46.4
55.1
62.0
((2%)
Vanilla GS
52.3
60.4
KFC (GS)
65.4
52.3
47.3
55.0
62.2
O(NK)
(
Initialization
53.3
61.0
w/o SQR
62.3
47.8
45.3
51.8
$7.3
O(NK)
(ii)
+ LowRank
53.7
61.8
w/o SFD
63.6
49.4
44.6
52.5
60.9
O(NK)
(1')
Downsample
53.9
61.6
(iii
+ LowRank + Downsample
54.7
62.2
Nar-KFC
67.7 57.9 48.9
58.1
64.4
O(NK)
(iv)
+ Refinement (KFC)
55.0
62.2
uniformly sampled frames yields improvements of 2.9% on Video-MME and 5.1% 1% on MLVU, indicating that adding narrative context, despite with frames not being query-specific, can effectively boost overall video understanding. To retrieve query-relevant and diverse keyframes, our Greedy Search (GS) strategy achieves results comparable to the optimal Integer Quadratic Programming (IQP) method (55.0% vs. 55.1 % on Video-MME and 62.2% vs. 62.0% on MLVU), while being significantly more efficient with ()NK) complexity. Details of our IQP implementation and comparisons with GS are provided in Appendix $E.2. Further ablations show that removing the query-relevance score SQR leads to 3.2% drop Video-MME and 4.9% on MLVU with greedy search. This emphasizes that retrieving query-relevant frames is critical in long videoQA. Mcanwhile, incorporating frame diversity SED further stabilizes and enhances performance across benchmarks. When threading all keyframes with interleaved narratives, Nar-KFC achieves the best overall results on all metrics, underscoring its solid effectiveness representing long video contents.
Component analysis of greedy search (GS). Starting from the vanilla GS, which iteratively selects the frame with the highest cumulative score relative the already selected frames, we progressively incorporate several techniques (Tab. 4) to enhance its effectiveness to near-optimal solution: (i) initialization with the frame most relevant to the query brings a modest yet consistent gain (from on MLVU); (ii and iii) applying low-rank denoising and downsampling further improves performance by producing a more compact and less noisy score matrix S; and (iv) adding the final refinement step, KFC (GS) achieves the best results of 55.0% on Video-MME and 62.2% on MLVU. This highlights the cumulative benefit of combining compact frame representations, reduced redundancy, and an iterative selection mechanism.
52.3% 3% Video-MME, and 60.4%-
Comparisons with other keyframe selection methods. We compare KFC with several keyframe extraction baselines in Tab. 5, all utilizing the Intern VL2 backbone and 8 frames. Details are in Appendix $E.4. Methods that apply top-K frame-query matching using SigLIP (Zhai et al., 2023), or BLIP-2 (Li et al., 2023a) embeddings perform worse than uniform sampling. possibly due to keyframes being concentrated within a narrow temporal window. For those localize-then-answer meth-
Table 5: Comparisons with dif- ferent frame selection meth- ods on Video-MME.
V-MME /sbl/s
InternVL22
$1.9/
+ CLIP (top-K)
47.7 50.0
+ SSgg.IP (top-K)
47.3/51.0
ods, i.e., TempGQA (Xiao et al., 2024) and ViLA (Yu et al., 2023), performance heavily depends on the quality of segment localization, which can be unreliable. Recent approaches including DPP (Sun et al.,
+ -LIP-2 (lop-K)
47.8/
TempGQA
50.4/51.1
SeViLA
52.2/
+ DPP
52.2/53.5
2025), AKS (Tang et al., 2025), and BOLT (Liu et al., 2025b) generally yield better results by incorporating frame diversity. However, these methods rely handcrafted and heuristic sampling strategies, lacking
+ AKS
52.8/
+ BOLT
53.3/
+ KFC (Ours)
53.5/ 55.0
8

---

Published as a conference paper at ICLR 2026
Table 6: Analysis of video input components on Video-MME (no sub). Superscript numbers indicate the quantity, Average time and tokens per video are reported.
Table 7: Temporal structure analysis between narratives and keyframes on Video-MME (no sub) benchmark.
Components
V-MME Latency (s)
TFLOPs L
Tokcui
Temporal Structure
V-MME
Narratives
51.1
0.98
109.6
4,725
Frames (uniform)
51.9
1.03
146.3
6,280
[Narrative] > [Keyframe - Query)
55.5
Frames (KFC)
53.5
1.31
[Keyframe) - (Narrative Query)
55.3
Interleave +210 (Nar-KFC)
56.3
2.13
202.6
11,005
Intericave -Nr-KKFC-++(0 (Query)
56.3
Qwen2-VL-28 Owen2-VL-78 InterVL2.5-22 IntenMV2.5-68
Owen2-VL-726
Overall Short
Medlum Long
60
70
Accuracy
60
56
54
40
35
70
140
210
280
Number of narrations
Captioner
Figure 4: Effect of the total numher of inserted narra- tives, corresponding to the narrative interval A, across videos of different lengths.
Figure 5: Impact of different captioners for gener- ating narratives. Video-MME, (sub.) results are for IntemVL2-SB (left) and Qwen2-VL-7B (right).
a principled and generalized frame selection guidance. In comparison, our proposed KFC consistently outperforms all baselines, demonstrating clear superiority in subset frame selection.
Effect of narrative quantity. Due to the varying length of videos, we do not directly ablate the effect on fixed interval value A. Instead, we control the total number of narratives appended. as shown Fig. 4 on Video-MME: (sub.). Narratives are incrementally added across 7 intervals between keyframes. The overall accuracy improves steadily from 55. 1% to 58. as more narratives are available, with more performance gains medium and long videos. However, since adjacent frames often contain similar visual information, adding more narratives results in diminishing returns due to redundant descriptions. We thus use 210 narratives as the default.
Effect of narrative quality. Fig. 5 presents the impact of different captioners on quality of generated narratives and the resulting performance of Nar-KFC the Video-MME (sub.). We evaluate five MLLMs of varying sizes and sources as captioners. Narratives extracted from the largest captioner, Owen2-VL--72B, achieves the best accuracy, i.e., 58.9% on Intern VL.2-88 and 58.3% on Qwen2-VL- VL-7B, highlighting the benefit of higher-quality narratives. Nevertheless, the overall performance gap across all captioners is small (less than 1%). This suggests that keyframes play dominant role in long video understanding, while captions serve as auxiliary and supportive context. We thus use the lightweight Qwen2- VL- as the default captioner for other benchmarks.
Efficiency and effectiveness between narratives and keyframes. We decompose Nar-KFC into standalone narratives and frames Tab. Although translated from 210 frames, pure narratives perform worse than even 8 uniformly sampled frames (51.1% 1% vs. 51.9%), which reflects that sub- stantial information is lost during the cocatioon conversion. Nevertheless, narratives exhibit advantages with the shortest latency (0.98s) and the fewest tokens (4,725 per video). Combining narratives with KFC-selected keyframes (Nar-KFC) achieves both the best accuracy and also main- tains reasonable efficiency. We discuss detailed computational overhead in Appendix $D.5. In addition, Tab. 7 investigates the temporal structure between narratives and keyframes. Placing all keyframes either before after the narratives degrades the performance by 0.8% and 1.0%. likely due to disrupted temporal sequences. In contrast, interleaving narratives and frames, as in Nar-KFC, yields superior results. These findings further validate our primary goal: constructing temporally continuous representations for long video understanding.
4.4 QUALITATIVE RESULTS
Fig. 6 presents two qualitative examples of our method. In the first example (left), our KFC effectively identifies frames that are both query gulevarelevant and content-diverse, resulting in the correct answer. In the second example (right), we demonstrate that Nar KFC substantially improves reasoning in a complete relay race scenario by threading temporally interleaved keyframes with coherent narratives.

---

Published as a conference paper at ICLR 2026
ihht food
boih the hero's braakfastt hunch?
Phich the video reached finish line first?
KFC
"C. Bread."
$ DD. South Africa feam..
0
male otlete foo Brazil is celdiriing on trnck.
Aauup of atilits track, one bolking haus.
(i male athirio ralay man paoing
0
Bread."
Jel mie aiikes roky
from Canadaa and one from USA. are runsing tawards the fisish line.
jerny
culebrating
KFC
E woop of people inclading iin man laking a selfie with Canadian flas are
gallored in stadian..
"A. Milk
BB. Canadian
Figure 6: Qualitative results. (left) Comparison of frames selected by uniform sampling, top-K sampling, and our KFC. (right) Key narratives generated by Nar-KFC that lead to the correct answer. Zoom in for details.
This enables accurate inference the final winner, whereas KFC fails due to limited number of frames. More examples can be found in Appendix $F.
CONCLUSION
In this paper; we propose keyframe capturing strategy (KFC) and a narrating keyframe method (NarKKFC) to boost existing MLLMs for long video understanding, under the constraint of limited context length in language models. Our approach constructs long video representations that are query- relevant, content-diverse, and temporally continuous, all achieved in training-free manner. This significantly improves the performance of current MLLMs on widely-used long video benchmarks. Our findings strongly validate the potential of MLLMs as effective long video comprehenders.
ACKNOWLEDGMENTH
This work was supported by Strategic Research Grants from City University of Hong Kong (Project.
No. 7005840).
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Lconi Aleman, Diogo Almeida. Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023.
-ean-Baptistte Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Lain Barr, Yana Hasson, Karel Lene, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems (NeurIPS), 35:23716 16-23736, 2022.
Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: frontier large vision-language model with versatile abilities. arXiv preprint NWV.23388.1966. 1(2):3, 2023.
Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo. Qidong Huang. Jie Huang, Fei Huang. Binyuan Hui, Shutong Jiang, Zhaohai Li. Mingsheng Mei Li. Kaixin Li, Zicheng Lin, Junyang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Yang Liu, Dayiheng Liu, Shixuan Liu, Dunjie Lu, Ruilin Luo, Chenxu Lv, Rui Men, Lingchen Meng, Xuancheng Ren, Xingzhang Ren, Sibo Song, Yuchong Sun, Jun Tang. Jianhong Tu, Jianqiang Wan, Peng Wang, Pengfei Wang, Qiuyue Wang, Yuxuan Wang. Tianbao Xic, Yiheng Xu, Haiyang Xu, Jin Xu. Zhibo Yang, Mingkun Yang, Jianxin Yang,
Yang, Bowen Yu, Fei Zhang, Hang Zhang, Xi Zhang, Zheng, Humen Zhong, Jingren Zhou, Fan Zhou, Jing Zhou, Yuanzhi Zhu, and Zhu. Qwen3-vl technical report, 2025a. URL hs petettss://ar/// //aaxivv.../////////a/ www.xvvvov....oo.o org. awbg////////. /os./511111 1.t...211.........1.........1111
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Siho Song. Kai Dang, Peng Wang, Shijie Wang, Jun Tang, al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502. 11.5002..199333 13923, 2025h.
10

---

Published as a conference paper at ICLR 2026
Christian Blicklú, Pierre Bonami, and Andrea Lodi. Solving mixed-integer quadratic programming problems with ibm-eplex: progress report. In Procoedings the twenty-sixth RAMP symposium, pp. 16-17, 2014,
Tom Brown, Benjamin Mann. Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few Advances in neural information processing systems (NeurIPS), 33:1877-1900 2020.
Keshigeyan Chandrasegaran, Agrim Gupta, Lea M Hadzie, Taran Kota, Jimming He, Cristóbal Eyzaguirre, Zane Durante, Manling Li, Jiajun Wu, and Fei-Fei Li. Hourvideo: 1-hour vidco-language understanding. Advances in Neural Information Processing Systems (NeurIPS), 37:53168- 53197, 2024.
Lin Chen, Xilin Wei, Jinsong Li, Xiaoyi Dong, Pan Zhang. Yuhang Zang. Zehui Chen, Haodong Duan, Zhenyu Tang, Li Yuan, et al. Sharegnt-video: Improving video understanding and generation with better captions. Advances in Neural Information Processing Systems (NeurIPS), 37:19472-19499 2024a.
Yukang Chen, Fuzhao Xue, Dacheng Li, Qinghao Hu, Ligeng Zhu, Xiuyu Li, Yunhao Fang, Haotian Tang. Shang Yang, Zhijian Liu, et al. Longvila: Sealing long-context visual language models for long videos. arXiv preprint arXiv:2408. 10188, 2024b.
The Chen, Jiannan Wu, Wenhai Wang. Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou 7hu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generie visual-linguistic tasks. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 24185-24199 2024c.
Chuanqi Cheng, Jian Guan, Wei Wu, and Rui Yan. Scaling iideo-language models to 10k frames via hierarchical differential distillation. arXiv preprint arXiv:2504.02433 220444488, 2025a.
Junhao Cheng, Yuying Ge, Teng Wang, Yixiao Ge, Jing Liao, and Ying Shan. Video-holmes: Can milm think like holmes for complex video reasoning? arXiv preprint arXiv:2505.21374 2025b.
Zesen Cheng, Sicong Leng, Hang Zhang, Yifci Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang, Ziyang Luo, Deli Zhao, et al. Videollama 2: Advancing spatial-temporal modeling and audio understanding in video-llms. arXiv preprint arXiv:2406.07476 2024.
Mostafa Dehghani, Basil Mustafa, Josip Djolonga, Jonathan Heek, Matthias Minderer, Mathilde Caron, An- dreas Steiner, Joan Puigeerver, Robert Geirhos, Ibrahim M Alabdulmohsin, al. Patch n' pack: Navit, vision transformer for any aspect ratio and resolution. Advances Neural Information Processing Systems (NeurIPS). 36:2252-2274. 2023.
Alexey Dasovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenbom, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, G Heigold, S Gelly, et al. An image is worth 16x 16 words: Trans- formers for image recognition scale. In International Conference Learning Representations (ICLR), 2020.
Haodong Duan, Jumming Yang. Yuxuan Qiao. Xinyu Fang, Lin Chen, Yuan Liu, Xiaoyi Dong, Yuhang Zang. Pan Zhang, Jiaqi Wang, et al. Vlmevalkit: An open-sounce toolkit for evaluating large multi-modality models. In Proceedings of the 32nd ACM International Conference on Multimedia (MM), pp. 11198- 1199811201, 2024.
Xinyu Fang, Kangrui Mao, Haodong Duan, Xiangyu Zhao, Yining Li, Dahua Lin, and Kai Chen. Mmbench- video: A long-form multi- benchmark for holistic video understanding. NeurIPS, 37:89098 89124, 2024.
Christoph Feichtenhofer. Haoqi Fan, Jitendra Malik, and Kaiming He. Slowfast networks for video recognition. In Proceedings of the IEEE/CVF international conference on computer vision (ICCV), pp. 6202 6211, 2019.
Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang. Zihan Wang. Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal 11ms in video analysis. In Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR), pp. 24108-241 24,18, 2025.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The lama 3 herd of models. arXiy preprint arXiv:2407.2 21733, 2024.
Kai Hu, Feng Gao, Xiaohan Nie. Peng Zhou, Son Tran, Tal Neiman, Lingyun Wang, Mubarak Shah, Raffay Hamid, Bing Yin, et al. M-Im based video frame selection for efficient video understanding. In Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR), pp. 13702-13712, 2025.
11

---

Published as a conference paper at ICLR 2026
Peng Jin, Ryuichi Takanobu. Wancai Zhang. Xiaochun Cao, and Li Yuan. Chat-univi: Unified visual representa- tion empowers large language models with image and video understanding. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13700-13710, 2024.
Kumara Kahatapitiya, Kanchana Ranasinghe, Jongwoo Park, and Michael S Ryoo. Language repository for long video understanding. In Workshop on Video-Language Models@ NeurlPS 2024, 2024.
Wonkyun Kim, Changin Choi, Wonseok Lee, and Wonjong Rhee. An image grid can be worth a vidco: Zero-shot video question answering using vlm. IEEE Access, 2024.
Bo Li, Peiyuan Zhang, Kaichen Zhang, Fanyi Pu, Xinrun Du, Yuhao Dong, Haotian Liu, Yuanhan Zhang, Ge Zhang, Chunyuan Li, and Ziwei Liu. Lmms-eval: Accelerating the development of large multimoal models, March 2024a. URL https: : /////tttuuul com/ Eevevvvoviiiiii wrrovinngMMM Lab/ httm//mmssseeeeeee -eval.
Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang. Feng Li, Hao Zhang, Kaichen Zhang. Peiyuan Zhang. Yanwei
Li, Ziwei Liu, al. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.0333 03326. 2024b.
Junnan Li, Dongxu Li. Silvio Savarese, and Steven Hoi. 22ip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. International conference on machine learning (ICML), pp. 19730-19742. PMLR, 2023a.
KunChang Li, Yinan He, Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang. Limin Wang, and Yu Qiao. Videnchat: Chat-centric video understanding. arXiv preprint arXiv:2305.06355 2023b.
Xinhao Li, Yi Wang, Jiashuo Yu, Xiangyu Zeng. Yuhan Zhu, Haian Huang. Jianfei Gao, Kunchang Li, Yinan He, Chenting Wang, et al. Videochat-flash: Hierarchical compression for long-context video modeling, arXiv preprint arXiv:2501 200554, 2024c.
Yanwei Li, Chengyao Wang, and Jiaya Jia. Ilama-vid: An image is worth 2 tokens in large language models. In European Conference on Computer Vision (ECCV), pp. 323-340. Springer, 2024d.
Jianxin Liang, Xiaojun Meng, Yueqian Wang. Chang Liu, Qun Liu, and Dongyan Zhao. End-to-end video question answering with frame scoring mechanisms and adaptive sampling. arXiv preprint arXiv:2407. 15047, 2024.
Bin Lin, Yang Yc, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, and Li Yuan. Video-llava: Learning united visual representation by alignment before projection. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 5971-5984. 2024a.
Ji Lin, Hongxu Yin, Wei Ping, Pavlo Molchanov, Mohammad Shoeybi, and Song Han. Vila: On pre- pre-training for visual language models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp. 26689-26699, 2024h.
Haotian Liu, Chunyuan Li. Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems (NeurIPS), 36:34892-34916, 2023.
Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jac Lee. Llava-next: wag.aaaaaaaaaiiiaaa....
Improved reasoning, ocr, and world knowledge, January 2024a. URL
iooobl cgllog/22000 22024-22222 10-30- 1avaa- nextt/
Huabin Liu, Xiao Ma, Cheng Zhong, Yang Zhang, and Weiyao Lin. Timecraft: Navigate weakly-supervised temporal grounded video question answering via bi-directional reasoning. In European Conference on Computer Vision (ECCV), pp. 92- 107. Springer, 2024b.
Nelson F I Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics (TACL), 12:157-173, 2024c.
Ruyang Liu, Chen Li. Haoran Tang, Yixiao Ge Ying Shan, and Ge Li. St-lim: Large language models are effecctiee temporal learners. In European Conference on Computer Vision (ECCV), pp. 1- 1-18 Springer, 2024d.
Ruyang Liu, Shangkun Sun, Haoran Tang, Wei Gao, and Ge Li. Flow4agent: Long-form video understanding via motion prior from optical flow. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 23817 27-2277, 2025a.
Shuming Liu, Chen Zhao, Tianqi Xu, and Bernard Ghanem. Bolt: Boost large dision-languaa model without training for long-form video understanding. In Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR), pp. 3318-3327, 2025b.
12

---

Published as a conference paper at ICLR 2026
Yuanxin Liu, Shicheng Li, Yi Liu, Yuxiang Wang. Shuhuai Ren, Lei Li, Sishuo Chen, Xu Sun, and Lu Hou. Tempcompass: Do video Ilms really understand vidcos? arXiv preprint arXiv:2403.0077, 2024c.
Ziyu Ma, Chenhui Gou, Hengean Shi, Bin Sun, Shutao Li. Hamid Rezatofighi, and Jianfei Cai. Drvideo: Document retrieval hased long video understanding. In Proceedings of the Computer Vision and Pattem Recognition Conference (CVPR), pp. 18936-18946, 2025.
Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Shahbaz Khan. Video-chatgpt: Towards detailed video understanding via large vision and language models. arXiv preprint arXiv:2306.05424 2023.
Karttikeya Mangalam, Raiymbek Akshulakov, and Jitendra Malik. Egoschema: A diagnostic benchmark for very long-form video language understanding. Advances in Neural Information Processing Systems (NeurIPS). 14446212-62244 2023.
Juhong Min, Shyamal Buch. Arsha Nagrani, Minsu Cho, and Cordelia Schmid. Morevqa: Exploring modular reasoning models for vidco question answering. In Proceedings of the TEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13235-1322 13245,, 2024.
David R Morrison, Sheldon H Jacobson, Jason Sauppe, and Edward C. Sewell. Branch-and- hound algorithms: A survey of recent advances in searching, branching, and pruning. Discrete Optimization, 19:79-102, 2016.
OpenAI. Chatgpt:
Optimizing language models for dialogue, 2023, URL httph:////s/// //pppasi//////a..//////// com/blog/ chatgpt.
Jongwoo Park. Kanchana Ranasinghe, Kumara Kahatapitiya, Wonjeong Ryu, Donghyun Kim, and Michael Ryoo. Too many frames, not all useful: Efficient strategies for long-form video qa. In Workshop on Video-Language Models@ @ NeurlPS 2024, 2024.
Long Qian, Juncheng Li, Yu Wu, Yaobo Ye, Hao Fei, Tat-Seng Chua, Yueting Zhuang, and Siliang Tang. Momentor: Advancing video large language model with fine-grained temporal reasoning. In International Conference on Machine I Learning (ICML), pp. 41340-41356 PMLR, 2024.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference machine learning (ICML), pp. 8748-8763. PmLR, 2021.
Harvey M Salkin and Cornelis A De Kluyver. The knapsack problem: a survey. Naval Research Logistics Quarterly, 22(1):127-144, 1975.
Xiaoqian Shen, Yunyang Xiong, Changsheng Zhao, Lemeng Wu, Jun Chen, Chenchen Zhu, Zechun Liu, Fanyi Xiao, Balakrishnan Varadarajan, Florian Bordes, et al. Longvu: Spatiotemporal adaptive compression for long video-language understanding. arXiv preprint 2rXiv:2410. 2024.
Yan Shu, Zheng Liu, Peitian Zhang, Minghao Qin, Junjie Zhou, Zhengyang Liang, Tiejun Huang, and Bo Zhao. Video-xl: Extra-long vision language model for hour-scale video understanding. In Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR), pp. 261 60-26666,, 2025.
Karen Simonyan and Andrew Zisserman. Two-stream convolutional networks for action recognition in videos. Advances in neural information processing systems (NeurIPS), 27, 2014.
Enxin Song, Wenhao Chai. Guanhong Wang, Yucheng Zhang. Haoyang Zhou, Feiyang Wu, Haozhe Chi, Xun Guo, Tian Ye, Yanting Zhang, et al. Moviechat: From dense token to sparse memory for long video understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11821, 2024.
Hui Sun, Shiyin Lu, Huanyu Wang, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang, and Ming Li. Mdp3: A training-free approach for list-wise frame selection in video-lims. arXiv preprint arXiv:2501 .02885, 2025.
Yucheng Suo, Fan Ma, Linchao Zhu, Tianyi Wang, Fengyun Rao, and Yi Yang. From trial to triumph: Advancing long video understanding via visual context sample scaling and self-reward alignment. arXiv preprint arXiv:2503.20 20.422,50.. 2025.
Xi Tang, Jihao Qiu, Lingxi Xie, Yunjie Tian, Jianbin Jiao, and Qixiang Ye. Adaptive keyframe sampling for long video understanding. In Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR). pp. 291 1812888, 2025.
Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju. Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, et al. Gemma: Open models hased on gemini research and technology. arXiv preprint arXiv:2403.08295 2024.
13

---

Published as a conference paper at ICLR 2026
Qwen Team. Qwen3 technical report, 2025. URL /ttps///ttt///arrrbrrrrrr///////
Peter Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Adithya Jairam Vedagiri IYER, Sai Charitha Akula, Shusheng Yang. Jihan Yang, Manoj Middepogu, Ziteng Wang, et al. Cambrian-1: 1: A fully open, vision- centric exploration of multimodal lims. Advances in Neural Information Processing Systems (NeurIPS), 37: 87310-87356, 2024.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 2023.
Zhongwei Wan, Xin Wang, Che Liu, Samiul Alam, Yu Zheng, Jiachen Liu, Zhongnan Qu, Shen Yan, Yi Zhu, Quanlu Zhang, et al. Efficient large language models: A survey. Transactions on Machine Learning Research (TMLR), 2024.
Haibo Wang, Chenghang Lai, Yixuan Sun, and Weifeng Ge. Weakly supervised gaussian contrastive grounding with large multimodal models for video question answering. In Proceedings of the 32nd ACM International Conference on Multimedia (MM), pp.
2024a.
Peng Wang, Shuai Bai, Siman Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang. Wenbin Ge, et al. Qwen2-vl: Enhancing language model's perception of the world at any resolution. arXiv preprint arXiv:2409.12110
2024b.
Xiao Wang, Qingyi Si, Jianlong Wu, Shiyu Zhu, Li Cao, and Liqiang Nie. Retake: Reducing temporal and knowledge redundancy for long video understanding, arXiv preprint arXiv:2412.20504, 2024c.
Xiao Wang, Qingyi Si, Jianlong Wu, Shiyu Zhu, Li Cao, and Liqiang Nic, Adaretake: Adaptive redundancy reduction to perceive longer for lidco-language understanding. arXiv preprint arXiv:2503, 12559, 2025a.
Xiaohan Wang, Yuhui Zhang, Orr Zohar, and Serena Yeung-Levy, Videoagent: Long-form video understanding with large language model as agent. In European Conference on Computer Vision (ECCV), pp. 58-76. Springer. 2024d.
Yi Wang, Kunchang Li. Yizhuo Li, Yinan He, Bingkun Huang, Zhiyu Zhao, Hongjie Zhang, Jilan Xu, Yi Liu, Zun Wang, et al. Internvideo: General video foundation models via generative and discriminative learning. arXiv preprint arXiv:2212.031
2022.
Ziyang Wang, Shoubin Yu, Elias Stengel-Eskin Jachong Yoon, Feng Cheng, Gedas Bertasius, and Mohit Bansal. Videotree: Adaptive tree-based video representation for Ilm reasoming on long videos. In Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR), pp. 3272 3283, 2025b.
Hao Wu, Huabin Liu, Yu Qiao, and Xiao Sun. Dibs: Enhancing dense video captioning with unlabeled vidcos via pscudo boundary enrichment and online refinement. In Proceedings of the IEEE/CVF conference on computer vision and pattern recogmition (CVPR), pp. 18699-187008 18708, 2024a.
Haoning Wa, Dongxu Li, Bei Chen, and Junnan Li. Longvideohench: A henchmark for long-context interleaved video-language understanding. Advances in Neural Information Processing Systems (NeurIPS), 37:28828- 28857, 2024b.
Junbin Xiao, Xindi Shang. Angela Yao, and Tat- Seng Chua. Next-qa: Next phase of question- answering explaining temporal actions. In Proceedings the IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp. 9777-9786, 2021.
Junbin Xiao, Angela Yao, Yicong Li, and Tat-Seng Chua. Can trust your answer? visually grounded video question answering. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13204 13214, 2024.
Wenhan Xiong, Jingyu Liu, Igor Molybog, Hejia Zhang, Prajjwal Bhargava, Rui Hou, Louis Martin, Rashi Rungta, Karthik Abinav Sankararaman, Barlas Oguz, et al. Effective long-context scaling of foundation models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume I: Long Papers) (NAACL), pp. 4643 4663, 2024.
Lin Xu. Yilin Zhao, Daquan Zhou, Zhijie Lin, See Kiong Ng, and Jiashi Feng. Pllava: Parameter- free llava extension from images to videos for video dense captioning. arXiv preprint arXiv:2404. 16994, 2024a.
Mingze Xu, Mingfei Gao, Zhe Gan, Hong- You Chen, Zhengfeng Lai, Haiming Gang, Kai Kang. and Afshin Dehghan. Slowfast-llava- A strong training-free haseline for video large language models. arXiv preprint arXiv:2407. 15841, 2024h.
14

---

Published as a conference paper at ICLR 2026
Antoine Yang, Antoine Miech, Josef Sivic. Ivan Laptev, and Cordelia Schmid. Zero-shot video question answering via frozen bidirectional language models. Advances in Neural Information Processing Systems (NcurIPS), 35: 1e.11124111, 2022.
Antoine Yang, Arsha Nagrani, Paul Hongsuck Sco, Antoine Miech, Jordi Pont-Tuset, Ivan Laptev, Josef Sivic, and Condelia Schmid. Vid2seq: Large-scale pretraining of visual language model for dense video captioning. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10714-10726, 2023.
Huanjin Yao, Wenhao Wu, Taojiannan Yang, YuXin Song. Mengxi Zhang, Haocheng Feng, Yifan Sun, Zhiheng Li, Wanli Ouyang, and Jingdong Wang. Dense connector for mllms. Advances Neural Information Processing Systems (NcurIPS), 37:33108-33 -3103040,, 2024a.
Linli Yao, Haoning Wu, Kun Quyang, Yuanxing Zhang, Caiming Xiong, Bei Chen, Xu Sun, and Junnan Li. Generative frame sampler for long video understanding. arXiv preprint arXiv:2503.09146 2025.
Yuan Yao. Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al. Miniepm-v: A gpt-4v level milm on your phone. arXiv preprint (.XXxx2448.018880 2024b.
Jinhui Ye, Zihan Wang. Haosen Sun, Keshigeyan Chandrasegaran, Zane Durante, Cristobal Eyzaguirre, Yonatan Bisk, Juan Carlos Niebles, Ehsan Adeli, Li Fei-Fei, et al. Re-thinking temporal search for understanding. In Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR), pp. 8579-8591, 2025.
video
Shoubin Yu, Jaemin Cho, Prateek Yadav, and Mohit Bansal, Self-chained image-language model for video localization and question answering. Advances in Neural Information Processing Systems (NeurIPS), 36: 76749- 76771, 2023.
Sicheng Yu, Chengkai Jin, Huanyu Wang, Zhenghao Chen, Sheng Jin, Zhongrong Zuo, Xiaolei Xu, Zhenbang Sun, Bingni Zhang. Jiawei Wu, et al. Frame-voyager: Learning to query frames for video large language (ICLR), pp.
In Proceedings of the Thirteenth International Conference on Learning Representations 2025.
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF international conference on computer vision (ICCV), pp. 11975-11986, 2023.
Boqiang Zhang, Kehan Li, Zesen Cheng, Zhiqiang Hu, Yuqian Yuan, Guanzheng Chen, Sicong Leng, Yuming Jiang, Hang Zhang, Xin Li, et al. Vidcollama 3: Frontier multimodal foundation models for image and vidco understanding. arXiv preprint arXiv:2501.133 13106, 2025.
Ce Zhang, Taixi Lu, Md Mohaiminul Islam, Ziyang Wang. Shoubin Yu, Mohit Bansal, and Gedas Bertasius. simple I1m framework for long-range video question-ani -asseerng.. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 21715-21737, 2024a.
Peiyuan Zhang, Kaichen Zhang, Bo Li, Guangtao Zeng. Jingkang Yang. Yuanhan Zhang. Ziyue Wang. Hao- ran Tan, Chunyuan Li. and Ziwei Liu. Long context transfer from language to vision. arXiv prepriat arXiv:2406. 16852, 2024b.
Yuanhan Zhang, Bo Li, haotian I. Liu, Yong jae Lee, Liangke Gui, Di Fu, Jiashi Feng, Ziwei Liu, and Chunyuan Li. Llava-next: A strong zero-shot video understanding model, April 2024c. URL WRRLhttps://llaaaaa github.icc//ooo htthub..cc/clg///00000 2024 04-30-- 1 lava setra-next-ooe -ext-videed vide..
Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li. Zejun Ma. Ziwei Liu, and Chunyuan Li. Video instruction tuning with synthetic data. arXiv preprint 2iti::24100022111 2024d.
Junjie Zhou, Yan Shu, Bo Zhao, Boya Wu, Zhengyang Liang, Shitao Xiao, Minghao Qin, Xi Yang, Yongping Xiong, Bo Zhang, et al. Mivu: Benchmarking multi-task long video understanding. In Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR), pp. 13691-13701, 2025.
Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: Enhancing vision- language understanding with advanced large language models. arXiv preprint arXiv:2304. 10592, 2023.
Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, Intervv1:: Exploring advanced training and test-time recipes for open- noon-souree multimodal models. arXiv preprint arXiv:2504. 10479, 2025.
Or Zohar, Xiaohan Wang, Yann Dubois, Nikhil Mchta, Tong Xiao, Philippe Hansen-Estruch Licheng Yu, Xiaofang Wang. Felix Juefei-Xu, Ning Zhang, et al. Apollo: An exploration of video understanding in large multimodal models. arxiv preprint MxXiv:2412.10366 2024.
15


Reviews:
[{"review_id": "AOIAAHYox7", "reviewer": "Reviewer_A1NG", "summary": "This paper introduces Nar-KFC, a plug-and-play module designed to improve long video understanding for Multimodal Large Language Models (MLLMs). Nar-KFC selects representative keyframes by jointly optimizing relevance and diversity through an efficient greedy search, then enriches temporal coherence by inserting textual narratives from non-keyframes. This dual strategy achieves both temporal consistency and content compression, enabling MLLMs to process long videos more effectively. Extensive experiments on multiple benchmarks show that Nar-KFC significantly boosts MLLM performance with minimal computational overhead.", "questions": "Please refer to the Weaknesses section. The authors should seriously address the concerns, especially the experimental aspects, as failing to do so may negatively impact the paper’s evaluation.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Well-written and easy to follow.\n\n2. Keyframe selection is formulated as an integer quadratic programming problem with a customized greedy search, providing clear theoretical grounding rather than relying on heuristic rules.\n\n3. The plug-and-play design is flexible and compatible with various MLLMs, training-free, which reduces computation cost and overfitting risk.\n\n4. Demonstrates consistent performance gains across different models (e.g., InternVL2, Qwen2-VL) and model sizes, showing its generalizability.", "weaknesses": "My main concerns are with the experiments:\n\n1. The number of benchmarks is limited. As a training-free module, Nar-KFC needs more diverse benchmarks to convincingly demonstrate its generality and robustness.\n\n2. The base models are outdated — the strongest one, LLaVA-Video, is already a year old. Including 1–2 more recent VLMs (e.g., Qwen2.5-VL, Qwen3-VL, Intern3-VL) would strengthen the claims.\n\n3. Some baselines in Table 1 seem questionable; for instance, LLaVA-Video’s VideoMME performance should be higher than 55.9 / 56.7.\n\n4. The paper should discuss and compare with more recent keyframe selection methods [1][2] to contextualize the contributions.\n\n[1] Flow4Agent: Long-form Video Understanding via Motion Prior from Optical Flow. ICCV25\n\n[2] From Trial to Triumph: Advancing Long Video Understanding via Visual Context Sample Scaling and Self-reward Alignment. ICCV25", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4oidwAdb8k", "reviewer": "Reviewer_TdJ3", "summary": "This paper introduces Nar-KFC, a novel and practical training-free framework designed to enhance long-video comprehension for existing Multimodal Large Language Models (MLLMs) constrained by limited context windows.", "questions": "1. The query-agnostic captioner may poses risks of noise and hallucination. Could the authors provide a failure analysis quantifying how often MLLM errors stem from misleading narratives?\n\n2. The keyframe selection relies entirely on CLIP, which may be unreliable for complex or fine-grained queries, and the MLLM baselines used are outdated. Could the authors adopt a more advanced VLM for selection and validate Nar-KFC on a recent MLLM (e.g., Qwen2.5-VL)? It would also be valuable to show performance differences when using newer VLMs, highlighting how the framework scales with stronger vision-language backbones.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.This paper presents Nar-KFC, a hybrid representation method that interleaves visual keyframes with textual narratives, offering a novel perspective on video compression and long-video understanding. The authors formulate keyframe selection as an Integer Quadratic Programming (IQP) problem, systematically optimizing both query relevance and frame diversity, making it a more principled alternative to heuristic approaches.\n\n2.The proposed method holds strong practical value: it is a plug-and-play, training-free module that can be directly applied to existing multimodal large language models (MLLMs) to significantly enhance long-video comprehension without retraining, opening a new direction for efficient hybrid video representation.", "weaknesses": "1. A significant potential risk of this method is that the lightweight captioner used for narrative generation may introduce errors or hallucinations. More importantly, this captioner operates in a \"query-agnostic\" manner; it merely describes the content of non-keyframe segments, and this content may be entirely irrelevant to the user's specific query. This results in the Nar-KFC method actively injecting a substantial volume of irrelevant noise into the MLLM's context . If a given narrative happens to contradict the visual content of the keyframes or the query itself, this could severely mislead the MLLM. \n\n2. The keyframe selection is entirely dependent on the quality of the upstream VLM (CLIP) used to compute the $S_{QR}$ and $S_{FD}$ scores. This reliance is potentially unreliable, as the model may fail to select rational keyframes for queries involving complex relations or fine-grained details. Furthermore, the MLLM models employed for the experiments (e.g., InternVL2, Qwen2-VL) appear to be outdated.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GjuoJCYLek", "reviewer": "Reviewer_co9R", "summary": "This paper introduces Narrating KeyFrames Capturing (Nar-KFC), a plug-and-play module designed to enhance long video understanding in multimodal large language models (MLLMs). Nar-KFC addresses the keyframe selection problem by formulating it as an integer linear programming task that jointly optimizes query relevance and frame diversity. To reduce computational cost, it also offers a greedy search alternative. Additionally, Nar-KFC leverages an off-the-shelf captioning MLLM to generate descriptive captions for the keyframes. A simplified variant, KFC, which omits the captioner, is also evaluated for ablation. Both Nar-KFC and KFC are tested across multiple backbone MLLMs and benchmark datasets, demonstrating their effectiveness and versatility.", "questions": "* Using Qwen2-VL as the MLLM and captioner at the same time leads to performance degradation, I wonder if the authors can provide more analysis on that matter.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The paper is clearly written and well-structured.\n* Nar-KFC demonstrates consistent performance gains across different MLLMs, effectively validating its utility.\n* Exploring improved frame sampling strategies for long video understanding is a valuable research direction.", "weaknesses": "* Unlike token compression approaches, Nar-KFC relies primarily on frame selection for information compression. The optimization criteria, i.e. frame-level diversity and query-frame relevance, are computed at the global frame level. This raises concerns about scenarios involving subtle or localized changes (e.g., small objects evolving over time), where high overall frame similarity might cause truly informative frames to be inadvertently discarded, as such nuances may not be captured by coarse frame-level metrics.\n* The most significant performance improvements appear to stem from the captioner module, which enriches keyframes with descriptive text. However, this component is computationally expensive, potentially offsetting the efficiency gains expected from reduced frame counts.\n* Since the method does not incorporate any token-level reduction mechanism, the number of selected keyframes is inherently constrained by the MLLM’s effective context length and its capacity to handle multiple images, shaped by its pre-training and SFT. This limitation could become problematic for extremely long videos with frequent scene transitions, where even a moderate number of keyframes may exceed effective modeling capacity. A more comprehensive comparison with token-reduction-based methods (rather than only keyframe selection approaches) would better contextualize Nar-KFC’s trade-offs between performance, robustness, and scalability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mSCALMN6CI", "reviewer": "Reviewer_N1Wd", "summary": "This paper addresses the challenge of using Multimodal Large Language Models (MLLMs) for long video understanding, where the high number of video frames conflicts with the limited context length of language models. The authors propose Nar-KFC, a plug-and-play module that enhances understanding efficiently. Their method involves two steps: first, selecting keyframes by optimizing for query-relevance and frame-diversity using an efficient greedy search; second, inserting textual narratives from non-key frames to ensure temporal coherence. Nar-KFC acts as a content-aware compression technique, and experiments on various benchmarks show it significantly boosts the performance of existing MLLMs.", "questions": "I noticed that all the baselines used in the paper are based on 7B or 8B models. How would the proposed method perform on larger or smaller models, and what impact would model size have on the approach?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 1, "presentation": 3, "contribution": 2, "strengths": "1. This paper presents a well-written and systematically organized study on a highly practical challenge.\n2. The proposed Nar-KFC method offers significant value for real-world deployment as it is a training-free, plug-and-play module that effectively tackles long-video understanding without imposing a substantial computational burden.\n3. Nar-KFC's effectiveness is validated through comprehensive experiments on multiple benchmarks, where it demonstrates a notable performance improvement for various MLLMs on tasks like question-answering.", "weaknesses": "1. Limited Task Diversity in Evaluation: The proposed method is evaluated exclusively on question-answering (QA) benchmarks. While frame compression may suffice for QA, its efficacy remains unverified for tasks demanding higher visual granularity, such as video captioning or video OCR. Broader evaluation across diverse tasks is needed to fully ascertain the method's applicability and potential limitations.\n2. Outdated Baseline Models: The experiments utilize MLLM baselines from the previous year (e.g., Qwen2-VL, InternVL-2). Given the rapid progress in the field, the plug-and-play value of Nar-KFC should be demonstrated on more recent models (e.g., Qwen2.5-VL, InternVL-3) to ensure its contribution remains relevant and complementary to enhanced base capabilities.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)