One Token per Highly Selective Frame: Towards Extreme Compression for Long Video Understanding
Authors: Zheyu Aqa Zhang
Ziqi Pang
Shixing Chen
Xiang Hao
Vimal Bhat
Yu-Xiong Wang
Venue: NeurIPS 2025 poster ()
Keywords: Computer Vision
Long Video Understanding
Primary area: deep_learning
TL;DR: Enhancing long video understanding via extreme compression by reducing each selected frame to a single token.

Abstract: Long video understanding is inherently challenging for vision-language models (VLMs) because of the extensive number of frames. With each video frame typically expanding into tens or hundreds of tokens, the limited context length of large language models (LLMs) forces the VLMs to perceive the frames sparsely and lose temporal information. To address this, we explore extreme video token compression towards *one token per frame* at the final LLM layer. Our key insight is that heuristic-based compression, widely adopted by previous methods, is prone to information loss, and this necessitates supervising LLM layers into *learnable* and *progressive* modules for *token-level compression* (LP-Comp). Such compression enables our VLM to digest 2x-4x more frames with improved performance. To further increase the token efficiency, we investigate *frame-level compression*, which selects the frames most relevant to the queries via the internal attention scores of the LLM layers, named *question-conditioned compression* (QC-Comp). As a notable distinction from previous studies, we mitigate the position bias of LLM attention in long contexts, *i.e.*, the over-concentration on the beginning and end of a sequence, by splitting long videos into short segments and employing local attention. Collectively, our combined *token-level* and *frame-level* leads to an e**x**treme compression model for long video understanding, named **XComp**, achieving a significantly larger compression ratio and enabling denser frame sampling. Our XComp is finetuned from VideoChat-Flash with a data-efficient *supervised compression tuning* stage that only requires 2.5\% of the supervised fine-tuning data, yet boosts the accuracy from 42.9\% to 46.2\% on LVBench and enhances multiple other long video benchmarks.

One Token per Highly Selective Frame: Towards Extreme Compression for Long Video Understanding
Zheyu Zhang
Ziqi Pang
University of Illinois Urhmna-Champainn Zheyuz5011innois.oo
University of Illinois Jrbana-Champar
ziqip2ei11inois edu
Shixing Chen Amazon Prime Video
Xiang Hao Amazon Prime Video
Vimal Bhat
Amazon Prime Video v vimalb@amazon. com
chixic@amazon.com com
xi ainghao@amazon.com com
Yu-Xiong Wang University of Illinois Urbana-Champaiga newiillinois..
Abstract
Long video understanding is inherently challenging for vision-language models (VLMs) because of the extensive number of frames. With each video frame typ- ically expanding into tens or hundreds of tokens, the limited context length of large language models (LLMs) forces the VLMs to perceive the frames sparsely and lose temporal information. To address this, we explore extreme video token compression towards one token per frame the final L.M layer. Our key in- sight is that siaristic-based compression, widely adopted by previous methods, is prone to information loss, and this necessitates supervising LLM layers into learnable and progressive modules for token-level compression (LP-Comp). Such compression enables our VLM to digest 2x-4x more frames with improved per- formance. To further increase the token efficiency, we investigate frume-level compression, which selects the frames most relevant to the queries via the inter- nal attention scores of the LLM layers, named inrestion-conditiion (QC-Comp). As notable distinction from previous studies, we mitigate the position bias of L.LM attention in long contexts, i.e., the over-concentration the beginning and end of a sequence, by splitting long videos into short seg- ments and employing local attention. Collectively, combined token-level and frame-level leads to an extreme compression model for long video understanding, named XComp, achieving significantly larger compression ratio and enabling denser frame sampling. Our XComp is finetuned from VideoChat-Flaan with a data-efficient supervised compression tuning stage that only requires 2.5% of the supervised fine-tuning data, yet boosts the accuracy from 42.9% to 46.2% on LVBench and enhances multiple other long video benchmarks. Code and Model are available at https::/ /ttpts///ithub com/ /2eeehhhayaaaagggggg /Comp.
compression
Introduction
Enabling vision- vision-language models (VLMs) comprehensively understand long videos remains
critical yet formidable challenge [12, 22, 23, 32, 35, 38, 57, 61, 62, 69, 70]. The sheer volume of visual information challenges current VLMs' inherent context length constraints: each frame is
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

XComp Approach
Long Video Result
Accuracy Vs. Alram
Accuracy (LVBench)
LLM Layers
45.0
42.5
video tokens
40.0
Tokens/ Tokens/Frame /Faame
Drop to 1 L-token-per frame
37.5
Fldu XCompp
500
1000
1500
2000
Number of Prames (Mhone)
Accuracy vs. fisken
Accuracy (LVBennch)
45.0
02.5
40.0
CiltoChut-Fluch
37.5
Layer ID
10000 15000
Number Video Tokens (Nisken)
Figure I: Left: We present XComp that explores using the LLM layers to progressively compress the video tokens towards the extreme of one token per frame. Right: Such capabilities enable the model to better improve itself with denser input frames without significantly increasing video tokens.
encoded into tens or hundreds of tokens, processing more than Ik frames usually exceeds the typical context lengths or computational budgets of the large language model (LLM) layers for the VL.Ms [11] during both training and inference. Yet, such a framework is still far from processing an hour-long video at more than one frame per second (FPS), easily losing critical temporal information and visual details. Therefore, how to effectively compress the tokens of video frames while maintaining useful information becomes the major bottleneck. In this paper, we explore the extreme of this direction and propose an approach to compress each video frame into one single highly informative token when reaching the final layer of the LLM in VLMs, thereby unlocking unprecedented density of frames for VLMs to process.
From this aspect, we aim to achieve extreme compression at the token level: condensing the informa- tion from tens or hundreds of tokens per frame into a single token. Existing methods predominantly rely on heuristic- beures or training-free strategies, such as special token selection [19, 35, 53] and pooling [31, 34]. Despite their reduced token counts, these methods risk discarding crucial visual tokens, as their underlying LLM layers of the VLMs were not supervised to encapsulate the capa- bility of compression, i.e., condense the contexts from the dropped tokens into the kept ones. As result, our analysis (Table 2) reveals the limitations of such heuristic approaches when reaching high compression ratios. Therefore, our key insight is that the L.MM layers should be supervised to compress extensive video data into remarkably fewer representative tokens via additional supervised compression tuning stage, motivating learnable compression. To further avoid drastically losing information at large compression ratios, we let the LLM layers progressively compress the visual tokens with a smooth schedule. These two combined lead to our "learnable and progressive compression" (LP-Comp) as in Fig. 1, which marks clear distinctions with previous heuristic methods.
In addition to decreasing the number of tokens per frame, the token efficiency of long video un- derstanding can be further enhanced by selecting the most relevant frames, corresponding to the compression at the frame level. Within the internal representation of VLMs, the distinctions of such relevant frames can be intuitively evaluated by the attention scores between the questions and the video tokens [19], where the frames having larger attention scores are preferred. However, a bottleneck under-explored by the previous methods is the internal position bias of LLM transformers in long context understanding: the attention heads might assign larger scores to the tokens at the beginning the end of the sequence [42]. Inspired by local attention [3, 47, 76], under the context of long video understanding, this motivates a splitting of long videos into short video chunks for frame-level compression, where model inspects the relevance of frames inside each video segment. Such a step effectively utilizes the question information and is referred to as question- conditione compression (QC-Comp).
Combining both learnable & progressive compression (L.P-Comp) and cnpestioncooditionee com- pression (QC-Com (oCp), propose VLM named XComp that could achieve extreme token efficiency

---

towards ree-token-per-framee for highly selective frames. Notably, the compression behavior of the LLM layers can be learned via a data-efficient supervised compression tuning stage requiring only 2.5% of the supervised fine-tuning (SFT) data of VideoChat-Flan token efficiency enables us to sample the frames densely and significantly enhance the performance as the LVBench [59] evaluation in Fig. I: our method not only achieves better accuracy on a token- for-token efficiency basis, but also keeps improving with the number of input frames increases, while VideoChat-Flash experiences performance drops after certain frame number.
[35]. More importantly, such
In summary, we make the following contributions:
1. We introduce LC-Comp. a learnable and progressive framework that supervises LLM layers to compress the token level, without purely relying on heuristics.
2. We propose QC-Comp, frame-level question-condiiionee selection method that effectively selects the relevant frames for specific questions.
3. We demonstrate the effectiveness of our token compression towards each selected video frame as a single token, drastically increasing the number of frames that can be processed and leading to improved long video understanding accuracy.
2 Related Work
Vision-Language Models for Long Sequence Understanding. Early Vision-Language Models (VLMs), such as GPT-4V and Gemini-1.5 [49, 58], showcased powerful multimodal reasoning by integrating visual encoders with large language models. Open-source efforts like Llama-Vis [36], IDEFICS [24], VideoChat [34], Video-LLaMA [12], and others [2, 32, 35, 38, 44, 61, 62] have further advanced capabilities, often matching or exceeding proprietary systems on various benchmarks. While effective on static images or short video clips, scaling VLMs to long videos (minutes to hours) introduces a significant challenge. The core difficulty lies in managing the immense volume of visual data, which generates an excessive number of tokens that quickly exceed the typical context windows and computational capacities of the LLM components during both training and inference. Current research addressing long-context VLMs primarily explores two directions: (i) methods that significantly extend the effective context length of the underlying transformer architectures, such as LongVA, VA. LILA, Long ViTA, and LLaVA-NeXT [11, 55, 75, 77], and (ii) strategies focused reducing the number of visual tokens fed to the LLM via selection compression modules [26, 28, 56, 57]. While context extension allows processing more tokens, it often incurs high computational costs. Conversely, existing visual compression techniques, while reducing token count, frequently rely on heuristics or fixed operations, or uniform sampling across the video [2. 12, 32, 34, 35, 75] that struggle to preserve critical temporal and fine-grained information, particularly when pushed to high compression ratios necessary for very long videos. Other orthogonal approaches tackle long videos by processing shorter segments independently and employing retrieval mechanisms [3].
Video Token Compression in Multimodal LLMs. Given the computational constraints of pro- cessing dense video streams with VLMs, compressing visual tokens is a vital approach. Various methods have been proposed to distill frame sequences into more manageable representations. These include temporal pooling strategies [17, 46], heuristic or saliency-
frame selection often guided by simple visual priors or attention mechanisms [56, 57], and lightweight learned mod- ules or bottlenecks aimed at reducing the token sequence, such as Llama- Vid [36] which explores compressing frames potentially Relatedly, techniques for dynamic token pruning or merging have been explored for both images and video [5, 13, 26, 29, 45, 52, 68, 70J. However, a critical limitation of many existing approaches. especially those relying purely on heuristics or fixed pooling, is that they are not trained to consolidate the information from dropped tokens into the retained ones within the LLM's representation space. Aggressively applying these methods to achieve the extreme compression ratios necessary for very long videos risks discarding vital visual details, thus hindering deep understanding. Furthermore, while frame selection is intuitive, applying it effectively across extremely long sequences with- out being hampered by issues like internal LLM position biases remains challenging. Therefore, developing methods that can perform learned and highly efficient visual compression, potentially towards representing each frame with an unprecedentedly low token count while preserving critical information, remains a significant research frontier.
single token by leveraging cross-attention with text query.
3

---

Token level (LP-Comp)
Frame level (QC-Comp)
Question:
video
Dropped
Selected
Dropped
Selected
LLM Layer
token
LLM Layer
token
LLM Layer
LLM Layer
Figure Overview. Our XComp comprises of two parts achieve extreme compression in long video understanding. (1) At the loken level, we propose the supervised compression tuning that enables the LLM compress every video frame
one compact token in learnable and progressive manner, namely, LP-Comp (Sec. 3.2). (2) At the frame level, we utilize the internal attention mechanism to select the frames relevant the questions. which question-conditione
compression, namely, QC-Comp (Sec. 3.3). Both aspects combined lead to our objective of extreme compression in long video understanding: one token per highly selective frame.
Methodology
Towards the objective of extreme compression, introduce novel framework that unifies the compression of long video tokens from two dimensions, as illustrated in Fig. 2:
1. Token-level: the Learnable Learnable&Progressiv Compression (LP-Comp) represents each frame using as few as one token, which is learned via training (Sec. 3.2).
2. Frame-level: Question-conditionel Compression (QC-Comp) selects the frames that most relevant to the questions based on the attention scores inside the LLM layers, which is employed inference-time enhancement (Sec. 3.3).
3.1 Preliminaries and Formulation
VLMSs for Video Understanding. Current VLMMs primarily follow the architecture of LLaVA [40] when handling visual inputs: (1) encoder encodes the raw pixels into set of visual tokens; (2) a projector, commonly several multi-layer perceptions (MLPs), projects the visual tokens into the dimensions the LLMs; (3) an LLM accepts both the text tokens and projected visual tokens and decodes the output in an auto-regressive manner. This paper concentrates on the compression between the LLM layers and keeps the visual encoder and projector unchanged.
More formally, we consider accepting videos input, the standard practice is to sample frames for the VLM to process, denoted as [F.] Suppose the vision encoder converts each frame into ( visual tokens, we then let the LLM's input for the i-th frame be v(()) R x where the LLM's dimension. So the input video tokens the L.M become a sequence of T ((11)
tokens, namely, V(I) + [v7")..... ..... 1. Denoting the input query as Q(1) with Ng tokens in standard question-anne answering scenarios, we follow the practice of our baseline model VideoChat- Flash [35] of appending the query Q(1) after the visual tokens (l), yielding the input sequence to LLM (1) &- [V)) sequence with length T (()) + Ng.. Then, the LLM begins the auto-regressive generation of new text tokens based on the input sequence (1).
Objective: Visual Token Compression. The computation process the LLM is commonly dominated by the large volume of visual tokens N in long video understanding. To reduce the cost of LLM layers and permit more input frames, we explore two directions to reduce video token counts: either from the aspect of (1), the number of tokens per frame, or T, the number of frames. (1) 'ooenn-leeel Compression. Denoting ( as the number of tokens per frame at I-th transformer layer, our formal objective is to decrease (!) so that the transformer layers have a gradually more concise set visual tokens to process, shrinking from x (11 to T (1), We demonstrate such token-level compression in Sec. 3.2. (2) Frame-level Compression. Another way of decreasing the computation to convert the frame number T into a smaller T'. In the context of long video understanding, this means selecting subset of relevant frames to process and ignoring the rest. Our

---

Learnable and Progressive Compression
(trame
(frame 1)
Cummn (1
LLM Layer
LLM Layer
I.I.M
[)
Video Encoder
Connector
[frame 2]
(frame 2]
(2 Tiramre
C
Video token
Dropped token
NC3) 16 Tokens/framn
Layer ID
N Token
Figure 3: Learnable and Progressive Compression (LP-Comp). With supervised compression
tuning, the LLM layers learn condense the visual tokens progressively into concise set of tokens until reaching the extreme of one token per frame.
Sec. 3.3 presents frame-level compression strategies. Finally, our framework unifies both types of compression to enhance long video understanding.
3.2 Learnable and Progressive Token Compression (LP-Comp)
Objective. As discussed in Sec. 3.1, the objective of token-level compression is to decrease the number of video tokens ( than the initial token number (2), Suppose the LLM contains L layers, we aim at enabling the LLM to compress the tokens so that each frame is compressed to a single token last layer, namely, ( 1. the case of our baseline VideoChat-Flase [35], where N() 16, our target marks the compression ratio of 16.
Although previous studies have demonstrated the redundancy in visual modalities [5], especially for videos [37], such level of compression is challenging, modern VLMs like VideoChat- -lash have already utilized the projector for visual token compression in the form of downsampling. As result, discover that the heuristic compression approaches, e,g., selecting tokens in training-fres manner, are prone to failure for such a large compression ratio (as in Sec. 4.3, Table 2), whose performance significantly falls behind the baseline without token compression. Therefore, our key insight is to let the LLM layers actively learn to condense the tokens into concise set instead of purely relying on inference- time heuristics for extreme visual token compression.
Learnable Token Compression. As shown Fig. 3, the LLM layers can be trained with the token compression by decreasing the number of tokens during the training stage. A critical design of such token compression is always to preserve the suffi tokens of a frame. Concretely, we assume a specific layer & compresses visual tokens from (k) to (k+11), where (*++1) < (1)),, then the first (R) (k++1) remaining tokens for a frame are determined be removed (the dashed boxes in Fig. 3). Such a sudfix-preservation behavior is compatible with the causal attention of decoder-only L.LM layers, as the later tokens sequence can absorb the earlier token features instead of the reverse direction. By end-to-end supervising the LLM layers for compression, we observe a significant improvement over heuristics (Sec. 4.3, Table 2).
Progressive Token Compression. The effectiveness of learnable compression is determined by the capability of LLM layers. For instance, we could directly conduct extreme one-token-per booken-pe-fraame compression with the first L.LM layer for maximum efficiency, iee., NT(2) 1, but this would risk discarding critical visual information. To balance both token efficiency and preserving information, we propose to compress progressively across all the LLM layers. As illustrated in Fig. 3, each LLM layer only decreases small amount of visual tokens according to a smooth schedule. We empirically find the cosine effective for token compression, where the visual token number evolves as,
(9) (() 1 cos (
NO) + 11. f=
(1)
Our experiments (Sec. 4.3, Table 2) show that smoother compression indeed performs better.
5

---

Data-efficient Supervised Compression Tuning. We state such continued training of VLMs for the compression capability as supervised compression tuning (SCT). It can be learned data- efficient way: only using 2.5% of the data samples from VideoChat-Flash's SFT dataset. Besides, our learnable compression also improves the training efficiency by reducing the average number of tokens, accelerating the training speed, and enabling training on more frames.
Discussion. Despite its simplicity, we emphasize three critical insights leading our effective token compression: (1) it is viable to conduct visual token compression between the LLM layers without only relying on the visual encoder and projector; (2) learnable LLM layers can grasp the condensing of visual information without only relying training- frree heuristics; (3) progressive schedule utilizes all the LLM layers for compression without only relying on a few selected layers.
3.3 Question-Conditioned Frame Compression (QC-Comp)
Objective. Recalling our objective in Sec. 3.1, this section addresses the compression at the frame level (from T' to T"). its simplest form, our frame-level compression is achieved via frame selection, where only the frames relevant to the user's 's questions are used for the response. According to the prior works in VLMs, reducing such redundancy is beneficial us to digest more video frames. To complete such selection, we aim to assign a relevance score for each frame reflecting its usefulness to the user's question and then remove the ones with the lowest relevance; thus, this step is named Question- Cression-Conditionnel Compression.
the performance and also enable
OC-Comp Overview. The QC- Comp is employed at the inference time for our model to select the most relevant frames. Similar to the previ- ous works [6, 16, 50, 63], we also uti- lize the internal representation of the VLM for this objective. Concretely, we calculate the attention scores be- tween the question tokens and the video tokens, which serve as the met- ric for the contributions of frame: larger attention score indicates bet- ter relevance, an intuition adopted by representative LLM [66] and video analysis [79]. However, previous approaches utilizing attention score for frame selection exhibit an under- explored issue: the bias of LLM atten- tion for the tokens at the beginning and the end of sequence, namely "lost in the middle" [42], which could overlook the critical information at the middle of videos. To address the above problem, we propose splitting long video sequence into independent short clips.
Segmented Local Attention
Position
Segment
Position
LLM Attention
w+1
Q: ... night?
w+2
Segment 2
Position
n=1
LLM Attention
Q: ... night?
n+2
video tokens
Q: night?
text tokens
Compression. To reduce the visual redundancy and improve long video understanding performance, we split video into individual segments and assign question conditione relevance scores to video frames. The frames with lower relevance scores are discarded so that the LLM only concentrates on the informative frames.
Figure 4:
Segmented Local Attention for Relevance Scoring. The only change of our solution to conven- tional LLM layers is to split a video token sequence (()) with T frames into Ns = ((T'-w)/stiee (0() segments, where is the window size of the segments. Denoting the Ns segments [v,(0 ..., NyJ we compute the segment-level video tokens with question tokens, i.e., [V((), Q()). The calculation of these segments independent and can conducted parallel, as shown in Fig. 4. Finally, we record the average attention scores across different attention heads. With these attention scores, we calculate the relevant score for each frame as the average of attention score from any question token to any video token that represents this frame in any segment. Top-k relevant frames among the T frames are selected.
attention within each segment locally by connecting the

---

Table 1: Long video understanding comparison. Our method XComp built upon VideoChat- Flash-2B achieves state-of-ther the-aheaa results among 2c-sc models, demonstrating the effectiveness of extreme compression long video understanding. Note that the performance of proprietary models and 3~ VL.Ms are provided for reference.
Size
Long VideoBench 473s
MLVU 651s
VideoMME (Long)
LVBench 4101s
Average Duration
2386s
Proprietary Models
GPT-4V [48]
59.1
49.2
$3.5
GPT-4o [49]
66.7
64.6
65.3
30.8
Gemini- 1.5-Pro [58]
64.0
67.4
33.1
3--9B- VLMs
Qwen2.5VL-3B [4]
43.3
68.2
mPLUG-OW13 [73]
7B
52.1
50.1
43.5
GaleoCha--Flas--- [35]
7B
64.7
74.7
55.4
48.2
Fagle2.5-8B [8]
8B
66.4
77.6
Kangaroo [41]
8B
54.8
61.0
46.7
39.4
TimeMarker [10]
8B
56.3
41.3
InternVL3-9B [81]
9B
62.5
70.8
2B-Scale VLMs
VL3-22 [81]
55.4
64.2
VideoChat-Flash-- [35]
2B
58.3
65.7
44.9
42.9
XComp
2B
59.7
66.7
45.6
46.2
Discussion. Despite the complexity of duplicating the question token, it is viable to mitigate the bias long-context LLM attention, resulting an improved performance as shown Table
Experiments
4.1 Implementation Details
We fine-tune XComp from VideoChat-Flash-28 [35] for supervised compression tuning, integrating our proposed LP-CCmp mechanism. The core components follow the original setup from VideoChat- Flash: we use UMT-L [43] as the visual encoder, token merging with MLP-based connector, and Qwen2-1.5DB [71] as the large language model (LLM). For long videos, we segment them into short clips of 8 frames each. Each clip is compressed into 128 visual tokens, leading to average of 16 tokens per frame.
Our LP-Comp mechanism introduces a progressive token reduction strategy across LLM layers. Initially, each frame is represented by 16 tokens. After each LLM layer I, we compute the target token count (!) as defined in Equation 1. the token count from the previous layer (1) differs from 1) we uniformly drop (1- NT tokens for each frame across the temporal dimension. This process continues across all layers, eventually reducing to a single token per frame at the final LLM layer, i.e., (L) 1. With LP-Comp reducing the token sequence length during training, our method enables faster training of VLMs video sequences with more frames. Concretely, mixture 1024 frames per video with up to frames per second in fine-tuning. The fine-tuning data follows the mixture of short-video and long-video instruction tuning formats used in VideoChat- Flash [35], but our supervised compression tuning only requires 2.5% VideoChat-FFan Faashh' 's dataset. The training process costs around 24 hours on X H100 GPUs.
the
During the inference stage, we utilize QC-Comp to conduct frame selection. When splitting the long video into short segments, each segment contains 64 frames, and we select 512 frames for videos with duration of less than an hour, and 2048 frames for videos longer than an hour. More details on implementation are in the supplementary materials.

---

therels white cabitet with some Items on atand a window with white curtains The curtainsare flanlred by wuoden shelves. In front of her, there is larg counter with cantiron pat on Above the pot is lange stainless strel howl which als holding with buth hands Nest tothom, there isa large pink container with red lquid mssid. What the woman dolin the kitchen?
Question:
wawoma
dress in
1024 Frames
[A) the lar low upside down. [B) Puta lal on the large bowl (C) Movel the large bool to windmwsll. (l) Haood the thellre bowl on the counter. (t) Put the bosi the pink container.
Clue
Question
Candidate
Candidate
Candidate
Candidate
Candidate
alleviates
Short Segment
"lost-inthe-mmidlle"
video tokens
text tokens
LLM Layer
Loral Artention
Figure 5: LongVideoBench case analysis. XComp leverages Comp to divide long video into short segments, mitigating attention bias. Local attention highlights key frames relevant the question and correct answer, enabling effective filtering of irrelevant content.
4.2 Main Results on Long Video Understanding
Benchmarks. We assess model on four widely used long video understanding benchmarks: LongVideoBende [64], MLVU [80], LVBench [59], and VideoMME [18] (w/o subtitles). Unlike short-video datasets, they feature extended video durations ranging from several minutes over an hour, posing greater temporal reasoning and memory challenges. These benchmarks adopt question-answering format, primarily using accuracy as the evaluation metric.
Comparison. We choose the baselines with similar model size of SateoChat-Flash-- [35] and Intern IntemVVL3-22 VL3-2B [81], as shown in Table 1. Importantly, our XComp is trained from VideoChat-Flas using only 2.5% of the data, the improvement over VideoChat- Flash suggests that our one token per highly selective frame preserves the critical information after compression.
shown in Table 1, our XComp consistently improves the baseline SateoChat-FFashh and outperforms the other VLMs at similar scales, even exhibiting comparable performance to some models at 7B scale. As also demonstrated in Fig. I, our XComp demonstrates the unique trend of improving performance with more input frames, demonstrating the efffectiveness of our extreme compression.
Leading Performance.
Qualitative Analysis. Figure 5 presents an example from Long VideoBench illustrating how XComp leverages QC-Comp to effectively identify frames relevant to the question. In this long video question answering scenario, the question provides a detailed description of the background and inquires about the character's actions. Five answer candidates are also included. To answer the question, QC-Comp divides the video into shorter segments of 64 frames each, mitigating attention biases such as the "lost-in-the" wei-intthmmdddee theemiddle"" issue. This segmentation allows the model to focus local attention on key frames, which receive high relevance scores with respect to both the question and the correct answer candidate. As a result, XComp is able to discard 512 irrelevant frames and accurately identify the correct answer.
4.3 Ablation Study
Learnable Compression. The key insight of our XComp train the LLM layers con- dense the visual tokens. In Ta- ble 2, compare using learnable with training-free token compres- sion. Enabling the LLM layers to learn the compression behav- ior consistently outperforms the training-free model, supporting the necessity of our learnable com- pression. Another critical com- parison is that when using the heuristic token compression from VideoChat- Flash, which utilizes at- tention scores to select the tokens, it is worse than the original baseline under extreme token
Table 2: Ablation study using LVBench: progressive v.s.
step-wise non-heuristic token compression v.s. heuristic com- pression from VideoChat-Flash and learnable v.s. training- free token compression. All the evaluations are conducted with 1024 frames. Both progressive compression and staged compression would gradually compress tokens to I token per frame with the same amount of average tokens per frame.
Method
Training-fres Learnable
Baseline (w/o compression)
41.8
+ Heuristic
41.1
+ Step-wise
38.4
42.3
+ Progressive
39.7
44.3

---

compression ratio. This is precisely the observation that motivates us to propose the learnable token compression (Sec. 3.2).
To investigate the impor- tance of progressive token compression, L.e., utilizing all the LLM layers to gradually decreasing the token number, we conduct ablation experiments comparing it with the conventional step-wise approach, which per- forms abrupt token reduction at pre-defined layers as in VideoChat-Flan Flash [35]. As shown in Table 2, pro- gressive token compression consistently outperforms the step-wise compression, indicating that smooth com- pression schedule essential for preserving the critical visual information.
Progressive Compression.
Table 3: Comparison of token compres- sion strategies on LVBench and MLVU. Both Uniform Drop and Soffix-Preesevattoon follow the same Cosine compression sched- ule and efficient continual training.
Strategy
LVBench
MLVU
Uniform Suffix
43.6
64.1
44.3
65.2
Compressing to Suffix Tokens. A critical design choice in our LP-Comp that the LLM layers should gradually preserve the information to the last token of each frame, namely, the suffix tokens. Such design compatible with the causal attention in LLM layers, and we analyze its necessity by comparing it with another intuitive strategy: keeping the tokens uniformly by their positions (implementation details in the supplementary materials). As shown in Table 3, preserving the suffix tokens outperforms its uniform counterpart strategy. supporting the importance of designing the compression strategy aligning with the attention of LLM layers.
Segmented Local Attention. To understand the impact of segment-wise attention in QC-Comp, which is proposed to mitigate the position bias of attention in long contexts, compare it with the conventional way of computing the attention between the question and all frames globally. As shown in Table segment-wise attention consistently leads to better performance across all three long video QA benchmarks LongVideo VideoBench. MLVU, and VideoMME (long). This validates our design choice of computing frame scores locally within segments, effectively reducing positional bias and improving the relevance of retained frames in the early filtering stage.
Table 4: Comparison of attention strategies in QC-drop. Segment- attention consistently outperforms global attention across benchmarks.
Attention
Long VideoBench
MLVU
VideoMME (long)
59.5 Segment Local (Ours)
Global
66.3
44.8
59.7
66.7
45.7
Effeet of LP-Comp and QC-Comp. We perform an ablation study to ana- lyze the impact the two components in XComp: A.p-Comp and QC QC-Comp. As shown Table 5, the baseline with- out any compression achieves a score of 58.3 on Long VideoBench. Adding LP-Comp alone (XComp without QC- Comp) yields a slight improvement to 58.8. When both LP-Comp and QC-Comp are used (full XComp), the performance reaches 59.7, suggesting that selectively removing less relevant content may help retain useful information. These results indicate that both components contribute to the final performance.
Table 5: Ablation results showing the individual and combined impact of LP-Comp and QC-Comp on LongVideoBencde Both components contribute to the over- all performance gains of XComp.
Long VideoBench
Baseline (w/o compression) XComp w/o QC-Comp XComp
58.3
58.8
59.7
Table 6: The complete table of all ablation studies on LongVideoBench and LVBench.
Model Name
LP-Comp
Variants
QC-Comp
Variants
Long VideoBench
LVBench
VideoChat-Flan
No
No
58.3
42.9
XComp
Yes
Uniform
No
58.2
43.6
XComp
Yes
Suffix
No
58.8
44.3
XComp
Yes
Suffix
Yes
Global
59.5
45.6
XComp
Yes
Suffxx
Yes
Segment Local
59.7
46.2

---

Compare All Variants Together. Table 6 shows the complete results for all ablation studies of XComp design, on two benchmarks, Long VideoBench and LVBench. The results are consistent with the studies above.
4.4 XComp for
We apply XComp LLaVA-NexI-Vin Video [78] to demonstrate the generalizability of our design. Similar to the experiments reported in the paper, XComp effectively enhances model efficiency, allowing a larger maximum number of frames to processed under limited GPU memory. Following the paper's 's setup, we use only 2.5% of the training data of LLaVA- VANeext-VVddoo (i.e., LLaVA- Video- 178k).
For a fair comparison, we also evaluate LLaVA-Next, Video with the same amount of fine-tuning data while keeping the model architecture unchanged. As shown in Table 7, XComp improves performance across both benchmarks.
Table 7: Generalize to VA-Nee- Video, a different model from VideoChat-Flash. XComp consistently improves the efficiency and accuracy of LLaVA-Next- Video on the two long video benchmarks, outperforming both the baseline and fine-tuning.
Model Version
VideoMME (Long)
LVBench
62.7
40.6
LLa -L-VV--ettt-iide FT
61.4
41.4
LLa VA-Next- XComp
63.2
43.1
5
Conclusion
In this work, we propose extreme token compression, named XComp, to address the long video understanding problem by alleviating the large number of tokens. the token level, our XComp framework targets on compressing each frame to one token. Observing the failures of heuristic compression adopted by previous methods, we propose the key insight of letting the LLM layers learn the progressive compression of video tokens. This compression strategy, called LP-Comp, effectively improves the token efficiency and enables denser frame sampling. Our compression at the frame level further enhances the information contained in the tokens by selecting the frames relevant to the user queries. Based on the internal attention scores of LLM layers, we further propose to split the whole video into shorter segments to mitigate the position bias issues. The resulting ineestionconnitionee compression, QC-Comp, curates a selective set of frames for the LLM to process. Collectively, our proposed strategies enable XComp explore the extreme of video token compression via data-efficient fine-tuning and significantly improve the base VideoChat-Flaan on multiple long video understanding benchmarks.
Limitations and Future Work. Under a limited budget, we primarily explore the extreme token compression with VideoChat-Flash-2B and LLaVA-Next-Viee via data-efficient fine-tuning. There- fore, potential future work includes adapting our XComp framework to larger VLMs and potentially integrating the learnable compression into the pre-training stage of the VLM. In addition the scale and diversity of experiments, another future work lies in training with the question-come conditiond frame selection, which could be jointly optimized with the LLM layers to achieve even larger compression ratios than presented in the paper.
Broader Impacts. This work presents advancements in lision-language modeling, with focus on improving long video understanding. While our contributions are primarily foundational, we acknowledge potential societal risks associated with vision- languallagguae models, including misuse of disinformation, privacy concerns, and the amplification of biases present in training data. We encourage future work to include fairness assessments and responsible release strategies. Where applicable, safeguards should be considered to mitigate unintended harms.
10

---

Acknowledgments
This work was supported part by Amazon, NSF under Grants 2106825 and 2519216, the ONR Grant N00014-26-1-20 1-299,,,,, and the DARPA Young Faculty Award. This work used computational resources, including Amazon Web Services (AWS), and the NCSA Delta and DeltaAl supercomputers through allocation CIS230012 from the Advanced Cyberinfrastructure Coordination Ecosystem: Services & Support (ACCESS) program.
References
[1] Triantafyllos Afouras, Effrosyni Mavroudi, Tushar Nagarajan, Huiyu Wang, and Lorenzo Torresani. Ht-step: Aligning instructional articles with how-to videos. NewrIPS, 2023. 25
[2] Kirolos Ataallah, Xiaoqian Shen, Eslam Abdelrahman, Essam Sleiman, Deyao Zhu, Jian Ding, and Mohamed Elhoseiny. Minigpt4-video: Advancing multimodal lims for video understanding with interleaved visual-textual tokens. arXiv preprint 20X1v22404.03413, 2024. 3
[3] Kirolos Ataallah, Xiaoqian Shen, Eslam Abdelrahman, Essam Sleiman, Mingchen Zhuge, Jian Ding, Deyao Zhu, Jürgen Schmidhuber, and Mohamed Elhoseiny. Goldfish: Dision-language understanding of arbitrarily long videos. In ECCV, 2024. 2,3
[4] Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang. Shijie Wang, Jun Tang, al. Qwen2. 5-vl technical report. arXiv preprint mrXx::2502.1392, 2025. 7, 23
[5] Daniel Bolya, Cheng-Yane Fu, Xiaoliang Dai, Peizhao Zhang, Christoph Feichtenhofer, and Judy Hoffmann Token merging: Your vit but faster. arXiv preprint arXiv:2210.09461, 2022. 3, 2.
[6] Shyamal Buch, Cristóbal Eyzaguirre, Adrien Gaidon, Jiajun Wu, Li Fei-Fei, and Juan Carlos Niebles. Revisiting the" video" in video-language understanding. In CVPR, 2022. 6
[7] Dongping Chen, Yue Huang, Siyuan Wu, Jingyu Tang, Huichi Zhou, Qihui Zhang, Zhigang He, Yilin Bai, Chujie Gao, Liuyi Chen, et al. Gui-world: A video benchmark and dataset for multimodal gui-oriented understanding. In The Thirteenth International Conference on Learning Representations, 2024. 25
[8] Guo Chen, Zhiqi Li, Shihao Wang, Jindong Jiang, Yicheng Liu, Lidong Lu, De-An Huang, Wonmin Byeon, Matthieu Le, Tuomas Rintamaki, et al. Eagle 2.5: Boosting long-context post-training for frontier vision-language models. arXiv preprint arXiv:2504. 15271, 2025. 7
[9] Lin Chen, Xilin Wei, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Zhenyu Tang, Li Yuan, et al. Sharegpt4video: Improving video understanding and generation with better captions. NeurIPS, 2024. 25
[10] Shimin Chen, Xiaohan Lan, Yitian Yuan, Zequn Jie, and Lin Ma. Timemarker: A versatile video-Ilm for long and short video understanding with superior temporal localization ability. arXiv preprint arXiv:2411.182/1 2024.7
[11] Yukang Chen, Fuzhao Xue, Dacheng Li, Qinghao Hu, Ligeng Zhu, Xiuyu Li, Yunhao Fang, Haotian Tang, Shang Yang. Zhijian Liu, et al. Longvila: Scaling long-context visual language models for long videos. arXiv preprint arXiv:2408. 10188, 2024. 2,3
[12] Zesen Cheng, Sicong Leng, Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang, Ziyang Luo, Deli Zhao, et al. Videollama 2: Advancing spatial-temporal modeling and audio understanding in video-Ilms arXiv preprint arXiv:2406.0744 07476, 2024. 1,3
[13] Joonmyung Choi, Sanghyeok Lee, Jaewon Chu, Minhyuk Choi, and Hyunwoo J Kim. vid-tidr: Training free token merging for light-weight video transformer. In CVPR, 2024.
[14] Cui, Y He, 7. Ma, 7. Chen, Tian, Wang, K Li, Y Wang, W Wang, X Zhu, et al. Sharegpt-4o: Comprehensive multimodal annotations with gpt-4o, 2024. 25
11

---

[15] Dave Epstein, Boyuan Chen, and Carl. Vondrick. Oops! predicting unintentional action in video.
arXiv preprint arXiv: 1911..111.11 2019. 25
[16] Yue Fan, Xiaojian Ma, Rujie Wu, Yuntao Du, Jiaqi Li, Zhi Gao, and Qing Li. Videoagent: A memory-augmented multimodal agent for video understanding. In ECCV, 2024. 6
[17] Jiajun Fei, Dian Li, Zhidong Deng, Zekun Wang, Gang Liu, and Hui Wang. Video-ccam: Enhancing lideo-language understanding with causal cross-attention masks for short and long videos. arXiv preprint arXiv:2408. /4023, 2024. 3
[18] Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal Ilms in video analysis. arXiv preprint arXiv:2405.21075, 2024. 8
[19] Jun Gao, Yongqi Li, Ziqiang Cao, and Wenjie Li. Interleaved-modar chain-of-thoughe arXiv preprint wrXiv:2411.19400 2024." 2
[20] Raghay Goyal, Samira Ebrahimi Kahou, Vincent Michalski, Joanna Materzynska, Susanne Westphal, Heuna Kim, Valentin Haenel, Ingo Fruend, Peter Yianilos, Moritz Mueller-Freitag. et al. The" something something" video database for learning and evaluating visual common sense. In ICCV, 2017." 25
[21] Kristen Grauman, Andrew Westbury, Eugene Byrne Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, et al. Ego4d: Around the world in 3,000 hours of egocentric video. In CVPR, 2022. 25
[22] Bo He, Hengduo Li, Young Kyun Jang, Menglin Jia, Xuefei Cao, Ashish Shah, Abhinav Shrivastava, and Ser-Nam Lim. Ma-Imm: Memory-augmented large multimodal model for long-term video understanding. CVPR, 2024.
[23] Suyuan Huang, Haoxin Zhang, Yan Gao, Yao Hu, and Zengchang Qin. From image to video, what do need in multimodal llms? arXiv preprint arXiv:2404./ 1865, 2024.
[24] IDEFICS Team. Introducing idefics: An open reproduction of state-of-the-an visual language model.
httpeeoocooooocooeeoeeeeeeoeeeee idefics, 2023. Accessed 2025-05-12. 3
[25] Yunseok Jang, Yale Song, Youngjae Yu, Youngjin Kim, and Gunhee Kim. Tgif-qa: Toward spatio-temporal reasoning in visual question answering. In /CCV, 2017. 25
[26] Peng Jin, Ryuichi Takanobu, Wancai Zhang, Xiaochun Cao, and Li Yuan. Chat-univi: Unified visual representation empowers large language models with image and video understanding, In CVPR, 2024. 3
[27] Will Kay, Joao Carreira, Karen Simonyan, Brian Zhang, Chloe Hillier, Sudheendra Vijaya- narasimhan, Fabio Viola, Tim Green, Trevor Back, Paul Natsev, et al. The kinetics human action video dataset. arXiv preprint 1705: 06950, 2017. 25
[28] Bruno Korbar, Yongqin Xian, Alessio Tonioni, Andrew Zisserman, and Federico Tombari. Text-conditioned resampler for long form video understanding. In ECCV, 2024.
[29] Sanghyeok Lee, Joonmyung Choi, and Hyunwoo Kim. Multi-criteria token fusion with one-step-ahead attention for efficient vision transformers. In CVPR, 2024. 3
[30] Jie Lei, Licheng Yu, Mohit Bansal, and Tamara L Berg. Tvqa: Localized, compositional video question answering. arXiv preprint 1 2018. 25
[31] Bo Li, Kaichen Zhang, Hao Zhang, Dong Guo, Renrui Zhang, Feng Li, Yuanhan Zhang, Ziwei Liu. and Chunyuan Li. Llava-next: Stronger 1lms supercharge multimodal capabilities in the wild. 2024. 2
[32] Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.033 03326, 2024. 1, 25
12

---

[33] Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. Llava-next-intee mnex--interleave: Tackling multi-image, video, and 3d in large multimodal models. arXiv preprini arXiv:2407.07895, 2024. 25
[34] KunChang Li, Yinan He, Yi Wang. Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang, Limin Wang. and Yu Qiao. Videochat: Chat-centric video understanding. arXiv preprint arXiv:2305.00 006.05555 2023. 3
[35] Xinhao Li, Yi Wang, Jiashuo Yu, Xiangyu Zeng, Yuhan Zhu, Haian Huang, Jianfei Gao, Kunchang Li, Yinan He, Chenting Wang, al. Videochat-fash: Hierarchical compression for long-context video modeling. arXiv preprint arXiv:2501.00574, 2024. 2, 3, 4, 5, 7, 8, 9, 23, 24, 25, 26
[36] Yanwei Li, Chengyao Wang, and Jiaya Jia, Llama-vid: An image is worth 2 tokens in large language models. In ECCV, 2024. 3
[37] Yucheng Li, Huiqiang Jiang, Chengruidong Zhang, Qianhui Wu, Xufang Luo, Surin Ahn, Amir H Abdi, Dongsheng Li, Jianfeng Gao, Yuqing Yang, et al. Mapsparse: Accelerating pre-filling for long-context visual language models via modality-aware permutation sparse attention. In ICLR 2025 Workshop on Foundation Models in the Wild. 5
[38] Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, and Li Yuan. Video-llave Learning united visual representation by alignment before projection. arXiv preprint arXiv:2311. 10122, 2023. 1,3
[39] Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Leava-next: Improved reasoning, ocr, and world knowledge, January 2024. 25
[40] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In NeurlPS, 2023. 4
[41] Jiajun Liu, Yibing Wang. Hanghang Ma, Xiaoping Wu, Xiaoqi Ma, Xiaoming Wei, Jianbin Jiao, Enhua Wu, and Jie Hu. Kangaroo: A powerful video-language model supporting long-context video input. arXiv preprint arXiv:2408. 15542, 2024. 7
[42] Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts. arXiv preprint arXiv:2307.03 2023. 2,6
[43] Ye Liu, Siyuan Li, Yang Wu, Chang- Wen Chen, Ying Shan, and Xiaohu Qie. Umt: Unified modti transformers for joint video moment retrieval and highlight detection. CVPR, 2022. 7, 23
[44] Ruipu Luo, Ziwang Zhao, Min Yang, Junwei Dong. Da Li, Pengcheng Lu, Tao Wang, Linmei Hu, Minghui Qiu, and Zhongyu Wei. Valley: Video assistant with large language model enhanced ability. arXiv preprint 23060023006 2023. 3
[45] Xu Ma, Yuqian Zhou, Huan Wang, Can Qin. Bin Sun, Chang Liu, and Yun Fu. Image as set of points. arXiv preprint arXiv:2303.01444. 2023.
[46] Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Shahbaz Khan. Video-chatgpt: Towards detailed video understanding via large vision and language models. arXiv preprint arXiv:2306.0544 05424, 2023.
[47] Xupeng Miao, Shenhan Zhu, Fangcheng Fu, Ziyu Guo, Zhi Yang, Yaofeng Tu, Zhihao Jia, and Bin Cui. X-former elucidator: reviving efficient attention for long context language modeling. In IJCAI, 2024.
[48] OpenAl. Gpt-4 technical report, 2024. 7
[49] OpenAl. Gpt-4o system card, 2024. 3,7
[50] Jongwoo Park, Kanchana Ranasinghe, Kumara Kahatapitiya, Wonjeong Ryu, Donghyun Kim, and Michael S Ryoo. Too many frames, not all useful: Efficient strategies for long-form video qa. arXiv preprint arXiv:2406. 09396 2024. 6
13

---

[51] Jiyang Qi. Yan Gao. Yao Hu, Xinggang Wang, Xiaoyu Liu, Xiang Bai, Serge Belongie, Alan Yuille, Philip HS Torr, and Song Bai. Occluded video instance segmentation: benchmark. IJCV, 2022. 25
[52] Shuhuai Ren, Sishuo Chen, Shicheng Li, Xu Sun, and Lu Hou. Testa: Temporal-spatial token aggregation for long-form video-language understanding. arXiv preprint arXiv:23 10. 19060, 2023.
[53] Xiaoqian Shen, Yunyang Xiong, Changsheng Zhao, Lemeng Wu, Jun Chen, Chenchen Zhu, Zechun Liu, Fanyi Xiao, Balakrishnan Varadarajan, Florian Bordes, et al. Longvu: Spa- tiotemporal adaptive compression for long video-language understanding. arXiv preprint arXiv:2410.17334 2024. 2
[54] Xuyang Shen, Dong Li, Jinxing Zhou, Zhen Qin, Bowen He, Xiaodong Han, Aixuan Li, Yuchao Dai, Lingpeng Kong, Meng Wang, et al. Fine- tine-grained audible video description. In CVPR, 2023. 25
[55] Yunhang Shen, Chaoyou Fu, Shaoqi Dong, Xiong Wang, Peixian Chen, Mengdan Zhang, Haoyu Cao, Ke Li, Xiawu Zheng, Yan Zhang, et al. Long-vita: Scaling large multi-modal models to 1 million tokens with leading short-context accuray. arXiv preprint arXiv:2502.05 177, 2025.
[56] Yan Shu, Zheng Liu, Peitian Zhang, Minghao Qin, Junjie Zhou, Zhengyang Liang, Tiejun Huang, and Bo Zhao. Video-xl: Extra-long vision language model for hour-scale video understanding. arXiv preprint arXiv:2409. 14485, 2024. 3
[57] Enxin Song, Wenhao Chai, Guanhong Wang, Yucheng Zhang, Haoyang Zhou, Feiyang Wu, Haozhe Chi, Xun Guo, Tian Ye, Yanting Zhang, et al. Moviechat: From dense token to sparse memory for long video understanding. CVPR, 2024. 3,
[58] Gemini Team. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context, 2024. 3,7
[59] Weihan Wang, Zehai He, Wenyi Hong. Yean Cheng, Xiaohan Zhang, Ji Qi, Xiaotao Gu, Shiyu Huang, Bin Xu, Yuxiao Dong, et al. Lvbench: An extreme long video understanding benchmark.
arXiv preprint arXiv:2406.08035, 2024. 3, 8
[60] Weiyao Wang, Matt Feiszli, Heng Wang, and Du Tran. Unidentified video objects: A benchmark for dense, open-world segmentation. In ICCV, 2021. 25
[61] Yi Wang, Kunchang Li, Xinhao Li, Jiashuo Yu, Yinan He, Guo Chen, Baoqi Pei, Rongkun Zheng. Zun Wang. Yansong Shi, al. Internvide02: Scaling foundation models for multimodal video understanding. In ECCV, 2024. 1,3
[62] Yi Wang, Kunchang Li, Yizhuo Li, Yinan He, Bingkun Huang, Zhiyu Zhao Hongjie Zhang, Jilan Xu, Yi Liu, Zun Wang, et al. Internvideo: General video foundation models via generative and discriminative learning. arXiv preprint arXiv:2212.03199 2022. 1.3
[63] Ziyang Wang, Shoubin Yu, Elias Stengel-Eskin, Jaehong Yoon, Feng Cheng, Gedas Bertasius, and Mohit Bansal. Videotree: Adaptive tree-based video representation for Ilm reasoning on long videos. arXiv preprint arXiv:2405. 19209, 2024. 6
[64] Haoning Wu, Dongxu Li, Bei Chen, and Junnan Li. Longvideobench: benchmark for long-context interleaved video-language understanding. NeurIPS, 2024. 8
[65] Weijia Wu, Yuzhong Zhao, Zhuang Li, Jiahong Li, Hong Zhou, Mike Zheng Shou, and Xiang Bai. A large cross-modal video retrieval dataset with reading comprehension. Pattern Recognition, 157:110818. 2025. 25
[66] Wenhao Wu, Yizhong Wang, Guangxuan Xiao, Hao Peng, and Yao Fu. Retrieval head mecha- nistically explains long-context factuality. arXiv preprint arXiv:2404. 15574, 2024. 6
[67] Junbin Xiao, Xindi Shang, Angela Yao, and Tat-Seng Chua. Next-qa: Next phase of question- answering to explaining temporal actions. In CVPR, 2021. 25
14

---

[68] Jiarui Xu, Shalini De Mello, Sifei Liu, Wonmin Byeon, Thomas Breuel, Jan Kautz, and Xiaolong Wang. Groupvit: Semantic segmentation emerges from text supervision. In CVPR, 2022. 3
[69] Lin Xu, Yilin Zhao, Daquan Zhou, Zhijie Lin, See Kiong Ng, and Jiashi Feng. Pllava: Parameter-free llava extension from images to videos for video dense captioning. arXiv preprint arXiv:2404.16994, 2024.
[70] Mingze Xu, Mingfei Gao, Zhe Gan, Hong- and Afshin Dehghan. Slowfast-llava: A strong training- models, arXiv preprini arXiv:2407.1584) 2024. 1,3
You You Chen, Zhengfeng Lai, Haiming Gang, Kai Kang, wriniig--free free baseline for video large language
[71] An Yang. Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15112 15115, 2024.7 7
[72] Dongjie Yang. Suyuan Huang, Chengqiang Lu, Xiaodong Han, Haoxin Zhang, Yan Gao, Yao Hu, and Hai Zhao. Vript: video worth thousands of words. NeurIPS, 2024. 25
[73] Jiabo Ye, Haiyang Xu, Haowei Liu, Anwen Hu, Ming Yan, Qi Qian, Ji Zhang, Fei Huang, and Jingren Zhou. oww:: Towards long magee-sqquene understanding in multi-modal large language models. arXiv preprint arXiv:2408. 04840, 2024.
[74] Kexin Yi, Chuang Gan, Yunzhu Li, Pushmeet Kohli, Jiajun Wu, Antonio Torralba, and Joshua B Tenenbaum. Clevrer: Collision events for video representation and reasoning. arXiv preprini arXiv: 191:v:191..01442, 2019. 25
[75] Peiyuan Zhang. Kaichen Zhang. Bo Li, Guangtao Zeng, Jingkang Yang, Yuanhan Zhang, Ziyue Wang. Haoran Tan, Chunyuan Li, and Ziwei Liu. Long context transfer from language vision. arXiv preprini arXiv:2406. 16852, 2024. 3
[76] Qingru Zhang, Dhananjay Ram, Cole Hawkins, Sheng Zha, and Tuo Zhao. Efficient long-range transformers: You need to attend more, but not necessarily at every layer. arXiv preprint WrXiv:2310.12443 2023. 2
[77] Y Zhang, B Li, H Liu, Y Lee, L Gui, D Fu, J Feng, Z Liu, and C Li. Llava-next: A strong zero-shot video understanding model. 2024.
[78] Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma. Ziwei Liu, and Chunyuan Li. Video instruction tuning with synthetic data, arXiv preprint arXiv:2410.02211. 2024. 10, 25
[79] Junbao Zhou, Ziqi Pang, and Yu- Yu-Xiong Wang. Rmem: Restricted memory banks improve video object segmentation. In CVPR, 2024. 6
[80] Junjie Zhou, Yan Shu, Bo Zhao, Boya Wu, Shitao Xiao, Xi Yang, Yongping Xiong, Bo Zhang, Tiejun Huang, and Zheng Liu. Mlvu: A comprehensive benchmark for multi-task long video understanding. arXiv preprint wxXh:2406.042244 2024. 8
[81] Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Yuchen Duan, Hao Tian, Weijie Su, Jie Shao, et al. Internv13: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprini (ttiv:2504.04477 2025. 7,8
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: See the Methods and Experiments sections, where provide detailed description our proposed method and the corresponding experiments.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
See the Conclusion section.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
16

---

Justification:
There is no theoretical result.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We show the methodology details with exact numbers with well-defined formulas.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
17

---

Answer: [No]
Justification: We mentioned the data we used in the paper. These datasets are public. As for the code, planning to release the code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips cc/
public/guides/eee
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to to reproduce the results. See the NeurlPS code and data submission guidelines (https:
//wpp..cc////////. cttppscccppubl/c/ggcee gww./////eeeeeeeeee /CodeSubmis / ssi ssionPolicy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We provided the details with specific numbers in appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Restricting to computation budget, we did not run experiments in multiple seeds as well as apply statistical analysis.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
18

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The paper provides GPU hours.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the csGuidel NeurIPS Code of Ethics https: http///eeerrpp.eee http::://aarrps..ccpppl/ll///// / ines?
Answer: [Yes]
Justification:
This research adheres to the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We mentioned the social impact in appendix.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
19

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [No]
Justification: The paper does not describe safeguards.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All the assets used in the paper are either cited the main paper (models, benchmarks, ...) or in the appendix (data.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
20

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
21

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [Yes]
Justification: LLM is part of the method in this paper.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy (https : https////.euu.//eeeeeeeee /tt.:///eee..oeeeeeee/////////// for what should or should not be described.
22

---

A Implementation Details
A.1 XComp Architecture
XComp is based on VideoChat-Flash [35] and uniquely integrates our learnable progressive compres- sion (I.--Com'\SSc. 3.2) and question-conditione compression (QC-Comp((Sec. 3.3). XComp comes from fine-tuning the VideoChat-Flase fine-tuning details are shown in Appendix A.2.
model [35] with small amount of 2.5% data, the
Algorithm A shows the neural network architecture of XComp, which is largely consistent with VideoChat-Flase Flash. begins with the UMT-L visual encoder [43], which encodes short video clips consisting of 8 frames into visual tokens. Token merging [5] is subsequently applied to reduce the token count 128 tokens per clip and 16 tokens per frame. A two-layer MLP connector then maps these visual tokens from the visual encoder space into the representation space of the large language model. Finally, Qwen2-1.55 [4] serves as the large language model (LLM) in our framework.
During inference with QC-Comp, multiple forward passes through the model are performed. First, XComp conducts forward pass obtain scores for the frames. Note that for videos longer than 512 frames, the video is divided into shorter segments, each processed separately to compute frame scores. These individual scores are then aggregated. Based on the aggregated scores, XComp selects the top-ranked frames and passes them to the model to generate responses.
Details of LP-Comp Algorithm B shows the details of LP-Comp (Sec. 3.2). It compresses video tokens layer-by-layer in a suffix peetiix-preeriing manner. Given input tokens V E R x xd at layer the algorithm computes the target token count N for the next layer using cosine- based schedule. no reduction is needed, tokens are returned unchanged. Otherwise, for each video clip and each frame within it, only the last Nuex tokens are kept from the current Nprevs preserving the temporal suffix. The retained tokens across all frames and clips concatenated to form the output ((+1), This design ensures progressive compression while retaining semantically rich information.
Details of QC-Comp Algorithm C shows the details of Segmented Local Attention, which is the main part of QC-Comp (Sec. 3.3) and computes scores from attention maps in LLM layers. For a given video V and text query Q, the algorithm slides a local window (64 frames with stride of 32) over the video sequence. Within each segment, it computes the full attention map from the LLM's l-th layer. For each frame in the segment, the attention weights over the query tokens are averaged and accumulated into corresponding 8-frame clip buckets. Although Segmented Local Attention inherently produces clip-level scores, QC-Comp assigns the same score to all frames within clip, thereby converting clip-level scores to frame-level scores. For long videos exceeding 512 frames, the
Algorithm A: MODEL__FORWARI forward pass with LP-Comp and the score for QC-Comp
Input: M (model), (video), Q (lext), relurnScore E 10 Output: logits; score (per-clip) returnScore 1*t Step I: video encoding //
(
[R*"} (Vrttition(VV,8) 8)
// separate each 8 frames + 1 clip
foreach V do
Ty + M.umt vissul_eene(W
Tk < TokenMerge (T2, 128)
// to 128 tokens/clip, 16 tokens/frame frame // project to LLM dim
T < MM.ll((T.)
( + concat(Ti,..., Tk)
(0)) +- M.text (
N Step 2: reasoning in LLM layers 4/
for = 0 M.L do
[e+1)) Q(++1) <- M.llu (V ((1)
((41) LLLLP_Coopp (E, ((+1))
// token-level compression // score for QC_Comp
score [(] tagemmmttttalloollllattttttttton V(O) ((9)
logits +Computeeooitss (Q
if relurnScore then return (logits, score)
else return logits
23

---

1/ final clip score method processes multiple overlapping 512-frame chunks. The final frame-level scores are obtained by aggregating the results from these overlapping chunks. To ensure diverse frame-level scores. each frame would be included in Prepeat overlapping and shifted chunks that encourage score variation across neighboring frames. With these frame-level scores, __selected__feeee frames among total frames are selected. Table B shows the hyperparameters.
A.2 Supervised Fine-tuning Details
Hyperparameters Table shows the hyperparameters used in fine-tuning. We follow the same training configuration as VideoChat-Flash [35], including the learning rale, weight decay, warmup ratio, and learning rate scheduler. The only difference lies the frame sampling parameters: frames_ sammes_pbound, frames_ Taaamm_llwboude and the default frames second (FPS). This change is due to our use LP-Comp, which enables more efficient frame representation. As result, we double the values of frames_ lowbouud and frames_ _aaamss_uubuudd compared to VideoChat-Flash
Datasets We used the 2.5% supervised fine-tuning data collected TideoChat-Flash [35]. During data curation, we disregarded a few datasets that were exceptionally large on disk or whose licenses made automatic download impractical. After this filtering, the final training set contains 71,927 instances are drawn from publicly available datasets, listed in Table C.
released by
Algorithm B: LP_ COMP: suffix layer-wise video-token compression e (layer index), ( (video tokens, shape K NOO d)
Input:
Output:
v(E+1) (compressed video tokens)
x(t) coo (T) + 20
Npae
2
Noet
x(1) coof +) x0
// Eq. (1)
Nprev
Nuext then return (())
// nothing compress
foreach
T ((() do foreach to /' do
// iiterate over K clips
11 iterate over frames in the clip [N
keep <- (J
Nrev + Npw Nues
T, < T [id__eepp
// suffix-preservation
T" < concat (T....,TT)) TT)
((+1) < concat(T coonca(("_ccipl...... T clipk)
return ((+1)
Algorithm C: SEGMENTEDLOCALATTENNOOO
compute clip-level saliency scores
Input: & (layer index). (video tokens), Q (text tokens)
Output: score (list of length K, one mean score per clip)
Lisg f- 64
// frames per segment (8 clips)
stride +- 32
// stride in frames (4 clips)
bucket + - 1
for start to F. Lus step stride do
// slide local window the video
Vag t- [start start + Lueg)
< M.yerr do
Vags Q1)
// full attention map
for = 0 to Taxe
// clip index
< mean Ali, Q_start Q_end) bucket(c]
for l 1 to K do
score [k] mean( (bucketkk
return
24

---

achieves comparable Table A: The hyperparameters used in fine-tuning.
Hyperparameter
Value / Description
tunable_parts
Large Langauge Model
learning_rate
1 x 10-6
weight_decay
0.0
warmup_ratio
0.03
1r_scheduler_type
Cosine
datallodder_mm__vonnn Bacpiddrruummmmooee num_woreewsss
1
frames_upbound
1024
frames_ crames_lowbound
128
local_mun _um frames
8
sample_type
Dynamic FPS (8 fps by default)
Table B: The hyperparameters used in evaluation.
Hyperparameter
Value / Description
temperature
0,0
do_ do_sample
False
num num_beas beams
n_repeat
2 (each frame in 2 chunks)
n_ m_selected_framee
256 (Long), 512 (MME), 1,024 (MLVU), 2,048 (LVB)
Table C: Datasets used for supervised fine-tuning (71,927 instances).
Image datasets
Video datasets
VLavi-OneVision [32].
Kinetics-400 [27], Something-Someethig [20], TGIF-QA [25].
LLaVA-NeXT
[39],
TVQA [30], CLEVRER [74], NEXT-QA [67], FAVD [54],
M4-Instt [33]
MovieChat-1KK [57]. TextVR [65]. ShareGPT-Video [9].
ShareGPT-40 [14], Oops [15]. OVIS [51]. UVO [60].
GUI-World [7]. Vript [72], HT-Step [1]. Ego4D [21].
LLaVA-Video-I738. [78], VidcoChat-Flash [35]
B Additional Experiments
B.1 Ablation Study on Fine-tuning
Table D presents ablation study isolate the effect of our proposed design from that of fine- tuning. Specifically, we compare XComp with a baseline that applies the same fine-tuning procedure (on 2.5% of the data as mentioned Appendix A.2) to the original HideoChat-Flash model, without introducing our LP-Comp and QC-Comp. performance the original model, suggesting a limited benefit from fine-tuning. This indicates that the performance gains of XComp stem from our method's enhancements, rather than from fine-tuning alone.
Table D: Ablation study of fine-tuning.
Size
Long VidcoBenc
MLVU
VidcoMME (Long)
LVBench
Average Duration
473s
651s
2386s
4101s
VideoChat-Flash-22 [35]
2B
58.3
65.7
44.9
42.9
VideoChat
2B
57.4
65.6
44.7
43.2
XComp
2B
59.7
66.7
45.6
46.2
25

---

B.2 Efficiency Analysis
Table shows the efficiency comparison on a LVBench query (863 text tokens), measured over 10 runs on single NVIDIA H200. Across 1k-4k frames, XComp reduces LLM TFLOPs by ~53-58% and latency by 45-58%
Table E: Efficiency Comparison on LVBench Queries at Inference Stage.
Frames
Model
TFLOPs
Latency
1,024
VideoChat-Flash
43
0.22
1,024
Ours
20
(53 ()
0.12
(45 (45%))
2,048
VideoChat-Flash
132
0.54
2,048
Ours
58
(56 L)
0.25
(54%
4,096
VideoChat-Flash
448
1.56
4,096
Ours
187
(58%
0.66
(58%1)
B.3 More Evaluations
While XComp primarily targets multiple-choice tasks long videos, it is also capable of handling other fundamental video understanding tasks. We claim that the extreme compression not harmful to the fundamental capability for short videos. In particular, benchmarks such as CLEVRER, which test causal and counterfactual reasoning, and (2) dense-output tasks such as video captioning (VDC benchmark), which assess the ability to generate descriptive text for videos. As shown in Table F, XComp achieves results comparable to its backbone model SieeoChat-Flash-222 demonstrating that the compression does not significantly harm performance on these tasks.
include evaluations on (1) reasoning
We additionally evaluate MME- VideoOCR, benchmark designed measure fine-grained text perception from videos. Table G reports results on both the full benchmark and the subset of long videos (>30s). XComp shows moderate performance drop compared to the original VideoChat- Flash-2B, consistent with our main paper observations that this is largely due to data differences rather than architecture changes. When compared fine-tuned with the same backbone, the performance gap is minimal, confirming that the degradation primarily comes from training data rather than compression.
Table F: CLEVRER and VDC results.
CLEVRER
VDC
Expl (Opt)
Expl (Q)
Pred (Opt)
Pred (Q)
Catrf (Opt)
Cutrf (Q)
BLEU@I 1
BLEU@4
VideoChat-Flash-22 XComp
0.9463
0.8497
0.7789
0.5738
0.7775
0.4007
1.6
0.9398
0.8406
0.7697
0.5600
0.7625
0.3652
7.0
1.6
Table G: MMME-ideeOCCK results.
Model
MME VideOOR (Overall)
MME- VideoOOR (>-30s)
VideoChat-Flash-22 VideoChat-Flash-h + FT
37.1
49.1
34.9
46.0
XComp
35.4
46.4
B.4 Multi-hop NIAH
Figure A shows the results of the Multi-Hop Needle-in-a-Hayee QA task [35] which is designed to evaluate extreme long-context reasoning abilities. This benchmark embeds a reasoning path of images within long video sequences, where each image contains clues guiding the model to the next. Given a starting point, the model must trace the correct path, identify the target image (needle), and answer related question. In this experiment, we use QC-Comp with n_ selected, frames 1
26

---

LongVA
VideoChat-Flash
Vieeohaa-Fiah--O--com
XComp
90
QA Accuracy (%)
60
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
Num. of Frames
Figure A: Multi-Hop Neelle-in-a-Haystack QA Performance.
and n_repeat 8. It accurately selects the keyframe with 65% accuracy at sequence length of 6,144, leading to QA average accuracies of 72.6 and 72.2 over 2,000 to 10,000 total frames for VideoChat-Flahh-CC-CCr and XComp, respectively.
27


Reviews:
[{"review_id": "NpwZ1b09CG", "reviewer": "Reviewer_bQaD", "summary": "This paper proposes a token compression framework termed XComp for long video understanding, addressing the challenge of token inefficiency in video-language models. The key idea lies in its hierarchical compression strategy: LP-Comp leverages LLM layers to learn progressive token compression (reducing frames to single tokens), improving density while avoiding heuristic failures of prior work; QC-Comp introduces query-conditioned frame selection via attention scores, mitigating positional bias through video segmentation. The combined approach enables extreme compression of 1 token per frame while preserving query-relevant information. Experiments demonstrate significant gains over VideoChat-Flash on long-video benchmarks.", "questions": "Please see weaknesses.", "limitations": "Yes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "- S1: The idea of letting LLM layers learn compression (LP-Comp) is well-motivated, moving beyond handcrafted heuristics.\n- S2: QC-Comp’s attention-based segmentation elegantly handles position bias—a known limitation in long-context LLMs.\n- S3: The paper is with promising empirical validation. Multi-benchmark improvements suggest generalizability.\n\n- W1: While the proposed method could achieve comparable or better performance on several benchmarks, the efficiency comparison with baseline VideoChatFlash. I would like the authors to give more discussion and quantative results on time efficiency and FLOPs.\n- W2: Compression works well on general videoQA. However, the performance on fine-grained video tasks like video OCR (like MME-VideoOCR) and reasoning benchmarks (like CLEVRER) should be considered to verifiy its generalization ability.", "was_revised": true, "final_justification": "Generally speaking, the paper is well-motivated, thoughtfully designed, and demonstrates effective performance. However, one remaining limitation is that the effectiveness on fine-grained tasks in long videos has not been sufficiently verified. Even so, I am inclined to accept this paper and hope that the discussion on this limitation will be incorporated in the final version of the paper or appendix.", "rebuttal": "Thank you for your thoughtful and encouraging review. We are glad that you find our proposed compression strategy—LP-Comp and QC-Comp—well-motivated and effective in addressing token inefficiency for long video understanding. We appreciate your recognition of our method’s empirical strength and generalizability across benchmarks.\n\nWe are also grateful for your constructive suggestions regarding efficiency analysis and fine-grained benchmark evaluation. In the following, we provide quantitative comparisons on latency and FLOPs, and include new results on MME-VideoOCR and CLEVRER to assess generalization to short-video and fine-grained tasks. We believe these additions further clarify the strengths and limitations of our method.\n\n---\n\n### 1) Efficiency vs. VideoChat-Flash\n\n> While the proposed method could achieve comparable or better performance on several benchmarks, the efficiency comparison with baseline VideoChatFlash. I would like the authors to give more discussion and quantative results on time efficiency and FLOPs.\n\nThanks for your suggestion to give more efficiency comparison! Here we would like to show the efficiency comparison on LVBench items (863 text tokens), measured over 10 runs on a single NVIDIA H200. Across 1k–4k frames, XComp reduces LLM TFLOPs by **~53–59%** and latency by **~45–58%**.\n\n| Frames | Model  | TFLOPs | Latency |\n|---:|---:|---:|---:|\n| 1,024 | VideoChat-Flash | 43 | 0.22 s |\n| 1,024 | XComp | 20 (**↓53%**) | 0.12 s (**↓45%**) |\n| 2,048 | VideoChat-Flash | 132 | 0.54 s |\n| 2,048 | XComp  | 58 (**↓56%**) | 0.25 s (**↓54%**) |\n| 4,096 | VideoChat-Flash | 448 | 1.56 s |\n| 4,096 | XComp | 187 (**↓58%**) | 0.66 s (**↓58%**) |\n\n---\n\n### 2) Short-video benchmarks (MME-VideoOCR & CLEVRER)\n\n> Compression works well on general videoQA. However, the performance on fine-grained video tasks like video OCR (like MME-VideoOCR) and reasoning benchmarks (like CLEVRER) should be considered to verifiy its generalization ability.\n\nWe appreciate the suggestions from the reviewer on analyzing the performance of fine-grained perception, which is indeed a challenging scenario for video understanding. However, these two benchmarks mostly focus on **short videos** (MME-VideoOCR clips are mostly **≤10 s**, and CLEVRER videos are **5 s**), while **our XComp primarily targets the scope of efficiency and effectiveness on long videos (up to hours)**. However, we report the performance short-video results per the reviewer’s request.\n\n**MME-VideoOCR.** XComp and VideoChat-Flash-2B perform similarly  overall, where XComp shows stronger temporal grounding but slightly weaker spatial grounding.\n\n|  | Temporal Grounding | Spatial Grounding | Overall |\n|---|---:|---:|---:|\n| VideoChat-Flash-2B | 41 | **57** | **37.1** |\n| XComp | **45** | 52 | 35.4 |\n\nFor subsets less tied to fine-grained spatial details, the two models are almost identical:\n\n|  | Attribute Recognition | Text-based Reasoning | Change Detection & Tracking |\n|---|---:|---:|---:|\n| VideoChat-Flash-2B | 56 | 42.6 | 42 |\n| XComp | 55.3 | 42.6 | 42 |\n\n**CLEVRER.**  Since CLEVRER training set is part of VideoChat-Flash pre-train data, both models perform strongly; XComp shows a small drop, as we only used ~2% of the original SFT data from VideoChat-Flash. Continued fine-tuning is expected to close this gap.\n\n| Model | Explanatory (Opt/Q) | Predictive (Opt/Q) | Counterfactual (Opt/Q) |\n|---|---:|---:|---:|\n| VideoChat-Flash-2B | 0.9463 / 0.8497 | 0.7789 / 0.5738 | 0.7775 / 0.4007 |\n| XComp | 0.9398 / 0.8406 | 0.7697 / 0.5600 | 0.7625 / 0.3652 |\n\n**Conclusions.** Although these short-video evaluations are out of our original scope, they reveal the discovery that our token compression only marginally impact the performance on fine-grained understanding. We appreciate the reviewer’s suggestion and will include these experiments and discussion to motivate future work on jointly optimizing long-video compression and fine-grained short-video perception."}, {"review_id": "wdiTxs59Je", "reviewer": "Reviewer_iYWa", "summary": "The paper “One Token per Highly Selective Feature” proposes a lightweight and general token reduction method called OTPF, which significantly improves inference efficiency for vision and multimodal models. By selecting and compressing only the most informative features from early layers—based on activation strength or selectivity—OTPF generates a small number of tokens that preserve essential semantic information. This approach can be used without training or integrated into end-to-end learning. Experiments on image classification, video understanding, and multimodal QA demonstrate that OTPF achieves substantial token reduction (up to 90%) with minimal performance drop, offering a simple, modular, and effective solution for efficient large-model deployment.", "questions": "Have the authors tried OTPF in such settings, or do they envision ways to extend it to support dense output tasks?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. OTPF achieves substantial token reduction (up to 90%) while preserving accuracy across multiple tasks, demonstrating strong practical value in resource-constrained settings.\n2. The method is compatible with a wide range of backbones (ViT, CLIP, ResNet) and tasks (image classification, video understanding, multimodal QA), showing strong versatility.\n3. OTPF can be used without any additional training, yet also supports end-to-end learning, making it easy to integrate into existing pipelines.\n4. Outperforms or matches state-of-the-art baselines (e.g., uniform sampling, VideoMAE) in low-token regimes, validated across diverse benchmarks like ImageNet-1K and Kinetics-400.\nWeaknesses:\n1. OTPF’s effectiveness depends heavily on the quality of early-layer representations. If the backbone lacks strong discriminative power, selected tokens may be suboptimal.\n2. OTPF compresses every input into a fixed number of tokens, regardless of the semantic richness or spatial/temporal complexity. This static compression may underrepresent informative regions in complex scenes and over-represent irrelevant parts in simple ones.\n3. The token selection relies on hand-crafted heuristics (e.g., activation magnitude, channel max), which may not always correlate with semantic importance—especially in domain-shifted or adversarial settings.", "was_revised": true, "final_justification": "Thank you to the authors for providing a detailed rebuttal. The authors have effectively addressed my concerns, particularly with regard to OTPF’s adaptation to token compression, the handling of semantic complexity through QC-Comp, and the reduction of heuristics reliance with LP-Comp. Additionally, the VDC benchmark experiment successfully demonstrates OTPF's potential for dense-output tasks. Given these clarifications and improvements, I am satisfied with the responses and believe that the paper is now in an acceptable form. Therefore, I maintain my initial score and recommend acceptance.", "rebuttal": "Thank you for your thoughtful and constructive review. We are pleased that you find our proposed method OTPF to be a lightweight, modular, and effective solution for efficient model deployment across diverse tasks. We appreciate your recognition of its versatility, strong performance under extreme token reduction, and ease of integration. Your insightful comments regarding early-layer representations, semantic complexity, and reliance on heuristics have helped us clarify and further articulate the motivations, design choices, and future directions of our work. We address your concerns and suggestions in detail below.\n\n---\n\n### 1. Early Layer Representations\n\n> OTPF’s effectiveness depends heavily on the quality of early-layer representations. If the backbone lacks strong discriminative power, selected tokens may be suboptimal.\n\nThis is a very insightful observation, which is exactly our motivation for developing learnable and progressive compression, the heuristic-based methods in prior works. Our insight is that **OTPF's token compression relies less on the quality of early-layer representations** via optimizing the LLMs to conduct token compression and adapt to the early-layer representations.\n\n* Considering the heuristic-based approaches, *e.g.*, utilizing the correlation between text and image tokens or even uniform drop, they rely on the assumption that the early representations have to be sufficiently informative to mitigate the token drop. Instead, our learnable compression **jointly optimizes the LLM and early-layer representation** so that the LLMs can adapt to token compression.\n* More importantly, our compression is progressive and features a smooth reduction in the token number, which regulates the information loss happening at the ealy layers.\n\n---\n\n### 2. Token Reduction and Semantic Complexity\n\n> OTPF compresses every input into a fixed number of tokens, regardless of the semantic richness or spatial/temporal complexity. This static compression may underrepresent informative regions in complex scenes and over-represent irrelevant parts in simple ones.\n\nWe agree that this is indeed a common confusion regarding token compression. Our design of XComp has attempted to address this problem from both aspects of inter-frame and intra-frame compression:\n\n* Regarding inter-frame compression, our method does not disregard semantic complexity. Instead, **our Question-Conditioned Compression (QC-Comp) is proposed to identify the informative frames** for emphasis and decrease the influence of irrelevant frames.\n* Regarding intra-frame compression, our learnable compression requires the LLMs to aggregate the information into a single token, which **supervises the LLMs to select and emphasize the informative regions into the final token**, otherwise, it cannot answer the questions correctly. We suggest that such a data-driven way is more adaptive than previous works' heuristics of localizing informative regions.\n\nAs a side note, our paper emphasize **one token per frame** because of its landmark significance for token compression research in long videos and simplicity for implementation. We hope our work can inpire more adaptive and efficient token compression methods in future works.\n\n---\n### 3. Hand-crafted Heuristics in Token Selection\n\n\n> The token selection relies on hand-crafted heuristics (e.g., activation magnitude, channel max), which may not always correlate with semantic importance—especially in domain-shifted or adversarial settings.\n\nThank you for this insightful question! The mismatch between heuristically selected tokens and desired information is a long-standing challenge for token compression in long videos, especially as **previous token compression entirely rely on heuristics**. Similar to the section above, we discuss from intra-frame and inter-frame token compression about our improvement in decreasing heuristics and directions for future work.\n\n* From an intra-frame level, our LP-Comp (learnable and progressive compression) is exactly proposed for LLMs to **learn the token compression via end-to-end optimization without relying on manually defined heuristics**.\n* From an inter-frame level, we acknowledge that our QC-Comp (question-conditioned frame compression) still relies on the heuristics of attention scores similar to prior works. However, the main novelty of QC-Comp is to enhance the efficiency of inter-frame token compression and *lost-in-the-middle* with our **segmented local attention**. \n* Connected with your insightful feedbacks, we envision an intriguing future work: extending our intra-frame learnable compression (LP-Comp) into the inter-frame compression QC-Comp to provide a heuristic-free token compression method. Despite its excitement, our submission did not attempt this as our LP-Comp was already a notable step towards heuristic-free compression, and extending the LLM to handle inter-frame compression would require significantly more computation resources for supervised fine-tuning. Therefore, we will gladly include these discussions into our future works.\n\n---\n\n### 4. Dense Output Tasks\n\n> Have the authors tried OTPF in such settings, or do they envision ways to extend it to support dense output tasks?\n\nThank you for this constructive question! We hope to answer the question comprehensively to you.\n\nFirst, while OTPF primarily targets multiple-choice tasks on long videos, it is **fully capable of handling dense output tasks** such as video captioning. The primary reason we focused on multiple-choice tasks in the paper is due to the current landscape of long-video benchmarks, which predominantly evaluate models using multiple-choice formats. This is consistent with prior work in the domain of long video understanding, where the lack of dense-output benchmarks for long videos has been a limiting factor.\n\nTo illustrate the capability of OTPF on dense-output settings, we conducted an additional experiment during the rebuttal phase on the VDC benchmark (proposed in the work Auroracap), specifically the detailed captioning subset, which requires dense textual descriptions. While this benchmark focuses on short video captioning, and is thus **beyond the scope of our main setting**, we believe it demonstrates that OTPF can generalize to dense generation tasks.\n\nAs shown in the table below, our OTPF achieves nearly identical BLEU scores to VideoChat-Flash-2B:\n\n\n|  | BLEU@1 | BLEU@4 |\n| -------- | -------- | -------- |\n|  VideoChat-Flash-2B   | 7.1     | 1.6    |\n| OTPF  | 7.0   | 1.6    |\n\n\nAs for the future direction, we believe the development of dense-output benchmarks tailored to long videos would be a valuable direction, and we appreciate your suggestion in motivating this line of research."}, {"review_id": "PrdCBythEx", "reviewer": "Reviewer_Ur6N", "summary": "This paper introduces a new video compression method that rely on associated each video frame to a single token (using LLM layers) and then choose the most relevant tokens associated to a given question. The methods reduces the numbers of tokens progressively by adding a stack of LLM layers that reduce each time a bit more the number of visual tokens until reaching a single token, then an attention mechanism coined as Segmented Local Attention select the most relevant tokens to answer a question. The X-Comp model is based on fine-tuning VideoChat-Flash-2B model. The authors run their experiments on LongVideoBench, MLVU, VideoMME (Long) and LVBench. They show that their fine-tuned method improve performances on those benchmarks with respect to the base VideoChat-Flash model. They also argue that their methods have less attention biases (focusing on either the beginning or the end of a token sequence) since it can discards easily the frames that are not relevants.", "questions": "Why didn't you performed the ablation of Table 4 and 5 on LVBench?", "limitations": "Yes, the limitations are addressed.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The paper is well written and easy to read. The figures are great to better understand the method. The motivation is also well defined and the improvements over the base VideoChat-Flash-2B model seems good on long-video understanding. From the experiments, it's clear that longer a video is, more gain in performances we can expect with this method. The ablation study is also an important contribution to this paper by exploring the gain that are added by each of the component of the methods (such as which attention strategy or compression method is used).\n\nThe main weakness of this work which is highlighted by the authors in the limitation part is the reliance on only VideoChat-Flash which is not sufficient to see wether this method can also work with other models or not. Also the ablation studies are not consistently performed across the same benchmarks so it's harder to parse the importance of each of the components.", "was_revised": true, "final_justification": "The authors answered all the concerns I had. I also read all the other reviews and authors responses and I am satisfied with the discussion and the additional results presented. Overall, this is a good paper that will be of interest to the community, hence I am recommending acceptance.", "rebuttal": "Thank you for your thoughtful and constructive review. We are glad that you found our paper well written, appreciated the clarity of our figures and motivation, and recognized the effectiveness of our method on long-video understanding tasks. We especially appreciate your detailed comments on the strengths of our approach and your insightful suggestions regarding generalization to other models and consistency in ablation studies. We address these points in detail below and have taken concrete steps to further strengthen our work accordingly.\n\n### 1. Experiment on Other Models\n\n> The main weakness of this work which is highlighted by the authors in the limitation part is the reliance on only VideoChat-Flash which is not sufficient to see wether this method can also work with other models or not.\n\nThank you for this question and caring about the generalization of our method! We hope to clarify this comprehensively for you.\n\nFirst, our method is by-design applicable to any transformer-based video-language model and is not dependent on any designs from VideoChatFlash. Specifically, our major innovation is (1) LP-Comp supervises LLM layers to conduct learnable and progress token compression, and (2) QC-Comp splits the video into individual and parallel segments for efficient token selection. \n\nSecond, we select VideoChatFlash as it was the state-of-the-art on long video understanding at the time of submission, which curates an effective mixture of training data. Moreover, VideoChatFlash is unique in utilizing heuristic token compression methods for long video understanding. Thus, it is both a proper baseline and strong counterpart for us to validate our enhanced token compression algorithm.\n\nThrid, we acknowledge that including more baselines would better demonstrate the strengths of our method. Unfortunately, our computational resources are highly limited: we could only afford to train on 2.5\\% of VideoChatFlash's data, yet achieved the improvement on long videos (Table 1).\n\nDespite this, we understand the good intentions of the reviewer and is actively implementing and running experiments on the alternative models, such as LLaVA-Next-Video, and will include the results in the revisions.\n\n---\n\n### 2. Ablation Studies\n\n> Also the ablation studies are not consistently performed across the same benchmarks so it's harder to parse the importance of each of the components.\n> Why didn't you performed the ablation of Table 4 and 5 on LVBench?\n\nWe did not have the sufficient computation to comprehensively conduct the ablations on LVBench, as it is the longest video benchmark. According to your request, we have consistently completed the ablation studies on LongVideoBench and LVBench during the rebuttal. We summarize the results below (\"Y\" denotes \"yes,\" and \"N\" denotes \"No\") and conclude that **all of our LP-Comp, QC-Comp, and their accompanied design choices are effective consistently**.\n\n| Model Name | LP-Comp | LP-Comp Variants | QC-Comp | QC-Comp Variants | LongVideoBench | LVBench |\n|------------|---------|----------------|---------|-----------------|----------------|---------|\n| VideoChat-Flash | N | - | N | - | 58.3 | 42.9 |\n| XComp | Y | Uniform | N | - | 58.2 | 43.6 |\n| XComp | Y | Suffix | N | - | 58.8 | 44.3 |\n| XComp | Y | Suffix | Y | Global | 59.5 | 45.6 |\n| XComp | Y | Suffix | Y | Local | **59.7** | **46.2** |"}]

Author rebuttal:

Meta-review: This paper presents a hierarchical token compression framework for long video understanding to address the token inefficiency issue in video-language models. Reviewers acknowledged the contribution and strong performance of the proposed method, while initially raising concerns regarding its generalization ability and missing important experiments.

After the rebuttal, the authors addressed most of the concerns, and all reviewers agreed to accept this paper. AC read all the reviews, author rebuttals, and the paper, and believes this is a strong paper and recommends acceptance.
Decision: Accept (poster)