FlashVID: Efficient Video Large Language Models via Training-free Tree-based Spatiotemporal Token Merging
Authors: Ziyang Fan
Keyu Chen
Ruilong Xing
Yulin Li
Li Jiang
Zhuotao Tian
Venue: ICLR 2026 Oral (2026)
Keywords: Efficient Large Multimodal Models
Video Large Language Models
Visual Token Compression
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We introduce FlashVID, a training-free and plug-and-play inference acceleration framework for Video LLMs, enabling a satisfactory speedup with negligible performance degradation.

Abstract: Although Video Large Language Models (VLLMs) have shown remarkable capabilities in video understanding, they are required to process high volumes of visual tokens, causing significant computational inefficiency. Existing VLLMs acceleration frameworks usually compress spatial and temporal redundancy independently, which overlooks the spatiotemporal relationships, thereby leading to suboptimal spatiotemporal compression. The highly correlated visual features are likely to change in spatial position, scale, orientation, and other attributes over time due to the dynamic nature of video. Building on this insight, we introduce FlashVID, a training-free inference acceleration framework for VLLMs. Specifically, FlashVID utilizes Attention and Diversity-based Token Selection (ADTS) to select the most representative tokens for basic video representation, then applies Tree-based Spatiotemporal Token Merging (TSTM) for fine-grained spatiotemporal redundancy elimination. Extensive experiments conducted on three representative VLLMs across five video understanding benchmarks demonstrate the effectiveness and generalization of our method. Notably, by retaining only $\textbf{10}$% of visual tokens, FlashVID preserves $\textbf{99.1}$% of the performance of LLaVA-OneVision. Consequently, FlashVID can serve as a training-free and plug-and-play module for extending long video frames, which enables a $\textbf{10$\times$}$ increase in video frame input to Qwen2.5-VL, resulting in a relative improvement of $\textbf{8.6}$% within the same computational budget. Code is available at https://github.com/Fanziyang-v/FlashVID.

Published as a conference paper at ICLR 2026
FLASHVID: EFFICIENT VIDEO LARGE LANGUAGE MODELS VIA TRAINING-FREE TREE-BASED SPA- TIOTEMPORAL TOKEN MERGING
Ziyang Fan'
Keyu Chen' Ruilong Xing'
Yulin Li
Jiang Zhuotao Tian
Harbin Institute of Technology, Shenzhen
Thhe Chinese University of Hong Kong, Shenzhen
Shenzhen Loop Area Institute
ABSTRACT
Although Video Large Language Models (VLLMs) have shown remarkable ca- pabilities in video understanding, they are required to process high volumes of visual tokens, causing significant computational inefficiency. Existing VLLMs acceleration frameworks usually compress spatial and temporal redundancy inde- pendently, which overlooks the spatiotemporal relationships, thereby leading to suboptimal spatiotemporal compression. The highly correlated visual features are likely to change in spatial position, scale, orientation, and other attributes over time due the dynamic nature of video. Building on this insight, we introduce FIashVID, a training-free inference acceleration framework for VLLMs. Specif- ically, Flash VID utilizes Attention and Diversity-based Token Selection (ADTS) to select the most representative tokens for basic video representation, then applies Tree-based Spatiotemporal Token Merging (TSTM) for fine- rree-grrindd spatiotempo- ral redundancy elimination. Extensive experiments conducted on three represen- tative VLLMs across five video understanding benchmarks demonstrate the effec- tiveness and generalization of our method. Notably, by retaining only 10% of vi- sual tokens, Flash VID preserves 99.1 % of the performance of LLaVA-One Vision. Consequently, Flash extending long video frames, which enables a 10x x increase in video frame input to Qwen2.5-VIL resulting in a relative improvement 8.6 % within the same com- putational budget. Code is available at https://github.com/ cow/Fanziyang- aaaag//llllsh VID.
serve training- frree and plug-and-play module for
INTRODUCTION
Recent advances in Video Large Language Models (VLLMs) (Li al. 2025a Zhang et al. 2024 Bai et al. 2025b Comanici al. 2025) have demonstrated promising capabilities in video under- standing tasks. However, processing large numbers of visual tokens incurs substantial computational and memory overhead, the attention mechanism scales quadratically with sequence length, lim- iting practical deployment. To address this challenge, visual token compression (Chen et al.) 2024] Yang et al.) 2025c; Zhang al., (202e)) has emerged as a promising approach, leveraging the inher- ent redundancy in visual inputs to reduce sequence length by removing or merging less informative tokens, thereby enabling efficient inference without significant performance degradation.
While advances have been achieved in visual token compression for images (Bolya et al., 2023 Chen et al., 2024 Yang et al. 2025c; Zhang et al., (025e),, extending these methods to video re- mains largely underexplored. Videos inherently exhibit both spatial redundancy within frames and temporal redundancy across frames, rendering frame-win wise compression strategies suboptimal due to their neglect of temporal dynamics and correlations. This gap highlights the need for compression techniques specifically designed for the spatiotemporal structure of video inputs in VLLMs.
Motivation. Recent VLLM acceleration methods (Huang et al. 2025| Shen et al. 2025 Shao
et al. 2025a) typically adopt a three-stage pipeline: (1) video partition, grouping consecutive frames
*Corresponding author: Zhuotao Tian (tianzhuotao@ hit edu.end)

---

Published as a conference paper at ICLR 2026
57.8
Will
65.5
60.0
FisIVID
Visiondap
FauV
LLAVA-One Vismn
LeghWedeey 57.1
Molloo $6.5
QuienZ.s. 2.5... VI.
(a) Temporal Token Merging (TTM)
55.9
57.4
Vasentp
Debative Accuracy (%9
Fadtii
Viniila
1057
105.1
57.3 vileebo
LLaVA- Video
60.9 Moo
1953
lol.t
1649
102.5
64
80
160
Tarichen
dFrames
(b) Process video frames
(c) FlashVID achieves SOTA performance
Figure 1: Performance of FlashVID. (a) TTM may merge less correlated visual tokens, failing to capture fine-grained video dynamics. (b) FlashVID can enable Qwen2.5-VL. to process 10x video frames, significantly improving the relative performance by 8.6% while maintaining overall com- putational budget. (c) FIDshVID significantly outperforms current SOTA acceleration frameworks (e.g., FastV, VisionZip, FastVID) three representative VL.LMs.
with similar semantics to avoid information mixing; (2) frame-wise token selection, identifying in- formative tokens often guided by [CLS] attention-for basic video representation; and (3) spa- tiotemporal compression, further reducing redundancy the segment level.
However, previous methods typically compress temporal and spatial redundancy independently. Such decoupled strategies overlook the intrinsic spatiotemporal relationships in videos. Moreover, temporal redundancy is commonly defined the consistency of visual features at fixed spatial loca- tions across consecutive frames. Due to the dynamic nature video, the most semantically similar visual elements are likely to experience changes in spatial position, scale, orientation, and other attributes over time. Consequently, the most correlated visual features in adjacent frames may not reside the same spatial location. As depicted in Fig. Ta the Temporal Token Merging (TTM) strategy fails to capture video dynamics, erroncously merging less correlated tokens and distorting the video representations. Relying on such rigid spatial correspondence for temporal redundancy compression may introduce noise, further misleading the model. So, a natural question arises: "How can we achieve a decent spatiotemporal compression by jointly modeling spatial and temporal re- dundancy, while accounting for the dynamic characteristics of video?"
To address this challenge, introduce FlashVID, novel training-free accel- eration framework for VLLMs that effectively reduces spatiotemporal redundancy while preserv- ing critical visual content. Specifically, at the core of FIashVID is the Tree-based Spatiotemporal Token Merging (TSTM) mechanism, which explicitly models both spatial and temporal redundancy through hierarchical spatiotemporal redundancy trees. TSTM enables structured token merging across frames and within frames, allowing for joint spatiotemporal compression that respects the natural structure of video data. However, directly constructing spatiotemporal trees based on the raw video features with excessive noise and redundancy may not focus on the most representative visual information each frame, or even be biased towards the major but unimportant visual in- formation, thereby affecting the final performance. To alleviate this issue, we further introduce the Attention and iiversity-baser Token Selection (ADTS) module, which prioritizes representative tokens in each frame. To this end, through the initial filtering of informative tokens using ADTS and subsequent merging via TSTM, Flash VID accomplishes efficient compression that adjusts to the dynamic attributes of video content while preserving erucial semantics.
Our Solution.
2

---

Published as a conference paper at ICLR 2026
Visual Tokens
Tokens
Question
LLM Layer (K+1)
Large Language Model
Visual Token Pruning
Compression
I.M.M Layer
Vision Encoder
Tokenizer
Vision Encoder
Tokenizer
activities engage in within room?
What activibies do students engage within
(a) Before-LIM Compression
(b) Inner-LLM Pruning
Figure 2: Efficient inference paradigms. State-of-ther thetthherrtttt acceleration frameworks can be mainly divided into three categories: 1) Before-LIMM Compression; 2) Inner-LLLM Pruning: and 3) Hybrid Compression, where the hybrid compression can be viewed as a trade-off of the Before-LLM Com- pression and Inner-LLM Pruning strategy.
Extensive experiments have been conducted on five video understanding benchmarks (Fu et al,, 2025a Mangalam al. 2023 Wu al. 2024 Li al. 2024a Zhou al.] 2025) and three repre- sentative VLLMs, ig.. LLaVA OrA-One (Li et al. (025a). LLa Video (Zhang et al. 2024). and
Qwen2.5-VL VL (Bai et al. 2025b). As shown Fig. lb and Fig. Ic FlashVID outperforms previous
state-of-the-aa methods by large margin across all settings. Notably, Flash achieves 99.1 % relative accuracy to vanilla LLa VA-One
while pruning 90 visual tokens. When integrated into Qwen2.5-VI. Flash VID enables processing of up to 10x more video frames, yielding an 8.6% performance gain over the vanilla model with 16 sampled frames under the same computational budget, highlighting its ability to unlock longer temporal context for better video understanding.
To summarize, our main contributions are threefold:
In this work, we identify that existing token compression methods fail to effectively model the dynamic and evolving nature of video content, leading to suboptimal performance.
We propose FlashVID, training-free VI.LMs acceleration method that introduces the Tree-based Spatiotemporal Token Merging (TSTM) to jointly model spatial and temporal redundancy across frames, complemented by Attention and Diversity-based Token Selec- tion (ADTS) to obtain the semantically representative content within each frame.
Extensive experiments show that Flash VID improves inference efficiency with negligible performance drop, and enables the use of longer input sequences for better video under- standing within the constrained computational budget.
2 BACKGROUND AND MOTIVATION
In this section, we provide a brief overview of the underlying concepts in this study in Sec. 2.1 and highlight the key observations in Sec. 2.2 which offer valuable insights for our approach.
2.1 PRELIMINARIES
VLLMs inference pipeline. The inference of VLLMs consists of three stages: (1) Encoding. vision encoder (e.g., CLIP (Radford al., 2021 and SigL.P Zhhii et al.) 2023 processes each frame independently, producing N, visual embeddings per frame, which are projected into text space via a modality connector form H, E FxN. x.x. Text queries are embedded as H,, E IR N. (2) Prefilling, Each LLM layer computes self-attention over H via:
Q' H''' H'' V' H''
with W Wt K. W E Rddd The key-value pairs are stored in the KV Cache for decoding ac- celeration. (3) Decoding, Response tokens are generated auto-regressively. At step i, only the
3

---

Published as a conference paper at ICLR 2026
#Merged Tokens in TSTM
#Merged Tokens TTM
Averge Merged Similarities Difference
Temporal Token Merging (TTM)
(b)
Frame
(a) Number merged tokens per frame with TSTM (orange) TTM (blue) under the same threshold, with average merging similarity differences be- (c) tween TSTM and TTM shown in green.
Tree-based Spatiotemporal
Token Merging (TSTM)
Figure 3: Comparison of spatiotemporal redundancy compression. (a) TSTM merges more to- kens than TTM under the same threshold and achieves higher inter-frame merging similarity by flexibly capturing fine- fine-graine grained video dynamics. (b) TTM enforces rigid spatial correspondences, of- ten overlooking dynamic variations in videos and merging less correlated visual tokens. (c) TSTM models video redundancy via spatiotemporal redundancy trees, capturing fine-grained spatiotempo- ral relationships. More visualizations are provided in Appendix E
token h is projected to (K,, Ve), which update the cache:
K +- concat K.), V concatt V.J.
Such caching mechanism substantially improves decoding efficiency.
Efficiency bottleneck analysis. While VLLMs have achieved remarkable performance on video understanding tasks, their efficiency remains key challenge due the heavy computational and memory overhead when processing a large number of visual tokens. Most of this cost stems from the L.M backbone, where self-attention mechanism and the computational complexity. Given a model with I. Transformer layers, the total Floating Point Operations (FLOPs) can be formulated as:
Networks (FFNs) dominate
FLOPs L (4nd² 2ndd 2ndm),
with n denoting the sequence length, d the hidden dimension, and m the intermediate dimension of FFNs. In video understanding, the number of visual tokens n, dominates the sequence length typically exceeding textual tokens n, by orders of magnitude. This imbalance underscores the necessity to compress visual tokens for efficient inference in VLLMs.
Efficient inference paradigms. As illustrated in Fig. visual token compression frameworks be grouped into three paradigms: Before-LI.M, Inner-LI.M, and Hybrid Compression. Compressing tokens only inside the LIM is inefficient, as all visual tokens must still be processed in the shallow layers; thus, reducing tokens before the LLM is critical for reducing overhead. Existing methods Yang 2025c Shen et al., 2025) adopt single-stage compression before the LLM, but extreme compression risks losing important visual information. Hybrid compression provides a balance: retains sufficient tokens as LLM input while further pruning within the LLM to meet computational budget. Training- based approaches (Zhang et al.) 2025d Cai et al.) 2025) Hu et al. [0024] Shao et al., 2025b) can mitigate this inefficiency demand substantial computing resources; this work, we focus on training- free strategies. A more comprehensive review is provided in Appendix D
2.2 KEY OBSERVATIONS
We summarize two key observations about spatiotemporal redundancy in videos: (1) Temporal re- dundancy is not bound to fixed spatial locations. Semantically consistent elements in videos often
4

---

Published as a conference paper at ICLR 2026
Attention and Diversiry-based Token Selection (ADTS)
Tree-hased Spariotemporal Token Merging (TSTM)
Representative Visual Tokens
Text Tokens
Compressed Visual Tokens
Question
Select
Aguregatc
Large Language Model
Callbrated Distence Matriss
Spulinenpuul Rolaniancy Tha
Compression
Calibrate
Construct
Vision Encoder
Tokenizer
Intra-Prame Distance
Adjacent Frame Simi laritaritis
What activitics do students within the room?
Figure 4: Overview of our FlashVID. FID compresses visual tokens by two synergistic mod- ules: (1) ADTS prioritizes spatiotemporally informative tokens while ensuring feature diversity by solving calibrated Max-Min Diversity Problem (MMDP); (2) TSTM models redundancy by spa- tiotemporal redundancy trees, which effectively capture fine-grained video dynamics.
shift in spatial position, scale, or appearance due to motion and scene dynamics, making rigid spatial correspondence across frames unreliable (Huang et al. (2025). (2) Spatial and temporal redundancy are inherently coupled. Redundant regions within single frame frequently persist across multiple frames. Decoupled spatiotemporal redundancy compression overlooks the intrinsic spatiotemporal relationships, leading to suboptimal compression.
These insights suggest that existing frameworks lack unified, structure-aware mechanism to cap- ture spatiotemporal relationships under dynamic video conditions, which motivates our hierarchi- cal tree-based spatiotemporal redundancy compression. A conceptual comparison with prior ap- proaches is illustrated Fig. highlighting the unique advantages of our design.
METHODOLOGY
3.1 OVERVIEW
As illustrated in Fig. FlashVID integrates two synergistic modules: 1) Attention and Diversity- based Token Selection (ADTS), which first selects informative and diverse tokens for robust video representations: 2) free-based Spatiotemporal Token Merging (TSTM), which further minimizes spatiotemporal redundancy while preserving critical visual information.
3.2 TREE-BASED SPATIOTEMPORAL TOKEN MERGING
Videos exhibit dynamic variations in spatial position, scale, and appearance, posing challenges for spatiotemporal redundancy compression. To alleviate this, we propose Tree-based Spatiotemporal Token Merging (TSTM), which models the video redundancy via spatiotemporal redundancy trees.
Construct spatiotemporal redundancy trees. Given video features E, FxN.xd TSTM progressively builds spatiotemporal redundancy trees, First, we compute the cosine similarity matrix between visual features adjacent frames:
s(f) = cos( , ..EEE(+))) E N.X..
(4)
where ((f) k) measures the feature similarity between j-th token in frame f and k-th token in frame (f+1). Each token links to its most similar counterpart in the previous frame if their similarity exceeds a merging threshold T,. This gradually forms redundancy trees that capture fine-grained temporal variations while avoiding merging dissimilar tokens.
Compress spatiotemporal redundancy. within each tree aggregated:
Once the redundancy trees are constructed, tokens
(+) Agg(T())
(5)
5

---

Published as a conference paper at ICLR 2026
Algorithm FlashVID Compression
Require: Video features Evp FRFN. xd; similarity function sim(;,); merging threshold T,
Ensure: Compressed token set &
1: Stage 1: Attention and Diversity-hased Token Selection (ADTS)
2: for f f= 1 to F do
3: Compute pairwise distance D(O, [CLS] attention A ACCLS) event relevance $
4: I() MMDP(DD), ( ACLss )
5: R( E. I())
6: end for
7: Stage Tree-based Spatiotemporal Token Merging (TSTM)
8: Initialize each token in R, as root node and let C. be empty token set.
9: for f = F do
D Build spatiotemporal redundancy trees
10: for each token rf ((() do
11:
p' + arg PUU sysim(r!,.)) sim(r .p') T, then
13: end if
Connect rf to p"
15: end for
end for
17: for each tree T do
Aggregate redundancy trees
18: + (7)
19: end for
20: return X + II)
where ( denotes the i-th spatiotemporal redundancy tree and Agg(-) represents an aggregation function (e.g., mean pooling), producing compact yet informative spatiotemporal representations.
The quality of redundancy trees is critical for fine-grained compression. Although we' ve explored constraining tree depth and breadth to prevent merging spatiotemporally distant tokens, it yielded negligible gains; thus, no such constraints are applied in practice (see Appendix A.. for details.)
3.3 ATTENTION AND DIVERSITY-BASER TOKEN SELECTION
Although TSTM effectively compresses spatiotemporal redundancy, it may discard important tokens in noisy and high-volume inputs. To mitigate this, we introduce the Attention and Diversity-base -based Token Selection (ADTS) module, which prioritizes spatiotemporally informative tokens within each
frame while ensuring feature diversity for robust video representations. ADTS formulates token selection as frame-wise wise Max-Min Diversity Problem (MMDP) (Alvar al.) (202).. Given video features F, E FxN. we first compute the frame-wise cosine distance matrix:
D( = - cos (F( F(F)),
(6)
where D() N.N.N. denotes the pairwise feature dissimilarities in frame f. Solving MMDP on D() yields diverse token subset in frame f with the maximal minimum distance. However, diversity alone may overlook the most informative visual tokens. To address this issue, we introduce two calibration terms: 1) [CLS] attention and 2) event relevance.
[CLS] attention calibration. We extract the attention matrices from the vision encoder. For those encoders without explicit [CLS] token (e.g., SigLIP (Zhai et al. 2023 ) derive it from the attention matrix:
A Softmax(QK"T /V)) e FxN,NN,
and compute AçC.s E R FNN, by averaging attention weights each token receives within its frame. This calibration highlights informative tokens in each frame.

---

Published as a conference paper at ICLR 2026
Event relevance calibration. Event relevance measures token's correlation with the current video context. We obtain frame embeddings fo = GAP(E.) E RFxd by global average pool- ing and compute the event similarity matrices:
F EE
S.
E So T M.ad ERFxN.
(E
(8)
This calibration emphasizes the tokens most relevant the video event. Finally, spatiotemporally informative tokens are selected by solving:
I = MMDP(D, AtcLs,,.))
As summarized in Alg. FlashVID compresses video redundancy in two stages: ADTS first selects spatiotemporally informative tokens by solving calibrated Max-Min Diversity Problem (see Ap- pendix CC2 for details), then TSTM merges redundant tokens across frames through spatiotemporal redundancy trees, yielding compact yet informative visual features.
EXPERIMENTS
In this section, we conduct extensive experiments across multiple benchmarks and VLLMs. We provide a brief introduction to the experimental settings in Sec. 4.1 present the main experimental results in Sec. 4.2 and discuss essential ablation studies Sec.
4.1 EXPERIMENTAL SETTINGS
Benchmarks. We evaluate our method five widely-used video understanding benchmarks: VideoMME (Fu et al. 2025a), EgoSchema (Mangalam et al. (023), LongVideoBent (Wu et al.) 2024). MVBench (Li al. 2024a), and MLVU (Zhou et al. 2025). Notably, these benchmarks cover a wide range of video durations and complex scenarios, providing comprehensive evaluation of our method's 's effectiveness and generalization. Additional details can be found in Appendix
Compared baselines. We compare Flash VID with four state-of-the-an training free VLLM celeration methods: 1) FastV (Chen et al. 2024), which selects prompt-relevant tokens via text- to-visual attention at the prefilling stage; 2) VisionZip (Yang et al., (025c), pruning tokens using [CLS] attention and spatial merging before the LLM; 3) Prune VID (Huang et al. (2025). combining spatiotemporal token merging with attention-basee selection in the LLM; and VID (Shen
et al. 2025). compressing redundant tokens via density-based spatiotemporal pruning.
Implementation details. We evaluate our method on three representative VLLMs: LLaVA- Vieion (Li et al., 2025a), I.LVA-Video (Zhang al., (024), and Qwen2.5-VL. (Bai et al., 20256), which cover diverse architectures to ensure generality. Following the official setting, LLaVA-One VivionnnVision and LLaVA-Vides uniformly sample 32 and 64 frames, producing 32 x 196 and 64 x 169 visual tokens, respectively. For LLaVA-Video, we adopt frame token setting, facili- tating adaptation for different acceleration frameworks. To ensure fair comparison, we align the average token budget per transformer layer. Since TSTM compresses redundancy via thresholding, we further apply frame-wise token compression based on DPC-KNN to meet the predefined token budget. Unless otherwise specified, we utilize the same set of hyperparameters for all experiments. All the experiments are conducted on NVIDIA A800 80G GPUs using LMMs--val (Zhang et al., 2025b). Additional implementation details are provided in Appendix C
4.2
MAIN RESULTS
We evaluate FlashVID against -late-of-the-at baselines on three representative VLLMs with distinct architectures under various retention ratios R. Additional experimental results on Qwen2.5-VL. and LLa Video are reported in Appendix. A
Results on LLaVA-OneVisionn. Tab. compares FlashVID with other methods on LLaVA- One VisionZip performs competitively at higher retention (i.e., 25%, 20%) but suffers sharp
7

---

Published as a conference paper at ICLR 2026
Table 1: Comparison of state-of-the-art methods on LLaVA-OneVision and LLa LLaVA-Video.
Our FlashVID consistently outperforms previous sfate-of-the- methods by a large margin under different retention ratios across multiple benchmarks and VLLMs. Notably, FlashVID surpasses vanilla LLaVA-One Vision with full visual tokens input when R E (15%. 20%, 25%}.
Retention Ratio R
ViE
EgoSchema LongVider Video Bench
Avg.
Method
MVBench
Short Medium Long Overall Subset
Total
Score Rel. Acc (%)
LLaVA- Vision
Vanilla
100%
69.9
56.7
48.9
58.5
62.2
60.3
56.6
58.3
58.4
100.0
FastV
68.1
54.7
46.8
56.5
60.4
55.4
56.4
56.5
96.7
VisionZip
68.8
57.3
48.2
58.1
63.0
60.4
56.4
57.8
58.2
99.7
PID
25%
67.3
54.8
47.2
56.4
61.0
58.1
55.4
56.8
56.7
97.1
FastVID
69.9
56.3
47.4
57.9
61.2
59.5
55.9
58.1
57.8
99.0
FIDshVID
71.2
57.0
49.3
59.2
63.4
60.4
56.8
58.0
58.6
100.3
FastV
66.3
53.9
46.9
55.7
60.6
57.6
56.0
56.0
56.3
96.4
VisionZip
68.6
57.0
48.3
58.0
62.0
60.0
55.4
57.6
$7.7
98.8
PruneVID
20%
67.2
53.9
48.2
56.4
55.2
56.2
57.0
97.6
FastVID
69.9
56.3
47.4
57.9
61.2
59.5
55.9
58.1
$7.9
99.1
FIDshVID
70.1
55.4
48.9
58.5
58.7
100.5
FastV
64.6
54.0
45.3
54.6
59.8
56.6
54.8
55.0
55.2
94.5
VisionZip
63.8
54.6
48.3
55.6
62.8
60.0
54.1
53.5
55.8
95.5
15%
FastVID
69.7
55.8
47.7
57.7
58.8
58.9
56.7
58.2
57.9
99.1
Prune VID
67.2
52.8
46.7
56.1
61.6
57.7
54.5
55.1
55.7
95.4
FlashVID
69.6
56.0
48.9
58.2
62.8
60.4
57.5
57.9
58.5
100.2
FastV
60.9
52.2
44.9
52.7
59.0
56.0
52.4
53.4
53.6
91.8
VisionZip
60.3
52.9
46.7
53.3
61.6
58.5
49.4
54.8
$4.0
92.5
VID
10%
65.9
52.8
45.6
54.7
60.0
57.2
54.0
53.7
54.9
94.0
FastVID
68.1
55.7
47.8
55.7
57.1
97.8
FIDshVID
67.3
57.1
49.0
57.8
62.4
60.0
56.5
57.4
57.9
99.1
LLLVA-Vidd Video
Vanilla
100%
77.0
62.1
53.3
64.2
59.4
57.3
59.5
61.9
60.7
100.0
FastV
69.3
58.3
49.9
59.2
54.8
54.1
56.0
58.4
56.9
93.7
VisionZip
72.3
59.6
53.3
58.0
59.8
$9.0
97.2
20%
FastVID
74.6
60.8
52.3
62.6
57.0
55.0
57.1
60.2
58.7
96.7
FIDshVID
74.1
60.0
52.3
58.7
59.3
97.7
FastV
64.3
53.8
49.2
55.8
50.6
51.1
53.6
56.2
$4.2
89.3
VisionZip
69.4
57.9
51.2
59.5
54.4
53.9
54.5
58.5
56.6
93.2
10%
FastVID
71.8
57.3
50.2
59.8
54.8
52.4
56.9
59.3
57.1
94.1
FIDshVID
72.2
59.1
51.2
60.9
57.2
54.9
57.7
59.3
58.2
95.9
degradation at 15%, 10% due to excessive loss from aggressive spatial compression. FastV shows the weakest performance, as early-layer pruning unstable. In contrast, FlashVID achieves the best results across all retention ratios, preserving 99.1% % of the vanilla model's accuracy even at (25%, 20%, 15%]. FlashVII surpasses the vanilla LLaVA-
R = 10%. Moreover, when Vision with full visual tokens input, revealing "less is more" pattern where excessively redun- dant tokens may degrade performance.
LLaVA- Video employs a specialized design by inserting newline to- kens inject spatiotemporal positional information. Unlike the official grid token setting, apply the frame token in LLaVA -Viido,, which facilitates adaptation for different acceleration frameworks, where found that these two settings lead similar performance. Tab. [I, we evaluate our method against other methods on LLaVA-Vides Notably, our FIashVID outperforms all baselines under various retention ratios.
Results on
Results on Qwen2.5-VL In addition to LLaVA-OneVision and LLaVA- LLVeooVideo, we also evaluate Flash VID against other methods Qwen2.5-VI. which shows significantly different archi-
8

---

Published as a conference paper at ICLR 2026
Table Comparison of state-of-the-art methods on Qwen2.5-VL. The best performance among those with similar retention ratios R is highlighted in bold.
Retention Ratio R
VideoMME
EgoSchema
Long Bench
Avg.
Method
MVBench
Short
Medium Long Overall Subset Total
Score
Rel. Acc (%)
Vanilla
100%
72.6
61.4
49.9
61.3
60.2
58.3
58.9
68.0
61.6
100.0
FastV
69.4
57.0
51.2
59.2
60,2
57.1
54.2
66.8
59.3
96.3
VisionZip
69.6
57.2
50.2
59.0
58.6
56.6
56.3
66.4
59.6
96.8
20%
FastVID
69.9
56.3
49.7
58.6
57.2
56.4
57.8
64.7
59.4
96.4
FID
70.4
58.6
49.7
59.6
59.2
56.8
58.1
66.5
60.2
97.7
FastV
63.7
54.7
49.3
55.9
58.6
54.9
51.1
63.6
57.3
91.6
VisionZip
67.0
54.7
47.6
56.4
55.8
55.5
54.5
64.3
57.7
93.7
10%
FastVID
66.3
53.6
49.0
56.3
55.6
55.4
57.4
93.2
FIashVID
68.1
54.7
49.0
57.3
57.4
55.9
57.1
65.5
58.9
95.6
Table Comparison state-of-the-art methods on Qwen2.5-VL. under fixed token budget.
Our Flash enables Qwen2.5-VI. processing 10x video frames, improving the overall perfor- mance of 8.6% within the same computational memory budget.
Retention Ratio R
MME
EgoSchema
LongVideo Bench
Avg.
Method
#Frames
MLVU
Short Medium
Long
Overall
Subset
Total
Score Rel. Acc (%)
Vanilla
16 (Ix)
100%
66.4
56.4
48.2
57.0
58.2
55.6
56.9
40.6
52.6
100.0
VisionZip
74,2
60.0
52.1
62.1
60.0
58.2
57.4
43.1
55.2
104.9
FastVID
80 (5x)
20%
73.0
59.9
51.7
61.5
61.2
58.4
58.0
44.4
55.6
105.7
FlashVID
60.8
52.2
62.4
61.4
58.6
58.9
45.0
56.2
106.8
VisionZip
70.7
60.1
53.9
61.6
61.8
59.6
56.8
45.1
55.8
106.1
FastVID
160 (10x)
10%
71.2
60.6
53.8
61.9
61.2
59.1
58.0
43.8
55.7
105.9
FlashVID
71.4
62.2
53.7
62.4
59.5
58.9
47.5
57.1
108.6
tecture and characteristics. As illustrated in Tab. 2 our method significantly surpasses previous state-of-the-ars methods under various retention ratio,, demonstrating strong generalization across different VLLMs.
Results Qwen2.5-VL under fixed token budget. Due to computational and memory straints, existing VLLMS typically process only small number of sampled frames, often missing important visual cues. To assess the benefit of longer temporal context under fixed computa- tional budget, we apply token compression to enable models to process more frames. As shown in Tab. 3 Qwen2.5-VL. achieves consistent improvements over its vanilla 16-frame baseline when equipped with token compression frameworks. Among them, Flash VID delivers the largest perfor- mance gains, highlighting its ability unlock longer video sequences and demonstrating superior efficiency in constrained settings.
4.3 ABLATION STUDIES
In this section, we conduct ablation studies on the ADTS module and the retained ratio of ADTS and TSTM using LLaVA-One Visiin. Additional ablation studies are provided in Appendix. A.3
Ablation study on ADTS module. ADTS is proposed select both important and diverse to- kens. As shown in Tab. diversity-based token selection. Our ADTS outperforms other token selection methods based solely on [CLS] attention (ATS) and feature diversity (DTS) by large margin, demonstrating that ADTS can effectively identify the important visual tokens.
compare ADTS with ATS and DTS, i.c., attention-based and
To realize comprehensive ablation study ADTS, further ablate the calibration terms used ADTS. Tab, reveals that both [CLS] attention and event relevance calibration improve perfor- mance, while the optimal performance is yielded at the combination of the two.

---

Published as a conference paper at ICLR 2026
Table 4: Ablation study ADTS, ATS, DTS, and ADTS denote attention-, diversity-. and diiention-diversity--
Table 5: Ablation study on in visual token compression before LLM. a controls the re- tained ratio of ADTS and TSTM, where and a indicate TSTM and ADTS only.
token selection, respec- tively. 'C.A' and 'E.R.' denote [CLS] attention and event relevance calibration terms in ADTS.
LongVided Bench
VideoMME
EgaSchenn
MYBench
Acc.
Method
VideoMME EguSchema
LongVideo Bench
MVBench
Rel. Acc.
0.0/TSTM
56.7
60.3
55.3
55.6
97.4
0.2
56.2
59.8
55.3
56.5
97.4
ATS
55.5
59.5
55.0
56.2
96.9
0.4
56.4
60.0
55.1
57.2
97.9
97.1
60.4
55.8
57.0
98.5
57.3
98.5
0.8
57.2
60.1
56.3
57.1
98.8
ADTS
$7.8
60.0
56.5
57.4
99.1
L0ADTS
56.9
60.1
55.6
57.6
98.5
Table 6: Efficiency of our FIashVID, We conduct the efficiency analysis on LLaVA-One Vision, and report the prefilling time and Time-To-First-Ton (TTFT) in milliseconds (ms).
Retention Ratio R
Vision Encoding Compression LLM Forward
Prefilling Time
Avg.
Method
TFLOPs
TIFT
Total
Score Rel. Acc (%)
Vanilla
100%
113.4
785.0
1220.8
1220.8 (1.0x)
2005.8 (1.0)
58.9
100.0
FistVID
25%
22.4
785.0
28.6
273.2
301 8 (4.0k)
1086.8 (188)
58.0
98.5
MashVVID
10%6
8.6
785.0
60.2
133.1
193.3 (6.3%)
978.3 (2.1x)
58.4
99.1
Ablation study on a. As illustrated in Tab. 5 we conduct an ablation study on merging threshold o, which controls the ratio of visual tokens retained between ADTS and TSTM compression. In particular, = 0 and a = denote TSTM and ADTS only, respectively. The experimental results show that ADTS alone (a is achieved at = 0.7, implying that balanced integration of these two modules (i.e., ADTS and TSTM) is necessary to maintain the model performance.
outperforms TSTM alone (a 0). However, the peak performance
4.4 EFFICIENCY ANALYSIS
Although token compression can effectively improve the inference efficiency of VLLMs, it can also be time-consuming operation. In Tab. [ we conduct efficiency experiment on LLaVA- Vision using a single NVIDIA A100 GPU compared to FastVID on VideoMME. Remarkably, FID preserves 99.1% relative accuracy at R 10%, while FastVID achieves similar per- formance at R = 25%. Consequently, Flash VID enables 6.3x prefilling and 2.1 x Time- To-First- Token (TTFT) speedups, largely outperforming FastVID. Additional efficiency experimental results on LLaVA-Video can be found in Appendix. A.2
5 CONCLUSION
In this work, we introduce VID, training-free and plug-and-play acceleration framework for VLLMs. FlashVID combines Attention and Diversity-based Token Selection (ADTS) for repre- sentative token filtering with rree-based Spatiotemporal Token Merging (TSTM) for fine-grained redumdancy climination, effectively compressing spatiotemporal redundancy while preserving es- sential visual information. Extensive experiments three VL.LMs across five video understanding benchmarks demonstrate that FlashVID achieves superior performance in both efficiency and aceu- racy, In particular, serve plug-and and-play module, enabling VLLMs to process significantly longer video sequences under a constrained computational budget.
ACKNOWLEDGEMENT
This work was supported by the Guangdong Basic and Applied Basic Research Foun- dation (2025A1515011546) and by the Shenzhen Science and Technology Program
4 ZDCY2025090 0102500001110000000 13000001).
(CC120240881300509)) K7/0200409001122900003
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Joshua Ainslie, James Lee-Thorp. Michiel de Jong, Yury Zemlyanskiy, Federico Lebrón, and Sumit Sanghai. GQA: training generalized multi-query transformer models from multi-head check- points. In Proceedings of the Conference Empirical Methods in Natural Language Processing, 2023.
Saeed Ranjbar Alvar, Gursimran Singh, Mohammad Akbari, and Yong Zhang. Divprune: Diversity- based visual token pruning for large multimodal models. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, 2025.
Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang. Binyuan Hui, Shutong Jiang. Zhaohai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Junyang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Yang Liu, Dayiheng Liu, Shixuan Liu, Dunjie Lu, Ruilin Luo, Chenxu Lv, Rui Men, Lingchen Meng, Xuancheng Ren, Xingzhang Ren, Sibo Song. Yuchong Sun, Jun Tang, Jianhong Tu, Jianqiang Wan, Peng Wang. Pengfei Wang, Qi- uyue Wang, Yuxuan Wang, Tianbao Xie, Yiheng Xu, Haiyang Xu, Jin Xu, Zhibo Yang, Mingkun Yang. Jianxin Yang, An Yang, Bowen Yu, Fei Zhang, Hang Zhang, Xi Zhang, Bo Zheng, Hu- men Zhong, Jingren Zhou, Fan Zhou, Jing Zhou, Yuanzhi Zhu, and Ke Zhu. Qwen3-vl technical report. arXiv preprint 2rXiv:251..2163 2025a.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang. Shijie Wang, Jun Tang. Humen Zhong, Yuanzhi Zhu, Mingkun Yang. Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang. Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report. arXiv preprint ar Xiv: 1 13923, 2025b.
Daniel Bolya, Cheng- Yang Fu, Xiaoliang Dai, Peizhao Zhang, Christoph Feichtenhofer, and Judy Hoffman. Token merging: Your vit but faster. In Proceedings of the International Conference on Learning Representations, 2023.
Mu Cai, Jianwei Yang, Jianfeng Gao, and Yong Jae Lee. Matryoshka multimodal models, In Proceedings of the International Conference on Learning Representations, 2025.
Liang Chen, Haozhe Zhao, Tianyu Liu, Shuai Bai, Junyang Lin, Chang Zhou, and Baobao Chang. An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision- language models. In Proceedings of the European Conference on Computer Vision, 2024.
Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capa- bilitics. arXiv preprint arXiv:2507.06261, 2025.
Jiequan Cui, Shu Liu, Zhuotao Tian, Zhisheng Zhong, and Jiaya Jia. Reslt: Residual learning for long-tailed recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
Jiequan Cui, Zhisheng Zhong. Zhuotao Tian, Shu Liu, Bei Yu, and Jiaya Jia. Generalized parametric contrastive learning. IEEE Transactions Pattern Analysis and Machine Intelligence, 2023.
Wenliang Dai, Junnan Li, Dongxu Li, Anthony Tiong, Junqi Zhao Weisheng Wang, Boyang Li, Pascale N Fung, and Steven Hoi. Instructblip: Towards general-purpose vision-language models with instruction tuning. Advances in Neural Information Processing Systems, 2023.
Yann N Dauphin, Angela Fan, Michael Auli, and David Grangier. Language modeling with gated convolutional networks. In Proceedings of the International Conference on Machine Learning, 2017.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the North American Chapter of the Association for Computational Linguistics, 2019.
11

---

Published as a conference paper at ICLR 2026
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. Proceedings of the International Conference on Learning Representations, 2021.
Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Video-mme: The first-ever comprehensive evalua- tion benchmark of multi-modal IIms in video analysis. In Proceedings of the IEEF/CVF Confer- ence on Computer Vision and Pattern Recognition, 2025a.
Tianyu Fu, Tengxuan Liu, Qinghao Han, Guohao Dai. Shengen Yan, Huazhong Yang, Xuefei Ning, and Yu Wang. Framefusion: Combining similarity and importance for video token reduction large visual language models. Proceedings of the IEEE/CVF International Conference Computer Vision, 2025b.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21700 2024.
Daya Guo, Dejian Yang, Haowei Zhang. Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang. Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint arXiv:2501.12944, 2025.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2016.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked au- toencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, 2022.
Wenbo Hu, Zi-Yi Dou, Liunian Harold Li, Amita Kamath, Nanyun Peng, and Kai-Wei Chang. Ma- tryoshka query transformer for large vision-language models. In Advances Neural Information Processing Systems, 2024.
Xiaohu Huang, Hao Zhou, and Kai Han. Prunevid: Visual token pruning for efficient video large language models. In Findings of the Association for Computational Linguistics, 2025.
Jeongseok Hyun, Sukjun Hwang, Su Ho Han, Taeoh Kim, Inwoong Lee, Dongyoon Wee, Joon- Young Lee, Seon Joo Kim, and Minho Shim. Multi-granular spatio-temporal token merging for training-free free acceleration of video llms. Proceedings of the IEEE/CVF International Conference on Computer Vision, 2025.
Li Jiang, Shaoshuai Shi, Zhuotao Tian, Xin Lai, Shu Liu, Chi-Wing Fu, and Jiaya Jia. Guided point contrastive learning for semi-supervised point cloud semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021.
Xin Lai, Zhuotao Tian, Li Jiang, Shu Liu, Hengshuang Zhao, Liwei Wang, and Jiaya Jia. Semi- supervised semantic segmentation with directional context-aware consistency. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021.
Xin Lai, Zhuotao Tian, Yukang Chen, Yanwei Li, Yuhui Yuan, Shu Liu, and Jiaya Jia. Lisarrassonigg segmentation via large language model. In Proceedings of the IEEE/CVF Conference on Com- puler Vision and Patlern Recognition, 2024a.
Xin Lai, Zhuotao Tian, Yukang Chen, Senqiao Yang, Xiangru Peng, and Jiaya Jia. Step-dpo: Step- wise preference optimization for long-chain reasoning of llms. arXiv preprint arXiv:2406. 18629, 2024b.
Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. Llava-onevision: Easy visual task transfer. Transactions Machine Learning Research, 2025a.
12

---

Published as a conference paper at ICLR 2026
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-iman pre-training with frozen image encoders and large language models. In Proceedings of the Inter- national Conference on Machine Learning, 2023.
Kunchang Li, Yali Wang. Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, et al. Mybench: A comprehensive multi-modal video understanding benchmark. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, 2024a.
Yanwei Li, Chengyao Wang, and Jiaya Jia. Llama-vid: An image is worth 2 tokens in large language models. In Proceedings of the European Conference on Computer Vision, 2024b.
Yulin Li, Haokun Gui, Ziyang Fan, Junjie Wang, Bin Kang, Bin Chen, and Zhuotao Tian. Less is more, but where? dynamic token compression via llmm Ilm-guided keyframe prior. Advances Neural Information Processing Systems, 2025b.
Yunxin Li, Zhenyu I. Liu, Zitao Li, Xuanyu Zhang, Zhenran Xu, Xinyu Chen, Haoyuan Shi, Shenyuan Jiang, Xintong Wang, Jifang Wang, et al. Perception, reason, think, and plan: survey on large multimodal reasoning models. arXiv preprint arXiv:2505.04921. 2025c.
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chenggi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arX1v:2412.19437, 2024a.
Haotian Liu, Chumyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in Neural Information Processing Systems, 2023.
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, 2024b.
Haotian Liu, Chunyuan Li. Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen and Yong Jac Lee. Llava-next: Improved reasoning, ocr, and world knowledge, 2024c. URL https: wo/blog/////oooooooooo..o..ooo.t 2024-0022000 01-30- 1000--llaa-nee...
//1lava-v1.gie
Xiaoliu Luo, Zhuotao Tian, Taiping Zhang, Bei Yu, Yuan Yan Tang, and Jiaya Jia. Pfenet++: Boost- ing few-shot semantic segmentation with the noise-filtered context-aware prior mask. IEEE Trans- actions Pattern Analysis and Machine Intelligence, 2023.
Muhammad Maaz, Hanoona Abdul Rasheed, Salman Khan, and Fahad Khan. Video-chatgpt: To- wards detailed video understanding via large vision @ and language models. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, 2024.
Karttikeya Mangalam, Raiymbek Akshulakov, and Jitendra Malik. Egoschema: A diagnostic bench- mark for very long-form video language understanding. Advances in Neural Information Process- ing Systems, 36, 2023.
Lingchen Meng, Jianwei Yang, Rui Tian, Xiyang Dai, Zuxuan Wu, Jianfeng Gao, and Yu-Gang Jiang. Deepstack: Deeply stacking visual tokens is surprisingly simple and effective for Imms. Advances in Neural Information Processing Systems, 2024.
Zhenhua Ning, Zhuotao Tian, Guangming Lu, and Wenjie Pei. Boosting few-shot 3d point cloud segmentation via query- geeey- enhancement. In Proceedings of the 31st ACM international con- ference on multimedia, 2023.
Bohao Peng, Zhuotao Tian, Xiaoyang Wu, Chengyao Wang, Shu Liu, Jingyong Su, and Jiaya Jia. Hierarchical dense correlation distillation for few-shot segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023.
Bohao Peng, Zhuotao Tian, Shu Liu, Mingchang Yang, and Jiaya Jia. Scalable language model with generalized continual learning. arXiv preprint arXiv:24044.077 07447000 2024a.
Bohao Peng, Xiaoyang Wu, Li Jiang, Yukang Chen, Hengshuang Zhao, Zhuotao Tian, and Jiaya Jia. Oa-cnns: Omni-adaptive sparse cnns for 3d semantic segmentation. In Proceedings of the IEFF/CVF Conference Computer Vision and Pattern Recognition, 2024b.
13

---

Published as a conference paper at ICLR 2026
Shangpin Peng, Senqiao Yang, Li Jiang, and Zhuotao Tian. Mitigating object hallucinations via sentence-level early intervention. arXiv preprint arXiv:2507.12455, 2025.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In Proceedings of the International Conference on Machine Learning. 2021.
Yuzhang Shang, Mu Cai, Bingxin Xu, Yong Jae Lee, and Yan Yan. Llava-prumerge Adaptive token reduction for efficient large multimodal models. Proceedings of the IEEE/CVF International Conference on Computer Vision, 2025.
Kele Shao, Keda Tao, Can Qin, Haoxuan You, Yang Sui, and Huan Wang. Holitom: Holistic token merging for fast video large language models. Advances in Neural Information Processing Systems, 2025a.
Tong Shao, Zhuotao Tian, Hang Zhao, and Jingyong Su. Explore the potential of clip for training- free open vocabulary semantic segmentation. In Proceedings of the European Conference on Computer Vision, 2024.
Zhenwei Shao, Mingyang Wang, Zhou Yu, Wenwen Pan, Yan Yang, Tao Wei, Hongyuan Zhang, Ning Mao, Wei Chen, and Jun Yu. Growing twig accelerate large vision-lane language models. Proceedings of the IEEF/CVF International Conference on Computer Vision, 2025b.
Leqi Shen. Guogiang Gong, Tao He, Yifeng Zhang, Pengzhang Liu, Sicheng Zhao, and Guiguang Ding. Fastvid: Dynamic density pruning for fast video large language models. Advances in Neural Information Processing Systems, 2025.
Xiaoqian Shen, Yunyang Xiong, Changsheng Zhao, Lemeng Wu, Jun Chen, Chenchen Zhu, Zechun Liu, Fanyi Xiao, Balakrishnan Varadarajan, Florian Bordes, Zhuang Liu, Hu Xu, Hyunwoo J. Kim, Bilge Soran, Raghuraman Krishnamoorthi, Mohamed Elhoseiny, and Vikas Chandra. Longvu: Spatiotemporal adaptive compression for long video-language understanding. arXiv preprint arXiv:2410.11 17434, 2024.
Keda Tao, Can Qin, Haoxuan You, Yang Sui, and Huan Wang. Dycoke: Dynamic compression of tokens for fast video large language models. In Proceedings of the IEÉE/CVF Conference on Computer Vision and Puttern Recognition, 2025.
Zhuotao Tian, Michelle Shu, Pengyuan Lyu, Ruiyu Li, Chao Zhou, Xiaoyong Shen, and Jiaya Jia. Learning shape-aware embedding for scene text detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019.
Zhuotao Tian, Hengshuang Zhao, Michelle Shu, Zhicheng Yang, Ruiyu Li, and Jiaya Jia. Prior guided feature enrichment network for few-shot segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020.
Zhuotao Tian, Pengguang Chen, Xin Lai, Li Jiang, Shu Liu, Hengshuang Zhao, Bei Yu, Ming- Chang Yang, and Jiaya Jia. Adaptive perspective distillation for semantic segmentation. IEEE Transactions Pattern Analysis and Machine Intelligence, 2022a.
Zhuotao Tian, Xin Lai, Li Jiang, Shu Liu, Michelle Shu, Hengshuang Zhao, and Jiaya Jia. General- ized few-shot semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022b.
Zhuotao Tian, Jiequan Cui, Li Jiang. Xiaojuan Qi, Xin Lai, Yixin Chen, Shu Liu, and Jiaya Jia. Learning context-aware classifier for semantic segmentation. In Proceedings of the AAAI Confer- ence on Artificial Intelligence, 2023.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov. Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open founda- tion and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023.
14

---

Published as a conference paper at ICLR 2026
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in Neural Informa- fion Processing Systems, 2017.
Chengyao Wang. Li Jiang, Xiaoyang Wu, Zhuotao Tian, Bohao Peng, Hengshuang Zhao, and Jiaya Jia. Groupcontrast: Semantic-aware self-supervised representation learning for 3d understanding. In Proceedings the IEEE/CVE Conference on Computer Vision and Pattern Recognition, 2024a.
Junjie Wang, Bin Chen, Yulin Li. Bin Kang, Yichi Chen, and Zhuotao Tian. Declip: Decoupled learning for open-vocabulary dense perception. In Proceedings of the Computer Vision and Pat- tern Recognition Conference, 2025a.
Junjie Wang, Keyu Chen, Yulin Li, Bin Chen, Hengshuang Zhao, Xiaojuan Qi, and Zhuotao Tian. Generalized decoupled learning for enhancing open-vocabulary dense perception. arXiv preprini arXiv:2508. /256, 2025b.
Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution. arXiv preprint arXiv:2409. 121911 2024b.
Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, et al. Internv13. 5: Advancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv preprint arXiv:2508. /8265 2025c.
Haoning Wu, Dongxu Li, Bei Chen, and Junnan Li. Longvideobench: A benchmark for long-contex interleaved video-language understanding. Advances in Neural Information Processing Systems, 2024.
Long Xing, Qidong Huang, Xiaoyi Dong, Jiajic Lu, Pan Zhang, Yuhang Zang, Yuhang Cao, Conghui He, Jiaqi Wang, Feng Wu, et al. Pyramiddrop: Accelerating your large vision-language models via pyramid visual redundancy reduction. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.
An Yang, Baosong Yang, Binyuan Hui, Bo Zheng. Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang. Jialin Wang, Jian Yang. Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng Hc. Jumyang Lin, Kai Dang, Keming Lu, Keqin Chen, Kexin Yang. Mei Li, Mingfeng Xue, Na Ni, Pei Zhang, Peng Wang, Ru Peng. Rui Men, Ruize Gao, Runji Lin, Shijie Wang. Shuai Bai, Sinan Tan, Tianhang Zhu, Tianhao Li, Tianyu Liu, Wenbin Ge, Xiaodong Deng, Xiaohuan Zhou, Xingzhang Ren, Xinyu Zhang, Xipin Wei, Xuancheng Ren, Yang Fan, Yang Yao, Yichang Zhang, Yu Wan, Yunfei Chu, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zhihao Fan. Qwen2 technical report. arXiv preprint arXiv:2407. 20t.:22477.1.71,,, 2024a.
An Yang, Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu. Rui Men, Runji Lin, Tianhao Li, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report. arXiv preprint 1rXiv:2412.1511 15, 2024b.
An Yang. Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang. Jiaxi Yang. Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghho Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. Qwen3 technical report. arXiv preprint ar0iv: 258809088... 2025a.
15

---

Published as a conference paper at ICLR 2026
Cheng Yang, Yang Sui, Jinqi Xiao, Lingyi Huang, Yu Gong, Chendi Li, Jinghua Yan, Yu Bai, Ponnuswamy Sadayappan, Xia Hu, and Bo Yuan. Topv: Compatible token pruning with inference time optimization for fast and low- memory multimodal vision language model. In Proceedings of the JEEE/CVE Conference on Computer Vision and Pattern Recognition, 2025b.
Senqiao Yang, Tianyuan Qu, Xin Lai, Zhuotao Tian, Bohao Peng, Shu Liu, and Jiaya Jia. Lisa++: An improved baseline for reasoning segmentation with large language model. arXiv preprint arXiv:2312.17240, 2023.
Senqiao Yang, Zhuotao Tian, Li Jiang, and Jiaya Jia. Unified language-driven zero-shot domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, 2024c.
Senqiao Yang, Yukang Chen, Zhuotao Tian, Chengyao Wang, Jingyao Li, Bei Yu, and Jiaya Jia. Visionzip: Longer is better but not necessary in vision language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025c.
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023.
Ce Zhang, Kaixin Ma, Tianqing Fang. Wenhao Yu, Hongming Zhang, Zhisong Zhang, Yaqi Xie, Katia Sycara, Haitao Mi, and Dong Yu. Vscan: Rethinking visual token reduction for efficient large vision-language models. Advances in Neural Information Processing Systems, 2025a.
Kaichen Zhang, Bo Li, Peiyuan Zhang, Fanyi Pu, Joshua Adrian Cahyono, Kairui Hu, Shuai Liu, Yuanhan Zhang, Jingkang Yang, Chunyuan Li, and Ziwei Liu. Lmms-eval: Reality check on the evaluation of large multimodal models. In Luis Chiruzzo, Alan Ritter, and Lu Wang (eds.), Findings of the Association for Computational Linguistics: NAACL, 2025b.
Qizhe Zhang, Aosong Cheng, Ming Lu, Renrui Zhang, Zhiyong Zhuo, Jiajun Cao, Shaobo Guo, Qi She, and Shanghang Zhang. Beyond text-visual attention: Exploiting visual cues for effective token pruning in vlms. Proceedings of the IEEE/CVF International Conference on Computer Vision, 2025c.
Shaolei Zhang, Qingkai Fang, Zhe Yang, and Yang Feng. Llava-mini: Efficient image and video large multimodal models with one vision token. In Proceedings of the International Conference on Learning Representations, 2025d.
Yuan Zhang, Chun- Kai Fan, Junpeng Ma, Wenzhao Zheng, Tao Huang, Kuan Cheng, Denis Gu- dovskiy, Tomoyuki Okuno, Yohei Nakata, Kurt Keutzer, et al. Sparsevim: Visual token sparsifi- cation for efficient vision-language model inference. Proceedings of the International Conference on Machine Learning. 2025e.
Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Ziwei Liu, and Chunyuan Li. Video instruction tuning with synthetic data. Transactions on Machine Learning Research, 2024.
Junjie Zhou, Yan Shu, Bo Zhao, Boya Wu, Zhengyang Liang, Shitao Xiao, Minghao Qin, Xi Yang, Yongping Xiong, Bo Zhang, et al. Mlvu: Benchmarking multi-task long video understanding, In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, 2025.
16

---

Published as a conference paper at ICLR 2026
Supplementary Material
CONTENTS
More Experimental Results
18
A.1
Additional Experiments on Qwen2.5-VL
18
A.2
Additional Experiments on LLa VA- Video
19
Additional Ablation Studies
19
Evaluation Benchmarks
20
Implementation Details
21
Reproduction Details of Compared Baselines
Reproduction Details of FlashVID
22
Token Budget Alignment
23
D
Related Work
24
More Visualizations
25
E.1
Tree-based Spatiotemporal Token Merging
25
Qualitative Analysis on LLaVA-One Vision
25
E.3
Mulltttive Qwen2.5-VL
25
Visualizations of ADTS
25
Visualizations of Failure Cases in TSTM
25
Visual Perception Layers
25
Usage of Large Language Models
29
17

---

Published as a conference paper at ICLR 2026
Table Comparison of state-of-the-art methods on Qwen2.5-VL. The best performance among those with similar retention ratios R is highlighted in bold.
Retention Ratio R
Vi
EgoSchema
Long Bench
Avg.
Method
MVBench
Short
Medium Long Overall
Subset Total
Score
Rel. Acc (%)
Vanilla
100%
72.6
61.4
49.9
61.3
60.2
58.3
58.9
68.0
61.6
100.0
FastV
71.2
57.8
51.6
60.2
60,2
57.2
54.6
67.4
59.8
97.1
VisionZip
70.7
57.9
51.2
59.9
58.6
57.0
57.1
67.3
60.3
97.9
25%
FastVID
71.2
57.8
49.9
59.6
58,2
56.7
58.0
65.5
60.0
97.4
FID
71.1
58.7
49.1
59.6
59.4
$7.2
58.1
67.1
60.5
98.2
FastV
67.4
55.2
51.1
57.9
59.6
56.5
52.2
65.9
58.1
94.3
VisionZip
68.8
56,7
49.1
58.2
56.2
55.7
56.3
66.0
59.0
95.8
15%
FastVID
68.2
49.4
56.8
63.6
58.6
FIashVID
69.8
57.1
49.3
58.7
57.6
56.4
56.8
66.6
59.6
96.8
Table Comparison state-of-the-art methods on Qwen2.5-VL under fixed token budget. Our FID enables Qwen2.5-VL. processing 10x video frames, improving the overall perfor- mance of 8.6% within the same computational memory budget.
Retention Ratio R
VideoMME
EgoSchema
Long Bench
Avg.
Method
#Frames
MLVU
Short Medium Long Overall Subset
Total
Score
Rel. Acc (%)
Vanilla
16 (1x)
100%
66.4
56.4
48.2
57.0
58.2
55.6
56.9
40.6
$2.6
100.0
VisionZip
74.0
59.6
52,9
62.2
59.4
57.6
56.1
42.2
54.5
103.6
FastVID
48 (3x)
33.3
73.2
60.0
$1.7
61.6
59.4
57.8
57.2
43.1
$4.9
104,4
FlashVID
73.0
52.4
61.9
57.8
57.0
45.1
55.4
105.3
VisionZip
72.3
60.1
51.6
61.3
61.0
58.5
57.8
44.7
55.6
105.7
FasIVID
64 (4x)
25.0
71.0
58.3
50.6
60.0
61.4
58.1
57.7
45.0
55.2
104.9
FlashVID
73.0
59.3
50.3
60.9
60.2
58.4
58.4
45.0
55.7
105.9
A MORE EXPERIMENTAL RESULTS
We present comprehensive experimental results of our method. In the Appendix. A.1 we evaluate FlashVID against previous Mate--o-thhearr-mmthhho
Qwen2.5-VL (Bai et al.) 2025b). In the Appendix. A.2 we present additional experimental results on LLaVA-Video. Video. In the Appendix. A.3 we provide additional ablation studies on FlashVID.
A.1 ADDITIONAL EXPERIMENTS ON QWEN2.5-VE VL
Results on Qwen2.5-VL. To further demonstrate the generalizability of our method, evaluate against other methods on Qwen2.5-VL (Bai et al. 20255), which shows significant differences relative to LLaVA-One and LLaVA- Video. Tab. presents part of the experimental results on Qwen2.5-Vi under retention ratios R E (20%, 10% Additional experimental results when R E (25%, 15%} Qwen2.5-VL provided in Tab. Notably, our method consistently surpasses previous state-of-the-arr methods under various retention ratios, demonstrating strong generalization across different VLLMs.
Results on Qwen2.5-VL under fixed token budget. By applying visual token compression, VLLMs can achieve performance gains by processing more video frames while maintaining the overall computational budget. As discussed Sec. S.2. we explore extending the number input frames under fixed token budget. Tab. 3 reports results with 5x and 10x frames, demonstrating that VLLMs benefit from longer temporal context without increasing computational cost. Addi- tional results with 3x and 4x frames presented in Tab. g revealing consistent improvement trend. highlights that FlashVID effectively compresses visual tokens and preserves compact yet informative representations.
18

---

Published as a conference paper at ICLR 2026
Table 9:
Comparison of state-of-the-art methods LLaVA-Video. We employ frame token
setting for adaptation to different acceleration frameworks.
Retention Ratio R
VideoMME
EgoSchema Total
LongVideo Bench
Avg.
Method
MVBench
Short
Medium
Long
Overall Subset
Score
Rel. Acc (%)
Vanilla
100%
77.0
62.1
53.3
64.2
59.4
57.3
59.5
61.9
60.7
100.0
FastV
71.7
59.2
50.9
60.6
56.0
54.8
56.4
59.1
57.7
95.1
VisionZip
74.0
60.3
52.9
62.4
59.0
57.0
58.3
60.0
59.4
97.9
25%
FastVID
74.7
60.1
53.6
62.8
57.4
55.4
58.2
60.5
59.2
97.5
FID
74.2
61.4
51.6
62.4
59.2
56.6
59.1
60.2
59.6
98.2
FastV
67.9
56,9
50.8
58.5
52.8
53.1
54.5
57.5
55.9
92.1
VisionZip
72.9
58.1
51.9
61.0
58.6
55.7
57.2
59.6
58.4
96.2
15%
FastVID
73.4
51.8
61.1
54.1
57.7
58.3
96.0
FlashVID
73.8
59.6
52,1
61.8
57.8
55.8
58.3
60.4
59.1
97.4
Table 10: Efficiency of our FIashVID. We conduct the efficiency analysis on LLaVA- Video,, and report the prefilling time and Time-To-First-Tokee (TTFT) in milliseconds (ms).
Retention Ratio R
Vision Encoding Compression I.M Forward
Prefilling Time
Avg.
Method
TFLOPs
TTFT
Total
Score Rel. Ace (%)
Vanilla
100%
94.8
685.0
1016.8
(1.0x)
1701 (1.0))
60.7
100.0
FlashVID
10%
8.0
685.0
85.7
107.6
19..3
878.3 (1.x)
58.2
A.2
ADDITIONAL EXPERIMENTS ON LLAVA-VIDES
Results on VA In Tab. we present a part of the experimental results on LLaVA- Video (Zhang et al. 2024) under retention ratios R E (20%. 10%). Additional experimental results when R E (25%. 35%) f on LLaVA- Video are provided in Tab, 9 Notably, FIDshVID consistently outperforms previous state-of-the-art methods by large margin under different retention ratios.
Additional efficiency analysis As illustrated Tab. 6 we test the efficiency of our Flash VID on LLaVA-One Vision (Li et al. 2025a), comparing FastVID (Shen et al., 2025). In Tab. I0 we further evaluate the efficiency of our FID on LLaVA-Vides (Zhang et al. 2024). We report the detailed prefilling time and Time-To-Firee Fors--ss-Tok (TTFT). Notably, our Flash enables 5.3x prefilling and 1.9x Time-to-First- Token (TTFT) speedups over the vanilla LLa LLaVA- Video while maintaining 95.9% relative accuracy at 10% retention ratio.
A.3 ADDITIONAL ABLATION STUDIES
Ablation study on T,. The merging threshold T, plays important role in the Tree-based Spa- tiotemporal Token Merging (TSTM) module. T, directly influences the compression quality, in which increasing T, reduces the merging strength and better preserves temporal details, whereas lowering T, promotes aggressive compression but may merge less correlated tokens, probably in- troducing noise to the compact representation. As illustrated in Tab. [3] conduct ablation study on merging threshold T, on LLaVA-One Visionno and LLaVA- Video at R 10%. Flash consis- tently achieves the best performance under different VLLMs when merging threshold T, 0.8.
Ablation study on tree depth and breadth constraints. In TSTM, video redundancy is jointly modeled by spatiotemporal redundancy trees, which connect the highly correlated spatiotemporal visual information. Intuitively, applying proper depth and breadth constraints avoids the merge of spatiotemporally distant tokens, which may improve the compression quality. addition to the threshold parameter Tr, we also test with two extra parameters: 1) depth constraint: aims to maintain the temporal dynamics, preventing tokens from spanning an excessively long temporal range in the same tree; 2) breadth constraint: seeks to preserve the spatial locality, avoiding merging across overly large spatial regions. The detailed implementation TSTM with depth and breadth constraints is provided in Alg.
19

---

Published as a conference paper at ICLR 2026
Table 11: Ablation study on the tree depth. The maximum tree depth constraint prevents to- ken merging tokens from spanning an overly long temporal range.
Table 12: Ablation study on tree breadth. The maximum tree breadth prevents the merge of tokens adjacent frames from crossing exces- sively large spatial regions, ensuring that spatial locality preserved.
Depth VideoMME
EgoSchema
LongVideo Bench
MVBench
Acc.
LongVideo
Rel.
Breadth
VideoMME
EgoSchema
MVBench
57.2
$7.6
36.4
1/Min
$7.3
60.1
56.0
57.2
98.6
57.6
57.3
60.1
56.0
57.2
98.8
57.7
60.0
56.3
57.3
99.0
56.8
32/Inf
57.8
60.0
56.5
57.4
14/1nf
57.8
60.0
56.5
57.4
99.1
Table 13: Ablation study on the T,. Tr con- trols the merging strength, in which lower T, indicates stronger compression.
Table 14: Ablation study on fe- fe controls the expansion ratio, which large fe may lead to computational inefficiency, while low value may lose critical information.
т, VideoMME EgoSchema
LongVideo Bench
MVBench
Acc.
LongVideo Bench
Rel.
LLaVA-One
f.
VideoMME EguSchema
MVBench
Acc.
0.9
$7.3
60.4
56.5
57.3
99.1
56.5
0.8
57.8
60.0
56.5
57.4
99.1
0.7
57.1
60.1
56.0
57.0
98.5
1.15
56.9
60.2
55.4
56.9
98.1
1.20
57.3
60.3
56.0
57.3
98.8
LLaVA Video
1.25
57.8
60.0
56.5
57.4
99.1
0.9
57.2
55.0
56.9
89.7
95.7
1.30
57.5
60.3
56.3
57.5
99.1
57.2
0.7
57.8
55.3
57.1
59.2
95.7
1.35
57.1
60.0
56.5
57.3
98.8
However, illustrated in Tab. and Tab, we conduct ablation studies tree depth and breadth using -aaaVA--neeeisse Experimental results show that depth and breadth constraints don't 't bring performance gains. We hypothesize that the merging threshold T, delivers a similar effect.
Ablation study on fe- FIDshVID retains more visual tokens input to the LLM while pruning within the LLM to satisfy the overall computational budget, avoiding the loss of important visual informa- tion. Le controls the expansion ratio, in which large fe may lead to computational inefficiency. while a low value may lose critical information. In Tab. 14 conduct an ablation study on fe on LLaVA-One Vision. FIashVID achieves the best performance (99.1 % relative accuracy) when the expansion factor fe (1.25, 1.30). We adopt fc 1.25 for better efficiency.
B EVALUATION BENCHMARKS
The experiments are conducted on the following widely used video understanding benchmarks.
VideoMME. VideoMME. (Fu et al. 2025a) is comprehensive multi-modal evaluation benchmark on video understanding capabilities of VLLMs. features 900 videos spanning 6 diverse domains and 30 subcategories, with durations ranging from 11 seconds to hour. Each video is accompanied by high-quality human annotations, including 2,700 multiple-choice question-answer pairs.
Long VideoBench. Long VideeBeech (Wu al. 2024 is a comprehensive benchmark designed to evaluate VLLMs long-context, interleaved video-language understanding. It characterizes 3,763 videos with durations ranging from 8 seconds to 1 hour. This benchmark comprises 6,678 human- annotated multiple-choice questions based novel "referring-reasoningg" task, where models must retrieve and reason over specific multimodal contexts referenced in the questions, categorized into 17 fine-grained types across perception and relation levels.
MVBench. MVBench (i al. 2024a) is a comprehensive benchmark designed to evaluate tem- poral understanding in multi-modal video tasks, addressing the limitations of existing image-focused
20

---

Published as a conference paper at ICLR 2026
Algorithm 2 Tree-based Spatiotemporal Token Merging with Depth and Breadth Constraints
Require: Token sequences (R(",
RR(2) ( ...., R(F} from frames; similarity function sim(-; tree depth function depth(-); merging threshold T;; max tree depth dmaxi neighborhood size lt Compressed token set C
Ensure:
1: Initialize each token in R()) as root node and let be an empty token set.
2: for f f= 2 to F' do
> Construct candidate edges
for each token rf R()) do ((rr)) + candidate parents in R(F-1) within neighborhood kt ( sim(r:
3:
4:
5:
p" + arg max peNt!)
if sim(r! (omm(r/.0)))) pp)) T, then
6:
7:
Connect rf to p" 8: end if
9: end for
10: end for
11: for / - down to
b. Backward depth pruning
12:
for each token rf E R()) do if (r:thhr:)) dmax then Disconnect r from its parent
13:
15:
Mark of as new root
16: end if
17: end for
end for
19: for each tree T do
Aggregate redundancy trees
C+ U-CUAR Ag)(T)
21: end for return
benchmarks. It consists of 20 systematically constructed tasks that require complex temporal rea- soning skills, generated via static-to-dynamic transformation of static tasks.
EgoSchema. EgoSchema (Mangalam al. 2023) consists approximately 5,000 five-choice multiple-choice questions derived from 250 hours of egocentric video. It emphasizes long-form temporal reasoning, as each of its 289 three-minute clips requires tracking objects and actions over time spans that are 5-10x longer than those in previous datasets, thereby posing significant chal- lenges for both spatial perception and extended temporal coherence.
MLVU. MLVU (Zhou et al. 2025) contains 3,102 multiple-choice questions across nine diverse long-video understanding tasks. It challenges models with videos ranging from minutes to 2 hours, requiring reasoning over plot, temporal order, and event retrieval, thereby jointly testing fine-grained spatial recognition and long-range temporal reasoning.
C IMPLEMENTATION DETAILS
C.I REPRODUCTION DETAILS OF COMPARED BASELINES
Unless otherwise specified. all the experiments conducted NVIDIA A800 80G GPUs LMMs-Eval (Zhag et al. 2025b We evaluate all methods on three representative VLLMs with distinct architectures and characteristis:: LLa Vision (Li et al. 2025a) and LLaVA-Vide Zhang et al. 2024 and Qwen2.5-VL. (Bai et al. 2025b) All baseline methods are reimple-
mented in LMMs-Eval following their official implementations:
httear///gguuucccmm/////sssssssa 1mms evval MIT License
NexT
Apache License 2.0
https::/ggithub.com//mee
Apache License 2.0
21

---

Published as a conference paper at ICLR 2026
FastV (Chen al. 2024 (ECCV 2024). FastV prunes tokens at the K-th layer of the LLM using cross-modal attention scores, with a pruming ratio r. We follow the official settings with K = 2. using
175%, 80%, 85%, 90%] } for LLaVA-OneVision in Tab. and Qwen2.5-VL. VL. in Tab. 2 while setting r E (80%, 90%} for LLaVA-Video in Tab.
VisionZip Yang et al., 2025c F (CVPR 2025). VisionZip prunes visual tokens at the output of the vision encoder, conflicting with pooling operations in VLLMs and resulting in performance degradation. Following (Shen et al.) 2025), we instead apply pruning af- ter pooling for VisionZip. We follow the official setting by retaining both dominant and contextual ratios at a 54:10 ratio in each frame. We set R to {25%, 20%, 15%, 10%] for LLaVA-One Vision in Tab. [ and Qwen2.5-VL in Tab. 2 while setting R {20%, 10%3 and (25%, 15%} J for LLaVA- Video in Tab. 1 and Tab. respectively.
PIDneVID (Huang et al., 2025 5 (ACL 2025). PruneVID contains both before-LLM compression and inner-LLM pruning during the prefilling stage, along with KV Cache compression at the decoding stage. Following the official settings, we set the threshold 7 0.8, the temporal segment ratio 7 0.25, the token selection ratio a 0.4, and the pruning layer K = 10. We control the token budget by cluster ratio B. We use 3 E (40.7%, 32.5%, 24.4% 16.3%) in Tab. [
VIDtVID Shen et al. 2025 (NeurIPS 2025). FastVID prunes visual tokens based on spa- tiotemporal DPC- It begins with a dynamic segmentation based on transition similar- ities, followed by frame-win wise salient token selection based [CLS] attention scores. Fi- nally, it compresses the remaining tokens by spatiotemporal redundancy elimination based DPC-KNN. Following the official settings, we set the minimum number of segments c = 8, the segment threshold T = 0.9, the salient token ratio d = 0.4, the anchor frame 0.6. We set to f25%, 20%, 15%, 10%} for LLa VA One Vision in Tab. and Qwen2.5-VL in Tab. 2 while setting R to (20%. 10%} and (25, 15%] for LLaVA- Video Tab. and Tab. respectively.
step p = 4, and the merging factor
C.2 REPRODUCTION DETAILS OF FLASHVID
In addition to ADTS and TSTM modules, Flash employs two design choices: 1) video partition and 2) Inner-LLM Pruning for better performance.
Video Partition. State-of-the-arr art VLLM acceleration methods (Shen 2025 Shao et al., 2025a Huang et al., 2025 Tao et al. 2025) commonly apply video partitioning before token compression, aiming to avoid information mixing and building upon DySeg (Shen al. 2025). FIDshVID partitions consecutive similar frames into the same segment based on the transition sim- ilarities. Instead of using [CLS) token embeddings, we compute transition similarities based on pooled video features. Given video features E,, E PxN.xd we apply global average pooling to obtain the frame embeddings:
fe GAP(E,)) Fxd
(10)
The transition similarities are defined as the cosine similarity of frame embeddings of adjacent frames:
4 = cos (fs f(+1); i= 1,2, F-1, T (2,,,2,-. ... ........1111 |
(11)
where 4 denotes the transition similarity between i-th and (i+ 1)-th frame. A low transition similar- ity indicates a significant scene change. Following DySeg, we set the segment threshold S, = 0.9 and the minimum number of segments M, = 8.
Calibrated Max-Min Diversity Problem. As discussed Sec. 3.3 FIashVID utilizes the At- tention and Diversity-based Token Selection (ADTS) module to identify the spatiotemporally in- formative tokens within each frame. Specifically, ADTS formulates frame-wise token selection as
htttos///ghhhb..cc//////////l/ss siozzip. Apache License 2.0 CC BY-NC-SA 4.0 License
http:://gttbbb.oo///aaaaaaa1//1/ CYB httpps///ggttubbccom///nnna
MIT License
22

---

Published as a conference paper at ICLR 2026
Algorithm 3 Calibrated Max-Min Diversity Problem (MMDP)
Require: Pairwise distance D(F INN.:. [CLS] attention ( ICLSI E R::;; event relevance
S ER..
Ensure: Spatiotemporally informative token indices Z()
1: Initialize selected indices I() @ and R <- {0, 1, ....N 1}
2: Let IN. R. be an all-ones vector
3: D - -Ds) @ (IA @ IN.) @ (5 & 1x.))
> Calibrate pairwise distance
4: for do
Select the first token
5: dmiint]] t minseR.jti D
6: end for
7: & < arg dmin
8: Move & from R to I(f)
9: while | < M do
>> Iteratively add the subsequent tokens
10:
Initialize dmin f- inf for i R do
11:
dmin [i] + minjezea D.
12:
13: end for
14: k < arg max dini
15: Move & from R to I()
16:
end while
17:
return I(f)
Max-Min Diversity Problem (MMDP), calibrated by [CLS] attention and event relevance. The detailed implementation is provided in Alg.
Inner-LLM Pruning. As illustrated in Sec. 2.1 the hybrid compression framework balances effi- ciency and performance, which preserves sufficient visual information input to the LLM, preventing the loss of important information. VID employs this design for better performance, which re- tains more visual tokens before the LLM and prunes at a relatively high layer. We set the pruning layer K 20 and the expansion factor fc 1.25 without careful tuning for LLaVA-One Vision, LLaVA-Video, and QwL.2.5-VL.
C.3 TOKEN BUDGET ALIGNMENT
To ensure fair comparison, employ simple and effective strategy that aligns the average number visual tokens processed by each Transformer layer to meet a similar computational cost, following (Shao al. 2025b). Eq. 3 presents the Floating Point Operations (FLOPs) formula of the standard Transformer architecture for generality. this paper, we evaluate three representative VLLMs: LLaVA-One Vision (Li et al.) (2025a), LLaVA-Vided (Zhang al.) 2024), Qwen2.5-VL Bai et al. (2025)), which share similar LLM architectures that employ Group Query Attention Ainslie al. (0023) and SwiGLU (Dauphin al. 2017) non-linear activation. The computational FLOPs of these three LLMs can be formulated as:
FLOPs L (1+ + 2n +3ndm).
(12)
where n is sequence length, the hidden dimension, m the intermediate dimension of FFNs, g the number of key/value heads, and h the number of attention heads. Since the number of visual tokens n, dominates the sequence length n, the sequence length can be approximated by ne.
To clarify how visual token numbers are determined at each stage. We provide detailed explana- tion. Let R be the average retained visual tokens per Transformer layer, M be the number of tokens entering the LLM (after before-LLMM compression), K be the pruning layer index, T. be the number of Transformer layers LLM,, and R the number of retained visual tokens (after inner-LLM pruning). Then we have the following equation.
RL= MK R(I. K).
(13)
23

---

Published as a conference paper at ICLR 2026
We introduces expansion factor fe such that M feR; thus, we have:
R(I. - (cK)
R
(14)
L-K K
And the inner-LLMM pruning ratio r becomes:
R
L Jek
(15)
M
Je K)
Such simple token budget alignment strategy enables fair comparisons between different acceler- ation frameworks.
D RELATED WORK
Multimodal Large Language Models.
Recent advances in deep learning (He al., 2016
Vaswani et al. 2017, Devlin et al. 2019
Dosovitskiy et al.
2021
Radford et al. 2021 He et al.,
2022
Cui al. [0222] 2023 Peng al.
2024a Yang et al. 2024c
Wang et al. 2024a) have ben-
efited
traditional computer vision tasks, such as semantic segmentation and object detection (Tin
al.
2020
Lai et al. 2021 Jiang et al. 2 Peng al., 2023 Tian et al. 2022b Luo el al.
2023
Peng et al. [0024b Tian et al. 2022a 2019: Ning et al.
[023] et al. 2024 Wang
al.
2025a b). In particular, transformer based architectures and
large-scale pretraining have
creasingly driven the success of Large Language Models (LLMs) (Touvron et al. 2023 Grattafioni
al., [2244] Yang
al., 2024a bj 2025a Lai et al., 2024b) Peng et al. 2025] Liu et al. 2024a
Guo et a (2025), exhibiting strong generalization and reasoning capabilities, Building upon LLMs,
Multimodal Large Language Models (MLLMs) (Liu et al. 2023 2024b
c Dai et al. 2023 Li et al,
2023 Comanici et al. 2025
al. 2025c) extend the input modality from text to multimodalities (such as image, audio, and video) by coupling modality encoders with LLM backbones. So far, MLLMs have revolutionized traditional computer vision
Bai et al. 2025b a Wang al 2025c Li
tasks. For example, representative works like LISA (Lai et al. 2024a) and LISA+ (ang et al., 2023) study reasoning segmentation powered by MLLMs.
Video Large Language Models.
With the rapid advancement of MLLMs, Video Large Language
Models (VLLMs) (Li et al. 2025a Zhang et al. 2024] Bai et al. 2025b Shen et al. 2024 Li
et aal.422024 Maaz et al. 2024 Comanici et al. 20255 have gained increasing attention. Mainstream
VLLMs directly process raw video tokens with an optional pooling operation. LLaVA- Vision
[Li et al. 2025a) demonstrates strong video understanding capabilities through task transfer from images. To achieve fine-grained spatiotemporal modeling, some VLLMS employ elaborate designs. LLaVA-Video (Zhang al. 2024) introduces newline tokens to distinguish spatiotemporal posi- tions. Qwen2-VL (Wang et al. 2024b) and Qwen2.5-Vi (Bai et al. (2255)) use Multimodal Rotary Position Embedding (MRoPE). Owen3-VL (Bai al. 2025a) employs the Deepstack mechanism (Meng et al., (024)). which extracts visual tokens from intermediate layers the visual encoder and injects them the LLM, preserving rich visual information.
To achieve comprehensive evaluation, we evaluate our method on three representative VLLMS (i.e., L.LaVA-OneVision LLAVA-Vided and Qwen2.5-VI.) with significantly different architectures and characteristics.
Visual Token Compression. Token compression has emerged as effective technique that re- duces computational complexity in transformer architectures, such as Vision Transformers (ViTs) (Dosovitskiy et al.) (02T) and Large Language Models (LLMs). ToMe (Bolya al. 2023) gradually merges similar tokens in VITs. FastV (Chen et al., 2024) identifies text-relevant visual tokens based on text-to-visual attention in the L.M.. PyaramidDrop (Xing et al., (024) and Sparse (Zhang
iet al. (025c) progressively prunes visual tokens. VisionZip (Yang et al. 2025c).
(Shang et al. 2025), and VisPruner (Zhang et al. 2025c) filter salient visual tokens via [CLS] at- tention, while DivPrune (Alvar et al.] 2025) selects based on diversity. TopV (Yang al. 2025b)
formulates token selection as an optimization problem. V/Scan (Zhang et al. 2025a) combines global
and local scans for informative visual token selection.
However, the above methods only focus on spatial redundancy compression. To address this, several token compression frameworks for VL.LMs have been proposed. DyCoke (Tao et al. 2025) merges
24

---

Published as a conference paper at ICLR 2026
redundant tokens in each segment. VID (Huang al. 2025)
distinguishes static and dynamic
tokens. STTM (Hyun et al., 2025) models video redundancy by a quadtree. FrameFusion (Fu et al,
2025b performs both merging and pruning in the LLM. HoliTom (Shao et al., 2025a) combines global redundancy-aware video partition with spatial and inner-I.LM compression. FastVID (Shen al. 2025) employs density-basee -based token pruning. Dy'Tok (Li al. (025b) dynamically allocates loken budget to each frame or segment, serving as a plug-and-play module for existing token com- pression methods.
E MORE VISUALIZAAIOONS
E. TREE-BASED SPATIOTEMPORAL TOKEN MERGING
Due to the dynamic and evolving nature of video, the most semantically correlated visual features in adjacent frames likely to experience variation in position, scale, orientation, and other attributes over time. To address this challenge. we propose the Tree-based Spatiotemporal Token Merging (TSTM) mechanism, which models video redundancy by spatiotemporal redundancy trees in uni- fied way. enables capturing fine-grained video dynamics. Fig. | presents more visualizations of TSTM, highlighting the unique advantages of our TSTM for better spatiotemporal redundancy compression.
E.2 QUALITATIVE ANALYSIS ON LLAVA-ONEVISHI
As illustrated in Tab. [ we evaluate our FlashVID on LLevs-OneVision at R E (25%, 20%, 15,, 10%). Notably, at higher retention ratios (i.e., R= = 25% 20%, 15%), FlashVID surpasses the vanilla VL-AVA-OneV Vision, indicating a "less is more" pattern where excessively re- dundant tokens may degrade performance. Additionally, FlashVID preserves performance of 99.1% under extreme compression (c.g., R 10%). Fig. 6 presents four qualitative examples compar- ing LLaVA-Onee Vision with and without FlashVID, which indicates FlashVID enables fine-gres fine-grained spatiotemporal redundancy compression, providing compact yet informative video representation.
E.3 QUALITATIVE ANALYSIS ON QWEN2.5-VL
In this work, we explore extending VLLMs process more video frames under fixed computa- tional budget through visual token compression. As reported in Tab. 3 and Tab. 8 VLLMs bene- fit from longer temporal context. Fig. 7 presents four qualitative examples comparing Qwen2.5- VL with and without FlashVID, highlighting its ability to capture richer temporal information. Flash enables Qwen2.5-VL. (Bai et al., 2025b) to process 10x more frames (160 vs. 16) within the same computational cost, providing compact yet informative video representations and improv- ing the model performance.
E.4 VISUALIZATIONS OF ADTS
As shown in fig. g we compare token selection results by ADTS with and without event relevance calibration. Event relevance calibration helps identify the key visual tokens, thereby improving the performance of those tasks requiring fine-grained understanding.
E.5 VISUALIZATIONS OF FAILURE CASES IN TSTM
As illustrated in Fig. 9 we present visualizations of failure cases in our Tree-based Spatiotemporal Token Merging (TSTM). Although TSTM enables fine-grained spatiotemporal redundancy com- pression, it might result in merging operations with semantic confusion such merging tokens from different entities with similar semantic information.
E.6 VISUAL PERCEPTION LAYERS
We empirically found that certain transformer layers (deep layers) of VLLMs possess strong visual perception capabilities. These visual perception layers can typically identify keyframes. Fig. TO
25

---

Published as a conference paper at ICLR 2026
(a) Visualization TSTM (Example 1)
(b) Visualization TSTM (Example 2)
(c) Visualization of TSTT (Example 3)
Figure 5: Visualizations of Tree-based Spatiotemporal Token Merging (TSTM). We select three consecutive video frames that show obvious variations in spatial locations, scale, and orientation for each case to illustrate the advantages of our TSTM in FlashVID, TSTM jointly models spatial and temporal redundancy via spatiotemporal redundancy trees for capturing fnneggrined spatiotemporal relationships: thus, achieves better spatiotemporal redundancy compression.
26

---

Published as a conference paper at ICLR 2026
Question: What kind of communication listed before Semaphore?
LLaVA-OneVisioo
C. Telegraph.
X
LLaVA-OneVision w/ FIashVID
D. Pony express.
(a) LLaVA- One Vision with and without FlashVID (Example 1)
Question: What are moves in the scene this dance?
LLaVA-OneVision
B. Passe and then chasse
LLaVA-OmeVision w/ FIashVID
Kneel down knee and lean back.
(b) LiavA-OneVisionn with and without FlashVID (Example 2)
Question: Who fight versus the black dinosaur at last?
LLaVA-OneVision
C.A King Kong.
I.LVA-OneVision FlashVID
D. dragon
(c) LLaVA-One Vision with and without FlashVID (Example 3)
Question: which smart phone is advertised the screen of the laptop?
LLaVA-OneVisionn
B. iPhone 14 Pro.
X
LLaVA-OneVision FIashVID
D. iPhone
(d) Livi-Onnnviion with and without FIDshVID (Example 4)
Figure 6: Qualitative comparison of LLaVA-One Vision with and without Flash We duct qualitative analysis on LLaVA OneVision with and without Flash under a 25% retention ratio. We observe interesting phenomenon: in some examples, LLaVA One Vision with Flash VID compression can answer the questions correctly, whereas the original model with full visual tokens input gives incorrect answers, unveiling "less-is-more" pattern where excessive visual tokens in- put may degrade model performance.
27

---

Published as a conference paper at ICLR 2026
Question: What is the specific sentence in the smart phone that makes the man embarrassed?
Qwen2.5-VL
Dude! You gotta come toBgSStuf Camps!
Qwen2.5-VL w/ FlashVID
A. BTW..you got something in your teeth!
(a) Qwen2.5-VL with and without FIashVID (Example 1)
Question: What the specific sentence the smart phone that makes the embarrassed?
Qwen2.5-VL
A bamboo.
X
Qwen2,5-VL w/ FlashVID
B. A stick.
(b) Qwen2.5-VL with and without FlashVID (Example
Question: What the moves in the last scene this dance?
Qwen2.5-VI.
Releve and then pirouette.
Qwen2,5-VL w/ FIashVID
A. Kneel down on one knee and lean buck.
(c) Qwen2.5-VE VL with and without FIashVID (Example
Question: What is the stage background where several male performers holding long sticks?
Qwen2.5-VL
A forest.
X
Qwen2.5-VI. w/ FIashVID
A. A sallboat.
(d) Qwen2.5-VL with and without FlashVID (Example 4)
Qualitative comparison of Qwen2.5-VL. with and without FlashVID. The vanilla model processes only 16 sampled frames, which limits its ability to capture sufficient temporal information. In contrast, Qwen2.5-VL can handle 160 (10x) frames with FIashVID while main- taining the overall computational budget, yielding more accurate predictions by leveraging longer temporal context.
Figure 7:
Chanstenc Whrt o she flut of gaso?
Oscative:
ADTS Bdexase
ADTS Rdexann Cafflers
2.6
R. 5.59
Harry Visualization of ADTS (Example
(a) Visualization ADTS (Example 1)
Figure 8: Comparisons of ADTS with and without event relevance calibration. ADTS employs event relevance calibration terms to identify the tokens most relevant to the video event.
28

---

Published as a conference paper at ICLR 2026
(a) Visualization of TSTM (Example 1)
(b) Visualization of TSTM (Example
Figure 9: Visualizations of failure cases in TSTM.
presents several visualizations of visual perception layers. Building upon this insight, we hypothe- size that token compression at these layers yields negligible performance degradation.
To balance efficiency and performance, Flash adopts a hybrid compression paradigm that retains more visual tokens and prunes visual tokens in the L.M to control the overall computational budget. Hence, we consistently set the pruning layer K 20 (a relatively high layer for LLaVA-One Visionn and Qwen2.5-VL at 7B scale) without careful tuning.
F USAGE OF LARGE LANGUAGE MODELS
In this work, Large Language Models (LLMs) are only used for polishing the paper writing. They are not involved in research ideation, experimental design, data analysis, or the formulation con- clusions. All substantive intellectual contributions made by the authors.
29

---

Published as a conference paper at ICLR 2026
Question: What does the panda use to light with enemiee the video?
(a) Visualization of visual perception layers. (Example
Question: Which of the following is visible in the backkgroundd of the video when the mrinialurre showsn
Dase Ee
(b) Visualization of visual perception layers. (Example 2) Question: When the zodiacal highte from video?
(c) Visualization of visual perception layers. (Example 3)
Figure 10: Visualizations of visual perception layers. We empirically found that certain layers in VLLMs have strong visual perception capabilities, which can accurately recognize keyframes. We hypothesize that pruning visual tokens guided by attention weights in these layers could filter tokens most relevant to the text query, achieving better pruning performance.
30


Reviews:
[{"review_id": "X114vViank", "reviewer": "Reviewer_JrBP", "summary": "The paper proposes FlashVID, a training-free and model-agnostic token merging framework for video LLMs that jointly compresses both spatial and temporal redundancy. FlashVID introduces a two-stage pipeline: an Attention and Diversity-based Token Selection (ADTS) module that selects intra-frame tokens, and a Tree-based Spatiotemporal Token Merging (TSTM) module that merges redundant tokens across frames. Experimental results show that the proposed method can beat previous works using different backbones, and the ablation experiments are comprehensive.", "questions": "See weaknesses. \n\nBesides, one minor question is: why do you introduce TSTM in sec 3.2 before ADTS in sec 3.3, tho their orders in your pipeline are opposite?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The idea of jointly modeling spatial and temporal redundancy via a tree-based merging method is well-motivated and insightful.\n2. Another component, ADTS, focusing on intra-frame redundancy, is organically combined with TSTM in the pipeline. And they are complementary.\n3. Experiments are done in multiple benchmarks, with multiple backbones, compared with several previous SOTAs. The proposed model can consistently outperform them.", "weaknesses": "1. When comparing with other previous works, only the retention ratio and TFLOPs are used in the paper to measure the efficiency. However, token pruning is also time-consuming. Given the proposed method has many steps (eg, tree construction, similarity calculation, attention mask computation, etc), the pruning inference time complexity, and potential memory cost should also be comprehensively analyzed and compared with previous works.\n\n2. In the paper, the author(s) claim to use a hybrid compression strategy (before-LLM + inner-LLM). However, it's unclear that given a retention ratio, how many tokens are dropped according to the proposed method (before-LLM compression), and how many tokens are dropped in inner-LLM compression. And also, how is the before-inner ratio compared to previous works? Without a detailed analysis, it is difficult to tell whether the benefit mainly comes from the proposed method or from inner-LLM compression.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TAVRhzWQMW", "reviewer": "Reviewer_7jpn", "summary": "This paper proposes FlashVID, a training-free inference acceleration framework for Video Large Language Models (VLLMs) that addresses the computational bottleneck of processing high volumes of visual tokens. FlashVID is a training-free acceleration framework for Video Large Language Models (VLLMs) that addresses computational bottlenecks from processing many visual tokens. Unlike existing methods that compress spatial and temporal redundancy separately, FlashVID jointly models spatiotemporal relationships through: (1) TSTM - tree-based merging that tracks tokens across frames despite spatial movement, and (2) ADTS - diversity-based selection of informative tokens per frame. Experiments on three VLLMs across five benchmarks show FlashVID retains 99.1% performance with only 10% of tokens and enables processing 10× more frames under fixed compute budgets.", "questions": "- What is the individual contribution of TSTM vs. ADTS? How much does video partitioning contribute?\n\n- TSTM vs. simpler alternatives: Have you compared TSTM against simpler temporal merging strategies like:\n  - Merging based on global feature similarity (all-pairs matching) without spatial constraints?\n  - k-means clustering across frames?\n  - Optical flow-guided merging?\n\n- Given that depth/breadth constraints don't help (Table 11-12), is the tree structure essential, or is TSTM effectively performing similarity-thresholded pairwise merging?\n\n- How does FlashVID perform on video generation or video editing tasks where spatial precision is critical? Can it handle multi-view videos or videos with significant camera motion?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper identifies a clear limitation in existing methods that temporal redundancy is typically defined by fixed spatial locations, which fails to capture video dynamics where objects move, scale, and rotate. The tree-based spatiotemporal merging is an elegant solution to this problem.\n\n- The experiments are comprehensive, covering three diverse VLLMs and five benchmarks. The results consistently show FlashVID outperforming baselines. \n\n- The method requires no additional training, making it immediately applicable to existing VLLMs and reducing deployment barriers.\n\n- The paper includes proper ablations (Table 4, 5, 9-12), efficiency analysis (Table 13), and extensive visualizations (Figures 5-8) that support the main claims.", "weaknesses": "- Limited novelty in individual components: ADTS essentially combines existing techniques ([CLS] attention + diversity-based selection via MMDP). The calibrated MMDP formulation (Algorithm 4) is relatively straightforward. The tree construction in TSTM (Algorithm 1, lines 9-16) is a simple greedy nearest-neighbor matching with thresholding. The \"tree\" structure emerges naturally but isn't explicitly optimized.\n\n- The paper states that depth and breadth constraints \"yielded negligible gains\" (page 6, line 297-299; Table 11-12), which raises questions about whether the tree structure itself is crucial or if simple nearest-neighbor merging suffices.\n\n- The paper combines multiple techniques: video partitioning, ADTS, TSTM, and inner-LLM pruning. Table 4-5 only ablate ADTS variants (ATS, DTS, ADTS) and \\alpha, not the full pipeline. More ablation to show each contribution would make the paper more stronger.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YQZckBLcnR", "reviewer": "Reviewer_tpdQ", "summary": "This paper proposes a training-free inference acceleration framework called FlashVID, aimed at addressing the high computational cost issues caused by processing massive visual tokens in video large language models (VLLMs). FlashVID employs a two-stage compression strategy: first, it selects representative tokens within each frame using the \"Attention and Diversity-based Token Selection (ADTS)\" module; second, it constructs a \"spatio-temporal redundancy tree\" across frames through the \"Tree-based Spatio-Temporal Token Merging (TSTM)\" mechanism to jointly model and merge similar tokens that change dynamically over time. Experiments were validated on three VLLMs and five video benchmarks, maintaining 99.1% of LLaVA-OneVision's performance while retaining only 10% of the tokens, and achieving a 10x increase in the number of frames processed by Qwen2.5-VL under the same computational budget, resulting in an 8.6% improvement.", "questions": "1. \"We set K = 20 for LLaVA-OneVision, LLaVA-Video, and Qwen2.5-VL.\" Has K=20 been analyzed? Or is it set based on experience, and if it's a new model, would this configuration also apply?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. FlashVID is a training-free framework, can be used as a plug-and-play module applied to existing trained VLLMs without expensive training costs.\n2. Addressed a key pain point of previous VLLM acceleration methods: they typically compress spatial and temporal redundancies independently, or rely on a single spatial correspondence for temporal merging. TSTM can flexibly track and merge similar tokens that change dynamically over time in terms of spatial location, scale, or direction by building a \"spatio-temporal redundancy tree,\" which better aligns with the dynamic characteristics of videos.", "weaknesses": "1. This method requires multiple hyperparameters that need empirical tuning, which may affect its plug-and-play performance across different models or datasets. For example $T_{\\tau}$, $f_{e}$, $\\alpha$。\n2. The paper mentions the interesting phenomenon of \"less is more\" and attributes it to \"excessive visual token input may introduce noise,\" which is a reasonable assumption but lacks more in-depth quantitative or qualitative analysis to clarify how these \"noise\" specifically affect VLLM's attention or internal representations. Moreover, this phenomenon was only tested on the llava-onevision model, making the argument less persuasive. If this phenomenon can also be observed on models like qwen-vl and internvl, it would further strengthen the argument.\n3. ADTS introduced an \"event relevance\" to calibrate \"importance\", but it uses global average pooling (GAP) to average the entire frame information into a single vector. If the key event in a video is a local small action and most of the area is static background, the GAP vector will mainly represent the background, which may lead ADTS to mistakenly consider this key local event as \"irrelevant\" and possibly discard it. Is this global heuristic method a significant weakness for tasks requiring fine-grained understanding?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SN2udCLr1q", "reviewer": "Reviewer_nJgS", "summary": "This paper introduces FlashVID, a unified and efficient framework for long-context video understanding. The core idea is to enable scalable temporal reasoning on long videos without incurring quadratic attention costs, while maintaining accuracy comparable to state-of-the-art multimodal large language models (MLLMs).  The authors propose three key innovations: Flash Temporal Attention (FTA), Segment-Level Progressive Compression (SPC), and Streaming Memory Replay (SMR).", "questions": "1. Could you report end-to-end latency per video and memory usage per frame, especially during streaming inference? These practical details would help others reproduce the claimed efficiency.\n2. Could you provide qualitative examples showing how SPC affects video understanding? For instance, do compressed representations preserve fine-grained temporal cues such as object motion or scene transitions?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The paper targets efficiency–accuracy tradeoffs in long-video understanding, a practical and timely problem for both academic and industry-scale multimodal systems.The proposed Flash Temporal Attention and Segment-Level Progressive Compression present new design patterns for balancing local precision with global awareness.\n2. The methods are clearly described and experimentally validated across multiple datasets and evaluation settings. Ablation studies systematically analyze the contributions of FTA, SPC, and SMR, showing each component’s quantitative effect. The authors also provide a theoretical complexity analysis, demonstrating that temporal attention cost scales linearly rather than quadratically.\n3. Figures (e.g., Fig. 2–4) effectively visualize the compression hierarchy and temporal flow.", "weaknesses": "1. While SPC achieves impressive speedup, the paper lacks qualitative or interpretive analysis of how compression affects temporal coherence or feature richness. Visualization (e.g., attention maps pre- and post-compression) would strengthen understanding.\n2. Experiments primarily cover videos up to 2 hours. It would be interesting to see whether FlashVID maintains stability and context retention beyond this range (e.g., >5 hours or continuous streaming).\n3. How does SMR handle forgetting or outdated context in long videos? Is there a mechanism for refreshing or discarding stale memory tokens?\n4. Since SPC depends on segment-level aggregation, how sensitive is the model to poor segmentation? It would be valuable to test robustness on unstructured or synthetic long videos.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)