VScan: Rethinking Visual Token Reduction for Efficient Large Vision-Language Models
Authors: Ce Zhang
Kaixin Ma
Tianqing Fang
Wenhao Yu
Hongming Zhang
Zhisong Zhang
Haitao Mi
Dong Yu
Venue: Accepted by TMLR ()

Abstract: Recent Large Vision-Language Models (LVLMs) have advanced multi-modal understanding by incorporating finer-grained visual perception and encoding. However, such methods incur significant computational costs due to longer visual token sequences, posing challenges for real-time deployment. To mitigate this, prior studies have explored pruning unimportant visual tokens either at the output layer of the visual encoder or at the early layers of the language model. In this work, we revisit these design choices and reassess their effectiveness through comprehensive empirical studies of how visual tokens are processed throughout the visual encoding and language decoding stages. Guided by these insights, we propose VScan, a two-stage visual token reduction framework that addresses token redundancy by: (1) integrating complementary global and local scans with token merging during visual encoding, and (2) introducing pruning at intermediate layers of the language model. Extensive experimental results across four LVLMs validate the effectiveness of VScan in accelerating inference and demonstrate its superior performance over current state-of-the-arts on sixteen benchmarks. Notably, when applied to LLaVA-NeXT-7B, VScan achieves a 2.91$\times$ speedup in prefilling and a 10$\times$ reduction in FLOPs, while retaining 95.4\% of the original performance. Code will be made publicly available upon acceptance.

Published in Transactions on Machine Learning Research (01/2026)
VScan: Rethinking Visual Token Reduction for Efficient Large Vision-Language Models
Ce Zhangl.*
Kaixin Ma²
Tianqing Fang²
Wenhao Yu?
Hongming Zhang²
Zhisong Zhang² Haitao Mi² Dong Yu²
Carnegie Mellon Universily
Tencent AI Lab
"Work done during an internship at Tencent Al Lab. Contact: cezhangocs. cmu edu.
Reviewed on OpenReview: httpps//ppparerrreer..ffffrrrrre
Abstract
Recent Large Vision-Language Models (LVLMs) have advanced multi-modal understanding by incorporating finer-gred grained visual perception and encoding. However, such methods incur significant. computational costs due to longer visual token sequences, posing challenges for real- time deployment. To mitigate this, prior studics have explored pruning unimportant visual lokens either at the output layer of the visual encoder or al the early layers the language model. In this work, we revisit these design choices and reassess their effectiveness through comprehensive empirical studics of how visual tokens are procossed throughout the visual encoding and language decoding stages. Guided by these insights, we propose VScan, two- stage visual token reduction framework that addresses token redundancy by: (1) integrating complementary global and local scans with token merging during visual encoding, and (2) introducing pruning at intermediate layers of the language model. Extensive experimental results across four L.LMss validate the effectiveness of VScan accelerating inference and demonstrate its superior performance over current seate-of-the-arte on sixteen benchmarks. Notably, when applied to LLaVA-NeXT-7BB VScan achieves a 2.91 speedup in prefilling and 10x reduction in FLOPs, while retaining 95.4% of the original performance. Code is com wtt.://...///.eeeeee////aaaatate htw.a//gggn//eeee/a////Be.e...
available at
1 Introduction
Large Vision-Language Models (LVLMs) have emerged as a transformative advancement in multi-modal learning, achieving remarkable proficiency across a broad range of vision-language lasks [38, 34, 33, 56). Recent advances in LVLMs [40, 32, 28, 5, 36, 45, 44] further enhance their capacity to process high-resolution images and multi-image/ vidco inputs, enabling fino-grained perception in tasks such vidco question answering [19, 61, 54], multi- image understanding [21, 29], and referential grounding [30, 46). However, processing such rich visual inputs necessitates substantial increase in the number of visual tokens, which often far exceeds the number of text tokens [36, 32]. For instance, LLaVA VA-NeA-' [40] encodes up to 2,880 visual tokens for high-resolution images, while Qwen-2.5- VL [5] can process up to 16,384 tokens for multi-image or video inputs- orders of magnitude higher than typical text-only sequences. This leads to significantly longer input sequences and, due to the quadratic complexity of self-attention [58), incurs substantial computational and memory overhead, thereby limiting the scalability and real-time deployment of LVLMs in practical applications [12, 64).
Recognizing that not all visual tokens contribute meaningfully to the final LVLM response, recent works [12, 62, 70] have proposed visual token reduction techniques aimed at improving computational efficiency by pruning visually redundant or textually irrelevant tokens. These methods generally fall inlo two categories: (1) Texi- agnostic pruming approaches [64, 59, 60] (Figure 1(a)), which prune visually redundant tokens based on their sig- nificance and uniqueness during the visual encoding stage, typically leveraging self-attention or [CLS] attention from the output layer of the visual encoder; and (2) Text-aware pruning approaches [71, 62, 65] (Figure 1(b)), which selectively remove tokens with low relevance to the text query during the early layers of language

---

Published in Transactions on Machine Learning Research (01/2020)
B
Prine at the suppit lover the
sust encodier us (GLS] antention
Layert Layer?
Prine at earty LLM decading layer
Resed on the last instruction amention
Image lepur
roduce recundoncy
Image Irpst
resuce textual irrelovancyy
Visual Enrooder
Projector
Layer Layer2
discard badig round detaits
Lagar L.
X
Visual Enextar
Projector
Laper Layer Laper
X
discerd essential cress- modal interactions
Liyer
Respanae Ground
Octput
Ourast
Layer
Text Where the people
Text Duery
standing the image?
Tokenizer
Where are the people
standing the imager?
Takenizer
(a) Text- Text-agnostic approaches
LLM Decoding Layers
(b) Text-aware approaches
LLM Decoding Leyers
Stage 1: Complamentary Global and Local Scan
Stage Middle Layar Pruning
Prefill Time
Lecal Scan
Lever
Logon
Loger
retain
LLaWh-feeT--1
1427
retain R,%
Rg
Image Impul
Tokena
X
Iol
bx2
Layer
Token Mergng
FLOR
Laakk BocT 201025
Vizual Encoder
Concet C
Projector
Responce Booch
Ours 2.099 T. 201
Global Scan
Output Layer
Sseul Tekers
Performance KY Gache
Ours 1311%
Where the standing in the image?
Tokenizer
LLM Decoding Layers
LLaVa-RexT-78
100
Purs
554
(c) VScan: A Two-Stage Token Reduction Framework
Vausi Takens
Teatual Towens
X
Pruned Tokino
Local Tokenss
Global Takens
Figure 1: Comparison of our VScan with representative text-agnostic approaches (e.g., Vi- sionZip [64]) and text-aware approaches (e.g., V [12]). In this work, we introduce VScan, a two-stage, training-free visual token reduction framework that can be seamlessly applied to various open- sourced LVLM architectures, delivering significant acceleration in inference with minimal performance loss.
decoding stage to preserve task-specific information while reducing computation. While these approaches have shown promising results, their performance is often constrained by their single-stage design and the lack of a systematic understanding of how visual tokens are processed and utilized throughout the entire LVLM pipeline.
In this work, we conduct an in-depth empirical analysis to reassess the effectiveness of these two prevailing pruning paradigms and distill insights that guide the design of more effective visual token reduction methods. Our study reveals two key observations: (1) In the visual encoding stage, the visual encoder attends locally significant tokens in the shallow layers, focusing on fine-gratd ane-grained local details, while at deeper layers, it gradually shift its focus to highly condensed set of tokens that encapsulate broader global context; (2) In the LLM decoding stage, early layers exhibit strong positional bias toward visual tokens appearing later in the sequence, neglecting their semantic relevance; as the layers deepen, cross-modal interactions begin to emerge, and output token probabilities typically converge in the mid-to-late layers where visual information is more effectively integrated into the language stream.
Building on these insights, we introduce VSean, a two-stage, training-free visual token reduction framework that enhances the efficiency of LVLMs by progressively pruning uninformative tokens during both visual encoding and language decoding stages, shown in Figure 1(c). In the visual encoding stage, VSean employs complementary global-local scan strategy to retain semantically important and spatially diverse tokens, followed by token merging to preserve comprehensive visual information. In the LLM decoding stage, VSean introduces middle layer pruning to further eliminate visual tokens with low relevance to the text query, while maintaining essential cross cross-moda modal interactions to minimize disruption to final task performance. Notably, VScan can be seamlessly integrated into diverse open-sourced LVLM architectures and is fully compatible with FlashAttention [18, 17], making it both practical and broadly applicable real-world applications.
We comprehensively evaluate the effectiveness of VScan LLaVA-1.5 [39], LLaVA-NeXT [40], Qwen- 2L [6], and Video-LLaVA [36] across sixteen image and video understanding benchmarks. Extensive experimental results demonstrate V'Sca's generalizable effectiveness across diverse LVLM architectures and LLM scales, highlighting its advantageous performance-elfiiinne trade-off. Specifically, VScan achieves a 1.77x speedup on LLa VA-1.5-B and a 2.91x speedup on LLa -eNNeXT during prefilling, while retaining 7..7% and 95.4% of the original performance, respectively.
The contributions of this work are summarized follows:
2

---

Published in Transactions on Machine Learning Research (01/2026)
We conduct comprehensive analyses to reveal how visual knowledge evolves throughout the entire LVLM, offering insights to inform the design of more effective visual token reduction strategies.
We introduce VScan, two-stage training- free visual token reduction framework that progressively climinates unimportant visual tokens to reduce both visual redundancy and textual irrelevance.
Extensive evaluations across sixteen benchmarks demonstrate that VScan consistently outperforms state-of-the-art methods maintaining robust performance under constrained token budgets.
2
Related Work
Efficient Large Vision-Language Models. Building powerful rugo-reeeessive LLMs [57, 15], recent LVLMs typically adopt an petee-proooooorreoodeeee architecture, where visual imputs are encoded into tokens and jointly processed with language sequences [38, 36, 4, 13, 56, 11]. However, as image resolution increases or the input scales to multi-image/videe the number of visual tokens grows proportionally, leading to a quadratic increase computation cost and runtime due to the self-attention mechanism [58, 7, 14, 67]. which limits the scalability of LVLMs in real-world applications [6, 12, 49, 31, 69, 68, 9]. To mitigate this issue, several LVLMs introduced specialized modules to enhance efficicncy-
the Q-Former in InstructBLIP [16] and the perceiver resampler [26] in OpenFlamiigo [1] that distill dense visual inpuls into a compact set of features before LLM decoding. Orthogonal to these architectural strategies, FlashAttention [18, 17] has emerged as a widely adopted, hardware-awar optimization that accelerates attention computation by minimizing redundant memory access, offering substantial speedups without compromising performance.
Vision Token Reduction in LVLMs. Another line of work aims to improve model efficiency on the sequence dimension pioneering works such as ToMe [6] and Fast [12] have explored strategies like visual token merging and text-guided pruning to improve the efficiency of LVLMs. Building these advances, subsequent approaches can be broadly divided into two main categories: (1) Text-agnoslic pruming approaches [52, 3, 70, 64, 59, 60], which identify and remove redundant or uninformative visual tokens during the visual encoding stage. For instance, VisionZip [64] selects dominant tokens based on [CLS] atlention scores, while FOLDER [59] introduces token merging with reduction overflow in the final blocks of the visual encoder. (2) Text-aware pruning approuches [71, 62, 41, 65, 55], which aim to remove visual tokens that are irrelevant the text query during the LLM decoding stage. For instance, SparseVLM [71] proposes an itera- tive sparsification strategy that selects visual-relevant text tokens to rate the significance of vision tokens, and PyramidDrop [62] performs progressive pruning at multiple decoding layers to balance efficiency and context presorvation. In this work, we present comprchensive analysis of how LVLMs process visual tokens during both the visual encoding and language decoding stages, and propose a corresponding two-stage approach, VScan, to effectively improve the inference efficiency of LVLMs while maintaining robust performance.
3 Empirical Analysis
In this scction, we provide comprehensive analysis of how LVLMs process visual tokens during both the visual encoding and language decoding stages, offering empirical guidance for designing more effective visual token reduction strategies.
Preliminary: Architecture of LVLMs. We consider an LVLM parameterized by 8, which consists of three major components: a visual encoder, a feature projector, and an LLM decoder. Given an image input, the visual encoder processes the image patches, and the projector converts them into n visual tokens Xv = { These visual tokens are then concatenated with the tokenized textual query xy and fed into the LLM decoder for rugooregressive next-token generation, represented ye ~ (y yea), where the next token Yi is sampled from the output probability distribution pa(-), and yet denotes the sequence of tokens generated prior to timestep t.
Rethinking Visual Redundancy Reduction. To address visual redundancy in token representations, recent studics [70, 64] have proposed text-agnostic approaches that retain visual tokens with high [CLS) attention at the output layer of the ViT-based visual encoder. While effective to some extent, this strategy raises important question: Is relying solely on outpul (CLS) allention traly sufficient to caplure all
3

---

Published in Transactions on Machine Learning Research (01/2026)
Query: What is the cooking utensi that is hang from the hook?
[CLS] Attention Maps Across Different Encoder Layers
Image Input
[CLS] Selection
Ref. CLP Selection
Image Input
Layer 6
Layer 12
Layer 18
Layer 23
GT Answer: Pan
Answer: Ladle @
Answer: Pan ?
Tokan #536: ground Tokan #234; person
person
Query: On which side of the picture the leather beg!
Salf-Atrantion Maps Across Different Encoder Layars
Image Input
[CL5] Selection
Ref. CLP Selection
Attantion co Token 11536
Layer
Layar
Allention to Tokem 12224
Layor
Layar
Right
Answer: Right
Figure 2: Empirical study visual redundancy reduction. (Left) We illustrate two failure cases where relying solely on the output [CLS] attention leads to incorrect predictions. For comparison, include reference token selections from CLIP-ViT-L-336p [51], following Gandelsman et al. [22], which highlight regions of interest relevant to the text query. (Rightt We visualize the [CLS] attention maps and self-attention maps of representative tokens (e.g., #536: ground, #234: person) across different encoding layers, illustrating how attention patterns evolve from localized focus in shallow layers broader global context in deeper layers.
lask-relevant visual information? Upon closer examination, we identify clear yet often overlooked limitation of these approaches: they tend to favor tokens corresponding to visually salient objects, while aggressively discarding background visual details that may carry essential semantic information. As illustrated by the examples in Figure 2 (Left), output [CLS] attention is incorrectly directed to the wall and person, ignoring the actual targets- the pan and leather bog to incorrect model responses.
To better understand and overcome this limitation, we analyze how visual information is processed across the visual encoding layers in LVLMs. Specifically, we visualize both the [CLS] attention and self-attention of representative tokens across different visual encoding layers, as illustrated in Figure (Rigght). Our observations are as follows: (1) In the shallow layers, the [CLS] attention maps capture fine-grained local details across the image. In contrast, in the deeper layers, the attention becomes increasingly concentrated the main entities, reflecting their global semantic relevance; (2) The self-attention maps for representative visual tokens reveal similar local-to-ghe tlobaa trend: in the shallow layers, these tokens primarily attend to nearby regions with similar semantic meaning, while in the deeper layers, their attention becomes more dispersed, integrating context from the entire image. These findings highlight a gradual transition the visual encoder from capturing low-level local details to modeling high-level, globally relevant semantics, suggesting that relying solely on the output layer may overlook the rich local information encoded in the shallow layers.
Rethinking Textual Irrelevance Reduction. While prior studies [12, 71, 37] have proposed effective text-aware approaches for pruning visual tokens at early layers during LLM decoding, a critical question remains: Are early layers the optimal stage for pruming visual tokens to minimize their impact on the model's final response? To investigate this, we conduct three empirical studies on POPE [35] and GQA [25], analyzing how the model's knowledge and prodictions cvolve during the decoding process:
Study 1: How does position bias in token selection evolve across LLM layers? Specifically, we visualize the distribution of the retained tokens selected by the attention score of the last instruction token [12] across LLM layers using LLaVA-11.5-7B As shown in Figure 3 (Left), early layers (e.g., layers 2 and 8) tend to select tokens at the bottom of the image, reflecting an inherent LLM position bias, as the last instruction token primarily attends to nearby lokens and focuses on local context [58, 50], and flattened visual tokens from the bottom of the image are positioned closest to the instruction tokens in the sequence. As the LLM layers deepen, this undesirable position bias diminishes and the focus shifts toward the center of the image, which more intuitive since the center of the image typically carries the most informative and task-relevant features [2, 8).
4

---

Published in Transactions on Machine Learning Research (01/2026)
Layer 2
Layer& 8
Layer 16
(3) 1,578
(b) Qwen-2.55 2.5V--255VL-78
Early Layers Focus on the Bottom
Middic Layors:hhie Shift Focus Center
Sutool Houds
Sorted
Figure 3: (Left) Study 1: Distribution of retained tokens at a 50% reduction rate in layers 2, 8, and 16 of LLaVA-155-B on POPE [35]; (Right) Study 2: Sum of visual attention across different attention heads and LLM layers using L.L.VA-1.5-7B and Qwen-2.5- on POPE (35).
(e) LLaVA-1.5-78
Pre-mature Layers
Mature Layers
Conf
monty
learts
Beart
eoter
mater
deach
Busch
Samples on BQA
sind
commuter
computer
Bay
Hey
Computer
laogle
tel
Shon
Rion
Man
Bed
Peemal
Samples PERE
10
16
18
19
20
21
23
24
26
27
20
20
10
31
LLM Layers
(b) LLaVA-NoXT-77
Pre -ree-mature Layers
Mature Layers
Conf.
Portall
Portal Portall
penas
Samplex on MMBarch
9
Pertall
Portal
Portal Pertal
age
Portal
deed latter
lutter
Pertall
15
Portall
Samples on MME
Pritual
HP
Portali
181
Portal
Portai
11
15
13
22
LIM
Layers
Figure 4: Study 3: Visualization of next-token predictions derived from the output hidden states of cach
LLM layer using (a) LLa VA-15-BB; (b) LLaVA-NeXT-7B. Darker colors indicate higher prediction confidence.
Study 2: From which layer does the LLM begin to gather and process visual information? We visualize the sum of attention received by all visual tokens from the last instruction token across different LLM layers using LLaVA-1.5-1 and Qwen-2.5-VL-TB in Figure 3 (Right). The red curve in each plot highlights the layer-wise attention patterns directed towards visual information. We observe that the middle LLM layers are primarily responsible for interacting with the visual tokens, whercas the early and deep layers focus predominantly on processing textual information.
Study 3: At which LLM layer do neat-token predictions begin to converge? In Figure 4, we provide an interpretation of the hidden states across different LLM layers using LLaVA-1.5-7B and LLAVA-NeXT-7B. Specifically, we feed the hidden states from each LLM decoding layer into the final linear projection layer to obtain vocabulary logits and intermediate next-token predictions. We observe that in more challenging
5

---

Published in Transactions on Machine Learning Research (01/2026)
open-ended tasks like GQA, the next-token predictions stabilize around LLM layer 20, whereas in simpler yes/no tasks such as POPE, the predictions converge earlier, around L.M layer 16. Our findings indicate that early layers are still forming core cross-modal semantics, and pruming them risks disrupting essential grounding. In contrast, by the middle layers, next-token predictions have largely stabilized, meaning that these layers contribute diminishing semantic change. This directly motivales pruning in the middle-to-late layers rather than the carly layers.
These findings collectively suggest that early layers are suboptimal for pruning due to position bias and limited engagement with visual content. In contrast, pruning at middle layers is more appropriate as it better preserves critical moosl interactions and minimizes disruption to model prodictions.
4
Method
We introduce VScan, a training-fres free approach that progressively prunes uninformative tokens in both visual encoding and LLM decoding stages to accelerate LVLM inference, illustrated in Figure ((c).
4.1 Reducing Visual Redundancy via Complementary Global and Local Scans
Motivated by the observations in Section 3, we design two complementary token selection schemes for the visual encoding stage, namely global and local scan, which select important tokens based on both local and global significance, enabling the capture of more comprehensive visual details.
Global Scan. Given that the final layers of visual encoders capture global information, we follow recent works [70, 64] to select global tokens that receive the most attention from the [CLS] token Victs] in the output layer (e.g., the penultimate layer in LLa VA-1.5 [39]). Specifically, the [CLS] attention computation for each attention head can be represented by
QiCLS/ KJ vD
Q cLS] = F|cLs W
Kv =v
SYOL)) Softmax
(1)
where Whth and Whth represent the projections weights for head h E [1, D denotes the hidden state size, and S(css) represents the [CLS] attention. The global tokens are then selected by
H
x = =a xx/
S Savg
2 27]. where save [CLS|
= H
1
Σ h=1
sh [cL))
(2)
Here, is soft threshold based top percentile of attention scores, set to retain target number of tokens. Note that for LVLMs without [CLS] token (e.g., Qwen-2.5-V VL [5]), can similarly select the tokens using self-attention, i.e., the average attention each visual token receives from others.
Local Scan. To complement the global tokens and capture finer local details, we divide the image into non-overlapping windows and select the locally important tokens with the highest [CLS] attention from the shallow layer 1 within each window. Specifically, we allocate token budgets uniformly across windows, and select local Lokens from each window as:
U w=1
U Tyy E
Savg CLS]
Tw
(3)
where w denotes the window index, x represents the set of all tokens within the window, and Tu is the soft threshold for window The final set of selected tokens is the union of global and local tokens, xpolectad x U xt, resulting in retention rate of R₁ %. By default, balance the selection such that xx$ = |x Le., half of the retained tokens are global and half are local.'
Token Merging. To alleviate information loss, we introduce a similarity-based token merging strategy that merges unselected visual tokens with their most similar selected counterparts. Specifically, for each unselected
To avoid duplication, we prioritize local takens and exclude them from global taken selection. Empirically, we find that this prioritization does significantly affect the final performance.

---

Published in Transactions on Machine Learning Research (01/2026)
token zi, identify its most similar selected token zi XV x-plected based on the highest cosine similarity. Once all unselected tokens are assigned to their closest selected tokens, apply average merging [6] within each group to obtain the final merged representation Xy compute the average token representation by
merged Specifically, for each selected token xy, we
nerged
Σ x + x
(4)
[ +1
NEE
where U denotes the set of unselected tokens associated with the selected token x and | indicates the cardinality of this set.
4.2 Reducing Textual Irrelevance via Middle Layer Pruning
After selecting visually significant tokens, we further refine the token set based on their relevance to the text query. Building on the empirical insights from Section 3, we design our approach to prune tokens the mature middle layers of the LLM, aiming to avoid position bias, preserve cross-modal interactions, and minimize the impact on final predictions. Specifically, compute the attention between all visual tokens and the last instruction token at middle layer k denoted as
Qr VD
Qr = = Ky = =xy nerged Wh Sh Text Softmax
(5)
We similarly average the attention scores across different attention heads and select R₂ % textually relevant tokens with the highost average text attention. This allows us to retain a set visual tokens that are both visually significant and lextually relevant, contributing the most to an accurate response.
Empirical Validation. We conduct comparative analysis the GQA benchmark using LLaVA-1.5-7B [39] to examine the elfect of pruning tokens at different LLM layers, while keeping the average reduction rate consistent across settings. To better highlight the impact of pruming depth, we first apply a global scan to reduce the visual tokens to 288/144 (i.e., Ri 50%/25%) and then perform pruning at various L.M layers to reach an average retention rate of 75% during the LLM decoding stage. As shown in Table pruning at middle LLM layers (e.g., layers 16 or 20) yields the best performance, whereas pruning at carlier layers (e.g., layer 2) leads to up to 1.9% drop in accuracy. These results align with our empirical insights in Section 3 and validate the effectiveness of pruning at middle layers to remove textual irrelevance in LVLMs.
Table 1: Comparative study of pruning visual tokens at different LLM layers. R1 denotes the retention rate the visual encoding stage, while k and R2 indicate the pruming layer and retention rate in the LLM decoding stage, respectively.
Settings
R; = 50%
Rg 25%
I 2, R2 73.3%
59.6
56.8
It R₂ 66.7%
59.6
57.2
It 12, R₂ = 60.0%
59.6
58.6
& == 16, Hg = -5.0%
60.7
58.7
& 20, R₂ -3.3%
60.6
58.7
I 24, 0.0%
60.2
58.4
Remarks KV Cache and Flash Our proposed VSean is fully compatible with standard KV caching mechanisms, pruning occurs before visual tokens added to the cache. Consequently, the KV cache stores fewer entries while its structure and format remain unchanged. VScan is also compatible with FlashAttention [17, 18), as we recompute the attention scores for the last instruction token using vanilla attention calculation [58] outside the standard LLM layers.
5 Experiments
In this section, we validate the effectiveness of our VScan on four widely used LVLMs, evaluating its performance across various benchmarks and comparing with other stato-of-the-art approaches.
5.1
Experimental Settings
Models. We evaluate the general effectiveness of VScan by applying to four popular LVLMS with diverse architectures. Following prior work in this field, we first compare performance on LLaVA-1.5-7B [39], a widely adopted academic baseline that maps each image input to 576 tokens, and L.a (40),
7

---

Published in Transactions on Machine Learning Research (01/2026)
which improves high-resolution understanding by encoding an image into up to 2,880 visual tokens. We further assess Video-I.aVV-7BB [36], which extends the LLaVA framework to videos, processing up to 8 frames with 2,048 visual tokens, Finally, we are among the first to report experimental results on the recent Qwen-2.5-VVL VL [5], tested across different LLM sizes (3B, 7B, 32B). This model incorporates dynamic resolution processing to handle images of varying sizes, supporting visual loken counts ranging from 4 to 16,384.
Benchmarks and Metrics. We conduct extensive experiments on 9 standard image understanding benchmarks, including visual question answering benchmarks such as GQA [25]. ScienceQA [43], VQAv2 [23]. TextVQA [53] and VizWir [24]; multi-mode reasoning benchmarks such as MMBench [42], MMBench-Ch CN [42].
MME [20], and POPE [35]. We also include evaluations on 3 more challenging referring grounding tasks using RefCOCO [30], RefCOCO+ [30], and ReSCOCOg [46], and report the accuracy achieved by different approaches. In these grounding tasks, a localization is considered correct if the predicted bounding box has an ToU score of at least 0.5 with the ground truth. Additionally, we evaluate our approach on video question answering benchmarks: TGIF [27], MSVD [10], MSRVTT [63], and ActivityNett [66]. We follow previous work (45, 62] to utilize both accuracy and the ChatGPT score² as key performance metrics for these video-based benchmarks. The evaluation protocol and prompts are detailed in Section B.4.
Baselines. We compare the performance of our approach with 6 state-of-the-art visual token pruning methods:
ToMe [o], V [12], SparseVT. [71], HiRED [3], PyramidDrop [62], and VisionZip [64]. (1) ToMe [6],
which uses bipartite soft matching to iteratively merge similar tokens within VIT layers; (2) FastV [12], which
drops visual tokens in early layers of the LLM, guided by text-oriented attention score; (3) SparseVLM [71],
which selects vision-relevant text tokens to evaluate the significance of visual tokens; (4) HiRED [3]. which
dynamically assigns token budgets to sub-images for high-resolution image inputs; (5) PyramidDrop [62], which divides the LLM into stages and drops portion of visual tokens at the end of each stage; (6) VisionZip [64], which selects a set of dominant tokens and merges unselected tokens contextually. To ensure a fair comparison, we directly report the results of these baselines from their respective original papers unless stated otherwise.
Implementation Details. We adhere to the default inference settings for each evaluated LVLM as specified in their respective codebases. Additionally, we perform local scan at shallow layer, specifically at = for LLaVV-series models and / = 8 for Qwen-2.5-VL, For LLM-stage pruning, we select the middle layer as k 16 for LLaVA-series modcls and k 14 for Qwen-2.5-VI-7B. For Qwen-2.5-VI-33 and -32B, we similarly select the exact middle layer for pruning. By default, we fix the retention rate at the LLM middle layer to Rg = 33.3%, and adjust Rg accordingly to achieve the target average reduction rate. For instance, to achieve an average retention rate of 11.1%, we set Ri = 16.7% and Rg = 33.3%. Note that these design choices are grounded in our analysis in Section 3, and we also provide comprehensive ablation studies in Section 5.3. Code will be made publicly available upon acceptance to facilitate reproducibility.
5.2 Results and Discussions
Results on LLaVA-1.5. In Table 2, we apply our approach to LLaVA-155-BB [39] and compare its performance with other baselines across 9 image understanding tasks. Following previous work [71, 64] in this field, we present a comparative analysis of performance across three settings, retaining an average of 192, 128, and 64 visual tokens across all 32 layers in LLaMA [57], corresponding to reduction rates of 66.7%, 77.8%, and 88.9%, respectively. We also calculate and present the percentage of performance retention in the final column, using the vanilla LL.-A-1.5 performance as the 100% upper limit. As shown in the table, with only 128 and 192 tokens per image instead of the original 576, our approach nearly retains the performance of the original LLaVA-1.5, with only negligible performance declines of 1.0% and 1.2%, respectively. Our approach becomes even more advantageous with higher reduction rates: With an aggressive 88.9% reduction rate, our approach results in only a 3.3% degradation in average performance across benchmarks, outperforming the second-best VisionZip [64] by substantial margin of 4.0%. These results suggest that our method effectively maintains high performance while reducing the number of visual tokens processed.
Results on LLaVA-NeXT. In Table 3, we further compare the performance achieved by our approach with other state-of-the-arts on the more advanced LLa LLaVA-NeXT-7B [40]. We present the performance of all approaches under a fixed budget of 320 tokens per image, corresponding to an 88.9% reduction rate. Our
Evaluated using 3. 5-turho: https: / / http//aaafffmoooooo openai. wwwpepe.co//ooc.pppeeeeee.ppeeee cwwm///ccc//nceeelggtt--a t.tbb....
8

---

Published in Transactions on Machine Learning Research (01/2026)
Table 2:
Performance comparisons on LLaVA-1.5-7B [39] across image understanding bench- marks. The best results in each setting are bolded, and the second-best best are underlined.
Method
Venue
GQA
MMB
MMB MME POPE
SQABIG
VQA
VQA Text
VizWiz
Average
Upper Bound, 576 Tokens (100%), 3.817 TFLOPs
LLaVA-L.5-BB (99
CVPR'2 24
61.9
58.1
1862
85.9
69.5
78.5
58.2
50.0
100.0%
Retain 192 Tokens in Average (4 66. 7%6,
11.259 TFLOPs
ToMe (6)
ICLR'23
54.3
60.5
1563
72.4
65.2
68.0
52.1
88.5%
VastV [12]
BCCV'24
52.7
61.2
57.0
1612
64.8
67.3
67.1
52.5
50.8
90.4%
SparseLL [71]
ICML '25
59.5
53.7
1787
85.3
68.7
75.6
57.8
50.5
97.6%
PrramidDrop [62]
CVPR
57.8
56.8
1797
82.3
75.1
56.5
51.1
97.2%
VisionZip [64]
CVPR'25
59.3
1783
85.3
68.9
77.4
57.3
97.8%
VSean (Ours)
60.6
63.9
57.4
1806
86.2
68.6
77.8
57.7
50.4
99.0%
Retain 128
Tokens in Average (4 77.8%),
~ 0..33 TFLOPs
ToMe [6]
ICLR 193
52.4
53.3
13:13
62.8
59.6
63.0
49.1
80.4%
FastV [12]
BCCV'24
49.6
56.1
56.4
1490
59.6
60.2
61.8
50.6
51.3
85.4%
SparseLMM [71]
ICML '25
58.4
64.5
51.1
1746
85.0
68.6
73.8
56.7
51.4
96.4%
PeramidDeop [62]
CVPR
57.1
61.6
56.6
1761
82.3
68.4
72.9
56.6
51.0
96.2%
VisionZip [64]
CVPR'25
$7.6
1763
83.2
75.6
56.8
96.2%
VSean (Ours)
59.8
63.0
58.0
1792
86.1
68.9
77.1
57.3
51.7
98.8%
Rittain 64
Tokens in Average (4 88. 88.9%).
~00. 5 TPLOPs
ToMe [6]
ICLR 23
48.6
43.7
1138
52.5
50.0
57.1
45.3
70.1%
FastV [12]
BCCV'24
46.1
48.0
52277
1256
48.0
51.1
55.0
47.8
50.8
76.7%
SparseLLM [71]
ICML '25
53.8
16.1
1589
77.5
69.8
68.2
53.4
50.1
90.4%
PyramidDrop [62]
CVPR
47.5
50.5
1561
55.9
69.2
69.2
50.6
30.7
86.6%
VisionZip [64]
CVPR
55.1
60.1
1690
77.0
69.0
72.4
55.5
92.7%
VSean (Ours)
58.3
62.1
55.7
1698
85.0
69.1
75.4
55.6
51.8
96.7%
Table 3: Performance comparisons on LLaVA-NeXT-7B [40] across 9 image understanding benchmarks. The best results in each setting are bolded, and the second-best are underlined.
Method
Venue
GQA
MMB
CM MMEE POPE SQAIGG ING
VQA VQA Text
VizWiz
Average
Upper Bowad, 2,880 Tokeas (100%), 20.8825 TELOPs
MaVA-N [40]
CVPR'24
64.2
67.4
60.6
1851
86.5
70.1
81.8
61.3
37.6
100.0%
Retain 320 Tokeus in Average (L 88.9%), ~ TFLOPs
FastV [12]
ECCV'44
35.9
61.6
51.9
1661
71.7
62.8
72.9
55.7
53.11
88.7%
HIRED
AAAA22
39.3
64.2
55.9
1690
83.3
06.7
75.7
58.8
54.2
93.9%
ProamidDoop (62)
CVPR 195
73.5
91.4%
VisiouZip (64)
CVPR'25
63.1
82.1
76.2
94.0%
VSean (Ours)
60.7
65.3
57.8
1767
85.1
66.9
77.1
58.0
53.8
95.4%
approach continues achieve superior performance on LLa L. [40], attaining the best performance on 6 out of 9 benchmarks, and achicving 95.4% of the original L.L.A-NNXT--BB [40] performance with only 11.1% of the token budget without additional training.
Results Qwen-2.5-VL. VL. To further validate the general effectiveness of our approach, apply it to the more advanced VL [5] with three different LLM scales and visualize the performance on three general visual underslanding benchmarks in Figure 5. The full numerical results are also provided in Appendix A.7. As shown, our approach consistently outperforms FastV [12]. PDrop [62] and VisionZip [64] across all retention rates (from 11.1% 50%) and model scales. We also observe that on MME, both Fast and PDrop exhibit degraded performance under low token budgets as the LLM scales from 7B 32B. We speculate that this is due to larger LLMs introducing stronger language priors, which can hinder the accurate

---

Published in Transactions on Machine Learning Research (01/2026)
MMBench
MMBench-CN
MME
85
328
85
328
24
320
22
Accuracy (55)
38
3B
36
20
70
65
65
Total
18
FastV
FastV
FastV
FastV
RastV
60
PDrop
PDrop
--- PDrop
60
-._
PDrop --- PDrop
PDrop
16
PDrop
--- PDrop
PDrop
Ours
Ours
Ours
0
20%
40%
60%
80%
100%
0
20%
40%
60%
80%
100%
0
20%
40%
60%
80%
100%
Average Retention
Average Retention
Average Retention Rate
Figure 5: Performance comparisons on Qwen-2.5-VL [5] with different LLM sizes (3B/7B, 32B)
across 3 image understanding benchmarks. We present the performance of different approaches at 4 various retention rates, along with the original model performance without token reduction.
Table 4: Performance comparisons on 1reen22.5VL-77- [5] across 3 referring expression compre- hension benchmarks: RefCOCO, RefCOCO+, and RefCOCOg. The best results in each setting are bolded, and the second-best are underlined. 'Evaluation is based on our re-implementation.
RefCOCO
RefCOCO
RefCOCOg
Method
Venue
Average
val
testA
testB
val
testA
testB
val
test
Upper Bound, 4-16384
Tokens
(100%)
Qwen-2.5-VL-7B [5]
arXio' '25
89.45
92.56
85.16
83.50
89.02
79.15
86.76
87.24
100.0%
Retain 75%
Tokens in Average
25%)
Vt [12]
ECCV24
85.27
87.84
82.28
79.02
82.95
72.86
82.95
83.32
94.8%
PyramidDrop' [62]
CVPR
87.79
91.00
83.22
81.48
86.55
74.02
84.62
85.10
97.2%
VisionZip" [64]
CVPR
88.33
91.55
83.68
81.99
87.28
74.09
85.08
85.92
97.9%
VScan (Ours)
88.75
91.94
83.96
82.39
87.90
74.15
85.54
86.55
98.3%
Retain 50%
Tokens in Average (4 50%)
FastVt [12]
ECCV'4
73.85
73.38
74.21
66.75
68.88
62.65
71.06
71.86
81.2%
PyramidDrog Dropt [62]
CVPR'25
77.52
80.82
72.07
70.27
75.48
63.33
74.86
75.65
85.1%
VisiouZip' [64]
CVPR 255
80.92
84.55
76,91
74.02
79.52
67.89
78,
79.34
89.7%
VScan (Ours)
86.78
90.74
82.37
79.99
86.12
71.67
84.03
84.44
96.1%
Retain 25%
Tokens in Average
75%)
Va [12]
ECCV'24
43.57
46.81
40.86
39.47
43.78
36.02
43.04
42.69
48.5%
Proom [62]
CVPR
46.46
53.83
37.23
42.29
47.76
32.81
45.32
44.91
50.4%
VisionZip' [64]
CVPR125
60.12
66.95
55.41
58.33
64.88
48.52
57.09
56.44
67.8%
VScan (Ours)
74.32
79.05
68.22
67.22
73.72
58.95
69.42
69.43
80.7%
selection of icalical visual tokens. In contrast, our two-stage reduction strategy mitigates this bias and remains robust across LLM scales, demonstrating superior performance.
We extend our comparisons to more challenging grounding tasks and present the performance of different approaches Table 4. Compared to image understanding tasks, these grounding tasks require higher token budgets to preserve visual information necessary for precise localization. 75% reduction rate, for instance, halves the performance of FastV [12] and PDrop [62]. In this challenging scenario, our approach still robustly maintains 80.7% of the original performance. Additionally, performance with only 50% of the visual tokens. Qualitative results in Figure A2 further demonstrate that our approach effectively retains the critical tokens for accurate localization in challenging cases.
approach achieves 96.1% of the original
Results on Video-LLa VA. Finally, we validate the effectiveness of our approach on video understanding tasks and compare its performance against other approaches on Vidco-LLLaVA Specifically, we report the accuracy and GPT-evahuttd scores for each benchmark to assess the quality of the responses. The experimental results demonstrate that, even under a strict. 25% token budget, our approach
[36], as shown in Table
10

---

Table 5: Performance comparisons on Vidco-LLaVA-7B [36] across Published in Transactions on Machine Learning Research (01/2026)
TGIF
MSVD
MSRVTT
ActivityNet
Method
Ace.
Score
Acc.
Score
Ace.
Score
Acc.
Score
Video-LLaVA-7B [36]
47.0
3.40
70.5
3.92
58.3
3.51
42.2
3.37
4 video understanding tasks with
PastVt [12]
42.7
3.19
67.4
3.83
53.6
3.40
36.1
3.15
a 75% reduction rate. The best results are bolded, and the second- best are underlined. 'Evaluation is based on our re-implementation.
PyramidDrop'' [62]
44.1
3.26
66.7
3.81
56.1
3.45
37.4
3.15
VisionZip [64]
44.0
3.28
68.2
3.85
55.4
3.42
39.0
3.23
VScan (Ours)
46.9
3.35
69.8
3.93
57.1
3.48
42.6
3.34
Table 6: Ablation experiment using LLaVA-1.5-7B with an average reduction rate of 11.1% on GQA and MME. We ablate the effects of (a) retention rates R1 and R2; (b) the proportion of global and local tokens; and (c) encoding layer for the local scan, while keep all other settings fixed.
(a) Retention rates Rı and R2.
(b) Global & local tokens.
(c) Local scan encoding layer I.
R1
R2
GQA
MME
Global
Local
GQA
MME
Encoding Layer
GQA
MME
11.1%
100.0%
56.7
1651
0%
100%
58.0
1681
= (shallow)
57.1
1712
13.3%
66.7%
57.1
1683
25%
75%
58.1
1689
/ (shallow)
58.3
1698
14.8%
50.0%
57.5
1676
50%
50%
58.3
1698
= 12 (middle)
57.8
1692
16.7%
33.3%
58.3
1698
75%
25%
57.4
1688
18 (deep)
57.5
1678
22.2%
0.0%
52.7
1720
100%
0%
57.5
1665
23 (oudput)
57.3
1683
is able to preserve nearly the full performance of the original Video-LLaVA-TB. Moreover, it consistently surpasses prior methods across multiple benchmarks, highlighting the efficiency and robustness of our token reduction strategy in challenging vidco understanding tasks.
Additional Results. In Appendix A, present further experimental results, demonstrating that our VScan generalizes to more challenging benchmarks that require fine- fine-grained visual perception (e.g., Doc [47] and IQfA [48]) and also effectively accelcrates training while largely preserving model accuracy.
5.3 Ablation Studies
Varying Retention Rates R1 and R2. We analyze how different retention rate configurations affect performance by varying the retention rales Ri and R2 during the visual encoding and LLM decoding stages, respectively. As shown in Table 6 (a), we observe that relying solely on token selection in the visual encoder or applying overly aggressive pruning during LLM decoding lcads to suboptimal performance. Specifically, text-agnostic pruning (Rt) reduces computation early but lacks awareness of the text prompt, making it prone to removing visually subtle yet. instruction-critical cues. In contrast, text-aware pruning (R2) benefits from intermediate cross-modal alignment and can better retain tokens relevant to the query, though pruning too aggressively at this stage may interrupt the model's semantic refinement and harm reasoning stability. Instead, more balanced and gradual two-stage pruning strategy, i.e., R₁ = 16.7% and R2 33.3%, achieves the best performance. These results also validate the effectiveness of combining both visual token reduction strategies to jointly improve efficiency and maintain accuracy.
Mixing Global and Local Tokens. Tn Table (b), we examine the impact of mixing global and local token selections by varying their proportions. We find that selecting an equal ratio of global and local tokens yields the best performance, achieving 58.3% on GQA and 1698 on MME. In contrast, retaining only local global tokens results in 0.3% and 0.8% performance drop on GQA, respectively. These results highlight the complementary roles of global and local tokens, which together capture rich visual information and help preserve the model's visual reasoning capabilities.
Different Encoding Layers for Local Scan. In Table 6 (c), we explore the effect of performing local token selection at different layers of the visual encoder. Consistent with our empirical findings Section 3, we find that applying the local scan at a shallow layer 6) yields the best performance. However, performing the local scan at very early (1 2) the output layer (1 = 23) leads to noticeable performance drop, with
11

---

Published in Transactions on Machine Learning Research (01/2026)
performance on GQA falling to 57.1 and 57.3, respectively. This confirms that local tokens selected from shallow layers are more effective at capturing fine-grained visual details and better complement. the global token set.
5.4 Efficiency Analysis
In Table 7, we evaluate the practical acceleration effects of VScan. Specifically, we report the total inference time and the time roquired for the pre-filling stage when processing all samples in the POPE benchmark. By ro- taining only 11% of the visual tokens, VScan achieves x.37x speedup in overall efficiency and 1.77x speedup in prefilling efficiency on LLa LLaVA-15--7B, while maintaining robust performance with only 0.9% decline.
Our approach achieves even more significant acceleration on LLaVA- NeXT-7B, where it delivers 2.05x speedup in inference and a 2.91x speedup in the prefill stage. Addi- tionally, our approach can also ef- fectively compress KV cache stor- age across different backbones. It is also important to note that VScan is compatible with Fltenttn [17], which can further enhance efficiency. For instance, the inference time of
Table 7: Efficiency comparisons on the POPE benchmark. We report the theoretical FLOPs, actual runtime, KV cache compression rate (%), and the achieved accuracy.
Method
FLOPs
Total Time 1 Prefill Time KV Cache 4
Accuracy t
VIVVA-L5--B
3.817 T
1113 (1.00))
416 (1.00))
100.0%
85.9
Ours (33%)
1.253
937 s (1.19x)
301 8 (1.38x)
39.9%
86.2
Ours (11%)
0.415 T
812 s (1,97x)
235 (1,7X)
19.9%
85.0
LAVT--NeT-7B
20.825 T 2204 (1.00x)
1420 (1.00x)
100.0%
86.5
+ Ours (33%)
6.459 T 1701 (1,35x)
994 8 (1.33))
34.8%
86.1
Ours (11%)
2.099 T 1120 (2.05x)
488 (2.93x)
13.1%
85.1
our approach with 11% retention rate on LLaVA-NeXT-7B can be further reduced from 488 to 473 seconds.
6 Conclusion
In this work, we present a comprehensive empirical study understand how visual information is processed across both the visual encoding and LLM decoding stages. Building these insights, we propose VScan- two-stage, training- free visual token reduction framework to accelerate LVLM inference while maintaining robust performance. Specifically, we design complementary global and local scanning strategics to select informative visual lokens that preserve rich visual details during visual encoding, and further refine this loken set via middle layer pruning in the LLM decoding stage based on textual relevance. Extensive experiments across 4 LVLM architectures and 16 image and video benchmarks demonstrate that our approach consistently outperforms existing state-of-the-art methods, achieving a superior trade-off between efficiency and accuracy.
Limitations. One potential limitation of this work is the inherent trade-off between efficiency and accuracy: while the proposed VScan siguificantly reduces inference cost of LVLMs, aggressive token pruning may still distort visual information and lead to degraded performance, particularly on challenging tasks that demand fine-grained understanding or compositional reasoning.
References
[1] Jean-Baptiste Alayrac, Jeff Donahue, Pauline Lue, Antoine Miech, lain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei, Marianne Monteiro, Jacob T, Menick, Sebastian Borgeaud, Andy Brock, Aida Nematzadeh, Sahand Sharifzadeh, Mikot aj Binkowski, Ricardo Barreira, Oriol Vinyals, Andrew Zisserman, and Karén Simonyan. Flamingo: a visual language model for few-shot learning. In Advances in Neurol Information Processing Systems, volume 35, pp. 23716 23736, 2022.
[2] Dosovitskiy Alexey, Philipp Fischer, Jost Tobias, Martin Riedmiller Springenberg, and Thomas Brox. Discriminative unsupervised feature learning with exemplar convolutional neural networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(9):1784-17477 2015. 4
3 Kazi Hasan lbn Arif, JinYi Yoon, Dimitrios S Nikolopoulos, Hans Vandierendonek, Deepu John, and Bo Ji. Hired: Attention-guided token dropping for efficient inference of high-resolution vision-language
12

---

Published in Transactions on Machine Learning Research (01/2026)
models in resource-constrained environments. In Proceedings of the AAAI Conference Artificial Intelligence, pp. 1773 1781, 2025. 3, 8, 9
[4] Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond. arXiv preprint arXiv:2308. 12966, 2023. 3
[5] Shuai Bai, Kegin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-v technical report. arXiv preprint arxiv:2002. 12.2.1139 13923, 2025. 1, 2, 6, 8, 9, 10, 18, 19, 24
[6] Daniel Bolya, Cheng- Yang Fu, Xiaoliang Dai, Peizhao Zhang, Christoph Feichtenhofer, and Judy Hoffman. Token merging: Your vit but faster. In International Conference on Learning Representations, 2023. URL https://openrevien,
net//foruniddJrroRRBa
3, 7, 8,
[7] Gianni Brauwers and Flavius Frasincar. A general survey on attention mechanisms in deep learning.
JEFE Transactions on Knowledge and Data Engineering, 35(4):3279 327933288 2021. 3
[8] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In Europeun Conference on Computer Vision, pp. 213 229. Springer, 2020.
[9] Wenhao Chai, Enxin Song, Yilun Du, Chenlin Meng, Vashisht Madhavan, Omer Bar-Tal, Jenq-Neng Hwang, Saining Xie, and Christopher Manning. Auroracap: Efficient, performant video detailed captioning and a new benchmark. In International Conference on Learning Representations, 2025. URL https://openrevieet
[10] David Chen and William B Dolan. Collecting highly parallel data for paraphrase evaluation. In Proceedings of the 49th Annual Meeling of the Associalion for Compulalional Linguislics, pp. 190-200, 2011. 8, 24
[11] Keqin Chen, Zhao Zhang, Weili Zeng, Richong Zhang, Feng Zhu, and Rui Zhao. Shikra: Unleashing multimodal Ilm's referential dialogue magic. arXiv preprinl arXiv:2306.155 15195, 2023. 3
[12] Liang Chen, Haozhe Zhao, Tianyu Liu, Shuai Bai, Junyang Lin, Chang Zhou, and Baobao Chang. An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models. In European Conference on Computer Vision, pp. 19-35. Springer, 2024. 1, 2, 3, 4, 8, 9, 10, 11, 18
[13] Zhe Chen, Jiannan WI, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, Bin Li, Ping Luo, Tong Lu, Yu Qiao, and Jifeng Dai. Internvl: Scaling up vision foundation models and aligning for generic visual-linguiste tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 24185-20 24198,, 2024. 3
[14] Rewon Child, Scott Gray, Alec Radford, and llya Sutskever. Generating long sequences with sparse transformers. arXiv preprint arXiv:190,.10500, 2019. 3
[15]
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Alex Castro-Ros, Marie Pellat, Kevin Robinson, Dasha Valter, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Zhao, Yanping Huang, Andrew Dai, Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoc V. Le, and Jason Wei. Scaling instruetion-fnettned language models. Journal of Machine Learning Research, 25 (70):1-53, 2024.
13

---

Published in Transactions on Machine Learning Research (01/2026)
[16] Wenliang Dai, Junnan Li, Dongxu Li, Authony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. Instructblip: towards general-purpose vision-language models with instruction tuning. In Advances in Neural Information Processing Systems, pp. 49250 49267, 2023. 3
[77] Tri Dao. Flashattention-2: Faster attention with better parallelism and work partitioning. In International Conference on Learning Representations, 2024. URL https://oppnrrevier htt.://rww.....eeeoooooodddda/aa 2, 3, 7, 12
[18] Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Flashattention: Fast and memory- efficient exact attention with io-awareness. In Advances in Neural Informalion Processing Systems, volume 35, pp. 16344-16359, 2022. 2, 3,
[19] Xinyu Fang, Kangrui Mao, Haodong Duan, Xiangyu Zhao, Yining Li, Dahua Lin, and Kai Chen. Mmbench-video: A long-form multi-shot benchmark for holistic vidco understanding. In Advances in Neural Information Processing Systems, volume 37, pp. 89098-89124, 2024.
[20] Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, Yunsheng Wu, and Rongrong Ji. Mme: A comprehensive evaluation benchmark for multimodal large language models. aix preprint arvv22306. 13394, 2023. 8, 24
[21] Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A Smith, Wei-Chiu Ma, and Ranjay Krishna. Blink: Multimodal large language models can see but not perceive. In European Conference on Computer Vision, pp. 148-166. Springer, 2024. 1
[22] Yossi Gandelsman, Alexci A Efros, and Jacob Steinhardt. Interpreting CLIP's image representation via text-based decomposition. In International Conference on Learning Represenlations, 2024. URL https://openrewe
4, 18
[23] Yash Goyal, Tejas Khot, Douglas Summers- Staa,, Dhruv Batra, and Devi Parikh. Making the v in vqa matter: Elevating the role of image understanding in visual question answering. In Proceedings of the IEEE Conference on Compuler Vision and Pallern Recogrition, pp. 6904 6913, 2017. 8, 23
[24] Danna Gurari, Qing Li, Abigale J Stangl, Anhong Guo, Chi Lin, Kristen Grauman, Jiebo Luo, and Jeffrey P Bigham. Vizwiz grand challenge: Answering visual questions from blind people. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3608- 1188-617, 2018. 8, 24
[25] Drew A Hudson and Christopher Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognation, pp. 6700-6709, 2019. 4, 8, 23
[26] Andrew Jaegle, Felix Gimeno, Andy Brock, Oriol Vinyals, Andrew Zisserman, and Joso Carreira. Perceiver: General perception with iterative attention. In International Conference on Machine Learning, pp. 4651-4664 PMLR, 2021.
[27] Yunseok Jang, Yale Song, Youngjae Yu, Youngjin Kim, and Gunhee Kim. Tgif-qa: Toward spatio- temporal reasoning in visual question answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2758-2766, 2017. 8, 24
[28] Mengzhao Jia, Wenhao Yu, Kaixin Ma, Tianqing Fang, Zhihan Zhang, Siru Ouyang, Hongming Zhang, Meng Jiang, and Dong Yu. Leopard: A vision language model for text-rich multi-image tasks. arXiv preprint arXiv:2410.0117
2024. 1
[29] Dongfu Jiang, Xuan He, Huaye Zeng, Cong Wei, Max Ku, Qian Liu, and Wenhu Chen. Mantis: Interleaved multi-image instruction tuning. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https://opennreriew.nt
1
[30] Sahar Kazemzadch, Vicente Ordonez, Mark Matten, and Tamara Berg. Referitgame: Referring objects in photographs of natural scenes. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 787-798, 2014. 1, 8, 24
14

---

Published in Transactions on Machine Learning Research (01/2026)
[31] Dan Kondratyuk, Liangzhe Yuan, Yandong Li, Li Zhang, Mingxing Tan, Matthew Brown, and Boqing Gong. Movinets: Mobile video networks for efficient video recognition ion. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 16020 16030, 2021. 3
[32] Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, and Chumyuan Li. LLaVA-onevision Easy visual task transfer. Transactions on Machine Learning Research, 2025. ISSN 2835-8856. URL https: cetps://openns /oppnreveeeeeeeee net formmor zKv8qULV6n.
[33 Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre- training with frozen image encoders and large language models. In International Conference on Machine Learning, pp. 19730-19742. PMLR, 2023.
[34] Yanwei Li, Yuechen Zhang, Chengyao Wang, Zhisheng Zhong, Yixin Chen, Ruihang Chu, Shaoteng Liu, and Jiaya Jia. Mini-gemini: Mining the potential of multi-modality vision language models. arXiv preprint 2rXiv:2403. 18811 2024. 1
[35] Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination in large vision-language models. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 292 305, 2023. 4, 5, 8, 24
[36] Bin Lin, Yang Yc, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, and Li Yuan. Vidco-Ilava: Learning unitod visual representation by aligument. before projection. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 5971-5984, 2024. 1, 2, 3, 8, 10, 11, 24
[37] Zhihang Lin, Mingbao Lin, Luxi Lin, and Rongrong Ji. Boosting multimodal large language models with visual tokens withdrawal for rapid inference. In Proceedings of the AAAT Conference on Artificial Intelligence, volume 39, pp. 5334 5342, 2025. 4
(38] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In Advances in Neural Information Processing Systems, volume 36, pp. 34892 34916, 2023. 1, 3
[39] Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognàtion, pp. 26296 2036662666,222 2024. 2, 6, 7, 8, 9, 18, 24
[40] Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jac Lee. Llava-next: Improved reasoning, ocr, and world knowledge, January 2024. URL https://llava-vll, wwtpaaawwwwl..g.....a.a.aa 2log/2024-01- 30-1300av llava-next/. next// 1, 2, 7, 8, 9, 24
41] Ting Liu, Lianglao Shi, Richang Hong, Yue Hu, Quanjun Yin, and Linfeng Zhang, Multi-stage vision token dropping: Towards efficient multimodal large language model. ar preprint ar arXiv:2411.0080 2024. 3
[42] Yuan Lin, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, Kai Chen, and Dahua Lin. Mmbench: Is your multi-modal model an all-around player? In European Conference on Computer Vision, pp. 216-233. Springer, 2024. 8, 24
[43] Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chur Zhu, Oyvind Tafford, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering. In Advances in Neural Information Processing Systems, volume 35, pp. 2507-2521, 2022. 8, 23
[44] Fan Ma, Xiaojie Jin, Heng Wang, Yuchen Xian, Jiashi Feng, and Yi Yang. Vista-llama: Reducing hallucination in vidco language models via equal distance to visual tokens. In Proceedings of the IEEE/CVF Conference on Compuler Vision and Pallern Recognition, pp. 13151 13160, 2024. 1
[45] Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Khan. Video-chatgpt: Towards detailod video understanding via large vision and language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, pp. 12585-12609 2024. 1, 8
15

---

Published in Transactions on Machine Learning Research (01/2026)
[46] Junhua Mao, Jonathan Huang, Alexander Toshev, Oana Camburu, Alan L Yuille, and Kevin Murphy. Generation and comprehension of unambiguous object descriptions. In Proceediiggs of the IEFE Conference on Computer Vision and Pattern Recognition, pp. 11 20, 2016. 1, 8, 24
[47] Minesh Mathew, Dimosthenis Karatzas, and CV Jawahar. Docvqa: A dataset for vqa on document images. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 2200- 2200-2209, 2021. 11
[88] Minesh Mathew, Viraj Bagal, Rubèn Tito, Dimosthenis Karatzas, Ernest Valveny, and CV Jawahar. Infographievqa. In Proceedings of the LEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1697-1706, 2022. 11
[49] Sachin Mchta and Mohammad Rastegari. Mobilevit: Light- general and mobile- friendly vision transformer. In International Conference on Learning Represenlations, 2022. URL
wttp://r...eeeoooo////ddddaaaaae OsUt8.....
50] Niki Parmar, Ashish Vaswani, Jakob Uszkoreit, Lukasz Kaiser, Noam Shazeer, Alexander Ku, and Dustin Tran. Image transformer. In International Conference on Machine Learning, pp. 4055 4064. PMLR, 2018. 4
[51] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and πlya Sutskever. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, pp. 8748-8766 PMLR, 2021. 4, 18
[2]] Yuzhang Shang, Mu Cai, Bingxin Xu, Yong Jae Lee, and Yan Yan. Llava-prumerge: Adaptive token reduction for efficient large multimodal models. ar preprint arXiu:2403. 15388, 2024. 3
[33] Amanprect Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. Towards vqa models that can read. In Proceedings of the IEEE/CVF Conference Computer Vision and Pallern Recognition, pp. 8317 8326, 2019. 8, 23
[54] Enxin Song, Wenhao Chai, Weili Xu, Jianwen Xie, Yuxuan Liu, and Gaoang Wang. Video-mmlt A massive multi-discipline lecture understanding benchmark. arXiv preprint arXiv:2504.14693, 2025. 1
[55] Xudong Tan, Peng Ye, Chongjun Tu, Jianjian Cao, Yaoxin Yang, Lin Zhang, Dongzhan Zhou, and Tao Chen. Tokencarve: infermation-presorving visual token compression in multimodal large language models. XrXiv preprint urXiv:2503. 01005003.1001,, 2025. 3
[6)]
Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05500 2024. 1, 3
[57] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinct, Maric-Anne Lachaux, Timothée Lacroix, Bapliste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, el al. Llama: Open and efficient. foundation language models. ar preprint arXiv:2302.18971, 2023. 3, 8
[88] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems, pp. 6000 6010, 2017. 3, 4,
[9] Haicheng Wang, Zhemeng Yu, Gabriele Spadaro, Chen Ju, Victor Quétu, and Enzo Tartaglione. Folder: Accelerating multi-modal large language models with enhanced performance. arXiv preprint ar Xiv:22501 02430, 2025. 1,
[60] Zichen Wen, Yifeng Gao, Shaobo Wang, Junyuan Zhang, Qintong Zhang, Weijia Li, Conghui Hc, and Linfeng Zhang. Stop looking for important tokens in multimodal language models: Duplication matters more. Xix preprint arXiv:2502.11000 11494, 2025. 1,
16

---

Published in Transactions on Machine Learning Research (01/2026)
[61] Haoning Wu, Dongxu Li, Bei Chen, and Junnan Li. Longvideobench: A benchmark for mong-contett interleaved video-language understanding. In Advonces in Neural Information Processing Systems, volume 37, pp. 28828 28857, 2024.
[22] Long Xing, Qidong Huang, Xiaoyi Dong, Jiajie Lu, Pan Zhang, Yuhang Zang, Yuhang Cao, Conghui He, Jiaqi Wang, Feng Wu, and Dahua Lin. Conical visual concentration for efficient large vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pallern Recognition, pp.
2025. 1, 3, 8, 9, 10, 11, 18, 22, 24
[63] Jun Xu, Tao Mei, Ting Yao, and Yong Rui. Msr-vtt: A large video description dataset for bridging video and language. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp- 5288 5296, 2016. 8, 24
[64] Senqiao Yang. Yukang Chen, Zhuotao Tian, Chengyao Wang, Jingyao Li, Bei Yu, and Jiaya Jia. Visionzip: Longer is better but not necessary in vision language models. In Proceedings of the IEEE/CVP Conference on Computer Vision and Pattern Recognition, pp. 19792- 19802, 2025. 1, 2, 3, 6, 8, 9, 10, 11
[65] Xubing Ye, Yukang Gan, Yixiao Ge, Xiao-Ping Ping Zhang, and Yansong Tang. Atp-llava: Adaptive token pruning for large vision language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 24972 24982, 2025. 1,
[66] Zhou Yu, Dejing Xu, Jun Yu, Ting Yu, Zhou Zhao, Yueting Zhuang, and Dacheng Tao. Activitynet-qa: dataset for understanding complex web videos via question answering. In Proceedings of the AAAI Conference on Artificial Fntelligence, volume 33, pp. 9127 -9344, 2019. 8, 24
[67] Manzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, and Amr Ahmed. Big bird: Transformers for longer sequences. In Advances in Neural Information Processing Systems, volume 33, pp. 17283-17297, 2020. 3
[68] Ce Zhang, Zifu Wan, Zhehan Kan, Martin Q Ma, Simon Stepputtis, Deva Ramanan, Russ Salakhutdinov, Mouis-PPilippe Morency, Katia Sycara, and Yaqi Xie. Self-correeting decoding with generative feedback for mitigating hallucinations in large vision-language models. In International Conference on Learning Representations, 2025. URL attps://opentee
3
[69] Jianshu Zhang, Dongyu Yao, Renjic Pi, Paul Pu Liang, and Yi R Fung. VIm2-bench: A closer look at how well vims implicitly link explicit matching visual cues. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguislics, pp. 5510-7545, 2025. 3
[70] Qizhe Zhang, Aosong Cheng, Ming Lu, Zhiyong Zhuo, Minqi Wang, Jiajun Cao, Shaobo Guo, Qi She, and Shanghang Zhang. (CL.S) attention is all you need for training-free visual token pruning: Make vlm inference faster. arXiv preprint ar arXiv:2412.00 12.01818 2024. 1, 3, 6, 24
[71] Yuan Zhang, Chun-Kai Fan, Junpeng Ma, Wenzhao Zheng, Tao Huang, Kuan Cheng, Denis Gudovskiy, Tomoyuki Okuno, Yohei Nakata, Kurt Keutzer, and Shanghang Zhang. Sparsevim: Visual token sparsification for ellicient vision-language model inference. In International Conference on Machine Learning, 2025. URL https: //perrr//pperrrieww net:/orrmmid=000aaaPa
1, 3, 8,
17

---

Published in Transactions on Machine Learning Research (01/2026)
VScan: Rethinking Visual Token Reduction for Efficient Large Vision-Language Models
Appendix
In the appendix, we provide additional details and experimental results enhance understanding and insights into method. The appendix is organized as follows:
Section A presents additional experimental results that further validate the elfectiveness and robustness of our approach across various settings.
Section B provides extended experimental details, including FLOPs calculation and full experimental configurations, to facilitate reproducibility.
Section lists the license information for all models, baselines, and benchmarks used in this work.
Section discusses the limitations of this work and explores its broader implications and impacts.
A Additional Experimental Results and Discussions
A.1 Empirical Validation of Global and Local Scan
To validate the effectiveness our global and local scan schemes, we construct adversarial subsets for GQA and POPE, namely AdvGQA and AdvPOPE, which contains failure cases similar to those shown in Figure 2 (Left), where the text queries play important role and relying solely on the global scan to select visual tokens leads to errors. To accurately select these samples, we follow Gandelsman et al. [22] to decompose the image represenlations and pinpoint the tokens or regions most relevant to the query. Specifically, we utilize both the text and visual encoders of CLIP-ViT- L-336px [51] identify the visual lokens relevant to the text query reference. Two examples of the visual tokens selected by CLIP are shown in Figure 2 (Left). A sample is included in the adversarial set if the response is correct when using the 64 tokens selected by CLIP, but becomes incorrect when using the 64 tokens selected solely by the global scan. Following this, we collected 886 and 515 adversarial samples in AdvGQA and AdvPOPE, respectively.
LLaVA-1.5-78
G&LL Scan w/ Merging
90
PyramidDrop FastV
G&L Scan
80.8
80
72,12
Accuracy (9)
70
67.5
61.6
60
50-
40
37.5
37,2
33.0
30
20
AdvGOA
AdvPOPE
Figure Al: Performance comparisons on AdvGQA and AdyPOPE. We report the results for each approach with 64 visual tokens retained using LLaVA-1.5-7B [39].
In Figure A1, present performance comparison of incorporating both global and local scans with FastV [12] and PyramidDrop [62), which select visual tokens based on text attention and are expected to handle samples in the adversarial set effectively. We observe that incorporating both global and local scans achioves performance comparable to these text-guided approaches, despite being text-agnostic and selecting important tokens solely based on visual siguificance. These results validate that combining both scanning strategies and token merging helps preserve the maximun amount of visual information, effectively preventing information loss.
A.2 Qualitative Results
In Figure A2, we present qualitative examples from the RefCOCO benchmark using Qwen-2.5- VL [5]. For each image, we show the model's predicted bounding boxes in response different referring expressions, along with visualizations of the retained visual tokens after token pruning. These examples illustrate that method can accurately localize the target objects described in queries, while significantly reducing the
18

---

Published in Transactions on Machine Learning Research (01/2026)
Image
The second from the left.
Man blue second from right.
Person yellow shirt
Image Input
center.
Person
car on right.
Figure A2: Qualitative results on RefCOCO benchmark using Qwen-2.5-VL. [5]. We present the predicted boxes for different queries on 2 images, along with visualizations of the retained tokens.
number of visual tokens used for inference. This demonstrates the model's ability to preserve semantically important information under token-efficient settings.
A.3 Results on More Real-World Benchmarks
To comprehensively assess the effectiveness of VScan, we evaluate its performance on a range of real-world benchmarks, including DocVQA, InfoVQA, baseline. As shown in Table A1, VScan achieves competitive results while significantly reducing the number of visual tokens. With only 192 tokens, VScan maintains performance nearly identical to the full-token LLaVA-1.5-7B baseline, even surpassing it on MM-Vet and MMMU. When further reducing tokens to 128 and 64, VScan still preserves strong accuracy, demonstrating graceful degradation across tasks. These results highlight the robustness and efficiency of our method, showing that substantial token reduction can be achieved without sacrificing performance on diverse real-world benchmarks.
MM-Vet, and MMMU, using the LLaVA-1.5-7B
We further compare VScan with Fast and PDrop under different token retention rates on DocVQA and Info using both LLaVA-NeXT-T7- and Qwen2.5-VL-7B backbones. As shown Table A2, VScan consistently outperforms FastV and PDrop al the same retention levels. For LLaVA-NeXT-7BB VSean achieves 72.6 DocVQA and 36.3 InfoVQA with only 33% of tokens, closely matching the full-model baseline (74.4/37.1) while retaining higher accuracy than Fast and PDrop. Similar trends hold pruning levels, where VScan shows lower degradation compared to stronger drops for the baselines. Results on Qwen2.5- VL-7B further confirm this advantage: VSean reaches 94.0/79.7 at 33% retention and maintains 83.9/73.4 even at 11%, substantially outperforming FastV and PDrop. These results demonstrate that VScan generalizes across architectures and provides a more robust trade-off between efficiency and accuracy.
aggressive
Table A1: Performance of our VScan across diverse real-world benchmarks. We report results on DocVQA, InfoVQA, MMF- World, MM-Vet, and MMMU. Compared with the L.A baseline, VScan achieves competitive performance while substantially reducing the number of visual tokens retained.
Method
Retention Rate
DocVQA
InfoVQA
Woal-delloolld
MM- Vet
MMMU
LLVVA-1.--BB
28.1
25.8
24.9
31.1
35.3
VScan
33.3%
27.5
25.7
24.0
31.8
35.7
VScan 11.1%
22.2%
25.7
25.4
22.7
30.5
36.1
VScan
23.9
23.7
22.2
29.7
35.8
19

---

Published in Transactions on Machine Learning Research (01/2026)
Table A2: Comparison VScan with V and PDrop on DocVQA and Info Results are reported under different token retention rates (33%, 22%, 11%). 'Evaluation is based on our re-implementation.
Method
DocVQA
Info
Method
DocVQA
InfoVQA
LLaVA-NeXT-7B
74.4
37.1
Swve2.5-VL--7
95.7
82.6
Vt (33%)
69.8
33.5
Fast (33%)
88.7
75.3
PDrop' (33%)
71.8
35.0
PDropl (33%)
90.1
76.2
VScan (33%)
72.6
36.3
VScan (33%)
94.0
79.7
Fast (22%)
64.7
32.8
Fast (22%)
81.6
70.8
PDrop (22%)
67.4
33.4
PDropl (22%)
83.8
73.7
VScan (22%)
71.6
34.2
VScan (22%)
87.8
77.1
FastVt (11%)
61.2
28.1
Fast (11%)
74.9
65.2
PDrop (11%)
64.0
31.6
PDrop (11%)
77.4
67.0
VScan (11%)
68.8
34.5
VScan (11%)
83.9
73.4
(a) LLVAA-NeXT-7B.
(b) 10wen2.5-VI--7..
A.4 VScan for Accelerating Training
We further validate the effectiveness of VScan in reducing training cost while preserving the model performance. As shown in Table A3, VScan reduces training time by 41.7% from 96 GPU hours 56 GPU hours, achieving 96. 7% of the original performance. For comparison, also report. inference-time VScan results, which reach
95.2% of bascline performance. Notably, applying VScan during training yiclds 1.5% improvement over using VSean solely at inference.
Table A3: Training efficiency and performance of VScan compared with the LLaVA-1.5-BB baseline. VScan significantly reduces GPU hours while maintaining strong performance across benchmarks.
Method
GPU Hours
GQA
MMB
MMB-CN
MME
POPE
Avg.
LLaVA-1.5-BB
96
61.9
64.7
58.1
1862
85.9
100.0%
VSean (fine-tume)
56
59.2
63.0
57.9
1721
84.6
96.7%
VScan (inference)
58.3
62.1
55.7
1698
85.0
95.2%
A.5 Remarks Multi- Turn Conversations
Adapting the middle-layer pruning component of VScan to support multi-turn conversations is straightforward: When presented with new questions, VScan can reassess token importance and reselect textually relevant visual tokens from the existing token pool through global-local scans. Although this re-selection introduces additional computation compared to text-agnostic methods (which are inherently compatible with multi-turn dialogue), the overhead is negligible relative to the overall prefill time.
A.6 More Efficiency Comparisons
Table A4 summarizes the trade-off between efficiency and performance various visual-token pruning methods across different retention ratios (33.3, 22.2%, and 11.1%). VSean consistently achieves superior accuracy while maintaining competitive FLOPs and inference time, demonstrating its effectiveness in preserving visual information under aggressive pruning.
A.7 Full Numerical Results on 2.5-
We report the full numerical results Qwen-2.5-VT--7B in Tables A5, A6, and A7.
20

---

Published in Transactions on Machine Learning Research (01/2026)
Table A4: Efficiency and performance comparison under different token retention ratios. We report the total FLOPs (T), inference time (s), and accuracy achieved by the model at given retention rate.
Method
33.3%
22.2%
11.1%
LLaVA-115-7B
3.817T
416s / 85.9
FastV
1.253T
298s
/ 60.7
0.836T
266s
57.2
0.421T
231s
44.5
PDrop
1.253T
307s
82.3
0.834T
/ 278s / 82.3
0.415T
240s
55.9
VisionZip
1.253T
2938
85.3
0.834T
2638 / 83.2
0.415T
229s
77.0
VScan
1.253T
301s
86.2
0.834T
/ 274s / 86.1
0.415T
235s
85.0
Table A5: MMBench accuracy under different visual token retention ratios.
Higher is better.
Model
Method
11%
22%
33%
50%
100%
Fast
70.13
78.09
81.01
85.40
87.37
32B
PDrop
73.22
79.04
82.74
86.03
87.37
Ours
78.09
82.47
85.31
86.17
87.37
Fast
69.07
74.74
77.73
80.27
84.19
7B
PDrop
71.08
75.43
78.04
81.96
84.19
Ours
76.89
80.50
80.67
82.56
84.19
Fast V
56.79
67.27
72.42
74.98
78.87
3B
PDrop
55.93
63.22
70.30
72.48
78.87
Ours
69.67
70.62
72.81
75.64
78.87
B More Experimental Details
B.1 Remarks on Ensuring Fair Comparisons
In our main result table, ensured fair comparisons between our VScan and text-agnostic approaches such as VisionZip. Instead of aligning on the final token retention ratio, we matched the average token retention across all LLM layers, which roughly correlates with the inference complexity/speed. For example, at 11.1% retention rate, VScan retains 96 tokens before the LLM using a global-local scan strategy and then prunes to tokens the middle LLM layer (layer 16 of 32). This yields an average of lokens retained across all LLM layers matching the average retention of the text-agnostic methods, ensuring that inference speed is consistent.
Additionally, in the comparisons, the compared text-aware approaches, such Fast and SparseVL.M, actually have an advantage over our method, as their reported results are based on the final token count. Since we were unable to fully reproduce all of these methods, we directly reported their results as presented in their respective papers.
To account for this, the Qwen-2.5-VL and Vidco-LLaVA experiments presented, we reproduce Fast and PyramidDrop while aligning the average token retention across all LLM layers to ensure fair comparison (marked in the tables with t). For instance, for FastV, to achieve average retention rate of 11%, we prune to 5.2% of the original token count starting from LLM layer 2, resulting in an average retention of (100% x 2 + 5.2% x 30) 32 11.1%.
Therefore, under the same average retention rate, the efficiency metrics are essentially similar. This means that, at the same retention rate, higher performance directly translates into a more favorable nerformanc--offcione trade-off. In this regard, we have conducted thorough experiments demonstrating that our approach achieves state-of-the-art performance-effinn efficceencc trade-off across benchmarks and models.
21

---

Published in Transactions on Machine Learning Research (01/2026)
Table A6: MMBench-CNN accuracy under different visual token retention ratios. Higher is better.
Model
Method
11%
22%
33%
50%
100%
Fast
67.61
76.12
79.81
83.42
86.00
32B
PDrop
71.31
77.65
80.72
84.28
86.00
Ours
77.23
80.76
84.45
84.78
86.00
Fast
66.92
70.53
76.98
77.41
81.79
7B
PDrop
68.59
69.85
76.12
78.15
81.79
Ours
74.48
76.37
78.69
79.04
81.79
Fast 54.73
66.32
71.13
74.05
77.41
3B
PDrop
57.20
65.92
71.58
74.79
77.41
Ours
63.66
68.47
72.45
74.51
77.41
Table A7: MME total scoresunder different visual token retention ratios. Higher is better.
Model
Method
11%
22%
33%
50%
100%
FastV
1536
1924
2124
2316
2432
32B
PDrop
1536
1797
2078
2279
2432
Ours
2015
2189
2288
2378
2432
Fast
1970
1990
2240
2257
2291
7B
PDrop
1982
2064
2218
2236
2291
Ours
1990
2135
2275
2270
2291
FastV
1620
1800
1953
2053
2139
3B
PDrop
1692
1709
1986
2139
Ours
1763
1838
1977
2037
2139
B.2 Computational Complexity
We follow PyramidDrop [62] to compute the theoretical floating-point operations (FLOPs) introduced in the LLM decoding layers during the pre-filling stage for processing the visual tokens. Specifically, in each of the K decoding layers, self-attention calculation with a causal mask is applied, followed by multiple feed-forward network (FFN) layers. The total FLOPs can thus be computed as:
Total FLOPs = =x (Aned + 2n + 3ngdm),
K
(6) where K the number of transformer layers, ng is the number of visual tokens at LLM layer k. d is the hidden state size, and m the intermediate size of the FFN. This calculation suggests that reducing the number of visual tokens can significantly decrease the FLOPs required during inference.
B.3 Experimental Design for Study 3
The goal Study 3 determine at which LLM layers the next-token predictions stabilize, i.c., when the model's forward computation becomes semantically saturated. Once the prediction has converged, further layers mainly refine numerical precision rather than introduce new semantic information. Understanding this stabilization point is crucial because pruning before convergence can disrupt semantic formation, whereas pruning after convergence is largely safe and provides meaningful computational speedups.
Here our experimental setup: For each decoding layer in the LLaVA-L.5-77 LLM backbone, we extract hidden state h, and pass it through the original final lincar projection layer obtain the logits, i.c., ss W hy. We then apply a softmax compute the prediction confidence and report the top-confidence token for each layer.
22

---

Published in Transactions on Machine Learning Research (01/2026)
You intelligent chatbot designed for evaluating the correctness of generative outputs for question-anse answer pairs. Your task is to compare the predicted answer with the correct answer and determine if they match meaningfully. Here's how you can accomplish the task:
#INSSTRUCTIONS:
Focus the meaningful match between the predicted answer and the correct answer.
Consider synonyms or paraphrases as valid matches.
Evaluate the correctness of the prediction compared to the answer.
User Input:
Please evaluate the following video-based question-answer pair:
Question: {question}
Correct Answer: {answer)
Predicted Auswer: (pred)
Provide your evaluation only as yes/no and score where the score is integer value between 0 and 5, with indicating the highest meaningful match. Please generate response Python dictionary string with keys 'pred' and 'score', where value of 'pred' is string of "yes" or 'no' and value of 'score' in INTEGER, not STRING. DO NOT PROVIDE ANY OTHER OUTPUT TEXT OR EXPLANATION. Only provide the Python dictionary string. For example, your response should look like this: ('pred': 'yes', 'score': 4].
form
Table B8: GPT-aided evaluation setup. We present the prompt and user input format for evaluating the LVLM responses in video underslanding lasks.
B.4 GPT-Aided Evaluation on Video Understanding Tasks
Following the evaluation protocol of Video-LLaVA, we employ GPT--35-Turbo to assess model responses on vidco undorstanding tasks, evaluating them based on both accuracy and quality score. Specifically, we adopt the prompt shown in Table B8 to guide GPT in rating each response:
Accuracy: A binary yes/no judgment indicating whether the response correct.
Score: An integer ranging from 0 to 5, where 5 represents the highest degree of relevance and informative- ness and indicates the highest meaningful match.
C License Information
We list the license information for all the used assets follows.
Benchmarks. We evaluate on a comprehensive set of 16 benchmarks spanning image QA, video QA, multimodal reasoning, and referential undorstanding tasks:
GQA [25]: compositional visual question answering. This benchmark is released under the CC BY 4.0 license.
ScienceQA [43]: multimodal science questions with diagrams and text. This benchmark is released under the MIT license.
VQAv2 [23]: real-world image-
image-based based QA with balanced answers. This benchmark is released under the CC BY 4.0 license.
Text VQA [53]: reading and reasoning over scene text images. This benchmark is released under the CC BY 4.0 license.
23

---

Published in Transactions on Machine Learning Research (01/2026)
VizWiz [24]] real-world visual questions from blind users. This benchmark is released under the CC BY 4.0 license.
MMBench [42] / MMBBench-CN [42]: multilingual multimodal reasoning. These benchmarks are released under Apache License 2.0.
MME [20]: fine-grained multimodal evaluation object, OCR, and commonsense. This benchmark is released under Apache License 2.0.
POPE [35]: probing object hallucinations in vision-language models. This benchmark is released under the MIT license.
RefCOCO / RefCOCO+ [30], RefCOCOg [46]: referential expression grounding. These benchmarks are released under Apache License 2.0.
TGIF [27]: video QA with spatiotemporal reasoning. The license of this work is not specified.
MSVD [10]: short video captioning and QA. This benchmark is released under the MIT license.
MSRVTT [63]: large-scale video-text retrieval and QA. This benchmark is released under the MIT
license.
ActivityNet-QA [66]: complex event-centric video QA. This benchmark released under Apache License
2.0.
Models. We apply approach to four widely used LVLMs.
LLaVA-155 [39] is released under the LLaMA 2 Community License.
LLaVA-NeXT [40] is released under Apache License 2.0.
Qwen-2.5-VI VL [5] is released under Apache License 2.0.
VideoLLaVA [36] is released under Apache License 2.0.
Code. Our codebase builds upon PyramidDrop [62], licensed under MIT, and FasterVT.M. [70], licensed under Apache 2.0.
D Limitations and Broader Impacts
Limitations. One key limitation of this work is the inherent trade-off between efficiency and accuracy: while the proposed VScan significantly reduces inference cost of LVT.Ms, aggressive token pruning may still distort visual information and lead to degraded performance, particularly on challenging tasks that demand fine-grained understanding or compositional reasoning.
Broader Impacts. The development of efficient. LVL.s has significant potential influence a wide range of applications, from autonomous systems and robotics healthcare, education, and accessibility, By optimizing visual token reduction with VScan, with processing large visual inputs, enabling faster and more efficient inference in real-time applications. This can lead to more widespread adoption of LVLMs in settings where rapid decision-making is crucial, such as autonomous vehicles, real-time video analysis, and interactive Al systems.
addressing the computational overheads associated
24


Reviews:
[{"review_id": "UR1XQFnrrY", "reviewer": "Reviewer_k8qv", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "**Summary**\n\nThis paper addresses the high computational cost of recent LVLMs, which arises from increasingly long visual token sequences produced by fine-grained visual encoding. The authors first conduct an empirical analysis of how visual tokens are processed across both the vision encoder and the language model, revealing limitations in existing pruning strategies that operate either only at the encoder output or at early LLM layers. Guided by these observations, the paper proposes VScan, a two-stage visual token reduction framework. The first stage integrates global and local scanning with token merging during visual encoding, while the second stage prunes redundant visual tokens at intermediate layers of the LLM. Extensive experiments on four LVLMs and sixteen benchmarks demonstrate that VScan achieves substantial inference acceleration with minimal performance degradation.\n\n**Strengths**\n1. The paper is clear and practical, proposing VScan to balance token importance in vision–language models efficiently. The manuscript is well-structured and presents the method and results in a coherent and accessible manner.\n\n2. Experimental validation is sufficient. The authors conduct comprehensive experiments on various tasks, demonstrating improvements to validate the effectiveness of the method. The ablation and empirical study are detailed.\n\n3. The visualizations presented in the Appendix are impressive and highly detailed.\n\n**Weaknesses**\n\n1. VScan performs pruning in both the visual encoding stage and the language model stage, which raises two concerns: first, some vision encoders do not provide a [CLS] token, such as SigLIP; second, pruning during the language decoding stage makes multi-turn dialogue impossible. This contradicts the claim that the method “can be seamlessly applied to various open-sourced LVLM architectures.”\n\n2. In Section 3, the authors use POPE and GQA as illustrative examples for analysis, but the conclusion drawn in Study 3 appears to differ from the earlier observations. I suggest adding at least two more datasets to support the conclusion and ensure its robustness.", "claims_and_evidence_yn": "No", "claims_explanation": "1. It appears that the reported accuracies for several baseline methods in the experiments may have been underestimated. For instance, SparseVLM's GQA accuracy under the 192 setting is reported as 57.6 in the paper, whereas the publicly available result is 59.5, which is a substantial discrepancy. Please re-check the evaluation pipeline and report the exact evaluation settings used for each baseline.\n\n2. This paper feels like a combination of several existing ideas, and I find that it lacks a clear and distinctive motivation. Global Scan is derived from VisPruner and VisionZIP, Local Scan is from PruMerge+, token merging is based on ToMe and SparseVLM, and reducing textual irrelevance via middle-layer pruning is taken from SparseVLM. Although there are some minor differences, the overall ideas are highly similar.", "audience_yn": "Yes", "audience_explanation": "In the Empirical Analysis section, the authors provide several visualizations and perspectives on visual-token pruning, which do offer some useful insights. Although these analyses are somewhat similar in spirit to FastV, the conclusions drawn here differ from those of prior work.", "requested_changes": "1. In the Efficiency Analysis section, the comparison is only made between the proposed method and the original model. Please include comparisons with the other methods used in the main experiments, especially in terms of TFLOPs and latency.", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "FjV1aMFcKc", "reviewer": "Reviewer_YT24", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents VScan, a general and dynamic approach for compressing visual tokens in Large Vision-Language Models (LVLMs). Unlike most prior works on visual token compression, VScan is grounded in thorough empirical analysis and addresses two key limitations of existing methods:\n\nOver-reliance on global compression: Previous methods typically perform a global token reduction based solely on the CLS token’s attention weights from the vision transformer. Such global reliance often overlooks fine-grained visual details, which are crucial for tasks like grounding-oriented visual question answering (VQA).\n\nLack of text-aware guidance: Many existing approaches conduct compression purely within the vision modality, neglecting the important role of textual context in determining which visual tokens are semantically relevant.\n\nTo overcome these issues, VScan introduces a two-stage compression framework. In the first stage, it performs an iterative global–local scan within the vision transformer, pruning redundant visual tokens while retaining both key global representations and essential local details. In the second stage, it applies text-guided filtering by leveraging attention between the remaining visual tokens and the final text guidance tokens, further removing redundant visual information.\n\nExtensive experiments and ablation studies demonstrate that VScan consistently outperforms recent visual compression methods for LVLMs, maintaining comparable or even superior performance to non-compressed baselines at equivalent compression ratios.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The methodology of VScan is well-motivated by comprehensive empirical analyses. The authors first demonstrate that as layers in the vision transformer deepen, both the [CLS] attention maps and the self-attention maps of representative visual tokens exhibit a local-to-global transition. This observation reveals that relying solely on the [CLS] token from the final layer causes the model to lose sensitivity to local visual details. Furthermore, the text-guided filtering stage is well-justified through analyses of LLM position bias and attention patterns, which motivate the design choice of using mid-level LLM layers for refining and pruning vision tokens. Finally, the effectiveness of VScan is thoroughly validated through extensive experiments and ablation studies on widely used benchmarks, consistently demonstrating its advantages over existing baselines in visual token compression for LVLMs.", "audience_yn": "Yes", "audience_explanation": "As discussed above, VScan is a well-motivated and generally applicable approach to visual token compression in LVLMs. It demonstrates strong performance across standard benchmarks, and with the authors’ commitment to releasing the code upon acceptance, VScan has the potential to serve as a solid baseline for future research in this area.", "requested_changes": "Questions for Clarification and Discussion\n\n1. Compatibility with KV Cache and Inference Optimization:\nIs VScan compatible with commonly used inference optimizations such as KV cache and FlashAttention? My understanding is that, although the proposed method effectively reduces the number of visual tokens, it may alter the cache structure used during autoregressive decoding. This could potentially limit the benefits of token reduction, as it might break cache reuse and lead to higher inference overhead or memory consumption. It would be helpful if the authors could clarify whether VScan can be integrated with these optimizations in practice, and if not, to discuss the trade-offs and possible solutions.\n\n2. Choice and Scope of Evaluation Benchmarks:\nWhile evaluating VScan on standard benchmarks is appropriate for comparison with prior works, the field of vision-language modeling has rapidly evolved, and many existing benchmarks have become less discriminative due to the increasingly strong text priors of modern LLMs. In some cases, these benchmarks can be solved largely from text alone, diminishing the significance of visual compression performance. Therefore, I would encourage the authors to discuss or explore more challenging, spatially grounded benchmarks that heavily depend on visual understanding. This would better demonstrate the extent to which VScan preserves true vision-dependent performance and highlight its strengths beyond simple compression gains.", "broader_impact_concerns": "No concerns are identified regarding the ethical implications of this work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ZClJZ8KxRm", "reviewer": "Reviewer_AvVV", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "In this paper, the authors analyze existing efficiency VLM methods and propose VScan, a two-stage visual token reduction framework. The first stage integrates complementary global and local scans with token merging during visual encoding; the second stage prunes tokens at intermediate layers of the language model. Experiments across four VLMs show that VScan accelerates inference and outperforms strong baselines, while maintaining competitive accuracy.", "claims_and_evidence_yn": "Yes", "claims_explanation": "As a researcher in this field, I have observed similar findings to those reported by the authors; therefore, I find the paper both convincing and clearly presented.", "audience_yn": "Yes", "audience_explanation": "The topic of Efficient VLMs is highly relevant to real-world applications and has gained significant attention in recent years.", "requested_changes": "1. Firstly, I think the authors’ empirical analysis is very interesting and well written, and I believe these analyses could further advance research in this field. However, regarding **Study 3: At which LLM layer do next-token predictions begin to converge?**, I am unclear about its purpose, experimental design, how the conclusions were derived, and how these conclusions contribute to subsequent method design. I hope the authors can elaborate further.\n\n2. I fully understand that due to the use of the Patch Merger, the improvement of the Efficient VLM method on the Qwen2.5VL model might not be very significant. Nevertheless, I still encourage the authors to present the Qwen2.5VL results in a **table format similar to Table 4**, rather than only using a line chart, as I believe this would better promote progress and development in the field.\n\n3. Since the entire method consists of the **text-agnostic** part in Section 4.1 and the **text-aware** part proposed in Section 4.2, I hope the ablation study can further clarify how each part respectively affects **speed** and **performance**. A deeper discussion of their respective advantages and disadvantages would make the paper much more meaningful.\n\n4. For **Table 5, Table A2, and Table A3**, I think that since the authors discussed many **text-agnostic** and **text-aware** methods in the introduction and writing of the paper, I suggest that in a future version, the benchmarks should also include **both categories** of methods, rather than comparing only one. This would more convincingly demonstrate the effectiveness of the proposed approach.\n\n5. In paper [1, 2], the researchers discussed that in most general scenarios, even simple resizing can achieve strong performance. How do the authors view this issue? I look forward to some discussion on this point.\n\n[1] VisionThink: Smart and Efficient Vision Language Model via Reinforcement Learning\n\n[2] Are We Using the Right Benchmark: An Evaluation Framework for Visual Token Compression Methods", "broader_impact_concerns": "None concern", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)