Script: Graph-Structured and Query-Conditioned Semantic Token Pruning for Multimodal Large Language Models
Authors: Zhongyu Yang
Dannong Xu
Wei Pang
Yingfang Yuan
Venue: Accepted by TMLR ()

Abstract: The rapid growth of visual tokens in multimodal large language models (MLLMs) leads to excessive memory consumption and inference latency, especially when handling high-resolution images and videos. Token pruning is a technique used to mitigate this issue by removing redundancy, but existing methods often ignore relevance to the user query or suffer from the limitations of attention mechanisms, reducing their adaptability and effectiveness. To address these challenges, we propose Script, a plug-and-play pruning method that requires no retraining and generalizes across diverse MLLMs. Script comprises two modules: a graph-structured pruning module that removes visually redundant tokens, and a query-conditioned semantic pruning module that preserves query-relevant visual information. Together, they enhance performance on multimodal tasks. Experiments on fourteen benchmarks across image and video understanding tasks show that Script consistently achieves higher model efficiency and predictive accuracy compared to existing pruning methods. On LLaVA-NeXT-7B, it achieves up to $6.8\times$ prefill speedup and $10\times$ FLOP reduction, while retaining 96.88\% of the original performance.

Published in Transactions on Machine Learning Research (11/2025)
Script: Graph-Structured and Query-Conditioned Semantic Token Pruning for Multimodal Large Language Models
Zhongyu Yang"
zy4028 uk danielauf1208 @gmail com w. w.pangghhwwcco ac. uk y. U-yuon @har ac. uk
BCML, Heriot- Wott University
Dannong Xu"
BCML, Heriot- Wolt University
Wei Pang
BCML, Heriot- Watt University
Yingfang Yuan
BCML, Heriot- Watt University
Reviewed OpenReview: https: httts:://ppenrevieu net/ forum? id= P66KkbgccH
openreview.
Input Image
Attention-based Pruning
Similarity-based Pruning
Divergence-based Pruning
Script [Ours]
Q: there
What is placed beside the limes
Description
Q: there flower on the table
wide-angle photo of modern, open-concept kitchen living foreground, a white kitchen island features black stovelop. with silver pot, chrome faucet, and plate of limes and pineapple. Recessed lighting and exposed space. To the left, wooden dining table with transparent chairs yellow flowers. the for gray sectional sofa faces floorto-calling windows, framing a stunning sunset and ailhouetted trees.
the
beams brighten
Q: How many fruits
for meal, centered with
the kitchen
Figure 1: Comparison of different token pruning methods. Attention-based and similarity-based methods prune tokens using attention scores and similarity scores, respectively. In contrast, divergence- based methods detect changes in model performance and retain tokens that cause minimal impact. Script (Graph-Structured and QueRy-CondItioned Token Pruning) combines graph-structured reduction of visual redundancy and query-conditioned semantic token selection to enable efficient pruning in MLLMs. In this example, Script successfully preserves key visual cues, such as the silver pot on the stove, the pineapple beside the limes, and the flowers on the table. Other methods fail to retain consistently.
Abstract
The rapid growth of visual tokens multimodal large language models (MLLMss) leads to ex- cessive memory consumption and inference latency, especially when handling high-resolution images and videos. Token pruning is technique used to mitigate this issue by removing redundancy, but existing methods often ignore relevance to the user query suffer from the
Equal contribution Correspond Author

---

Published in Transactions on Machine Learning Research (11/2025)
limitations of attention mechanisms, reducing their adaptability and effectiveness. To ad- dress these challenges, we propose Script, a plug-and-play pruning method that requires no retraining and generalizes across diverse MLLMs. Script comprises two modules: a graph- structured pruning module that removes visually redundant tokens, and a query-conditioned semantic pruning module that preserves query-relevant visual information. Together, they enhance performance on multimodal tasks. Experiments on fourteen benchmarks across image and video understanding tasks show that Script consistently achieves higher model efficiency and predictive accuracy compared to existing pruning methods. On LLaVA-NeXT- 7B, it achieves up to 6.8x prefill speedup and 10x FLOPP reduction, while retaining 96.8%% of the original performance.
1 Introduction
Recent advances in large language models (LLMs) (Tuvroon et al. 2023 Bai et al. 2023 Yang et al. 2024
2025a) have substantially enhanced language understanding and reasoning, forming the backbone of general- purposc multimodal systems, including vision-language models and embodied agents operating real-world scenarios. Building on this progress, multimodal large language models (MLLMs) (Li et al.) 2024a Chen iet al. 2024d|c cj Zhang et al., 2025) extend LLMs by integrating vision encoders, enabling joint reasoning over both visual and textual modalities. This integration empowers MLLMs to excel at diverse vision-language tasks such as visual question answering and image captioning.
However, deploying MLLMs in practical scenarios such as mobile agents or interactive assistants is con- strained by the high computational cost of handling high-rcsolution or temporally extended visual inputs. Patch-based vision encoders typically tokenize each visual input into hundreds or even thousands of to-
kens [Loo et al. 2024 Guo et al. 2024 Zhang et. al., 2023 Chen et al. 2024b Li et al. 2025b Liu et al., 2025 In contrast to compact textual inputs, visual representations often contain hundreds or thousands of tokens, which increases memory usage and inference latency. The explosion of vision tokens, compounded by the quadratic complexity of attention mechanisms (Vaswani et al. 2017); creates a signif- icant bottleneck for latency- and memory-sensitive scenarios such as mobile deployment, online inference, and based vision-language applications. To mitigate this inefficiency, visual token pruning has emerged as a promising strategy. An effective pruning method should minimize computational cost while preserving task performance within resourco-constrained environments.
2025 Bai et al.
Existing visual token pruning methods can be broadly categorized into three main paradigms: (1) attention- based methods, which retain tokens with high
importance, commonly referred to as attention scores (Chen et al. 2024a Xing et al. 2025 Zhang et al.) (0; (2) similarity-
methods, which identify
and climinate redundant visual tokens based on feature similarity [Bolya et al. 2023 Zhang et al., 2024b
Wen et. al.] 2025b Jeddi et al.) 2025); and (3) divergence-mininization methods, which prune tokens by minimizing the change in the model's output (Alvar al. 2025 Ye et al., 2025).
However, existing pruning methods face two fundamental challenges in query-condittoned scenarios. First, attention-based approaches depend on raw attention scores, which are susceptible to the attention sink issue (Bar- bero al.] 2025), often overlooking critical to- kens. Moreover, assigning similar scores to ad- jacent semantically similar tokens can re-
Purfomance Prefil Time KV Coche FLOP, b
[b)
NAL
NHB OR
LoVANeXT-7B 41.7
Scrigt 4:17
Reduction: 10x
POPS
LoVA-NeXT-78 1440.0
Script 169 OMB Reduction: 9x
KQA JM
GOA
LoVANNeTT-BB 245ns
Script
Reduction: 6.8x
duce pruning efficiency (Yang et al. 2025b).
VOWZ
MPYAT
As shown in Figure [ such methods fail to preserve the token representing the flower, de- spite its clear relevance to the query. Second, similarity- and divergence-based methods lack explicit query conditioning (Alvar al.] [2025] Li et al.) 2025a). Although they address visual
LLV 100%
FULE
Script
VOVAS
Figure 2: (a) Efficiency Analysis on LLaVA-NeXT-TR un- der 88.9% reduction. (b) Comparison with other baselines on 1A-155----77 under 94.4% reduction.
2

---

Published in Transactions on Machine Learning Research (11/2025)
redundancy or output stability, they generate fixed token subsets regardless of the imput query, which can lead to the omission of query-referencel objects, such as the pineapple and lime shown in Figure 1
To address these limitations, we propose Script, a training-fres frre and architecture-agnosii token pruning approach that combines graph-structured pruning (GSP) and query-conditioned semantic pruning (QCSP). On one hand, we build a bipartite graph to structure visual token redundancy, effectively identifying redun- dant tokens with lower computational cost. On the other hand, to eliminate reliance on attention scores and avoid issues such attention sink, Script explicitly models interactions between the query and visual tokens, identifying query-relevant tokens using Determinantal Point Processes (DPP), probabilistic model that favors diverse and scmantically meaningful subsets. Together, these two modules enable effective token pruning by jointly considering visual redundancy and input query relevance. As illustrated in Figure Script trims 88.9% tokens yet preserves 99.88% accuracy on POPE, and consistently outperforms other baselines on 10 benchmarks. In summary, our contributions are as follows:
We identify the core limitations of existing pruning paradignn in query-conditioned scenarios, high- lighting their inability to adaptively preserve task-relevant content.
We propose Script, a training-freo pruning method that combines graph-basod rodundancy reduction with query-aware token selection via DPP.
Experiments across fourteen benchmarks demonstrate that Script consistently overperforms other baselines, and achieves up to 10x speedup while retaining 96.88% of original performance.
2
Related Work
Scalability Challenges in MLLMs. MLLMs combine visual and textual modalities to support general-
purpose perception and reasoning Alayrac et. al. 2022: Zhu el al. 2024 Chen et. al. 2023 Yu et al.
2024 Liu et al.] 2024b Cai et al., 2024 However, visual inputs introduce substantial token overhead
due to spatial redundancy and high dimensionality, exacerbating the quadratic complexity of Transformer attention. For example, LLa-1-1.5 (Liu et al.) 2023) generates 576 tokens from a 336 336 image, which is already 4-5x longer than typical text-only prompts. High-resolution models like L.AVA-NeXT {Liul ct al. 2024a and video models like LongVA (Zhang et al. 2024a) push this further, producing tens even hundreds of thousands of tokens per input, leading to untenable memory and compute demands for many downstream applications. Consequently, such models often encounter memory bottlenecks, latency spikes, or even inference failures, especially under real-time or edge deployment scenarios (Papa et al. 2023 Li ct al. 2025e3, where computational resources are inherently limited. These challenges have motivated a growing body of work on token pruning, aiming to reduce input size while preserving task-relevant semantics and ideally without retraining or architectural changes.
Token Pruning Strategies. Efforts to mitigate visual token overhead in MLLMs can be broadly cate- gorized into four approaches, each with distinct assumptions and limitations. (1) Pre-fusion compression, which downsamples or selects tokens before vision-language fusion {Li et al. 2024b Hu et al. 2024). often requires model retraining or structural modifications, hindering plug-and-play usage. (2) Attention- based
pruning, which selects tokens using attention scores (Chen et al. 2024a Ye et al. 2025) suffers from at-
tention drift [Wen et al. 2025a), misguiding token importance, and limiting compatibility with optimized
backends such as FlashAttention (Dao) 2024]. (3) Pre-language fusion pruning, which drops tokens before language-level alignment (Shang et al. 2024 Song et al. 2025), is often tightly coupled to specific vision backbones, limiting transfcrability across architectures. (4) Similarity- based pruning, which climinates re- dundant tokens using intra-image feature similarity (Wen et al.] 2025b, Alvar et al. 2025), is generally efficient and model-agnostic, but often ignores query semantics, leading to the loss of task-relevant. content.
While existing approaches partially alleviate visual redundancy, few simultancously achieve the trifecta of query relevance, architectural generality, and training-free free deployment. In contrast, our proposed Script is explicitly query-aware, visual diversity, entirely model-agnostic, and operates without any additional training.
3

---

Published in Transactions on Machine Learning Research (11/2025)
Average Cosine Similarity
0.00
Local Neightors Random Pacoh
1.0
0.8
0:40
Derstty
0.6
0.4
Manhattan Distance
Cosine Similarity
64
0.0
(a]Average Cosine
(b)Similarity Distribution:
(c) All Images Entropy Heatmap
(d) All Images Similarity Heatmap
Similarity Spatial Distance
Local vs Random
Figure 3: Token redundancy visualized via similarity and entropy on 10,000 COCO images.
3 Preliminary
In this section, within the context of the task Text Visual Question Answering VQA), we empirically investigate two key aspects of visual tokens MLLMs: visual redundancy and query-conditioeed relevance. In addition, we present a theoretical formulation efficient query-conditoone token selection. The insights from both empirical and theorctical analysis inform the design of our Script.
3.1 Research Objective
lypically comprise three components: vision encoder fv. multimodal projector 9, and a large language model fg. After fo processes an input image X,, the projector maps the features into a visual-token sequence H, E Rexd where n is the token count and d is the token dimension. The language model fo processes the visual tokens H, and the tokenized query H, to generate an answer grounded in the image. Typically, n >> HH,), i.e., the visual sequence much longer than the textual query. Equation [ defines vision-token pruning by seeking the subset H; that preserves model performance while using the fewest tokens.
Existing MLLMs (Liu et al.
2023 2025
Xu et al. 2025)
H = arg min
C (Jo((H,; (,). H,)))
(1)
H.C.., |..==
Here, L mcasures the discrepancy between model outputs before and after pruning, serving a proxy for performance loss. The candidate subset. H. C H, contains m lokens, with m < n. To ensure query-aware pruming, it essential to explicitly capture the semantic interactions between visual tokens H, and their corresponding query tokens H,, These query-specific interactions are crucial for identifying the most query- relevant subset H.. At the same time, assessing the visual redundancy between H, and the full set H, is vital for promoting diversity in token selection, which is important for generalizing across wide range of tasks. Together, these two aspects contribute to forming an informative and compact token subset.
3.2 Rethinking Visual Redundancy
In natural images, spatially adjacent tokens often encode highly overlapping content, resulting what we refer as local redundancy. Carcfully identifying and removing such redundancy can significantly reduce compulation without compromising accuracy.
Table Comparison of redundancy estima- tion methods with a 90% pruning ratio on the POPE Benchmark. Sim. (%): Percentage of similarity pairs considered. Time: Inference time in milliseconds. IOU@K: Intersection- over-Union with top-K selections. Acc.: Clas-
To investigate local redundancy, we adopt the approach from (Wang et al.) 2024] and compute Manhattan dis- tances and cosine similarities between vision-token pairs. We group similarity values by distance and calculate the average for each group. As shown in Figure [a), cosine similarity is highest for nearby tokens and decreases until
sification accuracy (%). Specdup: Rolative time speedup compared to Full Graph.
Method
Sim. (%)
Time IOU@K Acc.
Speedup
Exhanstive
100
100
1.00
85.14
1.00x
Bipartite Random
SU
35
0.93
84.49
2.81x
34
0.52
78.63
2.82x
4

---

Published in Transactions on Machine Learning Research (11/2025)
distance of roughly 15. Beyond this, we observe that similarity increases again at longer distances, indicating that long-range token redundancy also exists and deserves attention. Furthermore, we visualize the similarity distributions for neighboring tokens (hop 1) and randomly sampled neighbor token pairs (hop > 1). As shown in 4 b), the number of neighboring tokens with similarity greater than 0.5 is consistently higher than that of random token pairs. These results indicate that neighboring (ie.. local) lokens encode more similar visual features, but long-range redundancy should also be taken into account, as it clearly exists.
Insight 1: Redundancy Exists Beyond Local Neighborhoods
Spatially adjacent tokens show high similarity (local redundancy), yet long-range pairs still share appre- ciable similarity, revealing long-range range redundaney. Effective pruning must. therefore model both local and long-rann range redundancy to avoid information loss.
To further validate local redundancy and its estimation via cosine similarity, we introduce local information entropy as distribution-level measure of neighborhood compactness and compare it with cosine similarity. For each token, we extract its 3 3 Moore neighbourhood N(v) = (h; |l - ppos())l S 1 (hop=1). We then project all vectors in the neighborhood onto their first principal component and discretize the resulting scalar values into 20 equal-width bins to estimate the probability distribution {p:}. The local information entropy defined as:
Ho. Σ
ps log Ps:
where = Pt + é and < 10-8. compact neighborhood typically exhibits low He, as its token repre- sentations are similar and concentrated in a few bins. In contrast, a diverse neighborhood naturally yields higher H., reflecting greater variation among neighboring tokens. As shown Figures 3 c) and (d), regions with low local similarity typically exhibit high entropy, confirming local redundancy and validating cosine similarity effective proxy.
Insight 2: Cosine Similarity a Valid Redundancy Proxy
Local information entropy inversely correlates with cosine similarity. Low-entropy (predictable) regions coincide with high neighbor similarity.
Motivated by the validated role of cosine similarity in capturing visual redundancy and the need for compu- tational efficiency, we design a lightweight bipartite graph-based estimator that approximates both local and long-range redundancy using cosine similarity alone. This estimator forms the core of the graph-structured pruning module detailed in Section 4.1
Insight 3: Lightweight Bipartite Graph Yields Scalable Redundancy Estimation
By weighting edges with cosine similarity, the even odd bipartite graph achieves redundancy estimation comparable to exhaustive computation, but with significantly lower computational cost.
Tokens represented as nodes, divided into even- and odd-indexed sets; each node in one set connects to all nodes in the other, with edge weights defined by cosine similarity. These weights enable efficient redundancy estimation and guide token removal. As shown in Table [ empirical results confirm the effectiveness of the bipartite graph-based approach. On the POPE Benchmark with a 90% pruning ratio, it achieves an accuracy of 84.49%, which close to that of the exhaustive approach (85.14%) while being nearly three times faster. Mcanwhile, it retains 93% IOU agreement with the exhanstive approach, indicating that most key tokens are preserved. In contrast, the random approach is equally fast but drops 78.63% accuracy and only 0.52 TOU, failing to preserve semantically relevant. tokens.
5

---

Published in Transactions on Machine Learning Research (11/2025)
3.3 Rethinking Query Relevance
Recent studies (Zhang et al. 2024b Yang et al. 2025b) address visual token redundancy using text-agnostic pruning strategies that retain tokens with high CLS) attention scores from the vision encoder's final layer. However, these methods often fail to incorporate the query explicitly. They are also limited by inherent issues in attention mechanisms, such as the attention sink effect, where a disproportionate amount of attention is assigned to the first token in a sequence, regardless of semantic relevance. As result, these methods tend to preserve tokens from visually salient or high-attention regions. Mcanwhile, they may discard less prominent areas, such as backgrounds textures, which can still be crucial to answering the query.
This observation raises critical question: Is output-layer [CLS] attention sufficient to capture all query- relevant visual information? Empirical observations suggest such attention-based strategies often overem- phasize dominant foreground objects while neglecting background elements relevant to the query. As shown in Figure I attention focuses on the pot and fruit in the foreground but overlooks the actual target, the flower in the background. This results incorrect outputs. Moreover, both divergence- based and similarity-bsse approaches also fail adapt to the user query in this example.
Insight 4: CLS Attention Misses Query-Relevant Background Tokens
Attention-basee relevance measures tend to focus on visually salient foregrounds and can miss query-relevant background tokens, underscoring the need for explicitly query-aware relevance scoring.
3.4
Rethinking Query-Relevane Visual Token Selection
Let the query tokens be Hg (h)²),, J, and define their mean embedding as:
h
(4)
(3)
The relevance score qi of each visual token h, computed as the cosine similarity between h; and the mean query embedding h
h h
T; h
for i = ===1,..... I,...,
(4)
We construct diagonal matrix Q = diag(ri, Tn) E Raxa to encode the query relevance of each visual token. To encourage diversity among selected tokens, we define a similarity matrix S(#) e Rax", where s(0) measures the pairwise similarity between vision tokens h; and hy. We assume ( is symmetric and positive semidefinile, which approximately holds when using cosine similarity among normalized features. This formulation naturally aligns with modeling token selection as a k-DPP (Determinantal Point Process), which promotes subsets that balance individual relevance with collective diversity. We therefore construct the following k-DPP kernel (note that is a positive-semicee sentiifi-eem kernel, not loss functioon) to integrate both query relevance and feature diversity:
L 1/2 (212221)))/2
where Q explicitly encodes query aligmment and ((v) impliciltly encourages feature diversity.
Insight 5: (()) Promotes Diversity
For any subset I, det(Lr) equals the squared volume of the parallelotope spanned by the selected token vectors (see Appendix B). If ((v) were the identity matrix, this volume reduces I 9i, i.e., relevance only. Introducing off-diagonal similarities lowers the determinant when two tokens similar, thus maximizing det encourages mutually orthogonal (diverse) token subsets.

---

Published in Transactions on Machine Learning Research (11/2025)
A k-DPP assigns to each subset I S [n] of fixed size k probability proportional to det(Lz), the determinant of the principal submatrix of L indexed by Z. This determinant reflects the volume spanned by the selected vectors, and is larger when the selected tokens are both informative and mutually distinct. Expanding the determinant yiclds:
det(Lz) det(Q7 1/2 2 dct(Qq 1/223 det(s())
II WE
det(s
(6)
This decomposition highlights the trade-off between query relevance (via II g:) and visual diversity (via )ct(s Based on this, we define the final surrogate objective for selecting an informative token subset:
I = arg
max IC ||
II qi det(s) IET
Insight 6: k-DPP Balances Relevance and Diversity via Geometry
The k-DPP determinant can be analogized lo the feature-space volume spanned by selected lokens, thus optimizing it jointly maximizes query relevance and inter-token diversity in a single objective.
4 Method
Motivated by the empirical analysis in Section 3 introduce Script, illustrated in Figure Script progressively refines token selection through two main modules: (1) Graph-Structured Pruning (GSP), which removes visually redundant tokens using a bipartite similarity graph; and (2) Query-Conditionas Semantic Pruning (QCSP), which consists of two steps: Query-Conditioned Relevance Scoring (QCRS), computing the semantic relevance of each token with respect to the imput query, and Diversity-Preservii Selection via DPP, selecting a compact, diverse subset of relevant tokens. The outputs from GSP and QCSP are then intersected obtain the final token subset H..
4.1 Graph-Structured Pruning
Transformer-Dasse visual encoders often generate dense token sequences with substantial visual redundancy across both local and long-range contexts, as identified Section 3.2 To mitigate this, we propose an inferenco-time pruning strategy based on bipartite similarity graphs without relying model paramcters.
Bipartite Graph Construction. Given visual token embeddings H, [hi..... hn] gaxd construct bipartite graph by partitioning tokens into two disjoint node sets Vere and Vast via alternating index assignment (e.g,, even indices to Vsre odd-indexed to Vaaa) (Buchholz) 2024). The bipartite graph is defined as G (Varer Vast, S(r)), where each node in Vare is fully connected to every node in Vasts and the edge weight s (v)) s(1) represents the cosine similarity between tokens t, and ty:
h hdst
S(t) 1
(8) In this way, G explicitly structures token redundancy, with similarity scores providing insights into both local and global redundancy. Notably, this bipartite graph design reduces computational cost by up to 75% compared to conventional similarity-basod pruning methods. As empirically validated in Scction 3.2 most redundancy lies in local neighborhoods and is thus well-captured by the bipartite approximation effectively. Overall, Vuee and Vas jointly offer structured, low-cost representation of H, for redundancy modeling.
hare [
7

---

Published in Transactions on Machine Learning Research (11/2025)
Input Image
(a)Script
Vision Ercoder Projector
Large Language Model
Graph-Structured
Structure tokens
Pruning
Query-Conditfoned
Semantic
Semantic Pruning
tokens
Text Query
Is there bag in the desk
Tokenizer
(b GSP
(c)QCSP
Legend
src
dst
Textual Tokens
Rerank based
Visual Tokens
Redundancy e
Pruned Tokens
Feature Similarity
Query Relevance
Contact
src Source Node
DPP ?
dst Destination Node
Semantically Prune
Extract Visual Tokens
X
DFF
Determinantal Point Process
Figure Overview of Script, a stree-stage pruning frameworks (a) overall architecture; (b) Query- Conditioned Semantic Pruning (QCSP); (c) Graph-Structured Pruning (GSP). Together, these modules remove semantically irrelevant and visually redundant tokens through joint selection process.
Redundancy Scoring. To identify redundant tokens, we retain all similarity edges above threshold т, forming pruned subgraph G,, where each node is connected to highly similar neighbors. For token 4 Varc: define its degree returning if the condition is true and 0 otherwise. Nodes with high degrees in G, are identified structurally redundant, typically indicating that tokens represent repeated visual content in either local or long-range contexts. To further distinguish tokens with similar degrees, we use the average similarity p(tz) between token t and its neighbors in G,, which captures their typical similarity. For cases where tę is an isolated node in G,; we define fallback similarity 7(4) based on the mean similarity between 4 and all tokens in Vast Ver under the full graph The final redundancy score is then defined as:
d(t,) Σjevan I( 2 r), where II denotes the indicator function,
[a(ta) exp (7(4(4)) -)), d(4.) > 0 in G,, (a(t)) in G,
score(t.)
(9)
otherwise,
where y is a tunable scaling factor. The score, computed using d(t.)) and p(4;) with exponential weighting, is designed improve ranking contrast between highly and moderately redundant tokens.
Pruning Rate and Token Removal. Based on the computed redundancy scores, we rank all visual tokens in descending order and discard the top p most redundant ones, where p denotes the pruning ratio. This yields a structurally pruned candidate set H., which then combined with the token set semantically aligned with the query, produced in the next stage, to jointly form H
8

---

Published in Transactions on Machine Learning Research (11/2025)
Table 2: Performance comparisons on LLaVA-1.5-7B (Liu et al. 2023) across 10 image under-
standing benchmarks. The best results in each setting are bolded, and the second- best are underlined.
MMthod
Venue
GQA
MMB
MMB
MME FOPE SQA VQA"
VQA TM
ViEWE
Vet
Acc.
Itelative
Upper Bonnd,
Pokena (10ME)),
3.917 TELOPS
LLaVA 1.5 7B Liu DE
2023
Nips 23
61.94
64,099
$6.10
15117 .05
66.96
69.41
18.50
58.20
50.32
31.82
65.47
100%
Redson
198
Tokons in Averoge
06. (15).
-1.353
YFLOPs
V
al.
ECCY
63,40
54.04
14901.6
12.55
54.09
54.97
TRIM
COLING'S 155
58.18
59:19
61,02
14005.29
78.19
67.87
74.86
64.29
48.23
30.82
59.28
53.41%
Vision2.pp
20255
CVPR 25
50:24
61.71
59.52
1480.07
80.03
68.37
70.44
04.89
03.97
30.74
09.28
93, 4116
DioPrure
CUPRES
02.64
1496.39
03.38
SparseVL
[4mrng 00
ICML
58.54
63. 46
65.20
1450007
80.49
68.07
75.01
06.83
31.24
02.08
97.82%
Script (Duns)
Proposed
60.82
63.85
57.55
1493.87
$7.02
69.49
77.02
57.81
53.80
31.40
63..45
100.00%
Atsougt
77.588)
TELOP
V Chen 15 al.
202
BCCY'8
36.85 85
62.40 00.46
53.011
1440.11 10
80.24
68.00 68.67
71.68
62.21
55.51
28.07
60.15
94.74%
TRIM Seeeean 4i22
COLING'S 95
36.85
68.42
18.66
1:50 995
77.95
68.44
74.42
63.74
47.62
29.35
68.35
91.93%
Vidim22p
1225
CVPR 25
$7.87
62.20
53.18
1449..75
85.20
68.22
11.62
54.12
54.39
27.30
60.87
95.000 05..00%
DivPrune
CUPR'25
12.16
1412.07
09.01
51.92
56.21
SparseVLA LMM (22ning 1
2024c
ICUIL 25
58.41
02.75
50.48
1442.44 41
86.22
08.82
73.86
56.29
51.83
29.12
61.21
40%%
Script (Qurs)
Proposed
60.27
63.30
65.88
1-431 24
87.32
68.79
76.65
66.45
53.97
31.00
62:51
98.49%
Tolona
Averoge
V
Chre e5 al. 20240
BCCY'24
$3.50
50.02
50,08
1306 55
75.34
68.11
58.71
51.62
54.74
27.06
56.72
59.36%
IRIM Socg
BOLING
56.30
66.08
46.39
12288 251
77.50
68.22
73.30
51.09
43.84
22.22
06.58
40.14%
Visiam21p
20125
CVPR'25 25
55.47
00.82
$1.29
81.88
71.59
52.72
51.81
50.32
58.465
DivPruse
CUPR25
57.24
000228
1966 525
86.61
08.22
72.35
64.61
67.66
27.38
60.12
93.14%
SporsWTM (Aain, 16 72
REMI 155
53.80
00.14
50.68
1705 16
80.02
69.46
68:24
53.67
50.11
21.92
57.67
60.66%
Scripl (Qurs)
Proposent
60.28
61.90
62:99
1412.08
86.955
68.65
75.00
65.20
54.31
29.96
61.44
96.88%
ReGatn
Tullen
Averuue
11. 016)
V
Chee ot al.
302
ROCV'
40.61
54.64
43.90
118..55
50
68.26
56.48
49.85
54.22
24.50
62.66
82.60%
TRIM
COLING'S
$4.35
$0.31
45.27
1215.02 .2
77.54
68.22
62.54
19.12
38.41
25.99
54.05
15.1
Visimn2p
CVPR22
58.65
1306
55.47
90.178
DivPrune
CVPR'2
$5.11
38.95
35,99
13013.25
53.93
08.57
09.41
51.46
57.26
25.75
87.16
SparsWVLL
LT (Zhing 14
272
ICML
51.74
50.64
08.64
1181.05
71.27
69.35
65.25
51.28
69.62
20.36
55.92
Script (Ours)
Proposed
57.58
61.28
1138.32
53.10
44..4%%
15 Tolone i0 Average
72
V
Ches et al.
BCCY'22
44.08
41.41
32.014
808.85
59.89
68.51
522005
42.15
52.16
16.24
45.35
71.45%
TRIM
COLING
45.28
38.46
1085.06 06
61,48
35.25
49.39
77.82%
Visinnlpp
Tang
CUPR
50.88
02.23
42.61
1179
76.11
66.95
02.48
45.77
54.78
20.68
63.00
53.68%
DioPruee
20yar 2 m0
CVPR22
51.10
52.09
31.76
1226.77
76.38
69.21
65..00
41.15
56.04
19.89
612.29
82.18%
Spars/VT
46.16
64.16
12.83
967.55
69.00
68.26
68.72
41.78
40.47
21.84
60.17
79.0458
Script (Ours)
Propoced
37.02
1258
00.12
47.65
23.71
4.2 Query-Conditiones Semantic Pruning
While the GSP module eliminates visually redundant tokens, it does not guarantee semantic alignment with the user query. To address this limitation, we introduce the QCSP module, which comprises QCRS for each token and diversity-preservin selection via DPP based on the relevance scores.
Query-Conditioned Relevance Scoring. We begin by computing an average query embedding h via mean pooling over Hg. Each visual token h, is then scored by cosine similarity to h(²), yielding a raw query relevance vector Traw |rı //....ral rn] (sce Eq. 4 To ensure consistency across samples, we apply min-max
normalization to obtain Fnorm E [0. 1)".
Query-Conditioned Kernel Construction. To support diversityy selection of query-relevant tokens, we construct a query-aware DPP kernel that integrates token relevance with visual similarity. Im- portantly, visual similarity in this context helps refine query-relevant token selection rather than explicitly promote visual diversity, which is addressed by the GSP module.
Specifically, let 5²(e) Rax be the cosine similarity matrix of Ey-normalized visual token embeddings:
h
h
= h
(10)
Then, we define a query-conditioned kernel E Ruxn by reweighting the pairwise visual similarity matrix (r(() with normalized token-to-query relevance scores Fnorm: L diag(Fnorm))- S'(v) -"(o)) -diag(nnom)- Intuitively, É captures diversity within token relevance, promoting token selections that are both query-aligned and diverse. Its symmetric and positive semi-definite structure makes it well-suited for k-DPP-based selection H
kk is not tunable hyperparameter but a direct control of the number of retained tokens. It conceptually equivalent to the pruning ratio p GSP, with the correspondence = (1 - p) n. where the number of original tokens.
9

---

Published in Transactions on Machine Learning Research (11/2025)
Table 3: Performance comparisons on LLaVA-NeXT-7B (Liu et al., 2024a) across 9 image under- standing benchmarks. The best results in each setting are bolded, and the second best are underlined.
Method
Venue
GQA
MMB
MMB
MME
POPE
VQA":
VQA ".
ViaWis
MMVel
Acc.
Relative
Upper Boand, 2,800 Tolons (1905),
20.855 TELOPs
LIVA--N-NT-75
(Tin et
f0
CVPR
62.63
65.51
57 65
1504 72
55.92
67.50
61.20
60.35
55.05
41.11
65:43
100%
Rodadn 641! Tolana Bu Aseruge
77.8%).
4.007
TPLOPs
FastV
2124
BCCV'4
60.14
1412.56
79.96
67.32
77.06
58.15
02.22
55.09%
TRIM
Song ml
2025
COLING'S
01
05.85
55.70
1473:62
96.92
66.71
78:36
54.85
54.12
37.10
63.46
96.95%
V Vakaul4p
Yaur 4t al
CVPR'25
00.84
64.52
06.28
14088.6
85.86
66.36
70.18
69.90
55.46
36.79
63.76
97.44%
DivPrime
Asvar
CVPR'S
00.58
65.77
57.11
1457, 48
56.00
06.76
70.26
57.91
35.02
38.16
61.01
97.82%
SparseVT.M
Zhong
20224
ICME
00:32
65.06
56
1426-83 83
$5.06
67.27
77.11
50.64
55.58
39.16
63.92
97.000
Scrips [Oues]
Proposed
62.64
65.98
37.76
1481,98
87.65
67:43
80.47
58.90 08.02
55.71
41.90
65.35
99.88%
Netoln 929 Voleens in Averspe (1 (8..9))
H 2.112
2PLOPS
PuslV
ol.
BCCV'24
52.17
53.87
44.51
1178.61
72.64
65.07
06.52
52.30
51.33
25.08
51.44
63.21%
TRIM
Song
2013
COLING'25
19.05
65.58
51.01
1426.40
96.17
66.22
74.27
51.02
53.94
32.77
61.02
50.27%
Vironn
Yang et a1
CVPR 185
59.11
62.80
53:30
1300.61
$4.44
66.71
70:41
58.53
55.22
36.25
62.00
94.70%
DicPrume
Alanr
CVPR'25
10.24
64.11
35.08
1418.46
$1.92
67:11
77.24
56.17
54.97
35.72
02.52
95.60%
SparseVLM
Zhung
21124
JCHL '25
58.75
04.18
54.86
1709-77
85.16
06.14
56.53
56.22
37.94
02:44
50.4295
Scrips [Ours]
Proposed
61.30
65.39
56.82
1452,85
87.22
67,80
79.45
57.24
55.51
38.12
63.96
97.78%
Reteis
160 Tilors
Average (1 94.
1.156
7PT.0P=
FastV
Chen
BGCV'24
46.19
47.96
08.81
64.117
61.42
48.92
50.34
76.93%
TRIM
Song et
2014
COLING'S 25
$7.36
61.53
47.79
1279.44
$4.58
65.51
71.04
45.94
52.91
29.82
58.05
88.71%
Vi
Vang et
CVPR 155
56,28
50.72
51.30
13.12.4
84.00
65.89
73.40
54.52
53.05
34.50
60.05
91.78%
DivPrime
Alar
CVPR
35.62
62.07
53.51
L359
$1.05
110.28
73.18
54.18
51.82
92.215%
SparsNN7.M
Zhnng
20224e
53:27
62.05
53:45
1302.98
$1.63
66.08
71.85
63:85
54.01
31.81
00:38
90.25%
Scripé (Oues)
Proposed
60.78
64.20
58.07
1428.89
87.19
67.44
70.71
65.28
65.19
36.38
02.80
96.08%
Greedy Token Selection via Kernel Decomposition To select compact yet informative subset of k visual tokens, perform approximate MAP (maximum a posteriori) inference under a k-DPP defined by the query-conditioned kernel L. This procedure allows us to find tokens that are not only individually relevant to the query but also collectively diverse in semantic content, reducing redundancy while ensuring broad query coverage. The objective is to select a subset S of & tokens such that the corresponding submatrix Lg maximizes det (s), max]s|= det (Ls). The determinant reflects both semantic relevance and diversity. A higher value indicates greater diversity in the selected subset.
To efficiently approximate the MAP solution for DPP, we adopt the Cholesky-based greedy algorithm (Chen et al.] 2018). The algorithm starts with an empty set and incrementally adds tokens. At each step, it selects the token That yields the highest gain, where the gain is defined as the increase in the determinant of the kernel submatrix after adding the token. The algorithm & for all i E (1....,n), where of represents the importance of token i for selection, and u, is a vector that stores the projection of token i onto the subspace spanned by the previously selected tokens. At each iteration, the token with the largest 2² is selected: j = arg max of. After the selection, for each & (S (j)), we compute:
initialized as follows: of = La, u; = E
L (u.. u.)
2i t Eeey of 2
(11)
+
where e; represents the projection of token i onto the direction defined by token j, ej is the j-th standard basis vector in IR", and is small positive constant (e.g., 10 10-9) introduced for numerical stability. The update v² +- of - ea encourages selecting tokens with small projections e,- A smaller e; indicates that the token less similar, that is, closer to being orthogonal to the already selected tokens, thereby enhancing the diversity of the subset.
4.3 Final Token Selection
GSP and QCSP each address complementary aspects of token selection. GSP explicitly targets visual redundancy, while QCSP captures query relevance. However, neither is sufficient on its own for robust selection. We therefore take their intersection to ensure that the final subset is both visually compact and semantically aligned with the query. If the intersection contains fewer tokens than the required number, we supplement it with additional lokens from QCSP by iteralively increasing & to ret rieve more tokens.
2MPP for DPP is an NP-hard problem.
10

---

Published in Transactions on Machine Learning Research (11/2025)
Table 4: Performance comparison of different pruning methods on Video-LLaVA-T1 with 64 frames per video. The best results in each setting are bolded, and the second best are underlined.
Mathod
MIVU
MVBench
LongVideooench
Video-MME
Varne
Relative
Metric
test
val
perception relation
w/o sub
shors
mediumm
long
Upper Bound,
159 Tokens (100%)
LLaVA Video 7B
67.75
58.15
59.07
65.03
53.04
63 63.08
76.45
61.11
50.19
62.19
100%
Retarin 64
64 Tokens
62. 15%)
V Chen et al.
2024a
ECCV2'
68.79
$6.81
56.21
60.00
52.15
61.89
73.45
59:45
52.71
59.56
95.77%
DieProne Alvar et al 2025
CVPR
64.01
54.82
57.97
64.32
53.75
61.3
72.40
59.43
51.02
59.90
96.3291
SparseV1. L.5 Ziaung et 272
ICME
65.33
56.30
61.20
51.57
60 60.94
73.20
58.78
51.12
39.45
85.50%
Script (Ours)
Purpose
66.13
67.74
64.25
53.37
62.36
74.67
02.08
98.22%
64
Tokens
81, 115)
Fust
2024a
ECCV'S
52:27
36
87.95%
DivPrume Alvar
3125
E5
61.44
$0.37
56.64
62.41
51.31
29.73
69.69
57.82
19.88
58.04
93.3295
SparsrVL.M Zhang et
2024c
ICML 25
60.37
54.13
53.47
58.51
49.79
59.00
18
56.49
50.35
56.82
01.37%
Script (Ours)
Purpoae
62:77
06.37
56.35
61:40
62.97
60.35
72.11
58.46
01.20
G8.99
95.00%%
Retain 64
10 Tokens
90.5%)
VastV Chen et al.
20246
ECCV
62.58
46.07
46.61
48.08
44.79
49
64.92
50.30
40.22
48.85
78.64%
DivPrume Alvar et
2025
CVPRESS
58.60
51.13
52.27
57.56
47.29
56
67.87
59.34
48.11
54.72
58. 68.989%
SparseV1.X 1.1 Zhang 4t
2024c
ICME
51.97
48.87
47.42
53.02
42.89
40 19:58
53.81
49.45
46,
49.25
79.1995
Script (Ours)
Purpose
53.45
46.60
57.20
60.30%
5
Experiments
5.1
Experimental Settings
We conduct comprchensive experiments to evaluate Script's performance across diverse MLLMs and bench- In all tables, the "Relative" column reports performance relative to the unpruned model.
m 3
Models and Baselines. We select four representative MLLMs: LLaVA 1.5 7B (Liu et al. 2023), LLa VA NeXT 7B (Liu et al.j (024aa,, Video LLaVA 7B (Lin et al. 2024), and Intern VL3 ZZuu al. 2025
These models cover both image and video modalities with diverse architectures, providing a comprehensive testbed for evaluating Script's generalizability. In addition, we compare with five state-of-the-art pruning baselines, including FastV (Chen et al. ) TRIM (Song et al. 2025), SparseVLM (Zhang et al. 2024c),
DivPrune [Alvrr et al. 2025), and VisionZip (ang el al. 2025b). Notably, most of these baselines are either query-agnostic or tightly bound to specific model architectures. Some of these architectures are included in our ovaluation, enabling fair and direct comparisons.
Benchmarks. We evaluate Script across fourteen widely adopted benchmarks spanning both image and video understanding tasks. Image understanding tasks: GQA (Hudson & Manning 2019). Sci- enceQA (Lu et al. 2022), VQAv2 (Goyal al. 2017). TextVQA (Singh et al.) 2019). VizWiz (Gurari et al.,
2018).
VM (Yu et al.) 2023), MMBench (Liu el al. 2024c) MMBench-CN CN Liu el. al. 2024c), MME (Fu
ct al. 2023). and POPE [Li et al. 2023b). Video reasoning tasks: LongVideo VideoBench
Wu et al. 2024
VideoMME Fuu et al. 2025). MLVU Zhou et al. 2025 and MVBench (Li et al. 2023a
5.2 Results and Discussions
LLaVA-1.5-7B. Table 2 compares visual token pruning methods on LLaVA-1.5-7B across different token budgets. Our proposed method, Script, consistently outperforms baselines such as Fast DivPrune, and SLarseVLM, demonstrating strong adaptability and robustness across pruning levels and diverse tasks. At the highest token budget (192 tokens), Script achieves perfect accuracy (100.00%), outperforming the next-best method, DivPrune, by over 4 percentage points (95.94%). When reducing tokens to an intermediate level (64 tokens), Script retains exceptional accuracy at 96.86% indicating minimal performance drop despite significant token reduction. This outperforms SparseVLM by around 6 percentage points, underscoring Script's effective token selection strategy. Even under the most aggressive pruning (16 tokens, corresponding
See the appendix for additional experiments, model settings, limitations, and broader impact.
11

---

Published in Transactions on Machine Learning Research (11/2025)
to 97.2% pruning), Script still retains high accuracy (89.51%), significantly outperforming all baselines, highlighting its strong ability to preserve essential semantic and visual information, making it suitable for resource-oonstriined deployment.
LLaVA-NeXT-7B. Table & presents a detailed comparison of visual token pruning methods applied to LaVA--NeTT-BB across different token budgets. Our method, Script, consistently delivers the best perfor- mance across pruming levels, outperforming all baseline methods. Under mild pruning (640 tokens, 77.8% roduction), Script achieves the highest average accuracy (65.1%) and the best relative performance (99.88%). indicating minimal information loss despite significant pruming. Under moderate pruming conditions (320 to- kens, 88.9% reduction), Script maintains robust performance, achieving 63.97% accuracy and 97.09% relative performance, outperforming the next-best baseline by 2.12% points. Even at the most aggressive pruning level (160 tokens, 94.4% reduction), Script still maintains superior results, reaching 62.80% accuracy and 95.98% relative performance. These results demonstrate Script's strong ability to retain essential visual and semantic cues under tight computational budgets. Video-LLaVA-7B: Table T the performance of SOTA pruming methods on Video-LLaVA-7 across multiple token budgets. Script consistently outperforms all baselines across pruning levels, demonstrating robust performance under growing compression ratios. Under mild pruning (64x64 tokens, 62.1% reduction). Script achieves 98.22% average accuracy, demonstrating effective preservation of informative video content essential for accurate multimodal reasoning. Furthermore, under moderate pruning (64x32 tokens, 81.1% reduction), Script maintains 95.00% accuracy, outperforming FastV by roughly 7 percentage points, which demonstrates its effectiveness even under tighter token constraints. Even under aggressive pruning (64 16 tokens, 90.5% reduction), Script retains strong performance with 89.30% average accuracy, significantly surpassing other baselines at similar compression levels. These results highlight Script's ability to preserve spatio-temporal semantics, supporting its applicability in real-world video-based multimodal tasks.
5.3 Ablation study
To analyze the individual impact of each module, conduct an ablation study to evaluate the contribu- tions of the GSP, QCRS, and DPP modules. All ex- periments are conducted under stringent constraints, retaining only 16 tokens on average (97.3% pruning), with a compute cost of 0.103 TFLOPs. As shown in Table 5 naively pruning tokens causes a sharp drop in relative accuracy (average: 63.73% underscoring the need for informed token selection strategies. Incorpo- rating GSP alone raises relative accuracy to 66.58%, demonstrating the benefit of using a bipartite graph to model redundancy. Integrating QCSP (QCRS + DPP) alone further improves performance, increasing relative accuracy to 91.19%. Notably, the full config-
Table 5: Ablation on LLava-1.5-7B evaluating the impact of GSP, QCRS, and DPP.
GSP QCRS DPP POPE MME GQA SQA Arc. Relative
LLa VA-1.5-78
Retain 570 Tokens (100%)
86 96
1507 0G 61.94
69
73:41
100%
Netain
16 Tokens in Averuge (6 97.3%).
~0.103
TFLOPs
49.92
695.70
38.69
63.76
78.78
61.73%
v
x
x
52.85
758.23
40.86
63.86
48.87
08.58%
1225.27
45.82
68
86.14%
v
v
x
77.75
1111.92
51.11
68.57
63:25
86.17%
83.38
1252
53.28
68.51
06. 95
91.19%
84.41
1258.55
54.42
69.12
67.73
92.19%
uration achieves 92.19% of the original accuracy while retaining only 2.7% of tokens. These results further
reflect the distinct yet complementary contributions of each module.
5.4 Case study
To analyze how different pruning strategies handle query-relevant vision tokens, we compare four approaches: attention-basod (via [CLS]), divergence-basod, similarity-based, and Dap-dasod. Figure 5 visualizes the normalized scores from each method on representative POPE benchmark. The scores reveal how each method prioritizes tokens based on distinct selection criteria.
As shown in Figure 5] the four strategics exhibit clearly different token selection behaviors. The attention- based method emphasizes globally salient regions, such as central objects, but often overlooks contextually relevant backgrounds like human bodies. In contrast, divergence- based and similarity- based methods struggle to prioritize tokens that align with the input query. The similarity- based method, in particular, ignores query information and produces relatively uniform token scores that primarily reflect visual diversity. While such diversity helps preserve overall image content, it fails to capture query-specific relevance. The Dap-based
12

---

Published in Transactions on Machine Learning Research (11/2025)
Input Image
Attention-based
Divergence-based
Similarity-based
DPP-based
Figure 5: Visualizations of Pruning Preferences. We compute four different pruning scores for sample from the POPE benchmark using LLaVA-1.5-7B, with the query: "Is there person in the image?" Red indicates high preference, while blue indicates low.
method assigns the highest relevance to human faces and moderately high scores to body-related lokens, demonstrating strong alignment with query semantics. However, similar to intention-basee approaches, it lacks diversity in the selected tokens. These observations highlight the importance of combining GSP and QCSP for more balanced and effective token pruning.
5.5 Efficiency Analysis
We comprehensively evaluate recent. token pruning methods on the POPE benchmark, focusing on computation cost, inference latency, memory us- age; and accuracy. As shown in Table 6 the uncompressed LLaVA- -7B requires 41.7T FLOPs, with 246ms prefill and 29ms decode latency per token. It consumes 1440MB KV cache and 16.7GB peak GPU memory, achieving an F1 score of 86.8. In con- trast, Script reduces FLOPs 4.1T (10x reduction), prefill latency 35ms (6.8x faster), and decode latency to 22ms. It also lowers KV cache to 160MB and memory to 13.5GB, while maintaining F1 score of 86.7. Without DPP, Script mainlains similar elficiency and achieves slightly lower F1 score of 86.5. Compared to other baselines, Script achieves the lowest prefill/decodd latency, the smallest KV cache, and the best overall F1 scorc. For instance, Vast and PDrop suffer major accuracy loss (F1: 49.5 and 60.8), while VisionZip and DivPrune perform better (F1: 82.3 and 84.7) but at higher computational and memory cost. Overall, Script achieves the best balance between efficiency and accuracy, outperforming all baselines across nearly all metrics.
Table 6: Efficiency comparisons the POPE benchmark. We report the theoretical FLOPs, actual runtime, KV cache com- pression rate (%), and the achieved accuracy.
Methoal
Token
FLOPs (T)
Prefill Time Decode Time Cache GPU Memory F1
(rux/taoknn) (may (ooke))
(F1)
MaVA-NeNT--B
320
(x4.6)
(x1.2)
160.3
49.5
330
(x9.3)
55 (xd..5
21(x1.2)
1002
15.8
018
Spa MMo
320
4.5 (29.3)
71 (x3.5)
26 (x1.1.1)
161.2
18.6
76.9
VistonZtp cCPPR2s)
320
4.2 (x9.9)
38 (x6.6)
22 (x1.3)
160.0
14.8
82.1
DioProme(capaas)
380
38 (x0.6)
22 (x1.3)
160.0
13.9
81.7
Scripe
320
4.1 (x300
(x6.8)
22 (x1.3)
13.5
86.7
(x10)
(x0.6)
(x1.3)
6 Conclusion
In this paper, we introduce Script which reduces inference cost by selecting a compact yet semantically meaningful subset of visual vokens, conditioned on the user query. Script combines visual redundancy and query relevance perspectives through four stages: (1) graph-structured filtering to remove visually redundant tokens, (2) query-conditioned relevance scoring, (3) diversity subset selection based relevance scores using DPP, and (4) intersection- baaeet fusion to retain tokens that are both visually compact and semantically aligned with the query. Tts architecture-agnosiic, plug-and-play design enables seamless inte- gration into existing MLLMs without requiring retraining or architectural changes. Extensive experiments on fourteen vision-lande language benchmarks show that Script consistently reduces computational overhead and latency while preserving, or even improving, task performance under aggressive pruning settings. Future work includes extending Script to additional modalities such as audio and vidco in spatial-temporal contexts, and exploring adaplive pruming schedules that dynamically adjust to input complexity.
13

---

Published in Transactions on Machine Learning Research (11/2025)
References
Jean-Baptiste Alayrac, Jeff Donahuc, Panline Luc, Antoine Miech, Tain Barr, Yana Hasson, Karel Lene, Arthur Mensch, Katherine Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei, Marianne Monteiro, Jacob L Memick, Sebastian Borgeaud, Andy Brock, Aida Nematzadeh, Sahand Sharifzadeh, Mikol aj Binkowski, Ricardo Barreira, Oriol Vinyals, Andrew Zisserman, and Karén Simonyan. Flaringoo a visual language model for few-shot learning. In Advances in Neural Information Processing Systems, volume 35, pp. 23716 227766,2366 2022.
Saeed Ranjbar Alvar, Gursimran Singh, Mohammad Akbari, and Yong Zhang. Divprune: Diversity- visual token pruming for large multimodal models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 9392-9401, 2025.
Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A frontier large languala model with versatile abilities. arXiv preprint arXiv:2308. 12966, 2023.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vi technical report. arXiv preprint (tb::2500.1992.. 2025.
Federico Barbero, Alvaro Arroyo, Xiangming Gu, Christos Perivolaropoulos, Michael Bronstein, Petar Veličković, and Razvan Pascanu. Why do llms attend the first token? arXiv preprint orXiv:2504.02792, 2025.
Daniel Bolya, Cheng- Yang Fu, Xiaoliang Dai, Peizhao Zhang, Christoph Feichtenhofer, and Judy Hoffman. Token merging: Your ViT but faster. In International Conference on Learning Representations, 2023.
Simon Buchholz. Learning partitions from context. In The Thirty-eighth Annual Conference on Newral Information Processing Systems, 2024. URL lhtps: hhttt::/// //tpar///////eeeeee///mm htt.::/wwww.ttoooooorrrrertrraoe
Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, et al. Internlm2 technical report. arXiv preprinl arXiv:2403. 17297, 2024.
Jun Chen, Deyao Zhu, Xiaoqian Shen, Xiang Li, Zechun Liu, Pengchuan Zhang, Raghuraman Krishnamoor- thi, Vikas Chandra, Yunyang Xiong, and Mohamed Elhoseiny. Minigpt-v2: large language model as a unified interface for vision-language multi-task learning, arXiv preprint arXiv:2310.09//7. 2023.
Laming Chen, Guoxin Zhang, and Eric Zhou. Fast greedy map inference for determinantal point process to improve recommendation diversity. Advonces in Neural Information Processing Systems, 31, 2018
Liang Chen, Haozhe Zhao, Tianyu Liu, Shuai Bai, Junyang Lin, Chang Zhou, and Baobao Chang, An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models. In European Conference on Computer Vision, pp. 19-35. Springer, 2024a.
Yukang Chen, Fuzhao Xue, Dacheng Li, Qinghao Hu, Ligeng Zhu, Xiuyu Li, Yunhao Fang, Haotian Tang, Shang Yang, Zhijian Liu, et al. Longvila: Scaling long-context. visual language models for long videos. arXiv preprint ar orXic:2408.10188 2024b.
Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.0555 0522711,, 2024c.
Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual- linguistic tasks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1185-24198,, 2024d.
14

---

Published in Transactions on Machine Learning Research (11/2025)
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ton Stoica, and Fric P. Xing. Vieuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023. URL https:://mmss..or//b0
Tri Dao. FlastAttetiion-2:: Faster attention with better parallelism and work partitioning, In International Conference on Learning Representations (TCLR), 2024.
Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, Yunsheng Wu, and Rongrong Ji. Mme: A comprehensive evaluation benchmark for multimodal large language models. arXiv preprint arXiv:2306. 189394 2023.
Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Vidco-mme: The first-ever comprchensive evaluation benchmark of multi-modal Ilms in video analysis. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 24108 24118, 2025.
Yash Goyal, Tejas Khot, Douglas Summers Summers-Stay, Dhruv Batra, and Devi Parikh. Making the v in vqa matter: Elevating the role of image understanding in visual question answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognation, pp. 6904-6913, 2017.
Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, et al. Hallusionbench: an advanced diagnostic suite for entangled lan- guage hallucination and visual illusion in large vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14375-14385, 2024.
Zonghao Guo, Ruyi Xu, Yuan Yao, Junbo Cui, Zanlin Ni, Chunjiang Ge, Tat-Seng Chua, Zhiyuan Liu, and Gao Huang. Llava-uhd: an Imm perceiving any aspect ratio and high-rosolution images. In European Conference on Computer Vision, pp. 390 406. Springer, 2024.
Danna Gurari, Qing Li, Abigale J Stangl, Anhong Guo, Chi Lin, Kristen Grauman, Jiebo Luo, and Jellrey P Bigham. Vizwiz grand challenge: Answering visual questions from blind people. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 3608-3617 2018.
Wenbo Hu, Zi-Yi Dou, Liunian Li, Amita Kamath, Nanyun Peng, and Kai-Wei Chang. Matryoshka query transformer for large vision-language models. Advances in Newral Information Processing Systems, 37: 2024.
Drew A Hudson and Christopher D Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVF conference on computer vision and pallern recognilion, pp. 6700 6709, 2019.
Ahmadreza Jeddi, Negin Baghbanzadeh, Elham Dolatabadi, and Babak Thati, Similarity-aware token prun- ing: Your vim but faster. arXiv preprint 1tt.:250331154999 2025.
Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi, and Ali Farhadi. A dia- gram is worth dozen images. In Computer Vision-ECCV 2016: 14th European Conference, Amsterdom, The Netherlands, October 11 11-14, 2016, Proceedings, Part TV 14, pp. 235-251. Springer, 2016.
Ivan Krasin, Tom Ducrig, Neil Alldrin, Vittorio Ferrari, Sami Abu-El-Haija, Alina Kuznetsova, Hassan Rom, Jasper Uijlings, Stefan Popov, Andreas Veit, et al. Openimages: A public dataset for large-scale multi-label and multi-class image classification. Dalasel available from 2017.
Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanic Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, et al. Visual genome: Connecting language and vision using crowdsourced dense image annotations. International journal of computer vision, 123:32 02332-3,,22 2017.
15

---

Published in Transactions on Machine Learning Research (11/2025)
Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. aiv preprint or orXim:2408.0088 033222 2024a.
Duo Li, Zuhao Yang, and Shijian Lu. Todre: Visual token pruning via diversity and task awareness for efficient large vision-language models, 2025a. URL http:://axxw..org//aaa wttp://v.wrr/b///////oooooosss 2ttp://555..877778 18757
Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, Limin Wang, and Yu Qiao. Mybench: A comprehensive multi-modal video understanding benchmark. 2024 IEEE/CVF Conference on Computer Vision and Pallern Recognétion (CVPR), pp. 22195-22200 2023a. URL
Yadong Li, Jun Liu, Tao Zhang, Song Chen, Tianpeng Li, Zehuan Li, Lijun Liu, Lingfeng Ming, Guosheng Dong, Da Pan, et al. Baichuan-omni-1.5 technical report. arXiv preprint arXiv:2501. 15368, 2025b.
Yanwei Li, Chengyao Wang, and Jiaya Jia. Llama-vid: An image is worth 2 tokens in large language models. In European Conference on Computer Vision, pp. 323 340. Springer, 2024b.
Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination in large vision-language models. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 292-305, 2023b.
Zongqian Li, Yinhong Liu, Yixuan Su, and Nigel Collier. Prompt compression for large language models: A survey. In Luis Chiruzzo, Alan Ritter, and Lu Wang (ods.), Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Techmologies (Volume 1: Long Papers), pp. 7182-7195, 7195, Albuquerque, New Mexico, April 2025c. Associ- ation for Computational Linguistics. ISBN 979-8-89176-1888 189-6. URL https::/acclattholloe hwwphhooogg...oo///o/ee org/2025.
naacl-long 1oncl-loog.366 368/
Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, and Li Yuan. Vidco-llava: Learning united visual representation by alignment before projection. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 5971-5984, 2024.
Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zätnick. Microsoft. coco: Common objects in context. In Computer Vision-EECCT 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp. 740 755. Springer, 2014.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lec. Visual instruction tuning. Advances in neural information processing systems, 36:34892 34916, 2023.
Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, ocr, and world knowledge, January 2024a. URL htttps///lava-zil http://llaaa-vl.iihhbb. wew.ggittuub.... io/
2024-01-30-11aaa
Ting Liu, Liangtao Shi, Richang Hong, Yue Hu, Quanjun Yin, and Linfeng Zhang. Multi-stage vision token dropping: Towards efficient multimodal large language model. arXiv preprint arXiv:2411.10803 2024b.
Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, Kai Chen, and Dahua Lin. Mibbenc: Is your multi-modal model an all-around player? In European Conference on Computer Vision, pp. 216-233. Springer, 2024c.
Yuliang Liu, Zhang Li, Mingxin Huang, Biao Yang, Wenwen Yu, Chunyuan Li, Xu- Ku-Cheng Yin, Cheng Lin Liu, Lianwen Jin, and Xiang Bai. Ocrbench: on the hidden mystery of oer in large multimodal models. Science China Information Sciences, 67 222122220, 2024d.
Zuyan Liu, Yuhao Dong, Jiahui Wang, Ziwei Liu, Winston Hu, Jiwen Lu, and Yongming Rao. Ola: Push- ing the frontiers of omni-modal language model with progressive modality alignment. arXie preprint arXiv:2502.0433 2025.
16

---

Published in Transactions on Machine Learning Research (11/2025)
Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering. Advances in Neural Information Processing Systems, 35:2507 2521, 2022.
Gen Luo, Yiyi Zhou, Yuxin Zhang, Xiawu Zheng, Xiaoshuai Sun, and Rongrong Ji. Feast your eyes: Mixture- of-resolution adaptation for multimodal large language models. arXiv preprint arXim:2403.0030 03000000,, 2024.
Ahmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq Joly, and Enamul Hoque. Chartqa: A benchmark for question answering about charts with visual and logical reasoning. ar preprint arx:::200 2022.
Lorenzo Papa, Paolo Russo, Irene Amerini, and Luping Zhou, A survey efficient vision transformers: Algo- rithms, techniques, and performance benchmarking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46:7682 7700, 2023. URL https://api semanticcccci Sarg/CorpuslD ID: 261531260
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8763 PmLR, 2021.
Yuzhang Shang, Mu Cai, Bingxin Xu, Yong Jae Lee, and Yan Yan. plava-puueerge: Adaptive token reduction for efficient large multimodal models. arXiv preprint wrXiv:2403.15388 2024.
Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. Towards vqa models that can read. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8317-8326, 2019.
Dingjie Song, Wenjun Wang, Shunian Chen, Xidong Wang, Michael X. Guan, and Benyou Wang. Less is more: A simple yet effective token reduction method for efficient multi-modal LLMs. In Owen Rambow, Leo Wanner, Marianna Apidianaki, Hend Al-Khalifa, Barbara Di Eugenio, and Steven Schockaert (eds.), Proceedings of the 31st International Conference on Computational Linguistics, pp. 7614-7623, Abu Dhabi, org/ 002/2025.
UAE, Jamuary 2025. Association for Computational Linguistics. URL main. /
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaci, Nikolay Bash- lykov, Soumyy Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint orXiu:2307.09288 2023.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin, Attention is all you need. Advances in neural information processing systems, 30, 2017.
Hongjie Wang, Bhishma Dedhia, and Niraj K. Jha. Zero-tprune: Zero-shot token pruning through leveraging of the attention graph in pre-trained transformers. In Proceedings of the IEEE/CVP Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
Zichen Wen, Yifeng Gao, Weijia Li, Conghui He, and Linfeng Zhang. Token pruning in multimodal large language models: Are we solving the right problem? arXiv preprint arXim:2502. 2025a.
Zichen Wen, Yifeng Gao, Shaobo Wang, Junyuan Zhang, Qintong Zhang, Weijia Li, Conghui He, and Linfeng Zhang. Stop looking for important tokens in multimodal language models: Duplication matters more. arXiu preprint xtXx:22502.1149 2025b.
Haoning WU, Dongxu Li, Bei Chen, and Jumnan Li. Longvideobench: A benchmark for long-context in- terleaved video- language understanding. Advances in Neural Information Processing Systems, 37:28828- 28857, 2024.
Long Xing, Qidong Huang, Xiaoyi Dong, Jiajie Lu, Pan Zhang, Yuhang Zang, Yuhang Cao, Conghui He, Jiaqi Wang, Feng Wu, et al. Pyramiddrop: Accelerating your large vision-language models via pyramid visual redundancy reduction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025.
17

---

Published in Transactions on Machine Learning Research (11/2025)
Jin Xu, Zhifang Guo, Jinzheng He, Hangrui Hu, Ting He, Shuai Bai, Keqin Chen, Jialin Wang, Yang Fan, Kai Dang, Bin Zhang, Xiong Wang, Yunfei Chu, and Junyang Lin. Qwen2.5-omni technical report. arXiv preprint 2r00::2503.2021, 2025.
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2.5 technical report. arXiv preprint ar 15.1151,, 2024.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Go, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yugiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. Qwen3 technical report. arXiv preprint orXiv:2505.0000 09388, 2025a.
Senqiao Yang, Yukang Chen, Zhuotao Tian, Chengyao Wang, Jingyao Li, Bei Yu, and Jiaya Jia. Visionzip: Longer is better but not necessary in vision language models. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, 2025b.
Weihao Ye, Qiong Wu, Wenhao Lin, and Yiyi Zhou. Fit and prune: Fast and training- free visual token pruning for multi-modal large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 22128 22136, 2025.
Tianyu Yu, Haoye Zhang, Yuan Yao, Yunkai Dang, Da Chen, Xiaoman Lu,, Ganqu Cui, Taiwen He, Zhiyuan Liu, Tat-Seng Chua, et al. Rlaif-v: Aligning mllms through open-source feedback for super gpt-4v trustworthiness. arxio preprint or Xiv::22405. 17220, 2024.
Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, and Li- juan Wang. Mm-vet: Evaluating large multimodal models for integrated capabilities. arXie preprint arXiv:2308.0 004.0000000 02490, 2023.
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre- training. In Proceedings of the IEFE/CVF international conference on compuler vision, pp- 2023.
Hang Zhang, Xin Li, and Lidong Bing. Video-llama: An instruction-tuund audio-visual language model for video understanding. arXiv preprint arXiv:2306. 02858, 2023.
Peiyuan Zhang, Kaichen Zhang, Bo Li, Guanggao Zeng, Jingkang Yang. Yuanhan Zhang, Ziyue Wang, Haoran Tan, Chunyuan Lá, and Ziwei Liu. Long context transfer from language to vision. arXiv preprint S.Kv:2406616855. 2024a.
Qizhe Zhang, Aosong Cheng, Ming Lu, Zhiyong Zhuo, Mingi Wang, Jiajun Cao, Shaobo Guo, Qi She, and Shanghang Zhang. [cls] attention is all you need for training-free visual token pruning: Make vim inference faster. arXiv preprint 212 2024b.
Shaolei Zhang, Qingkai Fang, Zhe Yang, and Yang Feng. Llava-mini: Efficient image and vidco large multimodal models with one vision token. arXiv preprinl urXiv:2501.0366 2025.
Yuan Zhang, Chun-Kai Fan, Junpeng Ma, Wenzhao Zheng, Tao Huang, Kuan Cheng, Denis Gudovskiy, Tomoyuki Okuno, Yohei Nakata, Kurt Keutzer, et al. Sparsevim: Visual token sparsification for efficient vision-language model inference. arXiv preprint arXiv:2410.0447 2024c.
Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Ziwei Liu, and Chunyuan Li. Vidco instruction tuning with synthetic data. Xixi preprint or Niw: 10.0271,, 2024d.
18

---

Published in Transactions on Machine Learning Research (11/2025)
Junjie Zhou, Yan Shu, Bo Zhao, Boya Wu, Zhengyang Liang, Shitao Xiao, Minghao Qin, Xi Yang, Yongping Xiong, Bo Zhang, et al. Mlvu: Benchmarking multi-task long video understanding. Tn Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 13691- 13701, 2025.
Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Ti, and Mohamed Elhoseiny. MmnGGPT-4: Enhancing vision- language understanding with advanced large language modcls. In The Twelfth International Conference on Learning Represenlations, 2024. URL
Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Yuchen Duan, Hao Tian, Weijie Su, Jie Shao, et al. Internv13: Exploring advanced training and test-time recipes for open-source multimodal models. Xix preprint ar orXiv22544.00479 10479, 2025.
19

---

Published in Transactions on Machine Learning Research (11/2025)
Script: Graph-Structured and Query-Conditioned Semantic Token Pruning for Multimodal Large Language Models Appendix
A
Notation Overview
To facilitate clarity and improve readability, we provide a detailed glossary of all mathematical symbols used throughout this paper, as shown in A.. The table serves as centralized reference point, systematically organizing every notation involved in the core components of the Script framework, including equations, algorithmic modules, and architectural definitions. The glossary covers all key components the proposed framework, including visual encoding, query representation, relevance scoring, similarity computation, DPP- based token selection, structural pruning, and auxiliary constants. This table is intended to improve the clarity of mathematical derivations, reduce rross-referencing overhead, and support accurate reproduction and further development of the method.
Table A.1: Detailed symbol glossary used throughout the Scripl framework.
Symbol
Type Dim
Definition
X, fu
Image video input
Raw visual input passed into the vision encoder f.. Vision encoder extracting patch-level features from X
Function
Function
Multimodal projector mapping visual features to token embeddings.
n, d,
Integers
#tokens (input, dim, query length, retained) respectively. Sequence of n visual tokens, each d-dimensional, before pruning.
H.
Matrix
Ha
Sequence of vectors
Tokenized user query of length Visual token subset after pruming (m < <n).
H.
Matrix (m d)
fe
Function (LLM)
Frozen language model consuming Hg):
L(-)))
Scalar loss
Task loss comparing model outputs before and after pruning. i-th visual token embedding in H.
Vector (E IRd))
Vector (E IRd))
Mean pooled embedding of the query tokens. Query-conditioned relevance score of token i. Relevance weighting matrix for visual tokens. Cosine similarity between tokens and Symmetric similarity malrix among visual tokens. DPP kernel combining relevance and diversity. Token indices selected for retention.
Scalar
Q = diag(g (onggg
Diagonal matrix
S.j
Scalar
Matrix (7 11)
L Q1/2Q'1/2
Matrix (n n)
I, [Z] = =m
Index set
I*
Index set
Optimal index subset maximizing log-det of DPP kernel. Query-conditioned DPP kernel with temperature scaling. Raw and temperature-scaled relevance scores.
L
Matrix
r,f
Vectors (E [0, 1)m)
Vie, Vast
Token subsets
Bipartite partitions used in graph-based redundancy pruning. Token similarity graph for structural filtering.
(Vare Vast. S)
Graph
df
Scalar
Residual variance of token under low-rank projection.
4
Vector (E IRT)
Coefficient vector for token in basis space.
Scalar
Normalizer in Cholesky-slyle update.
C
Matrix
Basis matrix for low-rank approximation of DPP kernel. Target rank DPP subset size.
T
Integer
Threshold (scalar)
Redundancy threshold for similarity pruming. Numerical stability term in projection update. Element- (Hadamard) product.
Constant (10-6)
Operator
Operator
Inner product used for cosine similarity. Euclidean norm used normalization.
Operator
20

---

Published in Transactions on Machine Learning Research (11/2025)
B Diversity in Determinantal Point Processes: Geometric Intuition and Theoretical Analysis
B.1 Notation.
To facilitate clear and consistent exposition, first define the key symbols and assumptions used throughout the paper and illustrate each with simple example explanation.
11-12 (Euclidean norm): The standard l2 norm for vectors in Rd Example: [1(3, 2) = 3 + 4 5.
Xx(A) (i-th largest eigenvalue of A): For a symmetric matrix A E R***, we order its eigenvalues as (4) 2 (2( 2 ... 2 (A)).. Since all our kernels will be positive semi-defintte (PSD), all
X 20.
1: (all-oncs vector): A k-dimensional column vector with all entries equal to one, i.c., (I,...,,)) 1)" e R²
Let T 01.-. Un Rd denote a collection of feature vectors. We assume that each is has unit norm: ||::||2 for all This means that each vector lies on the unit hypersphere in Rd.
Erample: The vector v (1,0..... has ||o 1, and thus is valid member of T.
We adopt this unit-norm convention to ensure that inner products v. u correspond exactly to cosine similar- ities, simplifying the interpretation of the similarity kernel introduced next. Meanwhile, this normalization remains fixed throughout the paper.
Similarity Kernel. Given collection of unit-norm vectors T , un] Rd as introduced above, define similarity kernel matrix L E _= whose -intyy captures the cosine similarity between and vy:
Lij uf By, for all 1 $ i, S n.
(B.1)
Since all is lie on the unit hypersphere, have of Uy [1. In particular, Lu v, 0 1, and the more similar two vectors are, the closer Lij is to 1.
We can express the full kernel compactly using matrix nolation. Let V = =Iv 02 ... val E Rdxx be the matrix whose columns are the feature vectors.
Then:
L= VV
(B.2)
This representation immediately implies that L is symmetric and positive semi-defnite (PSD), because for any x R", have:
-x x Vr V=
(B.3)
Therefore, satisfies the standard spectral properties of Gram matrices: all eigenvalues of are real and non-negative, and L admits orthonormal eigendecomposiition These properties critical for the DPP model discussed next.
B.2 Determinant Diversity Objective
We now connect the determinant of kernel submatrix with geometric diversity. Specifically, show that for any subset of & unit vectors, the determinant of their Gram matrix equals the square of the volume of the parallelotope they span.
Proposition (Determinant
-oolnnm Volume Equivalence). Let S Vi....,iyy in] S [m] be an inder subset of size Define VsS = [P,,.... 2 E Rd as the submatrix of feature vectors indeved by S, and Lg = V Vs R
their Gram matriz. Then,
det Ls det Vs) (Vol)
(B.4)
21

---

Published in Transactions on Machine Learning Research (11/2025)
where Vol, (Vs) is the k-dimennsonal volume of the porallelotope sponned by the column vectors of Vs.
Proof. Let Vs QR be the QR decomposition of Vs, where Q [d has orthonormal columns and R. E Rkak is upper triangular. Then,
- Vs= R R
(B.5)
using the fact that QTQ= 4a. Taking determinants on both sides gives:
det Ls = det(R R) (det Rt)a.
(B.6)
Geometrically, the k-dimensional volume satisfies
Vol+(Vs) det Vs) det Rl),
(B.7)
hence det Ls (Vol&(Vs))
This equivalence forms the foundation for interpreting det Lg natural diversity objective: it assigns higher values to sets of vectors that are more 'spread out' or 'orthogonal" in space.
B.3 Diversity Upper Bound via Hadamard Inequality
The determinant of any Gram matrix Lg formed from unit-norm vectors is upper bounded by 1. This follows from the classical Hadamard inequality, which gives a constraint on the determinant of a positive semi-definite matrix in terms of its diagonal entries.
Corollary (Hadamard Bound). Let Ls be the Gram matris defoned above, formed from k unit vectors. Then:
det Ls S 1, det Ls= =1
Dip 1 = for all p 9 and k S d.
(B.8)
Proof. Hadamard's incquality states that for any positive semi-definite matrix A E R with diagonal entries Oie.
det A S a11022 dgks
with equality if and only if the columns of A are orthogonal.
In our case, Ls = V Vs is PSD and satisfics diag( Lg) = (1,1,...,11) since each ||0/||2 1.
Therefore,
det Ls S 1,
(B.9)
with equality if and only if the vectors ......,, Ui are mutually orthogonal; this requires lo & d. If k: > d. then rank(Ls) S d < k and det Ls = 0 < 1.
The determinant det Ls quantifies how linearly independent or "spread out" the vectors in S are. The maximum value 1 occurs when the vectors are exactly orthogonal. Therefore, maximizing det Lg promotes diversity by encouraging the selection of vectors that are as close to orthogonal as possible.
B.4 Determinantal Point Processes and Diversity Maximization
We now formally introduce Determinantal Point Processes (DPPs), which are probability distributions over subsets that inherently promote diversity. In the fixed-size or k-DPP setting, each subset S of size k is assigned probability proportional to the determinant of its associated submatrix Ls:
(S) det L.g,
for S k.
(B.10)
22

---

Published in Transactions on Machine Learning Research (11/2025)
Specifically, we formulate this
det Ls
(L(S)
(B.11)
Σ det
LT
T
T-
where the denominator sums over all k-subsots of [n), ensuring a valid probability distribution.
This probabilistic model encourages the selection of sets whose vectors are geometrically diverse, since higher determinant values correspond higher volume (as shown previously). In practice, one often seeks the most probable subset under this modcl, known as the maximum posteriori (MAP) estimate:
SMAP = arg max det Lg. S||=
(B.12)
This optimization problem aims to find the subset of vectors that span the largest volume parallelotope in Rd, or equivalently, the subset exhibiting maximal geometric diversity. The DPP framework thus provides both a probabilistic model and a concrete objective- det fo- achieving diverse subset selection.
B.5 Redundancy Metrics and Determinant Bounds
While the determinant det Ls captures diversity, can also assess subset quality via redundancy metrics. These measure how similar the vectors in S are to each other, either in the worst-case or on average. Two commonly used metrics are:
Paaxx (S) max Lij,
2 Pavg (S) k(k - 1)
Σ Lij. icjES
(B.13)
ifJE8
Here, Ti = v. My the cosine similarity between unit vectors i and By. The value Pmaa((S) measures the most redundant pair (i.e., most similar), while Pavg(S) captures the overall similarity level.
We now relate these redundancy measures to det Ls using spectral bounds. The first bound is derived using Gershgorin's circle theorem.
Lemma 1 (Gershgorin-Bart Based Lower Bound). For any subset S of size k. define (S) maxi Ligl- Then:
Amin (Ls) ( 1 () (S), det Ls 2 [1- Poc
(B.14)
Proof. For each row of Ls, the Gershgorin radius r. Ejts Ligl S (K- 10 By Gershgorin's cirele theorem, every eigenvalue of Lg lies in at. least. one disk centered 1 with radius ri, hence
Amin (Lg) maxr, 2 1 ())
Since Lg is symmetric PSD, its determinant is the product of cigenvalues, so
det Ls I[ (Ls) 2 (imi((L)) 2 (k 1)p
Next, we upper bound det Ls via the arithmetic mean geometric mean inequality.
Lemma 2 (AM-GM Upper Bound). Let ,I,...,dE be the eigenvalues of L.g. Then:
det T.g < (r((Ls) k
= 1.
(B.15)
23

---

Published in Transactions on Machine Learning Research (11/2025)
Discussion. This bound follows directly from the arithmetic -eommgeeir mean inequality: the product of nonnegative eigenvalues is at most the k-th power of their average. Since I. is Gram matrix of unit- norm vectors, all diagonal entries are 1, so tr(Ls) = k. Hence, the right-hand side reduces to 1, independent of the actual off-diagonal similaritics.
Therefore, the GM-GM bound is extremely loose: only states the trivial fact that det L,s S 1, without reflecting the effect of redundancy or (avg(S), This motivates the refined spectral bound derived next, which captures how det Lg decays with increasing average similarity.
Motivation for Refined Bound. The AM-GM bound above provides worst-case envelope that does not incorporate any spectral structure of Ls beyond its average similarity. However, when all off-diagonal entries of Ls are equal lo Parg: the matrix exhibits an extremal spectral configuration that marimizes the determinant under a fixed average similarity. By explicitly analyzing this case, we obtain a refined upper envelope that better captures how the maximum achievable det Ls decays with increasing redundancy.
Lemma 3 (Refined Upper Bound via Spectral Construction). Let Lg be the Gram matriz of k unit vectors with average off-diagonal similurily Pawg(5) E (- 1). Then,
det Ls S (1 + (k- (8)) (1 Pavg, (Svn(5))(-1
(B.16)
Proof. Consider the class of symmetric PSD matrices Ls where all off-diagonal entries are equal to Pavgs and all diagonal entries are 1:
{ Pavg
if if i f
(Ls)
which is feasible iff Pavg E I- R- 1). This matrix has known spectral decomposition: one eigenvalue equals + (ke - Pagg and the remaining (h: 1) cigenvalues are 1
Pavg.
Hence, the determinant is:
det Ls = + (k 1)Pavg) (1 Pavg) )-1
Among all PSD matrices with unit diagonal and a fixed average off-diagonal similarity, the functional det()) concave and vermutation-invarian by symmetry, an optimizer (for maximizing det under this constraint) must be permutation-invariae i.c., the equicorrelation matrix above. Thus, the displayed value yields upper bound, establishing the claim.
This refined bound is an upper bound for all positive semi-definite matrices with unit diagonal and given average off-diagonal similarity, and it is tight: the equicorrelation matrix attains equality. Among such uniformly redundant matrices, the bound is achieved exactly, making it a sharp envelope for assessing diversity degradation as average correlation increases.
Compared to Lemma this refined bound is strictly tighter whenever (Sv > O, as it directly models the spectral behavior of uniformly redundant configurations. The bound thus provides a sharper envelope for analyzing diversity degradation in highly correlated subsets.
B.6 Global Optimality of the MAP Subset
Proposition 2 (MAP subset need not minimize redundancy). In general, a k-subset S arg max - det Lg need not ménimize either Pmax or Pavg among all size-k subsets.
Proof. Consider k: = with
( 9) CE i-1111
Then det I. 1 c² is maximized at e 0, where Pmax 0. For c 1. we have Pmax = = (strictly smaller), yet det (nor Pavg)-
Hence maximizing det L does not generally minimize Pmax
24

---

Published in Transactions on Machine Learning Research (11/2025)
0.8
det Ls upper bound
0.6
AM CM Bound:
Refined Bound: 4p)(11 p)*
0.4
0.2
0 0
0.2
0.4
0.6
0.8
Pavg
Figure B.1: Upper envelope on det Ls vs, average similarity Pava for l = 5. Unlike the trivial AM GM bound (which constantly 1), the refined envelope (1 + 4p)(1 p)} decreases with redundancy, revealing how the maximum attainable determinant shrinks as Pavg grows.
B.7 Approximation via Greedy MAP Inference
Exact MAP inference for a k-DPP is NP-hard, requires evaluating (*) determinants, However, the regularized objective
f(S) = log det(I Ls)
is known be monotone and submodular when L positive semi-definite, which allows for a natural greedy approximation algorithm. The function log det(1 + Ls) is monotone and submodular when L is PSD, as it ariscs from the entropy of Gaussian variables or from spectral submodular theory.
Let S denote the greedy solution obtained by ileratively selecting vectors that offer the greatest marginal increase in log det( Ls). Then, by the classical Nemhauser Wolsey theorem for submodular maximization:
log + Lg) 2 (1 1/e) leg + Ls-),,
(B.17)
where 5* is the exact optimal size-k subset for f(S) log det([ Lg).
Exponentiating both sides gives multiplicative guarantee on the regularized determinant:
det(I + Lg) (det(T + Ls+))
(B.18)
Translating such guarantees into bounds Pmax Pavg requires additional structural assumptions L (c.g., incoherence or equicorrelation); we therefore refrain from stating general redundancy guaranteos here.
B.8 Sufficient Condition for Equivalence
The counterexample in Proposition 2 shows that, in general, the MAP solution of a k-DPP need not min- imize redundancy measures such as Pmax Pavg. Nevertheless, under additional structural assumptions, maximizing geometric diversity and minimizing redundancy do become equivalent. We now state a sufficient condition under which this equivalence holds.
Proposition 3 (Equivalence under Equicorrelation). Suppose that for each condidate subset. S of size k, the associated Gram matria Lg has the form
if
(Ls)
p
i
25

---

Published in Transactions on Machine Learning Research (11/2025)
for some p E [0. Then:
arg max det Ls arg min Pmax (S) = arg min Pavg (S). s||=
[9]
=
other words, the MAP subset simultaneously marimizes geometric diversity and minimizes redundancy.
Proof. For such an equicorrelation matrix, the eigenvalues are known in closed form:
\1 1 (k- 1)p, X2 = d p-
Therefore,
det Ls (1 (k- (1 p)*
Since (S) (S) p in this setting, it suffices to analyze the monotonicity of det Ls with respect to p. Taking logarithms and differentiating.
d dp
h-1
h 0, for p E (0,1).
log det Ls
1 (k 1)p 1
Thus det Ls decreases strictly with p. and maximizing the determinant is equivalent to minimizing The claim follows.
This proposition establishes a clean sufficient condition under which DPP-based MAP inference exactly aligns with redundancy minimization: in equicorrelated settings, maximizing diversity volume and minimizing redundancy equivalent optimization goals.
B.9 Summary.
This section establishes a theoretical link between geometric diversity, measured through the determinant of kernel (Gram) submatrix, and redundancy metrics such as Pmax and Pavg. The analysis has revealed three central insights:
Maximizing det Ls systematically favors subsets of vectors that are nearly orthogonal one another, thereby reducing both worst-case and average redundancy.
2. In general, the maximum posteriori (MAP) solution of the k-DPP maximizes geometric diversity (via det Lg), but it does not necessarily minimize Puax Pavgi rather, large determinants enforce envelope-lype constrainds on admissible redundancy (Proposition 2
3. However, under additional structure most notably the equicorvelation condition formalized in Proposition 3 the MAP solution does coincide with redundancy minimization: maximizing det Ls then equivalent to minimizing Pmax and Pavg-
When exact MAP inference computationally infeasible, a greedy algorithm on the monotone submodular surrogate f(S) = log det( + Ls) provides constant- guarantees on the regularized determinant, thereby preserving much of the diversity of the global optimum.
Taken together, these results justify the determinant objective as a principled and effective diversity metric, one that offers both rigorous thcoretical foundations and practical algorithmic strategics for subset selection. This theoretical perspective provides solid basis for applications where balancing diversity and redundancy is essential.
26

---

Published in Transactions on Machine Learning Research (11/2025)
Application to Script. The diversity metric and DPP framework developed here directly ground the Script method proposed in the main text. Script is designed for practical, query-conditioned multimodal setting, yet its core selection principle remains the same: maximize the determinant of a positive semidefinite kernel submatrix in order to favor diverse and complementary elements.
In Script, the kernel defined as
L diag(r) S diag(r),
(B.19)
where S encodes token-token similarities (e.g., cosine similarity), and r R20o stores query-conditioned relevance scores. Because S is symmetric positive semidefinite and diag(r) is diagonal with nonnegative entries, the resulting kernel L remains positive semidefinite under the congruence transformation. This
ensures that. the spectral properties required for DPP inference are preserved, even when relevance weights vary across tokens.
Crucially, the token similarity structure in Script satisfics the equicorrclation-tyye type condition analyzed in Proposition 3 Consequently, the MAP subset.
= max det (Ls), [S|=
(B.20)
not only maximizes geometric diversity but also simultaneously minimizes redundancy. In other words, Script inherits the best of both worlds: the probabilistic diversity guarantee of DPPs and the redundancy minimization property under its structural assumptions.
This explains why Script can aggressively prune tokens while preserving both relevance and diversity. Em- pirical results presented in Section 5 confirm that maximizing det(Ls) with a query-conditioned kernel yields subsets that are simultaneoully relevant, non-redundant, and diverse, thereby improving both summarization and retrieval performance.
27

---

Published in Transactions on Machine Learning Research (11/2025)
C Details of experimental setup
C.1 Model Architectures
LLaVA-1.5 (Liu et al., 2023) The LLaVA series represents a foundational line of open-source vision- language models (VLMs), recognized for their simple design, low training cost, and strong performance. The original LLaVA architecture integrates a pretrained CLIP (Radford et al. 2021) as the visual encoder and Vicuna (Chiang et al., 2023) as the language model, connected via a linear projection layer. This enables the LLM to accept image grid features as input. Through visual instruction tuning, LLaVA gains the ability to handle multimodal tasks. LLaVA-1.5 enhances this framework by replacing the linear connector with a multi-layer perceptron (MLP), incrcasing input image resolution, and utilizing a broader and more diverse set of instruction tuning data. These modifications ions lead to substantial performance improvements. The model processes images at a resolution of 336,336, resulting in 576 visual tokens per image.
LAV (Liu et al.) 2024a) LLaVA-NeXT (also referred to as LLaVA-1.6) builds upon LL4
by introducing dynamic resolution mechanism aimed improving visual perception. Instead of using a fixed resolution, the model selects an optimal aspect ratio based the original image and increases its resolution by up to 4x. Importantly, the visual encoder remains unchanged. To handle the resoutinnoolutinn inputs, the image is divided into multiple sub-images of the original size. Each sub-image is encoded indepen- dently, and the resulting visual tokens are concatenated and passed to the language model. This approach enhances the model's performance in tasks such as visual reasoning, optical character recognition (OCR), and knowledge-intensinn questions. For consistency and fair comparison, we fix the resolution to 672x672 (4x the original), generating 2,880 visual tokens per image.
ViaVA-iider (Zhang et al. 2024d) LLaVA-Video is variant of the LLaVA family designed specifically for video understanding. It introduces the SlowFast frame sampling strategy to balance the nunber of frames and the density of visual tokens. The model utilizes SigLIP (Zhai et al. 2023) as the visual encoder and processes video frames at 384x384 resolution, encoding each frame into 729 visual tokens. To reduce computational load, a 2x2 average pooling operation is applied to the grid features, effectively reducing the number of visual tokens by a factor of 4. During evaluation, we uniformly sample 64 frames per video, resulting in a total of 10,816 visual tokens. This design allows LLa Video to efficiently model both spatial and temporal aspects of visual input.
InternVL3 (Zhu et al. 2025) One of the most advanced open-source MLLMs at present. Building upon its predecessor, InternVL2.55 it retains the ViT-MLP-LLM, architecture, integrating Vision Transformer with a large language model through an MLP connector. InternVL3 features a native multimodal pre- training paradigın, jointly acquiring linguistic and multimodal capabilities in a single stage. incorporates Variable Visual Position Encoding to handle extended multimodal contexts and employs advanced training techniques like supervised fine-tuning and mixed preference optimization. InternVL3 demonstrates superior performance across a wide range of multimodal tasks, including tool usage, GUI agents, industrial image analysis, and 3D vision perception.
C.2 Evaluation Benchmarks
C.2.1 General Image Benchmarks
VQAv2 (Goyal et al. 2017) An open-ended visual question answering benchmark that evaluates a model's ability to understand images, natural language, and commonsense knowledge. It contains 265,016 images from the COCO dataset (Lin et al., 2014) and abstract scenes, with each image paired with an average of 5.4 questions. Each question is annotated with 10 ground truth answers and 3 plausible alternatives. We use the test-dev split for evaluation.
GQA (Hudson & Manning 2019) A large-scale and widely used VQA benchmark based on real-world images from the Visual Genome dataset KKishna et al. 2017), specifically designed to test compositional reasoning and fine-grained visual understanding. It provides over million balanced question-answer pairs, with each image accompanied by a detailed scene graph explicitly describing objects, attributes, and rela- tionships. For our experiments, we evaluate on the standard test-dev balanced split.
28

---

Published in Transactions on Machine Learning Research (11/2025)
WizWiz (Gurari et al., 2018) A real-world VQA benchmark created from images taken by blind users, paired with spoken questions and 10 crowd-annotated answers each. It introduces two main challenges: answering questions and detecting unanswerable ones, due to issues like poor image quality and ambiguous content. We use the test split for evaluation.
ScienceQA (Lu al., 2022) multimodal, multiple-choice QA benchmark covering diverse scientific domains. It includes 21,208 questions categorized across 26 topics, 127 categories, and 379 skills. Nearly half of the questions include image text context, and the majority are supplemented with grounded lectures and detailed explanations. We evaluate using the test split for evaluation.
POPE (Li et al. 2023b) A benchmark focused on evaluating object hallucination in MLLMs. Using images from COCO, it formulates binary questions regarding the presence of specific objects in the scene. Precision, recall, and F1 score are used to quantify hallucination. We use the test split for evaluation.
MME (Fu al., 2023) A broad benchmark assessing the perceptual and cognitive abilities of multimodal large language models comprises 14 sublasks across perception (e-g., counting, color, position, OCR) and cognition (e.g., commonsense reasoning, translation, code understanding). All binary instruction-answer pairs are manually constructed to avoid data leakage, ensuring rigorous evaluation.
MMBench (Liu et al., 2024c) A comprehensive benchmark designed to evaluate a wide range of mul- timodal capabilities. It features a large and diverse set of questions, surpassing prior benchmarks in scale and coverage. A novel CircularEval strategy, powered by ChatGPT, converts open-ended responses into structured formats for consistent scoring. Chinese version, MMBench-CN, CN, is also provided.
MM-Vet (Yu ct al., 2023) A benchmark emphasizing the integration of diverse multimodal skills. It defines six core capabilities: recognition, OCR, knowledge, language generation, spatial reasoning, and mathematics, via 218 carefully designed challenging examples. Evaluation is conducted using ChatGPT to ensure consistency across varied real-world answer formats.
HallusionBench (Guan et al. 2024) An image-contexl reasoning benchmark crafted to expose two frequent failure modes of large vision language models: language hallucination (answers driven by strong linguistic priors that contradict the image) and visual illusion (misleading visual features that produce confident yet wrong responses). Comprising carefully designed examples that remain challenging for GPT- 4V and LLaVA-1.5, it enables fine-grained diagnosis of how VL.Ms over-trust. language or under-exploit vision, offering insights for building more faithfully grounded models.
C.2.2 Eext-Oriented Benchmarks
VQA (Singh et al., 2019) A benchmark designed to evaluate models' ability to read and reason about text embedded in images. Sourced from the Open Images v3 dataset (Krasin et al., 2017): ; it includes scenes rich in lextual content such as signs and packaging. The benchmark emphasizes integration of OCR with visual and linguistic reasoning. We use the validation split for evaluation.
AI2D (Kembhavi et al., 2016) A diagram-based question answering benchmark consisting of over 5,000 grade school science diagrams, annotated with more than 150,000 structured labels and ground-truth syntac- tic parses. It also includes over 15,000 multiple-choie questions aligned with the diagrams, enabling research on visual reasoning and diagram understanding in scientific contexts. We use the test split for evaluation.
ChartQA (Masry et al., 2022) A large-scale benchmark designed for question answering over charts, focusing on complex rcasoning that involves both visual interpretation and logical or arithmotic operations. It includes 9.6K human-written questions and 23.1K questions generated from chart summaries. Unlike prior remplate-based benchmarks, ChartQA challenges models to perform multi-step reasoning using both the visual content and underlying data tables of charts, highlighting the need for advanced multimodal understanding. We use the test split for evaluation.
OCRBench (Liu et al. 2024d) A comprehensive evaluation benchmark assessing the OCR capabilities of large multimodal models. It comprises 29 datascts across diverse text-related visual tasks, including text recognition, scene text-centric VQA, document-orientt VQA, key information extraction, and handwritten mathematical expression recognition.
29

---

Published in Transactions on Machine Learning Research (11/2025)
C.2.3 Video Benchmarks
MLVU (Zhou et al., 2025) The first large-scale benchmark for long video understanding. features videos ranging from minules to hours and spans nine lasks covering holistic, single-detail, and multi- detail understanding. Both multiple-choice and open-ended questions are included. We report performance using the M-Avg metric.
MVBench (Li et al. 2023a) A benchmark tailored for evaluating temporal reasoning in video compre- hension. It includes 20 carefully designed tasks requiring dynamic understanding across multiple frames. MVBench introduces a static-to-dynamic transformation approach to systematically test temporal under- standing. We use the test split for evaluation.
Bong (Wu et al., 2024) A large-scale benchmark for ovaluating undorstanding of long-form videos. It consists of 3,763 videos (up to 1 hour long); each accompanied by subtitles and 6,678 human- written multiple-choice questions across 17 categories. A key feature is the referring reasoning task, where questions target specific video segments. We use the validation split for evaluation.
Video-MME (Fu et al.; 2025) A comprehensive video benchmark featuring 900 expert-curated videos spanning 256 hours across six primary domains and 30 subfields. Videos range from 11 seconds to hour in duration and include video, audio, and subtitles (not used during evaluation). It provides 2,700 expert- annotated QA pairs designed probe complex temporal and multimodal reasoning abilities.
C.3 Comparison Methods
C.3.1 Text-based Methods
Fast (Chen et al., 2024a) The first work to identify inefficiencies in visual attention within MLLMs. Based this observation, FastV proposes simple, training-fres frre acceleration method: after the second transformer layer, it pruncs portion of visual tokens with the lowest visual-text attention scores. This strategy significantly reduces computational cost during inference without retraining.
SparseVLM (Zhang et al.) 2024c) Inspired by multi-stage pruning methods such as PyramidDrop, SparscVLM introduces more fine-grained strategy that incorporates textual guidance. observes that not all instruction tokens are equally informative for pruning visual tokens. Therefore, it first selects text tokens highly relevant to the visual content "raters," ! and uses their attention distribution guide which visual tokens should be preserved pruned, resulting in improved model efficiency and accuracy.
C.3.2 Vision-based Methods
TRIM (Song et al., 2025) Pruning based solely on visual input, while ignoring textual context, may lcad suboptimal decisions. TRIM addresses this issue by utilizing CLIP-based similarity. It computes cosine similarities between image tokens (from the visual encoder) and text tokens (Irom the text encoder), and uses these similarity scores to rank visual tokens by importance. Low-similarity tokens pruned to accelerate inference without significant performance loss.
VisionZip (Yang et al.) 2025b) A visual-only loken pruning method that analyzes self-attention concen- tration in the visual encoder. VisionZip first selects dominant tokens with high self-attention weights. Then, it applies clustering on the remaining tokens to extract diverse contextual tokens. The union of dominant and contextual tokens is passed to the language model, aiming to preserve both saliency and diversity of visual content.
C.3.3 Similarity-based Methods
DivPrune (Alvar et al. 2025) DivPrune reformulates token pruning as a Maximum Minimum Distance Problem (MMDP), aiming to select the most diverse subset of visual tokens. Rather than relying solely on attention or similarity scores, explicitly maximizes the minimum pairwise distance among retained tokens, ensuring the selected tokens cover a broad semantic space. This diversity- diversity-preeerving strategy leads to robust performance under extreme token reduction.
30

---

Published in Transactions on Machine Learning Research (11/2025)
Table D.I: Performance comparisons on InternVL3-8B Zhu et al. (2025) across 10 image under-
standing benchmarks. The best results each setting are bolded, and the second-best are underlined.
Method
A121)
TextVQA
ChartQA OCRBench
HallBench
MME
MMB-EN
MMB-CN
Acc.
Average
Upper Bound, All 1280 Tokens (100%)
InteruVL3 8B
85.28
81.51
85.07
853
50.02
2393.22
83.82
82.54
84.22
100.0%
Rofain 256 Tokens (1 80.0%)
Fast
82.21
74.34
70.58
632
48.45
2348,31
83.96
82.04
77.98
92.6%
DivPrune
80.88
64.70
57.51
477
38.63
2249.17
80.63
80.27
70.41
82.80%
Script
82.87
75.97
72.20
640
48.98
2334.22
83.45
81.57
78.35
93.03%
Retain
128 Tokens (1 (.900%)
FastV
77.35
63.55
46.82
426
42.75
2250.31
81.09
80.20
68.41
80.9%
DivPrume Script
42.58
37.57
2166.22
78.49
77.55
64.31
75.75%
79.88
67.65
50.78
471
44.46
2282.98
82.12
80.53
70.98
84.29%
C.4 Implementation Details
For image-based benchmarks, adopt the official implementation of LLa loading the relcased check- points (e.g., LLa VA-1.5 and LLaVA-NeXT) and following the default preprocessing pipeline for image resizing (336x336 or 672),72), tokenization, and prompt formatting. All evaluations are conducted in a zero-shot setting unless otherwise specified. For aadoo-based benchmarks, we instead use the official implementation of LLAVA-NeXT which also supports LLaVA-Video, Vidcos are processed by uniformly sampling 64 frames, resizing each 384 x384, and pooling visual tokens as described in the original T.T.a VA-Vides paper. For evaluation, we further employ the lmms-eval toolki o which standardizes metric computation and dataset loading for long-form video understanding.
By default, Script in GSP is configured with = 0.3 and 7 5. All experiments are run on NVIDIA H100 GPUs (80GB) with 66 precision. Moreover, to ensure fair comparison across models and pruming strategies, we keep inference batch size and decoding settings (Lemperalure 0.2, lop-k 1) consistent. Finally, each benchmark is evaluated times, and we report the average as the final score.
C.5 Licenses
Table summarizes all benchmarks and software licenses employed in our study. The image understanding bench- marks such as GQA, VQAv2, Vex, and VizWiz are re- leased under the CC BY 4.0 license, while others, including ScienceQA, MME, and POPE, adopt the MIT license, and recent large-scale resources such as MMWe MMBench, and MVBench are distributed under Apache-2.0. For video rea- soning, we rely on Long BengVideeBench and MLVU (Apache- 2.0) as well as VideoMME (MIT). On the software side, multimodal models like LLaVA and V.AVA-E are cov- ered by the Llama Community License, whereas InternVL3 is open-sourced under Apache-2.0. We have reviewed the terms of each license, ensured that all datasets and models are publicly available, and strictly limited their usage to non- commercial, academic research purposes. No proprietary or closed-access resources are included, which guarantees re- producibility, transparency, and compliance with community standards.
Table C.I: License information for the scien- tific artifacts.
Data Sources
URL
License
Link
CC BY
ScienceQA VQAr2
Link
MIT
Link
CC BY 4.0
TextVQA
Link
CC BY 4.0
ViaW
Link
CC BY 4.0
MMVee
Link
Apache 2.0
MMBench
Link
Apache-2.0
MMBennh-CC
Aparhe-2.0
MME
MIT
POPE
Link
MIT
LongVideeBench
Link
Apache--20
VideoMME
Link
MIT
MIVU
Link
Apache-2.0
MVBench
Link
Apache 2.0
Software Code
URL
License
LLaVA
Link
Llama Community Licenee
LEXXA-NEXT
Link
Llaus Community Licence
InternV13
Link
Apeche-2.0
httppr//hhhabbbomm///iiaani/a///
31

---

Published in Transactions on Machine Learning Research (11/2025)
Table D.2: Performance comparisons on LLaVA-1.5-13B Liu et al. (2024a) across 10 image
understanding benchmarks.
Mothod
Veune
VQA
GQA
VieWix SQA VQA
POPE
MME
MMB MMB
MMVet
Acc.
Average
Upper Bound,
(1NN6),
TFLOPs
LLAVA-1---38
138 Liu
al.
2024
Nijs 227
80.10
63.36
03.36
72.86
61.24
56.001
1631.25
08.66
63.55
36.22
06.21
100%
Retai
Tiokesus
Aesrrue
77.855),
-0..00 TPLOPs
V Chen
2023
BCCV14
70.53
08.13
64.46
58.76
70.00
1490.56
06.01
02.23
32.75
63.17
95.41%
TRIM Song 0 m
234g
COLING'SS 155
76.34
09:14
49.87
72.14
50.08
56.38
1426-49
67.11
ts.14
25.13
63.06
56.24%
Viiion24p
CM/PR'55
76.85
57.79
02.03
03.68
58.30
62.77
1449.42
67.14
02.95
36.01
64.01
06.67%
DivPrrme
CVPR''
1657.97
34.41
SpanseV
L. Ziange 6t
JCML 20
77.61
69.46
51.74
74.22
59.53
85.02
1487.59
0s.14
02.35
25.92
044.7
97.917%
Script (Ours)
Proposed
77.87
59:27
$2.09
73.29
$8.45
$7.31
1498.30
67:15
61.35
36.12
64.88
97.64%%
Reloin
Toleess
Anneroge (4
1983,
~0.426
FFLOPs
V Clen et aL
2024a
BCCV 24
65.73
51.39
58.48
73.01
53.74
56.49
1240.54
39:12
$5.18
26.89
56.78
84.24%
TRIM
376m
GOLING 12
73.12
67.85
49.23
72.00
$2,10
86.60
1400-11
65.04
02.57
27.95
G0.73
91.72%
VisionZip
CVPR''5
1379.66
61.94
33.40
11%511%
DivPrume
Altar &
CVPR
75.20
$7.91
$4.49
71.67
57.54
$4.35
1454.29
61.13
59.87
20.31
022773
94. 94.74'%
SpunseVL.
Zhang et
201
ICMU 25
73.12
55.01
52.17
72.09
$7.14
77.91
1374.5
65.12
00:23
32.94
61.602
03.077
Script (Ours)
Proposed
76.87
87.19
1496.58
05.85
t8.72
366.211
966.96%
Retsin 32 Tolens 10
Amerogge (4 92, 92,335),
-00.205
27LO7%
Fast
Chen et m.
6CCV
61:13
48.36
51.088
72.87
54377
64.19
1158.38 33
54.27
10.22
23.65
612,11
79.95%
TRIM
COLANG'S 125
70.64
1254.57
63.13
VisimnZip
Yang et
2025 2775
CVP22'5
68.80
52.71
00
72.
53.20
66.85
1257 67
61.32
53.55 53.58
20.
57.81
87.622)
DivPrime
A Nar o a
CVPR
72.00
56.20
64.55
70. 80
54.76
79.12
1406 02
61. 47
57.
27.05
60.41
91. 25525
SpanxVL.
M.
ICML
71.57
54.05
51.54
70.86
53.74
77.45
1327.37
62.85 88
58.91
28.13
59.55
89.99 89.9475
Script (Ouis)
Propoaed
76.25
08.75
58.35
71 11.00
50.43
87.01
1421.10
63.79
50.36
20.77
62.41
94..6%%
Table D.3: Performance comparisons on LLaVA-Next-13B Liu et al. (2024a) across 8 image understanding benchmarks. The best results in each sctting are bolded, and the second-best are underlined.
Method
Veme
VQA GQA
VieWir SQA
VQA
POPE
MME
MMB
MMB
MMVet
Acc.
Average
Upper Bonnd,
Tahma (1007),
3.17
TRLOPs
LIVVA-NoXT-138 al.
3024a
Nip 335
82.30
64.55
10.15
73.15
63:30
85,20
1530.00
68.56
61.05
46.02
67.91
100%
Relwin 64/
Tolenta Averuge 77.875).
TFLOPs
FaeVV Clen et al.
3024a
RCGY22
79.34
60.89
56.44
71.75
60.74
80.24
1536.7
65.25
59.30
43.05
65.45
06.38%
TRIM
Soon al 236
COLING'AS
79.34
63.19
54.31
71.42
07.64
87.31
1543.96
68.75
63:21
42.84
66.26
97.67%
V Yi Yauk
CVPR 25
79.37
62.79
56,12
7U.83
61.91
80.
1520.22
08.31
02.61
46.98
07.13
58.84%
DivPrume
CVTR 25
80.34
63.54
56.73
72.12
59.42
86.44
1531.41
67.51
62.50
06.42
97.81%
SparnVL..
2imin
ICML
79.19
02.74
57.60
72.35
62.48
M7.
1078.74
08.85
61:30
41.35
67.41
90.11%
Scrips
(Ouis)
Proposed
81.14
64,22
57.02
72.08
61.40
H7.31
1522.61
08.35
61.91
47.18
67.87
99.96%
Xiefein 330
Zahens
Averoge
88,
88.095),
1/3 314 TILOP's
PartV Chen
ECCV'S
59.48
54.14
57.27
TRIM Sonr
COLING
75.04
61.13
50.12
60.50
62.18
87:43
1484.0
67.58
57.34
33.41
63.09
02.00%
Vasoe 4p Win-
3255
CVPR 25
76.18
60.37
54.84
70.12
60.17
82.
1417.15
65.45
62:53
41.21
64.50
94.96%
DisPrunc
CVPR 355
78.13
61.28
55,10
72.59
57. 07.46
85.
1665.01
65.91
61.50
30:02
64.98
95 089
Spor -T..
2hinng
195
76.75
70.94
1401.40
68:10
63-45
95-78%
Script [Ours]
Propowal
79664
63.11
55.31
71.46
G8.74
87.83
1500.15
05.77
61.48
42.392
66.16
9/7.12%
Mitein 167
Jalena Bn Averupe
~J. 156 FLLPPs
PadVV Chen al.
RCCV'2
62.67
50.79
$2.08
69.11
61.74
62.99
1229.18
56.48
53.16
80.00%
TRIM Sang
$300
COLING'A
72:15
58.01
51.12
69.19
49..12
87.00
13802.30
03.75
53.65
27.05
80.221
58.66%
ValoeZip Ving
CVPR 25
72.45
57.81
52.85
69.47
58.099
76. 385
12931.43
61.85
80001
35.39
65.77
50..667
DivPrume Xoval
CVPR 25
75.64
60,30
53.15
71.44
58.43
81.
16366.1
63.19
00.30
37.34
63.311
50.22%
Spar SaanWL.n
IGML M5
74.02
59.79
65.19
59.17
02.00
92.115%
Scripé (Ours)
Preposed
77.85
02.332
50.19
71.37
56.47
88.83
1476.99
63.49
59.91
40.314
64.96
95.65%
D Additional Experimental Results
D.1 Script for advanced open-source MLLM
In addition to LLaVA, we further apply Script to one of the most advanced open-source MLLMs to date, InternVL3. The results are shown in Table D.1 Here, we fix the input resolution to 896x896, yiclding 1,280 visual tokens. Notably, unlike its performance on the LLaVA series, DivPrune exhibits a significant performance drop on InternVI.3, as it does not account for the relevance to user instructions during pruning. In contrast, our Script jointly considers both diversity and relevance, consistently achieving the best perfor- mance across different reduction ratios. Specifically, even when 90% of the visual tokens are removed, our method retains 83.9% of the original performance, 3% higher than the second-best Fast demonstrating its effectiveness and adaptability in advanced MLLM architectures.
D.2 Script for Large Parameters of Model
To evaluate the effectiveness of our proposed method on larger language models, we apply Script to two models equipped with 13B L.LMs: L.AVA-1.5-13BB and LLaVA-NeXT-13B. The results are presented in
32

---

Published in Transactions on Machine Learning Research (11/2025)
Table D.4: Hyperparameter sensitivity analysis of Scripts on LLaVA-1.5-7B with 64 tokens retained. The hyperparameters include the graph threshold т (0.1, 0.3, 0.5, 0.7, 0.9), scaling factor (1, 10, 50, 100), and kernel choices (S vs. 5*). Bold is the default setting (Hyperparameterr)
Benchamark
Hyperparameter
Average Relative
VQA GQA VizWiz SQA IMG VQA Text POPE MME MMBEN EN MMB MMVet
Upper Bound, 576 Tokens (100%), 3.817
TFLOPs
61.94
64.09
58.10
1507.06
86.96
69.41
78.50
58.20
50.32
31.82
63.47
100%
(graph threshold,
0.3,
0.5, 0.7, 0.9)
59.07
61.30
51.43
1421.74
84.76
06.88
73.98
55.31
51.44
28.76
60.71
95.61%
0.3
59.28
61.90
52.93
1412.08
86.95
68.65
75.08
55.20
54.31
29.96
61.49
96.88%
0.5
59.33
61.90
52.71
1409.31
85.16
68.00
74.28
55.10
54.07
29.06
61.01
96.12%
0.7
58.41
60.92
52.48
1394.89
84.05
67.45
74.72
54.99
53.98
28.77
60.56
95.42%
0.9
58.77
60.52
51.94
1377.28
85.74
67.34
74.11
53.82
53.99
29.41
60.45
95.25%
[Sealing factor)
09.11 1
61.50
52.77
1399.22 229.22
86,80
66,40
74.55
53.90
53.50
29.87
60.83
96.85%
59.28
61.90
52.90
1412.08
86.95
68.65
75.08
55,20
54.31
29.96
61.49
96.88%
10
59.30
61.80
52.95
1415.20
86,75
67.32
74.68
55.70
51.10
29.20
61.25
96.51%
50
89.35
61.90
52.90
1407.75
86.60
67.75
73.88
54.75
54.20
29.55
61.13
96.31%
100
59.38
61.90
52.90
1412.02
86.90
68.20
74.35
54.66
54.30
30.30
61.35
96. 06.66%
Kernel Choices (S vs.
S*)
59.58
60.30
51.75
1399.20
85.40
66,25
75.30
54.40
53.11
28.77
60.17
95.28%
S"
59.28
61.90
52.93
1412.08
86.95
68.65
75.08
55.20
54.31
29.96
61.49
96.88%
Table D.2 and Table D.3 The larger language models lead to significant performance improvements and also make MLLMs less sensitive to visual token pruming. Among various pruming strategies, text-attention- based methods benefit the most from scaling up the language model, indicating that a larger LLM brings more accurate attention. Across different types of pruning methods, Script consistently outperforms all other approaches under various reduction ratios. With 77.8% of visual tokens removed, our method retains 97.64% and 99.95% of the original performance on LLaVA-1.5-133 and LLAVA-NeXT--3BB, respectively, demonstrating its effectiveness on larger language models.
D.3 Hyperparameter sensitivity & guidance.
To better understand the robustness of Scripts, we conduct a hyperparameter sensitivity analysis under the setting where 64 tokens are retained. Specifically, we vary the graph threshold т the scaling factor y in Eq. 9 and kernel choices (S vs. S'). As shown in Table D.4 our method demonstrates remarkable robustness to the choice of threshold: across 5 E [0.1, 0.9), the average relative performance consistently remains above 95%, indicating minimal sensitivity to this hyperparameter. Moreover, performance varies smoothly and predictably with 7, without instability abrupt drops, further confirming the stability and reliability of our approach under different settings. In practice, extremely low (0.1) or high (0.9) values lead to slightly worse performance, while moderate settings achieve a better trade-off. Among them, T = 0.3 yields the strongest overall results. For the scaling factor 7, all values produce competitive outcomes, but ? = 5 achioves the most consistent improvements across benchmarks. In terms of kernel design, S" consistently outperforms S, suggesting that it better captures redundancy patterns among tokens. In the comparison between S and S', the former represents the redundancy of tokens informed by visual similarity, while the latter incorporates both visual redundancy and query relevance. Based these observations, the default. configuration.
= 0.3, 7 and S'
It is worth noting that in Script, the parameter & in DPP is not treated as a tunable hyperparameter but as a direct control of the number of retained tokens. This is conceptually equivalent to the pruning ratio p in GSP, with the correspondence:
k: = (1-)),
(D.I)
where n is the number of original tokens. In other words, while p specifics the proportion of redundant tokens to be discarded, & explicitly determines the number of tokens to be preserved by ranking redundancy from low to high. Detailed results under different k settings are reported in Section 5 and Appendix [)
33

---

Published in Transactions on Machine Learning Research (11/2025)
Table D.5: Intersection dynamics (GSP n QQSPP)
Benchamark
Average
VQAY2 V2 GQA ViaWiz SQA IMG VQA Text POPE MME MMBEN MMB MMVet
Retain 192
Tokens Average (4 66. (6),
~1.259FFLO's
45.69
45.48
54.40
47.28
48.72
44.53
49.32
52.44
48.21
49.08
48.52
Retain 128
Tokens in Average () 77.8%), 00.833 TFLOPs
33.29
33.78
40.89
33.68
36.75
31.55
31.50
42.12
37.51
36.26
35.73
Retain 64
Tokens in Average (L 88.9%),
~0.415 TFLOPs
21.89
23.05
26.25
21.32
26.74
19.27
20.81
28.71
23.42
24.58
23.61
Retain 32 Tokens in Average (4 94.5%),
0.200 TFLOPs
15.75
16.45
17.49
15.53
19.38
11.95
14.85
15.51
18.83
18.44
16.42
Retain 16 Tokens in Average (L 91.3%),
~0.103 TFLOPs
10.73
11.16
10.83
11.77
11.57
6.82
9.84
11.72
12.62
12.94
11.00
D.4 Overlap analysis between GSP and QCSP.
GSP and QCSP are two complementary modules that operate from different perspectives and together determine which tokens are retained. Table D.5 quantifies the overlap size between the two modules under different retention budgets.
Overall, the overlap ratio decreases steadily as the pruning ratio increases. This trend is intuitive: when fewer tokens are allowed to remain, it becomes more difficult for both modules to consistently select the same tokens, given their distinct selection criteria. For example, retaining 192 tokens leads to an average overlap of 66.7%, while retaining only 16 tokens roduces the overlap to 97.3%, indicating that QCSP dominates token retention under stricter pruning.
Another noteworthy observation is that the overlap never reaches 100%, which necessitates the use of QCSP fallback mechanism to ensure the desired number of tokens is met. As the pruning becomes more aggressive, QCSP contributes proportionally more to the final selection. At the same time, our ablation studies confirm that GSP provides complementary benefits by capturing structural redundancy patterns that QCSP alone may overlook.
These results highlight the importance of the cooperative design: GSP guides pruning with a global structural view, while QCSP ensures query relevance and satisfies token budget constraints. Their interaction achieves balance between efficiency and accuracy, with QCSP gradually taking on a stronger role as the pruning ratio increases.
D.5 Statistical Significance Experiments
To assess the robust.nsss and statistical reliability of our method, we therefore conduct 20 indepen- dent runs using LLAVA-Next-7B under identical ex- perimental settings with varying random seeds, as shown Table D.6 All statistical comparisons are made consistently against the Script. To deter- mine whether Script is statistically better than other methods, accordingly perform paired one-sided t- tests and Wilcoxon signed-rank tests. We explicitly evaluate the null hypothesis Ho: Script performs the same as the others, against the alternative hypoth- esis H Script performs better. Since the resulting p-values are significantly lower than the significance threshold of 0.05, we can confidently reject the null hypothesis Ho- This suggests that the observed performance difference is statistically significant.
Table D.6: Statistical comparison between our method and other baselines on MME benchmark with ratain 160 tokens. StdDev denotes standard deviation. Re- ported p-values correspond to the paired one-sided t- test and Wilcoxon signed-rank test. All experiments are set with the significance level of a 5%.
Mothod
Mean (%)
StdDDv
1-test
Wilcoxon
BisVV
1076.81
2.62
1.01 x 10
9.54 x 10
TRIM VisionZip
1316.74
3.41
8.96 x 10-35
9.54 x 10-7
1349.28
36
9.54
DivPrume
1347.72
2.04
1.90
10
9.54 x 10-7
SparsoVL.M
1369.62
2.30
7.58 x 10
9.54 x 10-7
Script(Ours)
1487.98
0.94
1.00
1.00
34

---

Published in Transactions on Machine Learning Research (11/2025)
E
Additional Visualization Results
Visual Token Number
Input Image
Score
there a car ?
What is the number the athlete ?
Is there pizza ?
there cat and dish?
Is there cake and beer
What is food on the plate?
Figure F.1: Visualizations of relevance scores and retained tokens. Fach visualization illustrates the spatial attention allocated by models to regions corresponding to various textual instructions, demonstrating the capacity of pre-trained multimodal models to identify and focus on task-specific visual elements.
E.1
Case study
In this section, provide additional visualizations that comprehensively illustrate the relevance scores and retained visual Lokens as shown in Figures E.1 and E.2 These visualizations further emphasize the strengthn hs of models utilizing language-image pre-training, showcasing their enhanced capability to align textual in- structions accurately with the corresponding visual regions. Specifically, the visualizations depict clear and intuitive correspondences: for instance, when instructions involve detecting the presence of specific objects counting individuals animals, the relevance maps highlight pertinent regions directly associated with
35

---

Published in Transactions on Machine Learning Research (11/2025)
Visual Token Number
Input Image
Score
Is there bird
How many person in the picture
there a man playing with skateboard ?
How many cats in the picture ?
Is there man playing tennis ?
ETP
Is there man fire hydrant?
Figure E.2: Visualizations of relevance scores and retained tokens. Each visualization illustrates the spatial attention allocated by models to regions corresponding to various textual instructions, demonstrating the capacity of pre-trained multimodal models to identify and focus on task-specific visual elements.
these queries. Such precise spatial attention enables effective and efficient visual token pruning, significantly reducing redundancy by relaining only informative regions. This targeted relention not only improves com- putational efficiency but also boosts interpretability by transparently displaying the reasoning processes of multimodal large language models. Overall, these visualizations underscore the robustness and adaptability of pre-trained models in dynamically identifying and concentrating on task-specific visual details essential for accurate multinodal understanding.
36

---

Published in Transactions on Machine Learning Research (11/2025)
1.0
0.8
0.6
0.4
0.2
0.0
0.1
0.3
0.5
0.7
0.9
Structural Similarity Threshold
Figure E.3: Visualization of structural redundancy under different thresholds. We show patch-level rodundancy maps computed from CLIP-ViT features on images from the COCO datasct. Redundancy is defined as the average cosine similarity between each patch and its spatial neighbors. Red regions indicate high structural redundancy, while blue regions are more distinctive. As the threshold increases (from left right: 0.1 to 0.9), the resulting selection becomes increasingly sparse, focusing structurally salient regions.
E.2 Redundancy Visualization Across Thresholds
To better understand how structural redundancy varies across visual patches, we visualize the redundancy score maps under multiple thresholds (r 0.3, 0.5, 0.7, 0.9), as shown in Fig. E.3 For each image, a patch-wise redundancy score is computed by averaging the cosine similarity of each patch with its 8-connected spatial neighbors. Binary masks then obtained by applying different thresholds to these scores.
From these visualizations, several detailed observations emerge:
1. Very low thresholds saturate the map. At J = 0.1, almost every patch is marked redundant, producing a mask that is nearly saturated. This indicates that even weak local correlations are enough to pass the threshold, causing both background and structured regions to flagged. Such masks preserve nearly all information but fail distinguish between repetitive and unique content, providing little benefit for redundancy reduction.
2. Progressive sparsification with increasing thresholds. As increases to 0.3 and 0.5, the maps become less dense and begin to highlight differences between structurally homogeneous regions and struc- turally diverse ones. Patches in flat or repetitive textures remain marked as redundant, while more irregular patches start to drop out. This progression shows how the threshold acts as a filter, gradually shifting the representation from "inclusive" " to "selective."
3. Emergence of discriminative structures mid thresholds. Around 0.5, the contrast between redundant and non-redundant patches becomes clearer. Large homogencous regions consistently retained as redundant, but edges, boundaries, and irregular textures are less frequently included. This suggests that mid thresholds achieve a balance: they still capture redundancy for compression purposes, yet they allow distinctive features to remain available for recognition or further processing.
Strong filtering high thresholds. At 0.7 and especially 0.9, the redundancy maps become extremely sparse. Only the most distinctive patches, which sharply differ from their neighbors, survive non-redundant. This makes the mask highly selective, isolating unique structures while discarding most background and repeated patterns. However, such aggressiveness may also risk losing subtle contextual cues that are semantically useful.
5. Spatial clustering of redundancy. Across all thresholds, redundant patches tend not to appear as isolated points but rather form spatially contiguous clusters. This behavior indicates that redundancy is inherently a local phenomenon, strongly driven by the short-range similarity between neighboring patches. It also reinforces the idea that large uniform areas will consistently be flagged together, rather than in scattered fragments.
37

---

Published in Transactions on Machine Learning Research (11/2025)
Input Image
GSP
QCSP
Result
Ground Truth
(a)
No
Yes
Is there remote?
(b)
Yes
No
Is there person?
(c)
iPod
No
Yes
Is there a warning sign?
Figure E.4: Error case analysis. Representative failure patterns when pre-trained multimodal models process task-spocific queries, comparing query, Graph-Structured Pruning (GSP), Qunry-Conditioned Sc- mantic Pruning (QCSP), model prediction, and ground truth. (a) Model capabilily: success when textual guidance suffices (GSP correct, QCSP correct). (b) Test-visual misaligument: errors from limited CLIP encoder alignment (GSP correct, QCSP incorrect). (c) Visual feature deficiency: failures due to inadequate CLIP visual representations, leading to incorrect grounding and relevance (GSP incorrect, QCSP incorrect).
Summary of trade-offs. Threshold selection directly governs the density and informativeness of the redun- dancy maps. Low thresholds preserve nearly all content but provide little compression, while high thresholds isolate only the most distinctive patches but risk losing contextual information. Mid-level thresholds (e.g.. T == 0.5) offer favorable compromise, retaining structural diversity while still filtering out redundant clus- ters. This motivates our pruning strategy, which leverages structural redundancy to maintain discriminative information while reducing redundancy.
E.3
Error Analysis
The case study Figures E.1 and E.2 has provided an excellent visualization of a success case. To provide more balanced perspective, equally important to include and discuss failure cases. These highlight the inherent limitations of current pruning strategics and offer insight into potential directions for improvement. As shown in Figure E.4 highlight three illustrative types of errors, though in reality these calegories are not mutually exclusive and some overlap may exist.
(a) Model capability crror. This case illustrates the inherent limitations of the underlying LLM. Both GSP and QCSP assign relatively high retention scores the visual token corresponding the remote control. However, the final prediction is still incorrect. This suggests that even when
38

---

Published in Transactions on Machine Learning Research (11/2025)
query-relevant tokens preserved, reasoning failures may occur due to the LLM's limited capacity in associating visual evidence with the query semantics.
(b) Text visual misalignment. Here, QCSP assigns higher scores tokens related to the chairs under the parasol, while GSP distributes atlention in a scaltered manner. The model incorrectly concludes that person is present. This reflects semantic drift issue, where token relevance scoring does not fully capture the muanced mapping between textual concepts ("person") and visual regions ("chair"), leading to confusion in visually similar contexts. Such errors highlight the need for stronger cross moosl grounding mechanisms that can better align text queries with the correct visual entities.
(c) Compounded pruning error. In this example, both GSP and QCSP assign misleading values to tokens, causing the warning sign to be pruned or overlooked. As a result, the system outputs an incorrect response. Unlike case (a), the error here stems from cumulative misjudgments at the pruning stage rather than downstream reasoning. This indicates that pruning errors can directly suppress crucial evidence, creating an information bottleneck that the LLM cannot recover from.
In summary, these failure cases expose three distinct but complementary challenges: (i) intrinsic reasoning limits of the base LLM, (ii) misalignment between text semantics and visual grounding, and (iii) error propa- gation from aggregated pruning stralegies. Together, they emphasize that while Script substantially improves efficiency and often preserves accuracy, there remain scenarios where query-specific signals are underutilized entircly lost. Future work could addross these issucs by incorporating adaptive confidence calibration, multimodal consistency checks, or dynamic pruning thresholds that adjust based query difficulty.
F Limitations and Future Work.
One key limitation of our work is that the proposed pruning method requires direct access to encoded visual tokens during inference, which restricts its applicability to spercsource Multimodal Large Language Models. By contrast, widely used proprietary systems such as ChatGPT, Gemini, and Claude are closed black boxes where intermediate visual features are inaccessible. Although these models also incur high computational costs for visual reasoming, our method cannot be deployed in such settings.
Another limitation is that our evaluation is limited to vision-language inputs, assuming queries and context are restricted to text and visual tokens. We do not consider other modalities such as audio, depth, or IMU signals. Multimodal tasks like audio-grounded dialogue, audio-visual question answering, or egocentric video reasoning introduce challenges beyond vision-lanne language settings. For instance, audio streams are temporally dense and require aligument between speech content, prosody, and visual context, while depth and IMU data carry fine-grained geometric and motion cues that evolve continuously over time. Efficient pruning in such scenarios would require not only modality-specifie encoders but also redundancy metrics that capture temporal correlations (e.g.. repeated acoustic frames or redundant motion patterns) rather than purely spatial visual redundancy. While exploring these directions would broaden the scope of our approach, they fall outside the primary focus of this work and are left for future investigation.
Moreover, while our method is compatible with advanced open-source MLLMs such as Qwen2.5- VL and InternVL3, we observe that these models are more sensitive to visual token pruning compared to the LLaVA series. Under the same pruning ratio, they exhibit more significant performance degradation. This likely stems from the fact that such architectures already incorporate internal visual token compression tcchniques, such as pixel unshuffle or token merging, which reduce redundancy prior to pruning. As a result, additional pruning may lead to excessive information loss. Adapting pruning strategies to better suit these optimized architectures, for example, through pruning-aware training, modality-specific heuristics, or model-adaptive redundancy metrics, remains an important direction for future research.
Finally, our method requires manually selecting the threshold r. Although our sensitivity analysis (Table shows relatively stable performance across a wide range of values and we provide default rocommendations, the threshold remains fixed rather than adaptively determined. Future work could explore automated or barning-based strategies for setting t, such validation-driven calibration, distribution-awar rules (e.g..
39

---

Published in Transactions on Machine Learning Research (11/2025)
Otsu quantile-based selection similarity scores), meta-learning schemes that condition image/- query characteristics. Such strategies may further improve robustness under domain shift and heterogeneous token redundancy proflee.. We believe addressing this limitation will enhance the practicality and general- izability pruning in multimodal systems.
G Broader Impacts
Multimodal large language models have demonstrated remarkable success across a wide range of domains, including education, accessibility, robotics, and content creation, Despite their capabilities, these models often incur high inference costs, particularly when processing high-resolution images or extended vidco sequences. Such computational demands pose substantial challenges for real-world deployment.
In this work, introduce simple yet effective visual token pruming strategy that enhances inference efficiency without requiring any additional training. By selectively removing redundant visual inputs, our method significantly reduces computational overhead and deployment costs. This improvement facilitates the deployment of MLLMs on resource-constrained platforms, such as mobile devices and edge computing environments, thereby promoting broader accessibility and scalability in practical applications.
However, it important to note that improving computational efficiency does not inherently address the ethical challenges associated with MLLMs. Our method does not mitigate risks related to potential misuse, such as the generation of harmful content the dissemination of misinformation. As such, continued research and policy development are essential to ensure the responsible and safe use of increasingly efficient. multimodal language models.
40


Reviews:
[{"review_id": "lpXqCZWCZh", "reviewer": "Reviewer_ok18", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper’s subject is in the area of visual token pruning to enhance performance of multimodal large language models (MLLMs) while preserving high accuracy by integrating query relevance queues.\n\nThe approach comprises a graph-structured pruning module that removes visually redundant tokens (Thus enhancing performance (Memory and computation time requirements)) and a query-conditioned semantic pruning module that preserves query-relevant visual information contrary to many existing approaches that focus only on eliminating visual redundancy. \nThe vision-token pruning uses Cosine similarity distance.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Authors claims a new pruning approach (Script) that requires no retraining and generalizes across diverse MLLMs.\nExperiments presented are convincing and use of a bipartite graph as built using the cosine similarity yielding accuracy in par with other approaches while processing only one fraction of tokens hence reducing in a significant proportion, processing power and time needed.", "audience_yn": "Yes", "audience_explanation": "Overall, the proposed approach combines query semantic-relevancy with more used visual token pruning, while achieving on-par performance with other existing approaches.", "requested_changes": "- Regarding authors’ inferred Insight 1, one can also estimate that similarity of near vs long-range tokens depends on the type of image/scene analyzed and impacts the shape of the right side part of Figure 3(a) Average Cosine similarity vs spatial distance curve.\n- Generalization to various types of image data sets has to be carefully selected though as many similar textures/patterns, objects of interest repeat either in indoor images or outdoor scenery. These patterns might pertain to semantically similar objects (furniture, wall, decorations, utensils etc.) in indoor images or patterns/textures in outdoor scenery.\n- On the POPE Benchmark with a 90% pruning ratio, the authors’ approach achieves an accuracy of 84.49%, while the random approach is equally fast but drops to 78.63% accuracy.\n     - Would this be related to the markedly higher semantic importance/attention scores of certain tokens compared to the majority of the other tokens in the chosen images? \n     - Would this difference be lower for more homogeneous token images?", "broader_impact_concerns": "NA.", "additional_comments": "Is formula (2), pg 5 complete?", "extra_scores": {}, "extra_text": {}}, {"review_id": "sAlNNXzeIN", "reviewer": "Reviewer_uTeB", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a graph-based method to identify and remove locally redundant visual patches based on visual similarity, effectively compressing visual tokens while preserving semantic information. Extensive validation demonstrates superior performance and robustness across various large-scale models (including 13B parameter models) and datasets, outperforming prior methods in accuracy and efficiency even after substantial token reduction.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The claims made in the submission are supported by accurate, convincing, and clear evidence. The paper provides visualizations, such as relevance maps and token retention visuals, which illustrate how the pruning strategy effectively retains task-relevant visual information. It also demonstrates the robustness and effectiveness of the proposed method through extensive quantitative evaluations across multiple benchmarks and models.", "audience_yn": "Yes", "audience_explanation": "The paper presents a new method for visual token pruning in multimodal large language models, which addresses challenges related to inference efficiency and deployment costs.", "requested_changes": "The proposed approach relies on a threshold. It would be better to offer more detailed guidelines or adaptive strategies for selecting the similarity threshold τ, potentially including an automated or learning-based approach, to enhance reproducibility and ease of use.\n\nFont in Figure 7 can be smaller.", "broader_impact_concerns": "N/A", "additional_comments": "N/A", "extra_scores": {}, "extra_text": {}}, {"review_id": "kWS321mYmb", "reviewer": "Reviewer_CQeb", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "## Summary of the paper\n\nThis paper introduces \"Script,\" which is a training-free visual token pruning method designed to mitigate the high computational cost associated with the large number of visual tokens in Multimodal Large Language Models (MLLMs). \nThe authors identify the key limitations of existing pruning paradigms: attention-based methods are susceptible to biases like \"attention sink,\" while similarity- and divergence-based methods are often query-agnostic, leading to the removal of task-relevant information.\n\nTo address this, Script employs a two-main process. (1) Graph-Structured Pruning (GSP) constructs an efficient bipartite graph to model both local and long-range visual redundancy, allowing for the effective removal of semantically repetitive tokens. (2) Query-Conditioned Semantic Pruning (QCSP) leverages the user's text query to score the relevance of the tokens and uses a Determinantal Point Process (DPP) to select a final subset that is not only relevant but also diverse. \nA simple intersection-based fusion after the two main processes keeps tokens that are both visually compact (GSP) and query-aligned (QCSP).\n\n## List of strengths and Weaknesses\n### 1. strengths\n- Strong results and ablations: Script consistently matches or beats FastV, TRIM, VisionZip, DivPrune, and SparseVLM across token budgets; ablations show complementary gains from GSP and QCSP (e.g., 92.19% relative accuracy with only ~2.7% tokens retained).\n- The DPP kernel cleanly balances relevance and diversity; the derivation clarifies why det(L_I) discourages redundant selections\n- Thorough Analysis and Presentation: The paper is exceptionally well-written and structured. The motivation for the method is clearly established.\n### 2. weaknesses\n- Modality scope (text–vision only). It seems the author treats MLLM as a vision–language model.\nThe evaluation setup assumes queries and context are restricted to text and visual tokens; the authors do not model audio (or other modalities such as depth or IMU). Reported efficiency/accuracy gains on vision–language benchmarks therefore may not generalize to audio-conditioned or audio-visual tasks without adding an audio encoder, modality-aware token budgets, and temporally aware redundancy scoring and evaluation. \n- Missing sensitivity study for tunables (for example, $\\gamma$ in GSP’s redundancy score)\n- (Minor) Heuristic for Final Token Selection: The final token set is determined by the intersection of the GSP and QCSP outputs. The paper mentions a fallback mechanism to supplement this set if the intersection is too small, but a deeper analysis of the dynamics of this intersection (e.g., typical overlap size, frequency of fallback) would provide a more complete picture of the interplay between the two modules.", "claims_and_evidence_yn": "Yes", "claims_explanation": "This paper presents a strong and well-validated contribution. The authors first clearly articulate the limitations of existing pruning methods, including the \"attention sink\" phenomenon, which effectively motivates their proposed solution of query-conditioned pruning.\n\nThe proposed method Script is also well explained. The authors provide a clear mathematical foundation using Determinantal Point Processes (DPP) and detail an efficient greedy algorithm for implementation. The empirics are equally compelling, demonstrating consistent performance gains over strong baselines across multiple models (LLaVA-1.5, LLaVA-NeXT, Video-LLaVA) and token budgets.\n\nThe comprehensive efficiency analysis (reporting FLOPs, latency, KV cache, memory, and accuracy) shows that Script achieves a superior efficiency-accuracy trade-off.", "audience_yn": "Yes", "audience_explanation": "Token pruning for MLLMs is a timely topic for scalability and deployment. Script’s training-free, query-aware, and diversity-promoting approach, coupled with substantial speedups and KV-cache savings on widely used LLaVA testbeds and standard benchmarks, should interest both efficiency and multimodal reasoning researchers", "requested_changes": "- Hyperparameter sensitivity & guidance. Provide analysis for $\\tau$ (graph threshold), $\\gamma$ (Eq. 9 scaling), target $k$ (selected tokens), and kernel choices ($S$ vs. $S’$). Include recommended default values and ranges, and robustness plots across several datasets.\n- Video-specific notes. Briefly discuss temporal redundancy handling and whether GSP considers frame-wise vs. spatiotemporal neighborhoods; connect to the Video-LLaVA results table.\n- Intersection dynamics (GSP ∩ QCSP). Quantify the interplay between modules: typical overlap size, variance across datasets/prompts, and fallback frequency (how often/top-up amount when $|\\text{GSP} \\cap \\text{QCSP}|<k)$. Add a short table or histogram and discuss when each module dominates.\n- Qualitative Failure Case Analysis: The case study in Figure 5 provides an excellent visualization of a success case. To provide a more balanced perspective, it would be valuable to include and discuss a qualitative failure case. \n- Modality scope exploration. A short paragraph in Limitations clarifying that the method (and pruning scores) are tuned to text–vision. Outline what would be needed to extend to audio (encoder, modality-aware budgets, temporal/spectral redundancy scores) and why results may or may not directly transfer.", "broader_impact_concerns": "N/A. The authors have included a \"Broader Impacts\" section (Section G) that adequately addresses the potential societal implications of this work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Fki8tfiaR4", "reviewer": "Reviewer_w6Ja", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The work tackles the issue of visual token explosion in MLLMs, presenting a practical solution for this. The dual pruning strategy, balances efficiency with semantic preservation. Being plug-and-play with no retraining requirement makes the method widely applicable across different MLLMs. Extensive experiments on fourteen benchmarks, covering both image and video understanding, demonstrate strong and consistent results. The reported gains (up to 6.8× prefill speedup, 10× FLOP reduction, and retention of nearly 97% performance) are nice and show practical significance.\nThe abstract communicates motivation, approach, and contributions clearly and succinctly.\n\nThe integration of graph-structured redundancy reduction with DPP-based query-aware pruning is original and well-justified. By modeling token-query interactions, Script avoids the pitfalls of prior methods that either rely too heavily on attention scores or ignore query conditioning.\n\nThe paper communicates technical details clearly, with motivating examples (like flower/pineapple/lime cases) and well-constructed figures that highlight both shortcomings of existing methods and the advantages of Script.", "claims_and_evidence_yn": "Yes", "claims_explanation": "See above", "audience_yn": "Yes", "audience_explanation": "See above", "requested_changes": "N/A", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)