Enhancing Visual Token Representations for Video Large Language Models via Training-free Spatial-Temporal Pooling and Gridding
Authors: Bingjun Luo
Tony Wang
Hanqi Chen
Xinpeng Ding
Venue: ICLR 2026 Poster (2026)
Keywords: Visual Token Representation
Video Understanding
Multimodal Large Language Models
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: Our training-free method, ST-GridPool, boosts Video LLM performance and efficiency by intelligently compressing visual tokens based on their spatiotemporal importance.

Abstract: Recent advances in Multimodal Large Language Models (MLLMs) have significantly advanced video understanding tasks, yet challenges remain in efficiently compressing visual tokens while preserving spatiotemporal interactions. Existing methods, such as LLaVA family, utilize simplistic pooling or interpolation techniques that overlook the intricate dynamics of visual tokens. To bridge this gap, we propose ST-GridPool, a novel training-free visual token enhancement method designed specifically for Video LLMs. Our approach integrates Pyramid Temporal Gridding (PTG), which captures multi-grained spatiotemporal interactions through hierarchical temporal gridding, and Norm-based Spatial Pooling (NSP), which preserves high-information visual regions by leveraging the correlation between token norms and semantic richness. Extensive experiments on various benchmarks demonstrate that ST-GridPool consistently enhances performance of Video LLMs without requiring costly retraining. 
Our method offers an efficient and plug-and-play solution for improving visual token representations.
Our code is available in [https://github.com/bingjunluo/ST-GridPool](https://github.com/bingjunluo/ST-GridPool).

Published as a conference paper at ICLR 2026
ENHANCING VISUAL TOKEN REPRESENTATIONS FOR VIDEO LARGE LANGUAGE MODELS VIA TRAINING- FREE SPATIAL-TEMPORE POOLING AND GRIDDING
Bingjun Luo', Tony Wang', Hanqi Chen', Xinpeng Ding
'Tsinghua University
Shenzzeen University
'Xidian University
jingginluooouteeoommm com,
tonyrang5458@mail.ccomm xdigafeconeect.us .st .k
2550101019ema11..so szu, edu.cu en.
Standard LLaVA-Vidco
LLaVA-Video + Ours
MV Hlench
TempCompass
Iepul
Irpui
Vaico Tower
Vaiion Tower
VideoMME
EgeSchema
Pyranit Torpana
Norre-toard Sputtal Preling
Too
Test
Longuoge Model
Langsage Modkl
LinVVidtelienc
NEXT-QA
VLVVA-Vido
(a) Comparison of standard LLaVA- Video and our method.
(b) Performance improvement ST-GridPool.
Figure 1: The proposed ST Cr-GriiPool enhances visual token representations while maintaining com- putational efficiency in training-free manner, improving video understanding performance of widely adopted Video LLMs. Figure Ia illustrates the visual token construction of the standard Video LLMs and our method. Iigure presents the performance improvement of ST-GridPool on LLa and LLa VA-One Vision model across various video understanding tasks.
ABSTRACT
Recent advances in Multimodal Large Language Models (MLLMs) have signif- icantly advanced video understanding tasks, yet challenges remain efficiently compressing visual tokens while preserving spatiotemporal interactions. Existing methods, such as LLaVA family, utilize simplistic pooling or interpolation tech- niques that overlook the intricate dynamics of visual tokens. To bridge this gap, we propose ST-GridPool, novel training-free visual token enhancement method designed specifically for Video LLMs. Our approach integrates Pyramid Tem- poral Gridding (PTG), which captures multi-grained spatiotemporal interactions through hierarchical temporal gridding, and Norm-based Spatial Pooling (NSP), which preserves high-information visual regions by leveraging the correlation between token norms and semantic richness. Extensive experiments various benchmarks demonstrate that ST-GridPool consistently enhances performance of Video LLMs without requiring costly retraining. Our method offers efficient and plug-and-plas solution for improving visual token representations. Our code is available in https//gghuubb.mmmbbggiuuuuu///u
INTRODUCTION
Recent advances in Multimodal Large Language Models (MLLMs) have revolutionized multimodal understanding, delivering breakthroughs in image captioning, cross-modal retrieval, and video rea-
"Corresponding author.

---

Published as a conference paper at ICLR 2026
soning (Radford al. 2021| Li et al. 2023a Liu al. 2023 Zhang et al. 2024d). In these models, the quadratic complexity of self-attention mechanisms poses strict limitations on the input token length (Keles et al. 2023 Shao et al., 2025). However, video understanding inherently requires dense spatiotemporal analysis across thousands of visual tokens to capture precise motion dynamics and scene evolution (Xu al. 2024a). Therefore, it is pivotal challenge strengthen visual token representations for better video understanding within the bounds of token length and computational limitations.
As shown in Figure M existing MLLMs, such as the LLaVA family et 2023 Li et al. 2024a
Zhang et al., 2024d). typically employ straightforward 2D pooling or interpolation compress vi- sual tokens into an appropriate shape. While these approaches are computationally lightweight, they often overlook the intricate spatiotemporal interactions inherent visual tokens, leading to suboptimal performance. To address this problem, recent works like SF-LLaVA (Xu et al., 2024b) and TS-LLaVA (Qu et al.) 2024) have introduced advanced training-free techniques adapt Image LLMs for video understanding. These methods have demonstrated significant improvements age LLMs, even outperforming earlier Video LLMs (Xu et al.] 2024b). However, over the past year, the rapid evolution of Video LLMs has resulted in remarkable advancements in video understanding, rendering mere optimizations on Image LLMs insufficient. There is an urgent need for training- free visual token enhancement strategies specifically for Video LLMs.
Im-
To address these challenges, we propose ST-GridPool, a novel training-free visual token enhance- ment method tailored for Video L.Mss, which strategically refines visual tokens across both tempo- ral and spatial dimensions while maintaining computational efficiency. Our method is composed of two key components: Pyramid Temporal Gridding and Norm-based Spatial Pooling. In Pyramid Temporal Gridding (PTG), we design a hierarchical gridding strategy over the temporal dimen- sion, which grids and updates frame tokens from different segments of varying lengths. PTG en- ables multi-grained spatiotemporal feature extraction, capturing both short-term dynamics and long- term context without introducing additional trainable parameters. Norm-based Spatial Pooling (NSP), we systematically explore the positive correlation between the token norm and the seman- tic richness of visual tokens, leveraging this insight to propose a norm-based 2D dynamic pooling approach. This approach preserves high-norm regions while adaptively compressing low-energy backgrounds, ensuring that high-information visual regions are prioritized and retained. By inte- grating this mechanism, we maximize the preservation of semantically meaningful visual details in the pooling process, significantly enhancing the representation power of the resulting visual tokens. The proposed ST- ST-GridPool brings both components together to enhance visual token representations, achieving substantial improvements in video understanding tasks without requiring costly retraining or architectural modifications. This training-free paradigm offers a plug-and-plaa enhancement for existing Video LLMs like LLaVA- Video, demonstrating significant potential for video understand- ing applications. Our main contributions are summarized as follows:
We propose the first training-free visual token enhancement method specifically designed for Video LLMs. By optimizing the visual token compression process, our approach sig- nificantly improves video understanding performance while maintaining computational ef- ficiency.
Our Pyramid Temporal Gridding introduces a hierarchical gridding strategy that captures multi-grained spatiotemporal interactions across varying temporal lengths. Our Norm- based Spatial Pooling leverages the positive correlation between token norms and semantic importance to effectively preserve high-value visual information during token compression.
We conduct extensive experiments on 6 video understanding datasets using widely adopted Video LLMs, such as LLaVAVVideo Experimental results demonstrate that our method achieves consistent performance improvements across multiple models and diverse datasets.
RELATED WORK
2.1
VIDEO LARGE LANGUAGE MODELS
Video Large Language Models (Video LLMs) have emerged as a highly active research area on leveraging the powerful capabilities of large language models for video understanding. With the
2

---

Published as a conference paper at ICLR 2026
Input Video
Input Text
Response
Visuall Enceder
MM Projector
ST-GridPool
LLM
9- V. Baite
hacpppp
Pyramid Temporal Gridding
Norm-busee Spatial Pooling
Input Token Frames
Token Norm
W.W.4)
T1
T2
T3
T4
Tg
166
77
Ta
Sliding Window
Level-1
Token
Grids
P999
Sliding
Summrry Token
Susurnry Tokan
SefMie
Is
To Ta.
G11
Tg v4
G
Ty
Wrighted Sum
Weight
Stiding Window
IT2 TTo
T4
Token
62.1
T
Level-N Token Grids
Output
Figure 2: Overview of ST-GridPool. The method takes the token sequence Tı, , input and
outputs the pooled token ...... T which consists of main components: Pyramid Temporal Gridding and Norm-based Spatial Pooling.
rapid advancement LLMs, critical focus has emerged on harnessing their power for nuanced video analysis. Early efforts primarily adapt existing image-based LLM frameworks to the video field. VideoChat (Li et al., (023b) develops an end-to-end chat-centric system by bridging video foundation models and LLMs via learnable neural interface. PLLaVA (Xu et al. (022a) adapts lmage-language pre- pre-trained models for video tasks without additional parameters. Recent research focuses further enhancing video L.MSs from different aspects like cross-modal integration, long- context modeling, and efficiency optimization. LLaVA-One Vision (Li et al.) 2024a) pioneers a single model architecture that unifies image, multi-image, and video understanding. mPLUG-OW13 Ye et al. 2024) introduces hyper attention blocks to efficiently process long image sequences. NVILA (Liu al. 2024d) adopts "scale-then-oompres approach to optimize both accuracy and efficiency in visual token processing. Based on existing Video LLMs, our approach aims at constructing efficient yet information-dense visual token representations from massive video data.
2.2 VISUAL. TOKEN CONSTRUCTION FOR VIDEO LLM
Visual token construction serves critical bridge between raw video data and high-level seman- tic understanding in Video LLMs. Given the inherent redundancy of video content and the limited context windows of LLMs, extracting efficient yet informative visual tokens from long video quences is essential for enabling efficient and accurate video understanding and reasoning. Current visual token construction techniques can be categorized into the following three types. First, single- frame feature extraction methods (Li et al.) 2024a] Zhang et al. (024d) leverage image-language models like CLIP to extract keyframe embeddings. followed by spatial downsampling via average pooling bilinear interpolation. While computationally lightweight, these methods inherently ne- glect temporal dynamics and cross-frame dependencies. Second, temporal-aware modeling frame- works (Maaz et al. 2023 Xu al. 2024a) integrate spatiotemporal attention layers to aggregate frame-level features. capturing motion patterns at the cost of quadratic complexity for long videos. Third, token compression strategies (Liu al.) 2024d) dynamically prune redundant visual tokens through learnable spatial-temy temporal aggregation. To summarize, most token construction optimiza- tions require costly finetuning model-specific architecture modifications. There remains lack

---

Published as a conference paper at ICLR 2026
of parameter-free visual token enhancement methods designed for Video LLMs which can improve token informativeness without altering model parameters or increasing inference latency.
3
METHOD
Salient Object Detection
HKU-SS
Background
Density
VideoMME
Question: What color of the sivir that the speaker
Options:
Salient Object
Brows.
Gray.
Image
High-Norm Token
Token Norm
(a) Raw images, high-norm token and task of some (b) Token norm density distribution of the whole samples.
dataset.
Figure 3: Illustration of the visual token norm distribution discrepancy between salient object area and background area. (a) presents some samples with the raw image, the area of top 50% visual tokens terms of Le norm, and the saliency groundtruth question. (b) plots the density distribution of L2 token norm among all validation samples from HKU-IS.
3.1 PROBLEM DEFINITION
Given an input video V with R raw frames, conventional Video LLMs typically select N < R frames I1, 12, ... IN from it via fixed-interval uniform sampling. The selected frames are then extracted by the vision tower (-)), to generate frame tokens T. Ti.T2.... TN E #xxxd where H W is the spatial dimension, d is the feature dimension. In long-conter scenarios like video understanding. the raw visual token length N x H W is usually excessive, posing heavy computa- tional load and processing challenges for the downstream language model. Therefore, the raw visual tokens further reduced by the downsampling function Down(.) T T T The down- sampled visual tokens are then fed into the language model to generate the final response. In this process, the downsampling function Down(-) )) is quite crucial for the visual token representations, as must balance between reducing computational complexity and preserving critical spatiotemporal information, thereby achieving optimal video understanding performance.
3.2 METHOD OVERVIEW
As illustrated in Figure 2 the proposed ST-GridPool method consists of two components: Pyramid Temporal Gridding and Norm-based Spatial Pooling. Firstly, Pyramid Temporal Gridding (PTG) proposes hierarchical gridding strategy the temporal dimension, enabling multi-grained spa- tiotemporal feature extraction by combining and updating frame tokens from segments of varying lengths. Secondly, Norm-based Spatial Pooling (NSP) designs norm-based 2D weighted pooling mechanism that prioritizes high-norm regions during spatial pooling to preserve rich visual seman- ties. Through the joint operation these two components, ST-GridPool effectively enhances visual token representations in training-free free manner, achieving robust video understanding while main- taining computational efficiency.
3.3 PYRAMID TEMPORAL GRIDDING
Video LLMs like LLaVA- Video often adopt straightforward approaches by uniformly sampling and concatenating token sequences, implicitly assuming that temporal dynamics in videos are uniform
4

---

Published as a conference paper at ICLR 2026
across scales. However, real-world video content frequently encompasses spectrum of tempo- ral granularities, ranging from rapid micro-motions (e.g., hand gestures) to gradual displacements (e.g.. pedestrian walking), which demands multi-scale temporal modeling strategy. To address this limitation, we propose the Pyramid Temporal Gridding (PTG), a hierarchical module that par- titions the video sequence into multiple layers, each corresponding to distinct temporal segment lengths, PTG generates summary tokens at the end of each segment, enriching the temporal rep- resentation while maintaining computational efficiency. Given the original visual token sequence TI. T2.2.... ... .NN E I #Wxd, xd, the details of this module are as follows.
As shown Figure 2 PTG consists of L levels, each corresponding For I-th level (1 = I, ..... L), the segment length is defined as K2 = K 2'-1, where K is the base length of Level-1. The input visual tokens are divided into Nj segments, where N₁ = [N/K]] Therefore, the start frame index for the j-th segment in the 1-th level is:
specific segment length.
t. (j - 1) Kt, j == ..,
(I)
Each segment spans the frame range {tt.j, f1.5 1, 1,.... m..
Ki 1,N 1)). For example, for an input token sequence with the length N = 32, we set the base length K = 8, and number of
levels L = the input sequence will be divided into 3 layers: For Level-1, the sequence is divided into 4 segments, each with K1 = 8 frames; For Level-2, the sequence is divided into 2 segments, each with K₂ 16 frames; For Level-3, the sequence is divided into I segment, encompassing all K3- = 32 frames.
For the j-th segment at the (-th layer, summary token is generated to capture the temporal dynam- ics. First, m x n frames are uniformly sampled from the segment. The sampling indices are:
{&r k [Kg/ (m m)]
The corresponding token grids of these frames spatially concatenated to form intermediate token grid Gt.:
T +o
Te T +m++
T mm-1 T tu +2m-1
T +m
Gij=
(3)
T +n 1)m T Tn ljm+1
T mmn-
Here, each Th has a resolution of H x W, making the resolution of G1, mH x nW. Sub- sequently, bilinear interpolation is applied to resize Gt, back to the original resolution H x W, resulting in the final segment-end token grid Interp (Gr,) E #xx xd The last frame of the seg- ment is then updated with the generated summary token:
T Ki
update
Interp (G.)
By processing all segments across layers, the token sequence is updated incorporate multi-scale temporal dynamics, providing a richer representation for downstream tasks. The Pyramid Tempo- ral Gridding module not only captures fine-grained and coarse-grained temporal interactions but also ensures computational efficiency, making it robust foundation for spatiotemporal modeling in video understanding. The updated token sequence is subsequently passed to the Norm-based Spatial Pooling module for further refinement.
3.4 NORM-BASED SPATIAL POOLING
Video LLMs, such as LLaVA- Video, often rely on uniform 2D pooling or bilinear interpolation to downsample visual tokens in the spatial dimension. These methods, however, treat all spatial tokens equally, disregarding the inherent heterogeneity in their information content. As illustrated in Figure 3a| typical image frame is dominated by background regions, while semantically salient objects occupy only a small fraction of the spatial grid. By applying squal-weighted downsampling, these critical regions (rich in visual information) are inadequately prioritized, resulting in substantial information loss and token redundancy. This limitation underscores the need for a more refined approach to spatial pooling that dynamically weighs tokens based their semantic importance.
To address the aforementioned challenges, we design a novel spatial pooling method that dynami- cally prioritizes regions based on their information saliency while preserving critical visual features.
5

---

Published as a conference paper at ICLR 2026
To identify effective indicator of regional saliency, we conduct experiments the validation set of the HKU-IS salient object detection dataset (Li & Yu 2016). We perform both qualitative and quantitative analyses to examine the discrepancy in token norms between salient object
eas and background regions, as illustrated in Figure 3 In Figure [a we visualize the area of the top 50% visual tokens in terms of 1.2 norm alongside the salient object ground truth. The results clearly demonstrate that regions corresponding to salient objects consistently exhibit high token norms, while redundant background regions are associated with low token norms. Furthermore, in Figure 3b] we plot the norm distributions of visual tokens for both background and salient object regions. The quantitative analysis confirms a significant discrepancy in token norms between these two regions. This insight helps to establish token norm as reliable metric for assessing regional information saliency.
Inspired by these findings, we introduce Norm-based Spatial Pooling (NSP), dynamic pooling mechanism that leverages visual token norms to assign adaptive weights to each spatial location during the pooling process. By computing these weights based the L2 norm of visual tokens, NSP selectively amplifies the representation of high-importance regions, such as salient objects, while diminishing the influence of low-importance backgrounds. This weighting strategy ensures that semantically rich visual information is prioritized and preserved, leading to more efficient and effective token representations. Through this adaptive pooling approach, NSP significantly enhances Video L.LMs to focus on critical regions, ultimately improving the quality of visual token represen- tations for downstream tasks.
Specifically, the input to NSP is the visual token sequence T1, T2, TN E RHlxxxd xd derived from the PTG module. Here, N represents the number of frames, while H, W, and d denote the height, width, and feature dimension of each token, respectively. The pooling operation employs a kernel size of (kH. kw) and stride of (sH. sw). For an input visual token T, let t denote the sliding window corresponding to the f-th row and w-th column of the output pooled token T. (h, w). The elements of the sliding window are defined as:
mm T. SH m, w sw + n.),
where 0 S m < kn, 0 S < kww and h, w represent the spatial indices of the output feature map 0 represents the token at the upper-left
T For example, in a conventional 2x2 kernel, corner of the kernel window, while m = 1, n = 1 represents the token at the lower-right corner. For each visual feature t, within the sliding window, we first calculate its L,p norm, denoted as ||tma lp-- This norm is then normalized into a weight Om,n using the softmax function, ensuring that the weights sum to across the window:
(m
ky ==0
Σ exp((l)ta,5ll))
where 3 is a temperature parameter that controls the sharpness of the weight distribution. Finally, the pooling result for each sliding window is obtained as a weighted summation of the visual features:
T (h, w) = kn
kw
Σ
Om,n tm,
During the pooling process, the NSP mechanism leverages the intrinsic relationship between visual token norms and semantic saliency to dynamically prioritize high-importance regions. By integrat- ing a softmax- based weighting scheme, NSP adaptively enhances the representation of semantically rich visual features while suppressing less informative background regions. This approach not only preserves critical visual details but also maintains computational efficiency, making a plug-and- play enhancement for existing Video LLMs. As a result, NSP significantly improves the quality of visual token representations, enabling more robust and accurate video understanding.
EXPERIMENTS
4.1 EXPERIMENTAL SETTINGS
Baselines To validate the effectiveness of the proposed method, we employ two LLaVA family models as backbones: LLaVA-Onee One-OOeeeiinTP (Li et al. 2024a) and I.I. VA-VVooo--77 (Zhang al.)

---

Published as a conference paper at ICLR 2026
Long Video Understanding
General Video Understanding
Model
Long LongV.Bench V.Bench
TempCompast
VideoMME
EgoSchema
MVBench
NexT-QA
VideoLLaMA2.1-78
54.9
53.1
56.8
57.3
LongVA, VA-7B
52.6
68.3
IXC-2.5-7B
55.8
69.1
Intern Video2-7B
60.0
67.2
Oryx-1.5-7B
58.8
56.3
81.8
67.6
NVILA-8B
64.2
57.7
82.2
69.7
68.1
mPLUGGOWI3-8BB
53.5
52.1
78.6
54.5
Apollo-7B
61.3
58.5
64.9
LLaVA-One -nisonnnnVi
58.2
56.5
60.1
79.4
64.2
56.7
+ Ours
59.0
56.7
62.1
79.6
64.4
58.0
(+0.8%)
(+0.2%)
(+2.0%)
(+0.2%)
(+0.2%)
(+1.3%)
63.3
58.2
57.3
83.2
65.4
58.6
+ Ours
64.2
60.1
57.8
83.8
66.1
59.8
(+0.9%)
(+1.9%)
(+0.5%
(+0.6%)
(+0.7%)
(+1.2%%
Table 1: Overall comparison with state-of-the-art methods on long-form and general video under- standing benchmarks (%). The best performance among all methods underlined.
2024d). These models serve as the foundation for our evaluation, enabling us assess the per- formance improvements brought by our approach in video understanding tasks. Additionally, we
compare against a diverse set of baseline models ensure comprehensive analysis. These include VideoLLaMA2.1117 (Cheng et al. (222), Long LA-gVA-7B VA-7B (Zhang et al., 2024c), IXC-2.5-7B (Zhang al. 2024b), Intern Video2-7B (Wang al., 2024), Oryx-1.5-7B (Liu al. 2024e), NVILA-SB (Ye et 2024), and Apollo-7B (Zohar et al. 2024). These
(Liu et al. 2024d), mPLUG-Ow
models represent state-of-th of--oh---hhrr art advancements in video and multi-modas understanding, providing
robust framework for benchmarking our method.
Benchmarks To comprehensively evaluate the proposed method, we conduct experiments on benchmarks from both Long Video Understanding and General Video Understanding domains. For Long Video Understanding, 2025), and EgoSchema (Mangalam al. 2024), which focus on capturing temporal dependencies and reasoning extended video sequences. These benchmarks the ability of models to process and interpret long-range visual information. For General Video Understanding, we utilize NexT-QA (Xiao al. 2021), TempCompass (Liu al., (024c), and MVBench (Li et al. 2024b). These datasets are widely recognized for their diverse scenarios and challenging complexity.
adopt VideoMME (Fu et al.) (024). LongVideoBench (Wu al.)
specifically designed assess
4.2 IMPLEMENTATION DETAILS
Our experiments are conducted on NVIDIA L20 GPUs with Ubuntu 22.04. We utilize the Imms- eval (Zhang et al. 2024a) library for model modification and evaluation. To maintain consistency in computational load, we ensure identical input frame counts for the visual encoder and token counts for the LLM with the original models. For the LLaVA-One Vision model, we follow the original setup to feed 32 frames into the visual encoder, while for the LLaVA- VA-Video model, we adhere to the original setting of 64 input frames. We employ spatial pooling strategy with a kernel size and stride of 2. As for hyperparameter, we set the temperature parameter 3 = and the norm order p = 2, optimizing for stable performance across diverse video understanding tasks. This setup allowed us to validate the efficacy of our proposed methods while maintaining computational fairness in comparative evaluations.

---

Published as a conference paper at ICLR 2026
Method
VideoMME
L.V.Bench
EgoSchema
Upper Bound (Full Tokens)
LLaVA-Video
63.3
58.2
57.3
Token Budget Ratio: 30%
FastV
59.3
53.5
51.3
PruMerge
59.9
54.7
50.9
FasterVLM
60.1
55.8
52.6
VisionZip
58.3
53.2
53.0
FrameFusion
61.3
56.0
53.0
Ours
62.0
58.1
56.0
Token Budget Ratio: 50%
FastV
62.2
55.7
54.7
PruMerge
61.3
56.9
54.6
FasterVLM
61.7
56.4
56.2
VisionZip
60.6
56.8
54.2
FrameFusion
62.6
57.6
55.8
Ours
62.5
58.9
57.1
Table 2: Comparison with token reduction baselines on LLaVA-Video- (%).
4.3 COMPARISON STUDY
Comparison with SOTA We first present the overall results across all datasets in Table I which reveals the following key observations: (1) When integrated with the original LLaVA-One Vision and ILLVA- Video models, proposed method achieves consistent performance improvements across all evaluated datasets. (2) The ST-GridPool mechanism significantly enhances long-term video understanding, enabling method to surpass existing 7B models long-form benchmarks like VideoMME, Long LongVideoBench. and Egoschema (3) Our method consistently demonstrates com- petitive leading performance in general video understanding tasks. For example,
achieves 66.1 % on TempCompass, making it closer to SOTA performance of WVILA-8B. These ad- vancements highlight the adaptability of our approach across diverse video understanding scenarios while maintaining the training-frees advantage and efficient computational load.
Comparison with Token Reduction Methods To further evaluate the effectiveness of our pro- posed method as an efficient visual token compression strategy, we compare directly with sev- eral leading token reduction methods. As shown in Table 4 the experiments are performed LLaVA- VL-VA-ide---7 model using two different token budgets: 50% and 30%. The performance each method is evaluated on three long-video understanding benchmarks. (1) With the 50% token budget, our method shows highly competitive performance. It achieves the highest scores on the L. V.Bench and EgoSchema datasets, and its performance on VideoMME is comparable to the perffrmeronn method, FrameFusion. (2) The advantages of our method become more evident under the stricter 30% token budget, which involves a higher compression rate. Under these conditions, our method achieves the best performance on all three benchmarks, outperforming all other methods. (3) In summary, our proposed method performs well in standard compression scenarios and is also highly robust under high-compression (low-budget) conditions, This indicates that our approach more effectively identify and preserve the key information crucial for video understanding, especially when the token budget highly limited.
the
4.4 ABLATION STUDY
Effectiveness of Different Components To validate the effectiveness individual components in method, conduct ablation studies key components: Norm-batt based Spatial Pool- ing (NSP) and Pyramid Temporal Gridding (PTG). As shown Table 3, three key observations emerge: (1) The full integration of NSP and PTG achieves optimal performance, showing consistent
8

---

Published as a conference paper at ICLR 2026
Model
VideoMME
LongV.Bench
MVBench
Baseline
63.3
58.2
58.6
Ours w/o NSP
63.8
59.2
59.1
Ours w/o PTG
63.6
59.8
58.8
Ours
64.2
60.1
59.8
Table Ablation results of different components
00.4
00.2
64.0
651.3
63.0
02.5 0100
(a) VidcoMME
(b) Long VideoBench
Figure Ablation study results for different values temperature / and norm order p.
gains over both standalone components. This demonstrates their complementary roles in optimizing spatiotemporal token downsampling for video understanding, (2) Removing PTG (Ours w/o PTG) results performance drop, demonstrating its role in improving temporal feature aggregation. The absence of NSP (Ours w/o NSP) also leads to reduced performance, highlighting its contribution to optimizing spatial feature representation. (3) While baseline modifications with single components improve performance, their isolated effects remain suboptimal. Our unified framework leverages NSP for spatial saliency alignment and PTG for multiscale temporal reasoning, establishing their joint necessity for achieving state-of-the-art results.
Impact of 3 and Lp We analyze two critical hyperparameters in the Norm-aware Spatial Pooling (NSP) module: the temperature coefficient 3 and the norm order p of Typ To evaluate their impact, conduct experiments the LLaVA-Vides backbone with varying 8 {0.01, 0. 1, 5, 10) and
p = 21,2,3), and summarize results on VideoMME and Long VideoBench in Figure 4 Some observations be made as follows: (1) Performance first rises and then declines with increasing
3: Optimal results are achieved at / = 1, while extreme values (e.g.. 3 = or 3 10) degrade accuracy, likely due meo-ssno unstable feature activation. (2) Increasing p beyond leads to gradual performance degradation. The L2-norm achieves the highest scores across both datasets, balancing spatial sparsity and feature discriminability. (3) Despite sensitivity extreme values, our method shows consistent trends across different datasets. Variations within experiments yield subtle performance fluctuations, highlighting robustness. The interplay between and Lp reveals a delicate balance: B = 1 ensures appropriate sharpness in activation distributions, while La Lg-norm optimally aggregates spatial features without over-sparsity.
p
4.5 COMPUTATIONAL COST ANALYSIS
We analyzed the computational efficiency of ST-GridPool under strict 30% token budget. The ex- periments compare our method against the baseline LLaVA- V-VA--- model terms of inference time and peak GPU memory usage when processing different numbers of input frames. As shown in Fig. 5, our method achieves a dual optimization in computational efficiency. For inference time (left figure), ST-GridPool demonstrates a significant advantage, and the savings become more pro- nounced as the number of input frames increases. For GPU memory usage (right figure), our method also outperforms the baseline, consistently saving peak memory. In summary, ST-GridPool not only enhances video understanding performance but also substantially reduces both inference time and memory usage, proving its value highly efficient enhancement solution.

---

Published as a conference paper at ICLR 2026
Time Cost
Memory Cost
41.8
45.5%
6.50
275
25
35.8
7.7%
20.3
8.0%
40
6.2%
27.8
4.91
Redus Iaal (%)
(GB)
12
24.0
Averrgee Time
reduction retlo
3.13
2.64
3.77
Reduction
Figure Computational cost comparison between our method and the baseline ILaVA-Video model in inference time and GPU memory usage under 30% token budget.
4.6 QUALITATIVE ANALYSIS OF EXAMPLES
8
man wearing suit and puppla walkinng concrete road surounded by ures and parked cars. He has short hair and holding piece of hand. Whatt other appear
LLaVA-Video
LLaVA-Video Ours
a library
of green
backdrop
flag.
Figure 6: Response examples of LLLVA--Vide with and w/o Ours from Long VideoBench dataset.
We conduct qualitative analysis using diverse samples from the LongVideoBench dataset to eval- uate the performance of our method fine-graing fine-grained video understanding tasks. As illustrated in Fig- when processing long videos, our method effectively captures and correlates distant spatio- temporal information, enabling holistic understanding of extended events. These observations highlight our method's ability handle both short-term spatial reasoning and long-term temporal reasoning. By integrating multi-scale temporal gridding and adaptive spatial pooling, our approach achieves robustness and precision in complex video understanding scenarios. Such capabilities are crucial for applications requiring detailed analysis of dynamic and prolonged activities.
CONCLUSION
In this paper, we presented ST-GridPool, training-free visual token enhancement method designed to bridge the gap between efficient token compression and the preservation of intricate spatiotem- poral interactions in Video LLMs. Our framework integrates Pyramid Temporal Gridding (PTG) to capture multi-grane grained temporal dynamics through hierarchical segments and Norm-based Spa- tial Pooling (NSP), which prioritizes high-information regions by leveraging the positive correla- tion between token norms and semantic richness. Extensive experiments multiple benchmarks demonstrate that ST-GridPool significantly improves the performance of models like LLaVA- Video without requiring retraining, showing particular robustness under strict token budgets. Furthermore, our approach achieves dual optimization by substantially reducing both inference latency and peak GPU memory usage input frame counts increase. As scalable and plug-and and-play solution, ST- GridPool offers powerful paradigm for enhancing visual token representations without the need for costly architectural modifications.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Zesen Cheng, Sicong Leng, Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang. Ziyang Luo, Deli Zhao, et al. Videollama 2: Advancing spatial-temporal modeling and audio understanding in video-Ilms. arXiv preprint arXiv:2406.07476, 2024.
Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Video-mme: The first-ever comprehensive evalua- tion benchmark of multi-modal IIms in video analysis. arXiv preprint arXiv:2405.21075 2024.
Feyza Duman Keles, Pruthuvi Mahesakya Wijewardena, and Chinmay Hegde. On the computational complexity of self-attention. In International conference on algorithmic learning theory, pp. 597- 619. PMLR, 2023.
Wonkyun Kim, Changin Choi, Wonseok Lee, and Wonjong Rhee. An image grid can be worth video: Zero-shot video question answering using vlm, arXiv preprint arXiv:2403. 10.40...8400 18406, 2024.
Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.00 030.026000, 2024a.
Guanbin Li and Yizhou Yu. Visual saliency detection based on multiscale deep cnn features. IEEE transactions on image processing, 25(11):5012-5024 2016.
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language- -magge pre-training with frozen image encoders and large language models. In International conference on machine learning, pp. 19730-19742. PMLR, 2023a.
KunChang Li, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang. Ping Luo, Yali Wang, Limin Wang. and Yu Qiao. Videochat: Chat-centric video understanding. arXiv preprint arXiv:2305.06355 2023b.
Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang. Jilan Xu, Guo Chen, Ping Luo, et al. Mybench: A comprehensive multi-modal video understanding benchmark. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp- 22195-222066 2024b.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892-34916 2023.
Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang. Sheng Shen, and Yong Jae Lee. Lexva-next: Improved reasoning, ocr, and world knowledge, January 2024a. URL https::/
llava-vl. www.l..gttu................ wo/bllcg/2224001103000 ot -
Ruyang Liu, Chen Li, Haoran Tang, Yixiao Ge, Ying Shan, and Ge Li. St-llm: Large language models are effective temporal learners. In European Conference on Computer Vision, pp. 1-18. Springer, 2024b.
Yuanxin Liu, Shicheng Li, Yi Liu, Yuxiang Wang, Shuhuai Ren, Lei Li, Sishuo Chen, Xu Sun, and Lu Hou. Tempcompass: Do video Iims really understand videos? In Findings of the Association for Computational Linguistics ACL 2024, pp. 8731-8772, 2024c.
Zhijian Liu, Ligeng Zhu, Baifeng Shi, Zhuoyang Zhang, Yuming Lou, Shang Yang, Haocheng Xi, Shiyi Cao, Yuxian Gu, Dacheng Li, et al. Nvila: Efficient frontier visual language models. arXiv preprint arXiv:2412.04444 2024d.
Zuyan Liu, Yuhao Dong, Ziwei Liu, Winston Hu, Jiwen Lu, and Yongming Rao. Oryx milm: On- demand spatial- understanding arbitrary resolution. arXiv preprint S1ti:22499.196611 2024e.
Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Shahbaz Khan. Video-chatgpt: Towards detailed video understanding via large vision and language models. arXiv preprint AXXv:23306005222 2023.
11

---

Published as a conference paper at ICLR 2026
Karttikeya Mangalam, Raiymbek Akshulakov, and Jitendra Malik. Egoschema: A diagnostic bench- mark for very long-form video language understanding. Advances in Neural Information Process- ing Systems, 36, 2024.
Tingyu Qu, Mingxiao Li, Tinne Tuytelaars, and Marie-Francine Moens. Ts-llava: Constructing vi- sual tokens through thumbnail-ame babbbnall-add-ammpliinn for training- freee video large language models. arXiv preprint arXiv:2411. 111111111666, 2024.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference machine learning, pp- 8748-8763. PmLR, 2021.
Zhiqi Shao, Haoning Xi, David A Hensher, Ze Wang. Xiaolin Gong, and Junbin Gao. A spatial- temporal dynamic attention- mamba model for multi-type passenger demand prediction in multimodal public transit systems. Transportation Research Part E: Logistics and Transportation Review, 202:104282, 2025.
Yi Wang, Kunchang Li, Xinhao Li. Jiashuo Yu, Yinan He. Guo Chen, Baoqi Pei, Rongkun Zheng, Zun Wang, Yansong Shi, et al. Internvide22: Scaling foundation models for multimodal video understanding. In European Conference on Computer Vision, pp. 396 416. Springer, 2024.
Haoning Wu, Dongxu Li, Bei Chen, and Junnan Li. Longvideobench: A benchmark for long-conter interleaved lido-language understanding. Advances in Neural Information Processing Systems, 37:28828-28855 2025.
Junbin Xiao, Xindi Shang, Angela Yao, and Tat-Seng Chua. Next-qa: Next phase of question- answering to explaining temporal actions. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), pp. 9777-9786, June 2021.
Lin Xu, Yilin Zhao, Daquan Zhou, Zhijie Lin, See Kiong Ng. and Jiashi Feng. Pllava: Parameter- free Ilava extension from images to videos for video dense captioning. arXiv preprint arXiv:2404. 16994, 2024a.
Mingze Xu, Mingfei Gao. Zhe Gan, Hong- You Chen, Zhengfeng Lai. Haiming Gang, Kai Kang, and Afshin Dehghan. Slowfast-llava: A strong training-free baseline for video large language models. arXiv preprint 1rxh::2407.15841 2024b.
Jiabo Ye, Haiyang Xu, Haowei Liu, Anwen Hu, Ming Yan. Qi Qian, Ji Zhang, Fei Huang, and Jingren Zhou. mplug-ow13: Towards long imagc-sequence understanding in multi-modal large language models. In The Thirteenth International Conference Learning Representations, 2024.
Hang Zhang, Xin Li, and Lidong Bing. Video-llama: An instruction-tuned audio-visual language model for video understanding. arXiv preprint arXiv:2306.000 028558 2023.
Kaichen Zhang, Bo Li, Peiyuan Zhang, Fanyi Pu, Joshua Adrian Cahyono, Kairui Hu, Shuai Liu, Yuanhan Zhang, Jingkang Yang, Chunyuan Li, and Ziwei Liu. Lmms-eval: Reality check on the evaluation of large multimodal models, 2024a. URL https:tppa..
12772
Pan Zhang, Xiaoyi Dong, Yuhang Zang, Yuhang Cao, Rui Qian, Lin Chen, Qipeng Guo, Haodong Duan, Bin Wang, Linke Ouyang, et al. InternIm- xoannlmmxxomooee-22.55 A versatile large vision language model supporting long-contextual input and output. arXiv preprint arXiv:2407.03333 2024b.
Peiyuan Zhang, Kaichen Zhang, Bo Li, Guangtao Zeng, Jingkang Yang. Yuanhan Zhang, Ziyue Wang, Haoran Tan, Chunyuan Li, and Ziwei Liu. Long context transfer from language to vision. arXiv preprint arXiv:2406.1668 2024c.
Yuanhan Zhang, Jinming Wa, Wei Li, Bo Li, Zejun Ma. Ziwei Liu, and Chunyuan Li. Video instruction tuning with synthetic data. arXiv preprint arXiv:2410.02713. 2024d.
Orr Zohar, Xiaohan Wang, Yann Dubois, Nikhil Mehta, Tong Xiao, Philippe Hansen- Licheng Yu, Xiaofang Wang, Felix Juefei-Xu, Ning Zhang, et al. Apollo: An exploration of video understanding in large multimodal models. arXiv preprint arXiv:2412. 2024.
12

---

Published as a conference paper at ICLR 2026
A
COMPARISON WITH TOKEN REDUCTION METHODS
Method
VideoMME
L.V.Bench
EgoSchema
Upper Bound (Full Tokens)
NVILA
61.5
56.3
52.9
Token Reduction Ratio: 30%
FastV
57.9
53.0
49.7
PruMerge
58.2
53.4
47.5
FasterVL..
60.1
53.0
49.3
VisionZip
59.1
50.9
48.9
FrameFusion
58.8
54.9
51.3
Ours
59.9
54.6
52.0
Token Reduction Ratio: 50%
FastV
58.9
53,9
50.2
PruMerge
57.6
53.9
48.9
Faster VLM
60.8
53.0
50.5
VisionZip
60.5
54.4
50.3
FrameFusion
59.4
54.8
52.6
Ours
61.4
55.6
53.1
Table 4: Comparison with token reduction baselines on NVILA-Video-SB with 64 input frames (%).
To validate token reduction method, conducted experiments on the NVILA Video- 8B model with token reduction ratios of 30% and 50%, evaluating performance on three long-video bench- marks (VideoMME, Long VideoBench, and EgoSchema). To ensure fair comparison of computa- tional efficiency and resource usage with I.aVA- -A-eoode,--7,, we establish a consistent setup by also setting the input frame count for NVILA 64. The results in Table demonstrate our method's superior performance. At 30% reduction, our approach is highly competitive and achieves the top score on EgoSchema. Its advantage becomes even more pronounced method ranks first across all three benchmarks. Notably, at this high compression rate, our method's performance not only comes remarkably close to full-token upper bound but even surpasses on the EgoSchema benchmark. This strongly validates the effectiveness and generalizability of our approach, proving it can significantly reduce computational cost while maintaining, and in some cases even exceeding, the performance of the original full-token model.
50% reduction, where our
B COMPARISON WITH ALTERNATIVES
We further compare our method with training-fres alternative approaches, including IG-VLM (Kim al. 2024), SF-LLaVA (Xu al. 2024b), and TS-LLaVA (Qu et al. 2024). While these methods have shown promising results on image-language models like LLaVA-Next (Liu et al. 2024a), we adapt them LLaVA- VA-Video model under fair experimental settings: adjusting parameters main- tain identical input frames and token counts as the original models, As shown in Table our analysis reveals two key findings: (1) Directly transplanting these mocusso methods video-language models yields unsatisfactory outcomes. SF-LLaVA and TS-LLaVA achieve limited improvements on specific datasets, but their overall performance remains unstable and non-substantive. Notably, applying IG-VLM alone leads to significant performance degradation. (2) Our method achieves the most substantial gains across all metrics, outperforming all alternatives with improvements on VideoMME, Long VideoBench, and MVBench respectively compared the original baseline. This demonstrates the unique effectiveness our approach in aligning visual- language reasoning pat- terns for video understanding tasks.
13

---

Published as a conference paper at ICLR 2026
Model
VideoMME
LongV.Bench
MVBench
Baseline
63.3
58.2
58.6
IG-VLM
60.6
55.9
54.2
+ SF-LLaVA
63.4
58.9
58.9
+ TS-LLaVA
63.8
59.6
57.1
+ Ours
64.2
60.1
59.8
Table 5: Comparison with training-fre free alternative methods on the LLaVA-Video-77 baseline (%).
Method
VideoMME
Long V.nneh
MVBench
EgoSchema
Image-griae Srage-griddiig
58.3
56.3
56.5
56.6
Token-gridding (Ours)
59.0
56.7
58.0
62.1
Table 6: Comparison with image-gridding and token-gridding (ours) on the LLaVA-OneVision-77 (%).
C ABLATION STUDY ON IMAGE-GRIDDING VS TOKEN-GRIDDING
In Pyramid Temporal Gridding (PTG), we introduce token-level gridding, which diverges from the image-level gridding employed by previous methods such as IG-VLM (Kim et al. (0224) and TS- LLaVA (Qu et al.) 2024). To examine the performance differences between these approaches, conducted experiments using image-griddingg image level, akin to VL-. In contrast, where the PTG module processes information at the method applies liken-gridding token represen- tations. Results are shown in table ( and table [7] which demonstrate that the token-grit -rindgrrd strategy consistently outperforms the image-gra -rid approach across both LLaVA-One Vision-7B and LLaVA- Video-7B configurations. This superiority highlights the advantages of fine-grained token-level pro- cessing, which preserves richer spatio-temporal features and enables dynamics. These findings validate the effectiveness of our token-gricn token-gridding strategy, showcasing its ability enhance performance while maintaining computational efficiency.
precise modeling of video
D IMPACT OF TEMPORAL GRIDDING CONFIGURATION
We investigate the impact of temporal pyramid configurations in the Pyramid Temporal Gridding (PTG) module, focusing segment length at each layer. In experiments, each layer's segment length doubles the previous one (e.g.. (8, 16, 32)). Results on VideoMME and Long VideoBench, shown in Figure 7 reveal the following obsevations: (1) For fixed L, performance initially improves with increasing maximum segment length but declines after threshold, suggesting overly long segments may dilute fine- grained motion patterns. Medium-length segments balance local detail capture and global temporal context, whereas excessively long segments introduce noise from irrelevant temporal regions. (2) Increasing L systematically boosts performance, with the best global results achieved at L = 3 using (8, 16,, 32) segments. As grows from to 3, the ideal maximum segment length increases from 8 (L = 1) to 32 (L 3). Deeper pyramids enable hierarchical modeling of multi-scale temporal dependencies, leveraging shorter segments for local actions and longer ones for macro- event integration.
critical design choices: the number of pyramid layers (L) and the
IMPACT OF SPATIAL POOLING SHAPE
We explore the impact of spatial pooling shape (i.e., kernel size) in the Norm-aware Spatial Pool- ing (NSP) module. Experiments conducted the LLaVA-Video backbone with kernel sizes spanning (1, (2, 2). (6, 6). and results on VideoMME and LongVideole VideoBench are depicted in Figure It observed that both datasets achieve peak performance at kernel size of (2, 2). Smaller kernels (1, 1) yield suboptimal results, as overly localized receptive fields fail to capture contextual spatial relationships, limiting feature aggregation. Performance declines progressively for kemnlss
14

---

Published as a conference paper at ICLR 2026
Method
VideoMME
Long
MVBench
EgoSchema
Image-gridding
62.8
59.0
58.4
57.1
Token-gridding (Ours)
64.2
60.1
59.8
57.8
Table Comparison with image-grit and ooken-grid (ours) on the JLaVA-Video-7B (%).
1015
61.0
61.
58.5
61.0
02.5
16
1000
Masimen Segreett
Maximan
(a) VideoMME
(b) LongVideoBench
Figure 7: Ablation study results for different values level L and maximum segment length on VideoMME and Long VineeBeen..
larger than (2, 2), with (6, 6) delivering the lowest scores. This degradation may be caused by the oversmoothing effect and local structural misalignment of broad receptive fields.
F FINE-GRAINED MULTI-TASK ANALYSIS ON MV BENCE
72.0
3001
00.5
63.4
16.5
12.0
11:0
11.0
19.0
66.5
47.5
13.5
12.0
17.0
16.1
590
78.
815
850
71.5
41.5
Cidorrh Maaz
13.0
22.5
000
310
MaloCha
35.5
PLL.VA
65.5
58.5
26,8
49.
58.0
55.5
48.0
410
22.0
$2.0
43.0
96.0
61.0
560
820
450
61.9
STLEM
64.0
16.5
510
55.5
86.0
46.
583
31.5
41.5
680
44.3
78.5
56.
42.
80.5
73.5
98.
86.
418
58.5
$4.9
MideoChizz
Lree Kopdl
83.
37.
440
55.0
755
42.9
725
35.0
31.
65.3
33.5
66.5
87.5
83.5
71.
5500
83.5
51.0
60.
Table 8: Multi-task analysis results MVBench (%). The best performance among all methods is underlined.
To further analyze the fine-grained video understanding capabilities of our method, present the detailed performance of compared methods across sub-tasks on the MVBench dataset, as shown in table Based on the results, the following observations can be made: (1) Our method achieves comparable performance to the statc-of-the-art baseline models, demonstrating its robustness across a wide range of tasks. (2) Our method substantially outperforms baselines in tasks requiring spatial localization and temporal granularity, such as Action Localization (AL), Unexpected Action (UA), Scene Transition (ST), State Change (SC). These improvements highlight the effectiveness of our proposed Norm-based Spatial Pooling and Pyramid Temporal Gridding in enhancing spatial pre- cision and temporal granularity. (3) Despite the advancements, our method lags behind SOTA in tasks requiring manghorrioon reasoning fine me--graned motion analysis, highlighting opportunities to better integrate cross-frame frame dependencies.
G EXTENSIVE QUALITATIVE ANALYSIS OF EXAMPLES
We illustrate some additional examples from Long VideoBench dataset in fig. Visual comparisons demonstrate that our method (LLaVA-Vided) + Ours) significantly outperforms the baseline in capturing fine-grained spatiotemporal details. One key limitation of baseline models is their inabil- ity to resolve temporal dependencies, even when all event components are clearly present in the video. For instance, in the first example, when asked about event sequencing, standard Video LLMs misorder actions, confusing co-occurrence with causality. Our temporal gridding strategy tackles this issue by explicitly organizing spatiotemporal tokens into chronological lattices, allowing the
15

---

Published as a conference paper at ICLR 2026
00.11
01.11
- Pasing Stapent"
Pastig Stepent
61.0
00.0
6011
08.0
(a) VideoMME
(b) LongeBeeeen
Figure 8: Ablation study results for different pooling shape on VideoMME and Long VongeBBBe.....
model accurately disentangle sub-frame temporal relationships. The integration of norm-based recalibration enhances the model's ability to localize subtle visual cues. In the second example, the baseline incorrectly identifies clothing pattern "solid blue lines" due to its failure detect the faint dashed stitching. In contrast, our method successfully recognizes the discontinuous texture through normalized feature recalibration. These examples collectively highlight how our method combines spatial refinement and temporal coherence to achieve narratively consistent and visually precise video reasoning.
8 According fillowing soquences
LLaVA-Video
A person is white background initially soes circle with letter G. followed by a triongle with the lettr A. and Josty weed 'rchitocture appené
persoo in white background initially sees trianale with the letter & followed by a citcle with the lecter G. and lastly the weed Architecture appears.
8
video,
chamistry prablem. circled
subtilies 'sind
remove all this and just show', what are the characterastics of line that highlights the circled numbers?
A solid blue liv.
LLaYA-Vies Video Ours
A dashed blue line.
Figure The other output examples of LLVVe-Video model without and with our method on Long VideoBench dataset.
16


Reviews:
[{"review_id": "t8UaNIxQrn", "reviewer": "Reviewer_35ED", "summary": "This paper proposes ST-GridPool, a training-free visual token compression method tailored for video understanding with multimodal large language models. ST-GridPool introduces Pyramid Temporal Gridding to construct multi-granularity temporal grids of visual tokens and employs Norm-based Spatial Pooling to further reduce token count while preserving informative regions. The method is evaluated across multiple video understanding benchmarks, demonstrating its effectiveness", "questions": "Please refer to the weaknesses. I'll update my rating accordingly based on the authors' response.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "* The paper is clearly written and well-structured, making it easy to follow.\n* The proposed approach is intuitive and straightforward to implement; its plug-and-play design enhances practical applicability across different MLLMs.\n* In terms of performance, ST-GridPool demonstrates strong potential, consistently outperforming or matching existing token reduction methods across multiple benchmarks.", "weaknesses": "* The core idea of ST-GridPool bears strong resemblance to prior training-free methods such as TS-LLaVA (which combines gridding with token sampling) and IG-VLM (which relies solely on image gridding). However, comparisons with these approaches are relegated to the appendix and lack sufficient implementation details. Given that ST-GridPool appears to be a more complex integration of similar components, a more rigorous and fair comparison is warranted, ideally involving hyperparameter tuning for the baselines (e.g., number of grids, token compression ratios, or retained visual token counts) to ensure a level playing field. Without such controlled experiments, it is difficult to assess the true contribution of the proposed design.\n* The efficiency analysis currently only compares LLaVA-Video with and without ST-GridPool. While it is unsurprising that adding a token compression module reduces computational cost, this alone does not demonstrate relative superiority. To substantiate its efficiency claims, ST-GridPool should be benchmarked against other token reduction methods under identical settings. \n* As shown in Table 1, the performance gains from integrating ST-GridPool are sometimes marginal. A more granular breakdown of results, e.g., by question type, video duration, or scene complexity on benchmarks like VideoMME and LongVideoBench, would help identify failure modes or scenarios where the method underperforms. Additional ablation studies and evaluation on more diverse datasets would further strengthen the analysis.\n* The last row of Table 1 appears to be duplicated and should be corrected in the final version.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "W2EI4Wt3bm", "reviewer": "Reviewer_BLb5", "summary": "The paper proposes ST-GridPool, a practical training-free method to enhance visual tokens for Video LLMs. The approach is technically sound, combining a hierarchical temporal gridding strategy (PTG) and a saliency-aware spatial pooling mechanism (NSP). The method demonstrates consistent performance improvements on LLaVA-family models across six benchmarks and compelling gains in computational efficiency. The empirical evaluation is extensive, including strong ablations and comparisons to other token reduction methods.", "questions": "1.Is it possible to explore an adaptive mechanism that automatically determine the optimal values of non-trivial hyperparameters in your method?\\\n2.How does the proposed PTG interact with or differ from the built-in temporal reasoning mechanisms already present in modern Video LLMs such as NVILA?\\\n3.Could other unsupervised indicators serve as complementary or alternative signals to the token norm?\\\n4.How to interpret the observed trend of a larger improvement for Long Video Understanding tasks compared to General Video Understanding tasks?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.The proposed ST-GridPool method is training-free and plug-and-play, offering an efficient solution to enhance existing Video LLMs without the need for costly retraining. It demonstrates consistent performance improvements across various backbones on all six evaluated video understanding benchmarks, with notable gains in Long Video Understanding.\\\n2.The motivation behind Norm-based Spatial Pooling is well-founded, supported by a clear analysis that establishes a positive correlation between token norms and semantic saliency, contributing to the method's effectiveness.\\\n3.The empirical evaluation is comprehensive, featuring extensive ablation studies on individual components, key hyperparameters, and design choices such as pooling shape and gridding strategy.", "weaknesses": "1.The performance depends on several non-trivial hyperparameters, including the number of temporal levels, norm order, and temperature. Balancing these parameters effectively is crucial for their successful application in real-world scenarios.\\\n2.The paper does not clearly show whether PTG provides complementary benefits or duplicates functionality already present in recent Video LLMs.\\\n3.The NSP module relies heavily on an empirical correlation between token norm and semantic importance, yet offers limited empirical comparison or grounding.\\\n4.The experimental section primarily focuses on quantitative results but lacks deeper analytical discussion. The paper does not explore why the proposed ST-GridPool achieves varying degrees of improvement across different benchmarks or tasks.\\\n5.There are some typos in the paper. E.g., the percentage gains reported for \"LLaVA-Video-7B + Ours\" are identical to those reported for \"LLaVA-OneVision-7B + Ours\".", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jkiGPtoQYj", "reviewer": "Reviewer_JydV", "summary": "This paper introduces a well-motivated and effective training-free method, ST-GridPool, for enhancing visual token representations in Video LLMs. The approach is simple yet intuitive, combining PTG and NSP to improve performance without retraining while also reducing computational overhead. The empirical validation is extensive, with strong ablation studies supporting the design choices.", "questions": "Please see the weakness above.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. Both components are well-justified and technically sound. PTG provides an intuitive hierarchical approach to model events at different time scales. NSP is motivated by both intuitive and quantitative analysis, which is a clever insight for preserving important information during compression.\n\n2. A comprehensive evaluation is conducted, covering both long and general video understanding tasks. The method is tested across multiple backbone models, demonstrating its generalizability and effectiveness.\n\n3. The paper is mostly well-written, clearly structured, and easy to follow, with helpful visualizations that aid in understanding the proposed approach.", "weaknesses": "1. The method for updating the token sequence in Pyramid Temporal Gridding involves overwriting the last frame token of a segment with a generated summary token. This specific design choice should be justified.\n\n2. The computational analysis focuses on the net efficiency gains, but offers limited insight into the additional computational cost (e.g., latency, memory) introduced by the ST-GridPool module itself.\n\n3. A deeper theoretical explanation for why token norms (specifically L_2 norm) reliably correlate with semantic importance in this context would be beneficial.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "d5AQsVMMvu", "reviewer": "Reviewer_VLyg", "summary": "This paper proposes ST-GridPool, a training-free method to enhance visual token representations for Video LLMs, addressing the challenge of efficient token compression while preserving spatiotemporal interactions. It integrates two core components: Pyramid Temporal Gridding (PTG) for multi-grained temporal feature capture and Norm-based Spatial Pooling (NSP) for preserving high-semantic regions via token norms. Evaluated on 6 benchmarks (e.g., VideoMME, MVBench) with LLaVA-OneVision-7B and LLaVA-Video-7B, ST-GridPool achieves consistent performance gains without retraining, while reducing inference cost.", "questions": "1. Given the model's clear overall efficiency gains, could you provide a more detailed computational cost breakdown contributed by the PTG and NSP modules individually?\n2. Why do you choose to make the PTG summary token overwrite the last frame's token instead of a less destructive alternative like appending?\n3. How does your method balance the trade-off between prioritizing high-importance regions and preserving potentially critical, low-importance backgrounds in practical video understanding tasks?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper tackles a critical research problem about visual token enhancement for Video LLMs. The proposed ST-GridPool method is effective. It offers a training-free, plug-and-play framework that improves visual token representations without adding extra computational cost, a very appealing design choice.\nThe paper is well positioned within the literature and clearly explains how it differs from prior work. The motivation is convincing, the narrative flows nicely, and the writing is clear throughout. It’s an easy and enjoyable read.\nCapturing multi-grained spatiotemporal interactions across varying temporal lengths and leveraging the positive correlation between token norms and semantic importance is a really interesting idea.\nThe results are solid, covering diverse tasks, different backbones, and a range of metrics and hyperparameters. The experimental section is thorough and leaves little doubt about the method’s effectiveness.", "weaknesses": "The paper demonstrates the overall computational advantages (inference time and memory) of ST-GridPool. However, it does not provide a detailed breakdown of the computational cost for the PTG and NSP components individually. Furthermore, the potential interaction between these two modules is not discussed, e.g. how PTG's multi-scale processing affects the norm distribution of the input to NSP.\n\nThe Pyramid Temporal Gridding (PTG) module generates a summary token for each temporal segment and then uses it to update the token of the last frame in that segment. This update operation seems potentially destructive, as it replaces the representation of an actual, observed frame with a synthetic, interpolated summary. This design choice is not clearly motivated over less destructive alternatives, e.g., appending the summary token.\n\nThe NSP component is explicitly designed to prioritize high-importance regions (i.e., salient objects) and suppress low-importance backgrounds. This strategy could be detrimental in tasks that require holistic scene understanding or reasoning about subtle interactions within the background regions, which are now actively suppressed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)