PPLLaVA: Varied Video Sequence Understanding With Prompt Guidance
Authors: Shangkun Sun
Ruyang Liu
Haoran Tang
Yixiao Ge
Haibo Lu
Jiankun Yang
Chen Li
Venue: ICLR 2026 Poster (2026)
Keywords: Video LLM
Prompt-guided Pooling
PPLLaVA
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: The past year has witnessed the significant advancement of video-based large language models. However, the challenge of developing a unified model for both short and long video understanding remains unresolved. Most existing video LLMs cannot handle hour-long videos, while methods custom for long videos tend to be ineffective for shorter videos and images. In this paper, we identify the key issue as the redundant content in videos. To address this, we propose a novel pooling strategy that simultaneously achieves token compression and instruction-aware visual feature aggregation. Our model is termed Prompt-guided Pooling LLaVA, or PPLLaVA for short. Specifically, PPLLaVA consists of three core components: the CLIP-based visual-prompt alignment that extracts visual information relevant to the user's instructions, the prompt-guided pooling that compresses the visual sequence to arbitrary scales using convolution-style pooling, and the clip context extension designed for lengthy prompt common in visual dialogue. Extensive experiments have validated the performance of our model. With superior throughput, PPLLaVA achieves better results on image benchmarks as a video LLM, while achieving state-of-the-art performance across various video benchmarks, excelling in tasks ranging from caption generation to multiple-choice questions, and handling video lengths from seconds to hours.

Published as a conference paper at ICLR 2026
PPLLAVA: VARIE VIDEO SEQUENCE UNDERSTAND- ING WITH PROMPT GUIDANCE
Shangkun Sun
Ruyang Liu
Haoran Tang
Yixiao Ge
Haibo Lu'
Jiankun Yanglt It
Chen Litt
Peng Cheng Laboratory
Pekknng University
*XPeng Inc.
*Ministry of Industry and Information Technology the People's Republic of China,
China Electronics Standardization Institute, Beijing, China
(sunshk & ruyangâstu, {luhb, yangik]$pel..o aaappplaccco ac.cn
hrtang@ (rkaang@)pku.edu.co edu.cn Aiche@ccei...o cn reyixiao831egmat com
ABSTRACT
In the past year, video-based large language models (Video LLMs) have achieved impressive progress, particularly in their ability to process long videos through extremely extended context lengths. However, this comes at the cost of signif- icantly increased computational overhead due to the massive number of visual tokens, making efficiency major bottleneck. In this paper, we identify the root of this inefficiency as the high redundancy in video content. To address this, we propose a novel pooling strategy that enables aggressive token compres- sion while retaining cetnuucttio-rellvann visual semantics. Our model, Prompt- guided Pooling LL (PPLLaVA). introduces three key components: a CL.PP- based pisupl-prompt alignment module that identifies regions of interest based on user instructions, prompt-guided guided pooling mechanism that adaptively compresses the visual sequence using convolution-style pooling, and a clip context extension module tailored for processing long and complex prompts in visual dialogues. With up to 18x token reduction, PPLLaVA VA maintains strong performance across tasks, achieving tfate-of-theaar art results diverse video understanding bench- marks-rang from image-to-video tasks such as captioning and QA to long- form video reasoning-wine while significantly improving inference throughput.
INTRODUCTION
Recent advances in Multimodal Large Language Models (MLLMs), such as the LLaVA series Liu et al. 2024b); Li et al. (2024a), Qwen-VI VL series Yang et al. 2024); : Bai et al. (2025), and Intern series Chen al. (2024a); Zhu al. (2025) have led unified models capable of handling both images and vidcos. A prevalent approach to modeling videos is to directly feed all frame-wise visual tokens into the LLM. Thanks to the extended context lengths supported by advanced MILMs, they can effectively capture temporal dependencies over long video sequences, making them effective for long video understanding. However, this strategy also introduces substantial computational overhead due to the massive number of video tokens, posing major challenge for real-time or resource- constrained applications.
To address this, various token reduction strategies have been explored. Early methods employed temporal average pooling to compress frame sequences (Li et al.) 2023b; Maaz et al.] 2023 Luo
ct al. 2023: Liu et al. 2024d), but at the cost of losing temporal dynamics. More recent efforts for
long video modeling introduce specialized structures like visual memory (Ren et al.) 2024 Zhang
al. 2024a Zhou al. 2024) adaptive keyframe selection Wang et al.) (2024b Shen et al.
(2024 Liu et al. (2025). ) which improve long video handling but often lack flexibility for short
videos. In contrast, conditional token pooling aggregation (Li al. 2023d] Xu al. 2024a Jin
ct al.
2023 offers a more general solution, enabling substantial token compression while preserving
"Equal contribution.
1 Correspnnning authors.

---

Published as a conference paper at ICLR 2026
+440
How daes The this vides?
buttertyy
vide??
three deraraaiion crsphics then con be soon thes vides7
Skmn Voalt Takors
(h)
Figure (a) instance from VideoMME (Fu al. 2024). The crucial information pertains to only small portion of the video for different questions. (b) Performance comparison of PPLLaVA- LLa Video with LLaVA- Video baseline on VideoMME-Overil (wo subtitles), which shows number of visual tokens input to the LLM and the corresponding model performance.
spatiotemporal structure, and is thus increasingly favored in recent bascline MLLM designs Zhang al. 2024c); Bai al. (2025).
However, pooling inevitably leads to performance degradation compared to using the full set of vi- sual tokens. As result, most existing models adopt conservative pooling scales, typically reducing the sequence length by only factor of 4, to strike balance between efficiency and performance. But is it possible to further reduce token length without sacrificing modeling capabilities? We be- lieve the key lies in the intrinsic redundancy of videos. As shown in prior work (Han et al., 2022] Liu et al.) 2023a Ma et al., 2022 NVIDIA et al., 2026), key visual information is often concen- trated in a few frames- especially long videos. For video LLMs, this is even more evident: as illustrated Fig. aa), user instructions may only pertain to a small portion of the video, rendering effectively extract instrcction-reeevan visual
much of the remaining content irrelevant. If features while compressing the token length, we may maintain or even enhance performance. In this context, the design of vision-language mapping modules becomes critical. Models using Q-Formers Li et al. (202aa) Dai et al. (2023) perform token compression by converting rich visual inputs into small set of query tokens while enabling interaction with the instruction text. However, modern MI.LMs such L.LaVA series Li et al. (2024a), Owen-VI. series Bai et al. (2025). and Intern VI. Chen al. 2024a) series have moved away from Q-Formers in favor simpler architectures like linear projections or MLPs, which are easier to train and more efficient in inference. This raises an important question: Can we develop a pooling strategy that achieves the token efficiency and instruc- tion alignment of Q-Formers, while retaining the simplicity and scalability of current mainstream models?
To this end, propose Prompt-guided Pooling LLaVA (PPLLaVA), novel method that combines visual token pooling with instruction-ane -wwar visual feature extraction. Specifically, PPLLaVA first identifies prompt-relevant visual representations through fine-grained vision-prompt alignment. then uses the resulting prompt-vision relevance map as a 3D convolutional kernel to compress visual tokens into any desired diree-dimensional size, based on specified output resolution or stride. In addition, recognizing that CLIP pretraining imposes a limited text context length-ane that training video LLMs, especially for multi-turn dialogues, demands extended textual input-PPLLa VA incor- porates asymmetric positional embedding extensions to enhance the model's text encoding capacity. As result, PPLLaVA effectively extracts instruction-relevan visual features from both long- and short-form textual prompts while significantly reducing the visual token length. It achieves over 90% token compression, supports ultra-long video inputs, and simultaneously enhances performance on short-video tasks.
Experiments on the latest multimodal LLM benchmarks have validated the superiority of our method: PPLLaVA has achieved top results across wide range of test sets, including NexIQA Xiao et al. (022a), EgoSchema, ActivityNet (Caba Heilbron et al. 2015). VCG Bench (Maaz et al. 2023 MV Benchh (L al. 2023c), Long VideoBench (Wu et al. (024), and Video-MME (Fu al.
2024 These benchmarks encompass tasks such as video question answering, detailed video cap- tioning, and video multiple-choice questions, with video lengths ranging from seconds to hours. We
2

---

Published as a conference paper at ICLR 2026
conducted post-training with the integration of PPLLaVA on different VLM base models, including LLaVA-Next, LLaVA- Video, and InternVL3. These VLMs vary in terms of training token scales and visual encoders, yet PPLLaVA VA consistently yields further improvements, demonstrating the gen- eralization capability of our model. More importantly, PPLLaVA significantly improves efficiency while achieving better performance. As shown in Fig. compared to the baseline LLa -ideo, PPLLaVA achieves superior performance with only one-quarter of the token count. When the num- ber of tokens is aligned, the advantage of PPLLaVA becomes even more pronounced, outperforming by 6.86% and 4.4% at 1000 and 2000 tokens, respectively. It is worth noting that the performance of PPLLaVA does not solely rely on the semantic alignment provided by CLIP. On the contrary, CLIP-based encoders offer strong initialization, while the model learns during subsequent training how to adaptively extract critical visual information. As result, PPLLaVA still achieves outstand- ing performance even on MideoChttGPT-Bende where summarization and caption-style problems are prevalent without informative user questions.
2 RELATED WORK
Multimodal Large Language Models. Image-domain pretrained models have long served as the
foundation for vidco understanding (Carreira & Zisserman 2017 Luo et al. 2022 Liu et al. 2023b] Ye et al. 2026 Peng et al. 2026, Sun al.] 2225a bp. This partly due the inherent simi- larities between image and video modalities and partly because image pretraining datasets offer a level of quality, quantity, and diversity that video datasets often lack. Early MLLMs, such as the LLaVA Liu et al.) (2024a|b) b) and BLIP series [i et al. (2023a); Dai et al. (2023), were typically trained with images during SFT, thus requiring additional investigation into how to transfer them to the video domain. In contrast, the latest open-souree MLLMs such as LLaVA-Onee Vision Li et al. (2024a). Qwen2.5-VL, Bai et al. (2025), and InternVL3 Zhu et al. 2025), have already integrated image, -mult, and video training. Thanks to their support for extremely long contexts (of- ten exceeding 16k tokens), inputting long videos is no longer a challenge. However, the overhead introduced by accommodating long videos through extremely long contexts not only demands sub- stantial computational resources but also hinders lightweight video understanding and deployment on resource-constrained devices.
Video LLMs. In the past year, Video LLMs have experienced rapid development. Early Video LLMs (Li al. 2023b Zhang et al. 2023 Luo et al. 2023 Maaz et al. [023 [Lu et al., (2024d) typ- ically used average pooling to process video sequences with Image LLMs while employing modality perceivers to model temporal sequences. However, this approach significantly limited the model's ability to fully understand video sequences. Alternatively, some models (Liu et al., 2024e b) Yang al. 2025) rely on the LLM itself to model video sequences, achieving good video understanding results. Nonetheless, this method is limited to handling small number of frames and does not support the comprehension of long videos. Understanding long videos is also a hot topic in video LLMs et al. (2023) ; Li et al. 2023d); Zhang al. (2024b); Shen et al. (2024): ; Liu et al. (002c)); Xu et al. (2024b al; Zohar et al. 2024); Liu et al. (2024g); Shu et al. (2025) ); Zhang et al. (2025) ; Qu al. 2024). MovieChat (Song al. 2024) and Flash- V VStremm (Zhang et al.) (0224) use memory
structures to process streaming videos, while Long VA Zhhng et al. (2024b) and Kangaroo Liu et al. (2024c) extend the LLM's context length to accommodate more frames. Video-XL ( et al. (2025) leverages MLLMs' inherent KV sparsification capacity to condense the visual input. Most similar to our work, PLLaVV (Xu et al. 2024a), as well as LLaVA- parametric AdaptiveAvgP Pool function to compress visual tokens. In contrast, our method supports not only token compression but also the extraction of visual features pertinent to user prompts. This enables our model to perform more aggressive compression while preserving performance, making suitable for both short and long video inputs. Furthermore, our convolution-style pooling method enables flexible output sizes.
and Qwen2.5-VL, employs the non-
Key Content Extraction. Extracting key video content is also a common research topic, and these models also tend to leverage CLIP to extract semantic priors Shen et al. (2024); Wang et al. 2 Park et al." (2024). For example, VideoAgent Wang et al. (2024b) iteratively searches instructions by utilizing both generated dense captions and CLIP
for key frames relevant similarity. In comparison, these methods resemble frame selection strategies, whereas PPLLaVA represents complete model framework. Additionally, these models typically scale up runtime, as
3

---

Published as a conference paper at ICLR 2026
searching frame by frame is computationally expensive. In contrast, major advantage of PPLLaVA its ability to significantly enhance the efficiency of Video LLMs.
In fact, PPLLaVA functions more similarly to a Q-Former within LLaVA, but it offers several advan- tages over directly training a Q-Former: (1) PPLLaVA introduces far fewer additional parameters and computational overhead, amounting to less than one-tenth of Q-Former. (2) While Q-Former requires a three-stage pretraining process- process-ccntrastine contrastive learning, alignment training, and instruction tuning-PPLLaW VA can be utilized solely during instruction tuning, allowing for seamless transfer from the most advanced MLLMs. (3) PPLLaVA supports flexible output sizes for different modali- ties, whereas the number of queries Q-Former fixed once set. As result, different Q- Formers typically need to be trained separately for images and videos (Zhang et al. 2023 Li ct al. 2023c Liu et 2024f).
3 APPROACH
3.1
MOTIVATION AND ANALYSIS
In the previous section, we discussed that the videos are redundant in both length and content. Vista-LLaMA (Ma al. 2024) demonstrated that the extensive number of tokens in long videos makes it difficult for LLMs to capture video content. In this section, we further examine whether redundant video content impacts the performance of video LLMS and whether extracting key video content can enhance performance. Inspired by EgoSchema (Mangalam et al., 2024), we adopt the certificate length to measure the redundancy. The certificate length of video-QA QA pair is determined by the shortest video sub-clip that can answer the question. Instead of using manual annotation, we employed an automated method determine the certificate. Specifically, frames are sampled fps, and then the similarity between each frame and the question-answa text is calculated using CLIP-L-336 (Radford et al., 2021]. If the similarity exceeds 0.5, the frame considered relevant to the text. Finally, the proportion of relevant frames to the entire video is calculated as the certificate.
Based on the Vidco-MME dataset, selected the 100 video-QA pairs with the shortest certificate lengths termed Video-MME--rddun. We then evaluated the per- formance of various models on both the full Video-MME dataset and these selected samples. Additionally, for these 100 samples, we manu- ally selected the frames most
Table The study on the impact of video redundancy, we used the Vicuna- 7B version for all models. "Average" and "Manual" refer to the default average sampling and manual selection, respectively.
full average
redund
Model
Haames&Tokemn
average
manual
InstructBIIP
32&1004
39.2
36.1
39.5
LLaVA-Next
32&4608
41.1
36,9
42,0
LLaVA-Ners -i--eee--Vide
8&1155
42.9
39.0
43.5
VA-VA--e Video
32&:6088
45.0
41.5
46.1
PPL.LVVA (ours)
32&
49.8
47.6
50.5
relevant to the questions, alongside the default frame sampling method. This approach was used to test whether extracting key information enhances video understanding. As shown in Table T all models experienced a decline in performance on high-redundancy videos. As an earlier model, InstructBLIP performed as expected, not matching the overall performance of the more advanced LLaVA-Next. However, high-redundancy videos, InstructBLIIP, which has instruction-aware video feature extraction capabilities, declined slower than L.AVA-Next. Furthermore, when manually selected frames were used, all models showed significant performance improvements, highlighting the importance of extracting key video information for enhancing video understanding. Additionally, we clearly observed the importance of including more frames for long videos, such as those in the Video-MME dataset. These findings motivated us to explore token compression to accommodate more video frames while effectively extracting key information.
3.2 PPLLAVA
As shown in Fig. PPLLaVA, like most video LLMs, includes a vision encoder, a mapping layer, and a LLM. It also features additional text encoder paired with the visual encoder. Given T-frame video, we first pass it through the CLIP- ViT visual encoder, obtaining the visual feature V. TRxxxxxxx This feature is then fed into the Prompt-guided Pooling module, where it is
4

---

Published as a conference paper at ICLR 2026
System: You videu lunge
Large Language Model
Hoiding On the call Reading book v
language asoisttant, You able
Projection
(c (LUMA))
Query: What weating glasses doing took the phene handed to him?
Encoder
ComvolaationS Style
Text
Fine-tin
Pooling
alige
sumple
Frozen
Eincader
Visual
reshape
[TOK]
Figure 2: The overview of PPLLaVA for compressing the video based user prompts and generat- ing responses.
compressed by over 90%, resulting in V' E RT'''">H'>D V is fed into the MLP mapping layer as the final visual input. Importantly, V' not only contains significantly fewer tokens but also condenses information more relevant to the user's instructions. This ensures improved performance while efficiently processing the video input. Note that more advanced MLLMs in recent years often adopt more powerful visual encoders, such as SigLIP. However, in our model, they can be seamlessly switched. Without loss of generality, we will still refer to the visual encoder as CLIP in the following. Next, we will detail how V' is obtained.
Fine-grained Vision-Prompt Alignment. To extract video features relevant the prompt, we first utilize the original CLIP dual encoders to identify which video features are related to the text. Specifically, we input the user's question into the CLIP text encoder to obtain the text feature c Rº. Following the CLIP training pipeline, we only use the CLS token of the text. The attention score the (tth th hh video token relative to the text feature then calculated as:
eXp(Te:
(1)
I
MW E exp(Te Setspo(P
where "(t,w,)) h) represents the token at the (1, wt, h) position in V, T is the CLIP temperature scale, and Setip is the CLIP visual projection, which is typically not used in multimodal LLMss Note that (tt,w,)) typically refers to the patch token from the penultimate layer of CLIP, rather than the CLS token from the final layer used during CLIP training. However, since the spatial representations in CLIP's final layers are similar, applying fatipe still allows the patch tokens to be mapped into the interaction space with the text.
Promet-Guided Pooling, In the previous section, obtained token-level weights corresponding to the user's prompt, which we use as guidance for pooling the video. Unlike traditional tasks that require only ID-dimensional feature for contrastive learning (Ma et al.) 2022, Wang al.) 2022). our approach aims to preserve a certain 3-dimensional structure to enable the [LM to perform temporal modeling. To achieve this, we perform pooling with S {t) in manner similar to 3D convolution. Specifically, we define the spatiotemporal 3D convolution kernel and stride as (ke, kw, ks) and (d,, du, d,), respectively. The output dimension of V can then be expressed as:
I T hr W =
dc
W
kin
(2)
dy
H
Kh
H'
1.
dr.
Unlike conventional convolution kernels, our kernel parameters are derived from Moreover, the parameters of the kernel dynamic; the kernel slides over different positions in V, its parameters

---

Published as a conference paper at ICLR 2026
taken from the corresponding positions in S. Finally, the feature at position (t, h) in the output V' is calculated as:
-1
((,,.,))
((tdde+i,eee Ao.d +kY"
(3)
== k=0
(f h,dd, +-))
By flexibly adjusting stride and kernel size, we can control the output dimensions. This approach allows us to better accommodate videos of varying lengths and facilitates joint training with images, compared to fixed-output methods.
CLIP Context Extension. In our method, CLIP-text is the only additional parameter used. De- spite having significantly fewer parameters than Qformer, achieves better performance. However, CLIP-text has major limitation: its context length is too short (77 for CLIP and 64 for SigLIP). While this length is sufficient for objects or simple descriptions, is inadequate for long prompts propose
multi-turn dialogues in multimodal LLMs. To address this performance bottleneck, extending the context length of CLIP-text using asymmetric positional embedding extensions. In most cases, extending the positional embedding involves randomly initializing new embeddings the end. A more theoretically sound approach is to perform linear interpolation on the original posi- tional embedding at a rate r. Assuming the original and target positional embeddings are and P', respectively, the jth position of P' is:
P + (j (il) (PLJ+1 - PUj), j j=i.r,
(4)
where [i] means taking the floor of j. However, we found linear interpolation yielded inferior re- sults randomly initializing embeddings the end. We believe this because CLIP's positional embeddings are well-trained, and globally averaged interpolation disrupts the well-pre-traines in- formation. Given that short sentences dominate CLIP's training data, the earlier parts of positional embeddings are more thoroughly trained. Hence, adopted asymmetric interpolation, applying different interpolation rates different positions. In the early part of the new positional embedding, use large r value to shorten the interpolation distance, while in the later part, value to extend the interpolation distance. This asymmetric approach allows us to effectively extend the context length of CLIP-text while preserving as much pre-trained information possible.
smaller
3.3 TRAINING
PPLLaVA enables plug-and-play transfer of either image-domain LLMs or image-video unified MLLMss As a result, initialized from well cell-pretrained MI.LMs, we can bypass expensive contrastive or alignment pretraining and proceed directly to instruction tuning. In this stage, we fully fine- tune the LLM, the projection MLP, and the CLIP text encoder. Our instruction datasets include multi-turn and single-turn conversations presented in conversational format, along with various forms of visual input such as images, videos, and multiple images. For different types of data, we employed interleaving training approach. Rather than using batches composed of single data type, we mixed various data types within the same batch. This training method enables the model to simultaneously process both long videos with many frames and single-frame images, greatly enhancing its adaptability to visual sequences of varying lengths.
4 EXPERIMENT
In this section, we have performed comprehensive evaluations of PPLLaVA, covering settings, com- parisons, and ablations, while visualizations and limitations analysis can be found in the appendix.
4.1 EXPERIMENT SETUP
Implementation Details.
To verify the effectiveness of PPLLaVA across different model domains and to ensure fair comparisons with models from different periods, we implemented PPLLaVA on three separate models: the image-domain model LLaVA-Next Liu et al. (2024b), the video-domain model LLaVA- Zhang et al.] (2024c), and general-doman VLM model Intern VL3-8B (Zhu al. 2025). Moreover, these models employ different visual encoders CLIP, LIP Zhai et al.

---

Published as a conference paper at ICLR 2026
Table 2: PPLLaVA performance on 7 video benchmarks, including NexIQA, EgoChema, Activ- ityNet, VideoChatGPT-Bench MVBench, Long VingViddeBB,h, and VideoMME. All results are re- ported as 0-shot accuracy on 7B or 8B size model.
VideoMME
Models
NexIQA
EgoSchema
A-Net
VCG-Bench MVBench
L-V--Bench
Long Overall
Open-Source Video MLLMs
Video-ChatPPe (Maz et
2023
35.2
2.42
32.7
39.1
LLAMA-VID Liet al
2023
38.5
47.4
2.89
ChatUniVi Jin al. 2023
45.8
2.99
35.8
40.6
LLvVa-NeXT-Vidoo
Liu al.
2024b
70.2
43.9
53.5
3.26
50.5
46.5
VideoAgent Wang
2024h
71.3
54.1
VideoTree Wang et al.
M024c
75.6
61.1
Park
Park at al. 2024
72.9
61.1
STLLM [Liu al.
2024e
50.9
3.15
54,9
31.3
37,9
VideoLLaMA2 (Cheng el
2024
75.6
51.7
53.0
3.12
$7.3
43.8
46.6
LA Zhang al.
2024b
69.3
3.19
46.2
52.6
VidzoCher2 (Li
20230
2.98
51.1
36,0
33.2
39.5
PLLaVA Xu al. 2024a
56.3
3.12
46.6
LLaVA-One Vision
2024a
56.7
56.4
46.7
58.2
Video-XL
55.3
55.5
LLaVA-Video Zhane et
2024g
82.2
56.5
3.52
58.4
58.2
50.6
63.2
Apollo Zohar et al.
2024
58.5
61.3
Oryx-1.3 5 at.
2024g
81.8
3.62
57.6
58.8
IntemVI.3 al.
2025
75.4
58.8
66.2
Videol.JaMM3 (Zhang et
2025
84.5
63.3
61.3
69.7
59.8
66.3
PPLLaVA (LLaVA-Next)
74.9
60.1
56.
3.32
59.2
53.6
47.4
53.6
PPLLaVA (LLaVA-Vidco)
84.1
61.6
59.7
3.66
58.8
60.4
54.3
64.5
PPLLaVA (InterVVL33)
86.8
63.9
60.3
3.61
75.6
60.3
56.6
67.1
2023). and Intern VIT Chen et al. (2024b), respectively-which further validates the generalization ability of PPLLaVA across diverse visual features. For image and multiple-image inputs, the pooling kernel and strides are set to (1, 3). For video inputs, we uniformly sample 32 frames and set the pooling kernel and strides (2, 3), compressing the video tokens by 18 times more aggressive compression compared to the 4x reduction used in Qwen-VI. or LLaVA-Video For CL.IP context extension, when < 20, is set to 1, and when > 20, set 0.25. We train for one epoch using learning rate of 2e 5 and a batch size of 256. Since Intern VL338B employs Intern ViT-30M,, which undergoes extensive post-training after the initial contrastive pre- training, we re-conducted contrastive pre-training to ensure alignment between the visual and text encoders. Specifically, we used CLIP-L14-ted (with ViT-300M initialized from CLIP-L14) together with the frozen Intern ViT-30M. trained 10M image text pairs sampled from LAION Schuhmann et al.] (202T) and Wukong Gu et al. (2022). The post-training aligned CLIP-L14-text was then adopted the initialization of 16 A100 GPUs or 32 910B NPUs.
much
The full training takes 36 hours
Data Details. The instruction tuning data includes diverse modalities and sources. We randomly sampled 300k image data from the LLAVA-1.5 training set (Liu et al., 2024a) and used 594k multiple-image data from LLAVA-Interleave (Li et al. 2024b). The video data includes Kinetics (Kay et all 2017). SthSth-V2 V2 (Goyal et al. 2017). Next-QA (Xiao et al. 2021b). CLEVRER (Yi
al. 2019 and LLAVA- Video-300k, resulting in a total of 1.1 multimodal training samples.
We evaluate our model on 7 video LLM benchmarks, categorized into two types based on the eval- uation method: GPT-based evaluation and multiple-choice questions. The GPT evaluation mainly involves open-ended QA, including the VCG Bench (Maaz et al.) 2023) and ActivityQA (Caba Heil- bron et al. 2015). Consistent with most models, we used the GPT-3.5-turbo--
version for testing. The multiple-choiee question benchmarks include NexIQA Xiao et al. 2021a), EgoSchema Man- galam et al. (2024), MVBench (Li et al., 2023c). Long VideoBench (wu et al. 2024) and Video- MME (Fu al. 2024). For medium-to-loe
videos in Video-MME and Long VideoBench, sam- pled 64 frames instead of the 32 frames used in other datasets. Our test corpus encompasses videos of various genres and lengths, offering comprehensive evaluation of PPLIaVA's performance.

---

Published as a conference paper at ICLR 2026
Table The ablation study on model components. TP means throughput (seconds/video).
Context Length
VCG Bench
Video-MMR (w/ subs)
Model
C1
DO
CU
TU
CO
Avg
TP
Short
Medium
Long
Overall
TP
LLaVA-Next (Average Pooing)
576
3.05
3.07
3,71
2.62
3.01
3.09
2,9
53.1
41.3
36,0
43,4
3.1
LLaVA-Next (w/o Pooing)
4608
3.23
3.08
3.82
2.75
3.11
3.20
15.0
58.4
45.1
38.8
47.4
15.2
+Prompt-guided Pooling
1024
3.21
3.15
3.80
2.88
3.02
3.21
4.6
59.0
45.6
42.2
48.9
5.3
+CLIP Context Extension
1024
3.32
3.20
3.88
3.00
3.20
3.32
4.6
59.7
48.6
44.0
50.0
MMR
VideOMEE Throighput
Ovenil on VideoMME (%)
50
Throggput
Thmugliput (secunde vevviiee))
Overall VideoMMEE (%%
Throughput (scoonds (oooosss/Vidoo))
50
45
20
40
14
47
35
44
3
12
24
16
32
kw & d kn & dp
& d Figure 4: Temporal pooling effects. We set T = 32 and kw = du t₂ d, 3, varying the temporal kernel size and stride.
Figure 3: Spatial pooling effects. We set T = 16 and K = de L, varying the spatial kernel size and stride.
4.2 MAIN RESULT
Table provides a quantitative comparison across several video understanding benehmarks. The results indicate that PPLLaVA consistently surpasses previous state-of-the-art models on evalu- ated datasets. For example, compared with LLaVA-OneVision PPLLaVA-LLaVAVideo achieves performance gains of 4.7%, 1.5%, 3.1%, 4%, and 6.3% on NexIQA, EgoSchema, ActivityNet, Long Beng,,ee,,,,,, and VideoMME, respectively. Importantly, PPLLaVA excels in handling long video content. Under similar frame sampling settings but with significantly fewer tokens, it out- performs LLaVA- Video and LLaVA-One Vision by 3.7% and 7.6%, respectively, on VideoMME videos longer than 30 minutes. Compared with Intern VL3, PPLLaVA also achieves 1.6% im- provement on LongVideoBenche essential information from highly redundant video streams within limited context window. On benchmarks focused on reasoning, such as NexIQA and EgoSchema- where the videos are rela- tively Short-PPLLaVA still delivers strong results. This suggests that the model's ability identify and utilize key information plays a crucial role in enhancing video reasoning capabilities. Even on VCG-Bench where summarization and caption-type questions dominate, PPLLaVA still achieves leading results. This indicates that the improvements of PPLLaVA are not merely attributable to the semantic alignment provided by CLIP-like encoders; rather, the model has learned to adaptively extract critical video features, enabling it to perform well even when user queries contain limited information. Moreover, when compared to other models designed for key content extraction, includ- ing VideoAgent, VideoTree, and LVNet, PPLLaVA continues to maintain clear performance lead. This demonstrates the effectiveness of incorporating motion priors in improving video understand- ing across range of benchmarks. Finally, whether based image model (LLaVA-Next), video (Ineel(InerrVV// PPLLaVA consistently outperforms
This underscores PPLLaVA's strength in efficiently extracting
model (LLaV Video), or a general-domal its competitors under similar baseline, especially considering that it also reduces the visual token length by several times. This demonstrates the generalizability of the PPLLaVA architecture, which can be seamlessly integrated into various types of VLMs.
4.3 ABLATIONS AND ANALYSIS
Unless otherwise stated, in the ablation studies, for efficiency, PPLLaVA is uniformly based on the LLa VA-Next version, and the training data includes all video data, with single-image and multi- image data excluded.
Model Components. The core of PPLLaVA is its prompt-guided token compression. To assess the impact of this feature. conducted ablation experiments overall components. As shown in Table 3 while the LLaVA-Next Baseline's 's direct averaging method is the most efficient, its per- formance subpar. Directly feeding all tokens into the L.M yields reasonable results but suffers
8

---

Published as a conference paper at ICLR 2026
Table 4: The image results. * means sell-impleemnntation
Model
Resolution
MMMU(val)
MattVista
MMB-ENG
MMB-CN
MM- Vet
SEED-IMG
MME
POPE
LLaVA- L--13
336
36.4
27.6
67.8
63.3
36.3
68.2
1531/295
85.93
AlaVA--ext-77
6724672
35.8
34.6
67.4
60.6
711.2
1519/332
86.53
LaV
3364336
60.9
32.0
84,40
Chat-Uniy 1.5
3361336
62.7
28.3
85.40
LLaVA-Next- Video
336*336
34.2
28.9
64.7
56.7
44.0
64.6
1501/351
83.10
PPLLaVA
336*336
37.9
34.6
68.9
62.0
44.7
70.7
1539/277
88.46
from low throughput. Our Pooling module substantially improves both efficiency and performance. Extending the CLIP context further enhances results, particularly in long video understanding. The improvement both efficiency and effectiveness underscores the superiority of our model.
Pooling Size. PPLLaVA can flexibly implement pooling at any scale. However, the pooling kernel and stride increase, while efficiency improves, there will inevitably be performance degradation. Therefore, it's erucial to find balances efficiency and performance. As illustrated in Fig. 3 first explore the impact of pooling in the spatial dimension. is evident that when the pooling kernel and stride small, increasing them significantly improves efficiency, and thanks to the prompt- guided approach, the performance remains almost unaffected. In contrast, as shown in Fig. pooling in the temporal dimension yields smaller efficiency gains compared spatial scaling, with more noticeable performance degradation as the kernel and stride sizes increase. When the pooling kernel and stride are large, the efficiency gains tend to plateau, but the decline in effectiveness becomes significantly pronounced. Considering all factors, for video input, we ultimately selected a pooling kernel and stride of (2, 3, 3) ensure substantial improvement in efficiency while maintaining stable performance.
Image Performance, The VP method can also seamlessly applied to images. Although images do not have the same need for token compression videos, the guidance from user prompts can still similarly enhance performance. In Table 4 we present PPLLaVA's results on various pop- ular image LLM benchmarks. Since PPLLaVA was trained VLa image data based on LLaVeee we compared the results of these two models. We also compare the image perfor- mance with LLaVA-Next- Video and other image-vides unified models. As shown, PPLLa shows a significant advantage in image performance compared to video models, indicating that PPLLaVA has effectively retained pre-trained knowledge. Compared image models, PPLLaVA, model, still achieved better results on most benchmarks. Notably, our pooling method reduced the visual tokens one-ninth of the original count at the same resolution. This demonstrates that PPLLaVA can achieve both performance and efficiency improvements even on image-based tasks, highlighting its potential for lightweight multimodal LLM.
video
Beyond the weighted average pooling detailed in the main text, we experimented with sev- eral alternative pooling methods guided by the prompt. First, we applied sepa- rate spatiotemporal pooling, conducting pooling operations independently temporal and spatial dimensions before concatenation. We also explored com- binations of different pooling sizes to assess their impact. Lastly, imple- mented max pooling using weights de- rived from the prompt as guidance. As shown Table 5 spatiotemporal separate pooling demon- strates the worst performance, underscoring the importance of maintaining the 3-dimensional spa- tiotemporal structure during pooling. Max pooling, though slightly better, still falls short, suggesting that few prominent features are insufficient to represent the entirety of the video. The combination of various pooling kernels performs similarly to direct weighted averaging when the context length is comparable. Furthermore, we experimented with TOME-like (Bolya et al. 2022) token merging approach, which adaptively fuses visual tokens based on their NxN similarity. However, as shown at the bottom of T TOME not only achieves lower compression ratio compared to PPLLaVA but also performs worse in terms effectiveness. Consequently, we opted for weighted averaging PPLLaVe it provides optimal results while maintaining simpler structure.
Pooling Approach.
Table 5: The ablation study on the Pooling Approach. We report the overall performance of VideoMME (w/ subs).
Pooling Method weighted average separate S-T max pooling
kernell kerne12 tokens
Overall
(2,3,3)
1024
53.6
the
608
44.1
(2,3.3)
1024
52.0
multiple multiple Token Merging
(1,6,6)
(8,2,2
1088
52.8
(4,3,)
(2,4,4)
1088
53.2
2048
51.9

---

Published as a conference paper at ICLR 2026
5 CONCLUSION
In this paper, we propose Prompt-guided Pooling LLaVA (PPLLaVA), a novel pooling method that simultaneously achieves token compression and prompt-aware feature extraction. While re- cent video LLMs can handle long video inputs via extended context lengths, they often suffer from excessive visual tokens, resulting high computational costs and limited scalability. To address this efficiency bottleneck, model introduces three key components: Fine-grained Vision- Prommp Aligmment, Prompt-Guides Convolution Style Pooling, and CLIP Context Extension. These mod- ules enable aggressive reduction of visual context while preserving instruction-relevant informa- tion. Extensive experiments demonstrate the effectiveness of PPL.aVV across a wide range of tasks and video lengths, achieving state-of-the-ar performance with significantly improved effi- ciency-particularly excelling in long video understanding while maintaining strong results on short video and image tasks.
Ackmowledgements. This work was supported by National Science and Technology Major Project (2044ZD0INL00101) and National Natural Science Foundation of China (62302227).
REFERENCES
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502. 13923, 2025.
Daniel Bolya, Cheng- Yang Fu, Xiaoliang Dai, Peizhao Zhang, Christoph Feichtenhofer, and Judy Hoffman. Token merging: Your vit but faster. arXiv preprint arXiv:2210.0946)) 2022.
Fabian Caba Heilbron, Victor Escorcia, Bernard Ghanem, and Juan Carlos Niebles. Activitynet: A large-scale video benchmark for human activity understanding. Proceedings of the ieee conference on computer vision and pattern recognition, pp. 961-970, 2015.
Joao Carreira and Andrew Zisserman. Quo vadis, action recognition? a new model and the kinetics dataset. In proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6299-6308, 2017.
Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shen- glong Yc, Hao Tian, Zhaoyang Liu, al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05221,, 2024a.
Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual- visual-linguistic tasks. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 24185-24198, 2024b.
Zesen Cheng, Sicong Leng, Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang, Ziyang Luo, Deli Zhao, et al. Videollama 2: Advancing spatial-temporal modeling and audio understanding in video-Ilms, arXiv preprint arXiv:2406.0 074766 2024.
W Dai, J Li, D Li, AMH Tiong, J Zhao, W Wang, B Li, P Fung, and S Hoi. Instructblip: towards seneral-purpose vision-language models with instruction tuning. arxiv. Preprint posted online on June, 15:2023, 2023.
Chaoyou Fu, Yuhan Dai, Yondong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Video-mme: The first-ever comprehensive evalua- tion benchmark of multi-modal lims in video analysis. arXiv preprint arXiv:2405.2107. 2024.
Raghav Goyal, Samira Ebrahimi Kahou, Vincent Michalski, Joanna Materzynska, Susanne West- phal, Heuna Kim, Valentin Haenel, Ingo Fruend, Peter Yianilos, Moritz Mueller-Freitas et al. The" something something" video database for learning and evaluating visual common sense. In Proceedings of the IEEE international conference on computer vision, pp. 5842-5850, 2017.
10

---

Published as a conference paper at ICLR 2026
Jiaxi Gu, Xiaojun Meng. Guansong Lu, Lu Hou, Niu Minzhe, Xiaodan Liang, Lewei Yao, Runhui Huang. Wei Zhang, Xin Jiang, et al. Wukong: A 100 million large-scale chinese cross-modal pre-training benchmark. Advances in Neural Information Processing Systems, 35:26418-260 18-26431, 2022.
Tengda Han, Weidi Xie, and Andrew Zisserman. Temporal alignment networks for long-term video. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2906-2916, 2022.
Peng Jin, Ryuichi Takanobu, Caiwan Zhang, Xiaochun Cao, and Li Yuan. Chat-univi: Unified vi- sual representation empowers large language models with image and video understanding. arXiv preprint vrXiv:2311.08046 2023.
Will Kay, Joao Carreira, Karen Simonyan, Brian Zhang, Chloe Hillier, Sudheendra Vijaya- narasimhan, Fabio Viola, Tim Green, Trevor Back, Paul Natsev, et al. The kinetics human action video dataset. arXiv preprint (rXiv:1705.06955 2017.
Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.088 033326 2024a.
Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. Llava-next-interllavee: Tackling multi-image, video, and 3d in large multimodal models. arXiv preprint arXiv:2407.0755 0,7855, 2024b.
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models. arXiv preprint arXiv:2301.12597, 2023a.
KunChang Li, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang, Limin Wang, and Yu Qiao. Videochat: Chat-centric video understanding. arXiv preprint arXiv:2305.063555 2023b.
Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang. Jilan Xu, Guo Chen, Ping Luo, et al. Mvbench: A comprehensive multi-modal video understanding benchmark. arXiv preprint arXiv:2311.17003 2023c.
Yanwei Li, Chengyao Wang, and Jiaya Jia. Llama-vid: An image is worth 2 tokens in large language models. arXiv preprint arXiv:231 7043, 2023d.
Haotian Liu, Chunyuan Li. Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- fion, pp. 26296-26000 26306, 2024a.
Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, ocr, and world knowledge, 2024b.
Jiajun Liu, Yibing Wang, Hanghang Ma, Xiaoping Wu, Xiaoqi Ma. Xiaoming Wei, Jianbin Jiao, Enhua Wu, and Jie Hu. Kangaroo: powerful video-language model supporting long-context video input. arXiv preprint arXiv:2408. 15542, 2024c.
Ruyang Liu, Jingjia Huang, Wei Gao, Thomas H Li, and Ge Li. Mug-stan: Adapting image- language pretrained models for general video understanding. arXiv preprint arXiv:2311.1077. 2023a.
Ruyang Liu, Jingjia Huang, Ge Li, Jiashi Feng, Xinglong Wu, and Thomas H Li. Revisiting temporal modeling for clip-based image- image-to-vides vi-video knowledge transferring, In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 6555-6564, 2023b.
Ruyang Liu, Chen Li, Yixiao Ge, Thomas H Li, Ying Shan, and Ge Li. Bt-adapter: Video conver- sation is feasible without video instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13658-13667, 2024d.
Ruyang Liu, Chen Li, Haoran Tang, Yixiao Ge, Ying Shan, and Ge Li. St-llm: Large language models are effective temporal learners. arXiv preprint arXiv:2404 003008 2024e.
11

---

Published as a conference paper at ICLR 2026
Ruyang Liu, Shangkun Sun, Haoran Tang, Wei Gao, and Ge Li. Flow4agent: Long-form video understanding via motion prior from optical flow. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 23817-23827, 2025.
Xiaohong Liu, Xiongkuo Min, Guangtao Zhai, Chunyi Li, Tengchuan Kou, Wei Sun, Haoning Wu, Yixuan Gao, Yuqin Cao, and Zicheng Zhang. NTIRE 2024 quality assessment of Al-generated content challenge. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6337-6362, 2024f.
Zuyan Liu, Yuhao Dong, Ziwei Liu, Winston Hu, Jiwen Lu, and Yongming Rao. Oryx mllm: On- demand spatial-temporal understanding at arbitrary resolution. arXiv preprint arXiv:2409.1293 2024g.
Huaishao Luo, Lei Ji, Ming Zhong, Yang Chen, Wen Lei, Nan Duan, and Tianrui Li. Clip4clip: An empirical study of clip for end end video clip retrieval and captioning. Neurocomputing, 508: 293-304, 2022.
Ruipu Luo, Ziwang Zhao, Min Yang, Junwei Dong, Minghui Qiu, Pengcheng Lu, Tao Wang, and Zhongyu Wei. Valley: Video assistant with large language model enhanced ability, arXiv preprint arXiv:2306.07207, 2023.
Fan Ma. Xiaojie Jin, Heng Wang, Yuchen Xian, Jiashi Feng, and Yi Yang. Vista-llama: Reducing hallucination in video language models via equal distance to visual tokens. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13151-13160, 2024.
Yiwei Ma, Guohai Xu, Xiaoshuai Sun, Ming Yan, Ji Zhang, and Rongrong Ji. X-clip: End-to- end multi-graines contrastive learning for video-text retrieval. In Proceedings of the 30th ACM International Conference on Multimedia, pp. 638-647, 2022.
Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Shahbaz Khan. Video-chatgpt: Towards detailed video understanding via large vision and language models. arXiv preprint arXiv:2306.054224 2023.
Karttikeya Mangalam, Raiymbek Akshulakov, and Jitendra Malik. Egoschema: A diagnostic bench- mark for very fong-form video language understanding. Advances in Neural Information Process- ing Systems, 36, 2024.
NVIDIA, Arslan Ali, Junjie Bai, Maciej Bala, Yogesh Balaji, Aaron Blakeman, Tiffany Cai, and et al. Cao, Jiaxin. World Simulation with Video Foundation Models for Physical AI, February 2026.
Jongwoo Park, Kanchana Ranasinghe, Kumara Kahatapitiya, Wonjeong Ryoo, Donghyun Kim, and Michael S Ryoo. Too many frames, not all useful: Efficient strategies for long-form video qa. arXiv preprint arXiv:2406.09336, 2024.
Changhao Peng, Yuqi Ye, Shuangjun Du, Wenxu Gao, and Wei Gao. Dual-path condition alignment for diffusion transformers. In The Fourteenth International Conference on Learning Representa- tions, 2026. URL httttttss///peereeei/
Bowen Qu, Xiaoyu Liang, Shangkun Sun, and Wei Gao. Exploring AIGC Video Quality: A Focus on Visual Harmony Video-Text Consistency and Domain Distribution Gap. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6652-6660, 2024.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference machine learning, pp. 8748-8763. PMLR, 2021.
Shuhuai Ren, Linli Yao, Shicheng Li, Xu Sun, and Lu Hou. Timechat: A time-sensitive multimodal large language model for long video understanding. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 14313-14323, 2024.
12

---

Published as a conference paper at ICLR 2026
Christoph Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarczyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, and Aran Komatsuzaki. Laion-400m: Open dataset of clip-filtered 400 million image-text pairs. arXiv preprint arXiv:2111.021111 2021.
Xiaoqian Shen, Yunyang Xiong, Changsheng Zhao, Lemeng Wu, Jun Chen, Chenchen Zhu, Zechun Liu, Fanyi Xiao, Balakrishnan Varadarajan, Florian Bordes, et al. Longvu: Spatiotemporal adap- tive compression for long video-language understanding. arXiv preprint arXiv:2410. 1...4440....7..... 2024.
Yan Shu, Zheng Liu, Peitian Zhang, Minghao Qin, Junjie Zhou, Zhengyang Liang, Tiejun Huang, and Bo Zhao. Video-xl: Extra-long vision language model for hour-scale video understanding. In Proceedings of the Computer Vision and Patlern Recognition Conference, pp. 26160-26169, 2025.
Enxin Song. Wenhao Chai, Guanhong Wang, Yucheng Zhang, Haoyang Zhou, Feiyang Wu, Haozhe Chi, Xun Guo, Tian Ye, Yanting Zhang, et al. Moviechat: From dense token to sparse memory for long video understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattemn Recognition, pp. 18221-18232, 2024.
Shangkun Sun, Xiaoyu Liang, Songlin Fan, Wenxu Gao, and Wei Gao. Ve-bench: Subjective- aligned benchmark suite for text-driven video editing quality assessment. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 7105-7113, 2025a.
Shangkun Sun, Xiaoyu Liang, Bowen Qu, and Wei Gao. Content-Rich AIGC Video Quality As- sessment via Intricate Text Alignment and Motion-Aware Consistency, February 2025b.
Qiang Wang, Yanhao Zhang, Yun Zheng, Pan Pan, and Xian-Sheng Hua. Disentangled representa- tion learning for text-video retrieval. arXiv preprint arXiv:2203.07111 2022.
Xiao Wang, Qingyi Si, Jianlong Wu, Shiyu Zhu, Li Cao, and Liqiang Nie. Retake: Reducing tempo- ral and knowledge redundancy for long video understanding. arXiv preprint arXiv:2412.20504, 2024a.
Xiaohan Wang, Yuhui Zhang, Orr Zohar, and Serena Yeung-Levy. Videoagent: Long-form video understanding with large language model as agent. In European Conference on Computer Vision, pp. 58-76. Springer, 2024b.
Ziyang Wang, Shoubin Yu, Elias Stengel-Eskin, Jaehong Yoon, Feng Cheng, Gedas Bertasius, and Mohit Bansal. Videotree: Adaptive tree-based video representation for 11m reasoning on long videos. arXiv preprint 1rXi::2405.19209 2024c.
Haoning Wu, Dongxu Li. Bei Chen, and Junnan Li. Longvideobench: A benchmark for long-contex, interlcaved video-language understanding. Advances in Neural Information Processing Systems, 37:28828-2888 2024.
Junbin Xiao, Xindi Shang, Angela Yao, and Tat-Seng Chua. Next-qa: Next phase of question- answering to explaining temporal actions. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition, pp. 9777-9786, 2021a.
Junbin Xiao, Xindi Shang, Angela Yao, and Tat-Seng Chua. Next-qa: Next phase of question- answering to explaining temporal actions. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition, pp. 9777-9786, 2021b.
Lin Xu, Yilin Zhao, Daquan Zhou, Zhijic Lin, See Kiong Ng, and Jiashi Feng. Pllava: Parameter- free llava extension from images to videos for video dense captioning. arXiv preprint arXiv:2404. 16994, 2024a.
Mingze Xu, Mingfei Gao. Zhe Gan, Hong- You Chen, Zhengfeng Lai, Haiming Gang, Kai Kang, and Afshin Dehghan. Slowfast-llava: A strong training free baseline for video large language models. arXiv preprint 147iv:2407.158411 2024b.
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang. Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint 1515 2024.
13

---

Published as a conference paper at ICLR 2026
Yuhang Yang, Ke Fan, Shangkun Sun, Hongxiang Li, Ailing Zeng, FeiLin Han, Wei Zhai, Wei Liu, Yang Cao, and Zheng-Jun Zha. VideoGen- Agent-basee System for Video Generation Evaluation, April 2025.
Yugi Ye. Zijian Zhang. Junhong Lin, Shangkun Sun, Changhao Peng, and Wei Gao. Sautodrive\\texte
Unified chain of perceppion-prediiioon-ppiiin thought via reinforce- ment fine-tuning. In The Fourteenth International Conference on Learning Representations, 2026.
URL
net forum /tt
Kexin Yi, Chuang Gan, Yunzhu Li, Pushmeet Kohli, Jiajun Wu, Antonio Torralba, and Joshua B Tenenbaum. Clevrer: Collision events for video representation and reasoning. arXiv preprint arXiv:1910.01440 2019.
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 11975-11986, 2023.
Boqiang Zhang, Kehan Li, Zesen Cheng, Zhiqiang Hu, Yuqian Yuan, Guanzheng Chen, Sicong Leng, Yuming Jiang, Hang Zhang, Xin Li, et al. Videollama 3: Frontier multimodal foundation models for image and video understanding. arXiv preprint urXiv:2501.13106, 2025.
Hang Zhang, Xin Li, and Lidong Bing. Video-llama: An Instruction-tunce audio-visual language model for video understanding. arXiv preprint arXiv::306.. 02858, 2023.
Haoji Zhang, Yiqin Wang, Yansong Tang, Yong Liu, Jiashi Feng, Jifeng Dai, and Xiaojie Jin. Flash-vstream: Memory-based real-time understanding for long video streams. arXiv preprint arXiv:2406. 240.002500.88885. 08085, 2024a.
Peiyuan Zhang, Kaichen Zhang, Bo Li, Guangtao Zeng, Jingkang Yang, Yuanhan Zhang, Ziyue Wang, Haoran Tan, Chunyuan Li, and Ziwei Liu. Long context transfer from language to vision. arXiv preprint arXiv:2406. 16852, 2024b.
Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Ziwei Liu, and Chunyuan Li. Video instruction tuning with synthetic data. arXiv preprint 1xXiv:2410.02711: 2024c.
Xingyi Zhou, Anurag Arnab, Shyamal Buch, Shen Yan, Austin Myers, Xuehan Xiong, Arsha Na- grani, and Cordelia Schmid. Streaming dense video captioning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18243-18221 2024.
Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Internv13: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.10479 2025.
Orr Zohar, Xiaohan Wang, Yann Dubois, Nikhil Mehta, Tong Xiao, Philippe Cansen-Estruch, Licheng Yu, Xiaofang Wang, Felix Juefei-Xu, Ning Zhang, al. Apollo: An exploration of video understanding in large multimodal models. arXiv preprint arXiv:2412. 10360, 2024.
14

---

Published as a conference paper at ICLR 2026
Qu: How does the girl feel this video?
Qu: What the number three dinenssoonl graphics con sten?
QJ: When demenstrating the Germony nodern Christmas tree is in tidly decorsted with apples, candles and berniss, which kind the deccration has the largest number? Apppless. B.Candles. C.Berriss D. The some number
QZ What the gare of this vides? is nexe neport introdacing the history behind Oiristnce, It is documentory the evolution of Christmas. C. is travel slag exploring Christmas. D. ae futorial DIV Christmas
the falloxing used this vides? Cards, string.
the main content vides? shows magic doughter froiing nagic. anizing magicion's doughter.
teaching dsughter magic
What the object ilustrated the thind the vides?
which the auther depict the handshoka
Figure 5: The visualization of the attention weights used guide video pooling.
50
Figure 6: The visualization of the attention weights used to guide video pooling. The prompts for all videos are "Describe this video.".
A QUALITATIVE RESULTS
In Fig. 5 we visualize the attention weights used guide video pooling based on the user prompts. For the same video, we tried different questions. It can be clearly observed that the model's 's attention shifts noticeably depending on the question. For example, when the user asks about the girl's feel- ings, the attention is significantly focused on her face. Conversely, when asked about the number of 3D objects the video, the attention shifts more toward the 3D objects. These visualizations demonstrate that while reducing the visual context, PPLLaVA effectively captures the key informa- tion in the video. In Fig. 6. we additionally illustrate the attention weights for captioning- related questions, these questions theoretically provide less informational content. As shown in the fig- ure, prompts like "Describe this video," which lack specific references, result in attention weights being evenly distributed across the foreground. This indicates that our model still plays significant
15

---

Published as a conference paper at ICLR 2026
video demonstrating routine while holding basketball, performe while naintaininn rhythin and fludity The video coptures movenents and expressions, highlighting cordidence dance.
the this video baskethall player?
No, the man in the video is not basketball player: he demonstrating dorce fashion relcted movemonts.
Tell something about the this video.
The mon in the video dressed in stylish outfit and appears to be confidently perforning dance rap routine. He wearing layered cutfit with jacket, furtleneck, and trouzers, complemented by necklace and black accessory. video also includes text overlays in English script, may provide additional context or information about mon content being showcased.
Can you describe this video!
video shows u man stunding in room and perfa performing a series of dance moves or stops. appears to following a choreogrophed reutine or demonstrating set of movements The man chessed in striped shirt and beige pants, and he maintoiks fecused diencanor throughout the video, The room has weeden flooring and simple furnishings, including door and couple
background There room even, suggesting ting indoor lighting. The movements are deliberate and controlled, indiosting reheorsed practiced routine
sign the wall the bockground lighting
What peacticing this viden?
The man the video practicing a sequence of movements or steps, which could part of dance or exerose routine.
What makes this video fumy??
The individudl's exaggerated and humorous donce movements or gestures make the video funny.
Is the mun prectcing Thi Chi profesionally?
No, the mon's attire and demeznor auggest rar enn-grofessionll or instructiond Tal chi proctice.
you deserbe this video!
video captures mon standing in front sign pleque Asian characters on smiling and speaking conversing directly the comerd, The lighting consistent, and the mon appears to be indoors. The mon dressed in dark. colored jacket coot with visible coller and pink inner gorment or shirt. bockground static, and the focus remsire the throughou?
Deserihe the ftcial expression of this man.
friendly approachable expression slight gentle eyes appear to open and ergaged, and his mouth is slightly open or curved upwards in mannr that suygests positive or welcoming demeanor
man
What makes this video fanty?
exaggerated expressions gestures. focial expressions body languge are over the top und seem to be intentionally comodic.
Figure Qualitative result of multi-turn video conversation and reasoning.
16

---

Published as a conference paper at ICLR 2026
Can you this
The video and billboards. The womon is dressed in stylish black or red leather jacket ond appaars confidint she novigates the wel The video highlights the woman's the warm glow artificial lights neon sigra, creating a dynomic energetic ambiance.
capturee captures wamon walking down vibront city street at night, showcasing the lively ahmosphere and illuminated signs
sunglesses handbog, adding to her fashicrable
oppearance. The
Describe the scenes of video detail
The video shawcoses series of dramatic and etmespheric
rugged, natural environment with elements fantesy emironments, such rocky snowy landscopes, mystical polette suggest
characters settings with illuninated statues or runss. The lighting overcast atmosphere. The characters are dressed detailed, possibly historical or fontasy-inspinn attire, and some of them are wielding wespons or casting spells, The video coptures sends of action and explorction, with characters nevigcting through the unvironment and inferacting with their surroundings. The video also includes textual elements, such as Chinese chanacters subtities, which add the cultural narrative contextt of the scenes.
a time day
light moody.
me
this video attrativve details.
The video showcases coptivating and immersive environment with detailed textures and ctmospheric lighting that create seriee of depth and mystery, The visuds feature dramatic londscapes with rocky cliffs ethereal lighting effects, and sense of ancient ruins or fontasy settings. The color palette is domviated muted earth tones with occasional bursts of vibrant colors fantasy inspired dynomic
like glowing cras fiery effects, The characters and creatures within the video textures expressions convey
with intricate sequences with characters wielding weopons casting spells, contributing to the sense adventure and confilct the narnative.
otherworidly mythical beings. video
Figure 8: Qualitative result of video summary and detailed video description.
role in handling captioning-related questions. In Fig. and 7 we further present some examples of video dialogue. As shown in Fig. g for the famous Sora video, PPLLLVVA accurately and intricately describe details about the protagonist and the environment. For the more complex scene changes in the trailer for Black Myth Wu Kong, PPLLaVA remarkably captures the details of each scene and character. In Fig. 7 PPLLaVA maintains accuracy and consistency across multiple rounds of dialogue and is capable of making reasonable inferences on open-ended questions.
B LIMITATION
Limited model size and context. Although the 7B PPLLaVA has demonstrated impressive perfor- mance, even rivaling that of 34B video LLMs, biggest regret is that, due to having only eight A100 GPUs, we were unable to train larger model or extend the context length to uncover the full potential of this architecture. Current state-of-the-art MILMs,, such as VA--Oe-Oee Vision, typically adopt an 8K or longer context, which requires at least 64 A 100 GPUs for training- an expense cannot afford. On the other hand, the smaller context length ensures PPLLaVA's exceptional efficiency, representing a trade-off between efficiency and performance.
Dependency on prompt. PPLLaVA utilizes extract relevant information. However, when the user instruction carries limited information (e.g.. "Describe this video"), ), may impact the model's performance. of MLLM methods that rely on semantic priors Park et al. (2024): WWang et al. (2024b); Shen et al. (2024): Wang al. (202cc a). Nonetheless, there is evidence that PPLLaVA can partially mitigate the impact of insufficient information in user prompts: (1) Theoretically, numerous studies have shown that Q-former can adaptively extract key video features, even when given minimal or no user prompts. Since PPLLaVA functions more lightweight and flexible version of Q-former, it may also inherit Q- Q-former's ability to adaptively capture essential video features. (2) Quantitatively, in the VideoChatGPT--enn happening in the video?" and "What is the sequence of events in the video?" Despite this, our method still performs well VideoChatGPT. (3) Qualitatively, in Fig. 6 we visualize the attention weights
instructions compress visual tokens and
fact, this common limitation
over 50% of the questions caption-style prompts such as "What is
17

---

Published as a conference paper at ICLR 2026
used to guide video pooling for captioning-relater questions. As shown in the figure, despite the lack of sufficient information in prompts like "Describe this video," the attention weights still manage to distribute relatively evenly across the foreground while filtering out some meaningless background. This suggests that our model remains effective in handling captioning-related questions.
C
USE OF LLM
In this work, the LLM is used solely for language polishing and serves no other purpose.
18


Reviews:
[{"review_id": "REtpsf0qi6", "reviewer": "Reviewer_u53s", "summary": "This paper introduces PPLLaVA, a novel video understanding framework that improves both efficiency and performance by compressing visual tokens based on user prompts. The core idea is to reduce video redundancy using a prompt-guided 3D pooling mechanism, which dynamically compresses visual tokens while preserving instruction-relevant information. The model includes three key components: a CLIP-based vision-prompt alignment module to identify relevant visual regions; a prompt-guided pooling mechanism that performs 3D convolution-style pooling using prompt-based attention weights; a CLIP context extension module to support longer text inputs for multi-turn dialogues.  \nPPLLaVA achieves up to 18× token compression, and consistently outperforms strong baselines (e.g., LLaVA-OneVision, InternVL3) across 7 video understanding benchmarks, including long-form video tasks. It is also plug-and-play, transferable across different base models (LLaVA-Next, InternVL3, etc.).", "questions": "1. Have you considered making the prompt-guided pooling weights learnable (e.g., via lightweight attention or gating mechanisms) instead of directly using CLIP similarity?\n2. How does the model perform when user prompts are vague or generic (e.g., “What is happening?”)? Is there a quantitative analysis of performance degradation in such cases?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Innovative and Practical Solution: The prompt-guided pooling mechanism effectively tackles visual token redundancy in long video understanding, also enhancing performance in image-only tasks, indicating broad applicability.\n2. Strong Empirical Performance: PPLLaVA achieves state-of-the-art results across multiple benchmarks (e.g., Video-MME, NextQA, EgoSchema), often using fewer tokens and enabling faster inference.\n3. High Generalizability and Efficiency: The method demonstrates strong transferability across various base models (image-only, video-only, and unified VLMs), significantly improving throughput—up to 3× faster—with minimal to no accuracy loss.", "weaknesses": "1. The prompt-guided pooling weights, derived directly from CLIP similarity scores without learnable parameters or adaptive gating, may restrict expressiveness and adaptability.\n2. The paper lacks direct comparisons with other prompt-aware compression methods (e.g., VideoAgent, VideoTree) in terms of accuracy and efficiency, despite numerous SOTA model comparisons.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "W2cjDPbW8X", "reviewer": "Reviewer_ddaz", "summary": "This paper proposes PPLLaVA, a video LLM that introduces a prompt-guided pooling strategy for adaptive token compression, aiming to achieve unified and state-of-the-art performance on both short and long video understanding benchmarks.", "questions": "See Weaknesses", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The identification of redundant content as the core issue and the proposed prompt-guided pooling as a solution are well-motivated contributions to the field.\n\n- The paper is well-written and easy to follow.", "weaknesses": "1.  The main contribution of this work is the design of a Q-Former-like module, integrated into a pre-trained MLLM, to reduce the number of visual tokens. The authors use the CLS token from the text encoder and compute its similarity with all visual tokens to parameterize a 3D convolution operation. However, since textual queries may involve spatio-temporal information, computing similarity with individual visual tokens, which lack temporal context, could introduce non-negligible errors.\n\n2.  In the experiments, the authors continually fine-tune their method on three different MLLMs and compare it against their base versions. This comparison appears unfair, as their model has been exposed to additional training data. A more equitable evaluation would involve fine-tuning the baseline models on the same dataset for a valid comparison.\n\n3.  The authors omit discussion and comparison with existing state-of-the-art training-free token pruning methods. When considering such approaches, the proposed method does not appear to offer clear advantages.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ehwURcTS6l", "reviewer": "Reviewer_eVAF", "summary": "This paper addresses the high computational cost and inefficiency of Video LLMs caused by processing redundant visual tokens. It proposes PPLLAVA, a model featuring a prompt-guided pooling strategy. The method uses CLIP-based visual-prompt alignment to generate a Attention map, which then functions as a dynamic 3D convolution-style kernel to aggressively compress visual tokens while retaining task-relevant information. A CLIP text context extension is also introduced to handle long prompts. Experiments on seven benchmarks demonstrate SOTA performance and significantly improved inference throughput.", "questions": "See weaknesses above.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well-motivated and addresses a critical and practical problem in Video LLM deployment: the trade-off between performance and computational efficiency. The motivation analysis in Section 3.1 (Table 1), which demonstrates performance degradation on high-redundancy videos, provides a solid empirical basis for the proposed solution.\n2. The experimental validation is extensive and a significant strength of this work. The authors validate PPLLAVA's effectiveness across diverse video benchmarks.", "weaknesses": "1. The claim of model generality could be further substantiated. While the SOTA results on InternVL3 are strong , all in-depth ablation studies (e.g., Table 3, Table 5) are conducted exclusively on the LLaVA-Next version. Replicating key component ablations on the InternVL3 would be necessary to confirm the generalization. Furthermore, testing the method on other prominent open-source VLMs, such as the Qwen2/2.5-VL series, would significantly strengthen the paper's claims of being a general-purpose module.\n2. The comparison to alternative token compression strategies is insufficient. The paper's main ablation (Table 5) only compares against internal variations (e.g., max pooling) and a \"TOME\" method. This TOME approach, originally proposed in 2022, is a relatively dated technique. This limited ablation does not adequately benchmark the proposed method against the current state-of-the-art. The paper would be more convincing if it included direct baseline comparisons against other full, established, and more recent token reduction methods to clearly isolate the benefits of the prompt-guided approach.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tUo3gtpnf5", "reviewer": "Reviewer_Qq7T", "summary": "This paper presents PPLLaVA, a prompt-guided pooling framework designed for video-based large language models (Video LLMs). PPLLaVA focuses on compressing redundant visual tokens in video sequences while maintaining semantics that are critical to user instructions. The method achieves this through fine-grained vision–prompt alignment and a novel convolution-style pooling mechanism guided by textual prompts. In addition, it extends CLIP’s textual context to better support complex, multi-turn visual dialogues. Extensive experiments demonstrate that PPLLaVA attains state-of-the-art accuracy and efficiency across a broad range of image and video understanding benchmarks, delivering notable gains in both computation speed and task performance—particularly for long-duration video analysis.", "questions": "1.  Are the CLIP-based relevance scores $S=\\{s(t,w,h)\\}$ computed once (frozen) or updated during training?  If CLIP is frozen, how does the model adapt to new prompt distributions?  \n2. How was the interpolation rate (r=1 before 20, r=0.25 after 20) in CLIP Context Extension chosen, empirically or theoretically?\n3. The paper reports “throughput (s/video)” but not FLOPs, GPU memory, or latency at batch=1. Please report actual compute savings (e.g., FLOPs, peak memory, tokens/sec) to make the 18× compression claim quantifiable.\n4. How sensitive is PPLLaVA to the CLIP backbone (ViT-L/14 vs. ViT-H/14 vs. SigLIP)? Have you tested whether replacing CLIP with a weaker vision encoder (e.g., EVA-CLIP) preserves benefits?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "**1. Validity and Soundness**: \nPPLLaVA introduces a highly effective token compression strategy, achieving a reduction ratio of up to 18x. This aggressive compression directly confronts a significant bottleneck in Video LLMs: the prohibitive computational cost associated with long context lengths. Notably, this reduction does not compromise, and in some cases even enhances, performance compared to state-of-the-art models across diverse video understanding benchmarks (Table 2).\n\n**2. Strong Generalization**: \nThe model's robust generalization is further highlighted by the experiments in Table 2. These results show that PPLLaVA can be seamlessly integrated with various mainstream MLLM backbones (such as LLaVA-Next, LLaVA-Video, and InternVL3), yielding consistent performance improvements across the board. This adaptability underscores its powerful generalization and transfer capabilities.\n\n**3. Clarity and Concrete:** \nThe paper is characterized by its clarity. The mathematical exposition is particularly lucid, especially the detailed formulations governing the alignment and pooling mechanisms (see equations in Section 3.2). Furthermore, the end-to-end pipeline is effectively illustrated in Figure 2, providing a clear visual representation of the entire architecture.", "weaknesses": "**1. CLIP Context Extension Ablation:**\n The paper posits the specific contribution of the CLIP context extension (asymmetric positional embedding). However, isolated ablation results for this module are not provided in Table 3. It would be valuable to evaluate its effect in isolation from the prompt-guided pooling module and to further compare it with other methods designed to extend CLIP's text length, such as Long-CLIP[1] and VideoCLIP-XL[2].\n\n**2. Missing Efficency Analysis:**\nEfficiency claims (18× compression) are strong but lack FLOPs or latency analysis (just “throughput (s/video)” — not standardized).\n\n**3. Potential Over-reliance on CLIP-Based Semantics:**\nThe method's core relies on CLIP’s ability to associate text and visual fragments. While the authors acknowledge this (pages 2–3), there is no systematic analysis of whether this dependency limits extensibility to domains beyond CLIP’s pretraining. Furthermore, the CLIP context extension is only heuristically motivated and lacks ablation results isolating this component’s pure effect.\n\n**4. Minor Issue:** \nline 021, \"clip\" should be \"CLIP\"\n\n[1] Zhang B, Zhang P, Dong X, et al. Long-clip: Unlocking the long-text capability of clip[C]//European conference on computer vision. Cham: Springer Nature Switzerland, 2024: 310-325.\n\n[2] Wang J, Wang C, Huang K, et al. Videoclip-xl: Advancing long description understanding for video clip models[J]. arXiv preprint arXiv:2410.00741, 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)