QueryStream: Advancing Streaming Video Understanding with Query-Aware Pruning and Proactive Response
Authors: Kairui Zhang
Zhenyu Yang
Bing Wang
Shengsheng Qian
Changsheng Xu
Venue: ICLR 2026 Poster (2026)
Keywords: Multimodal Large Language Model
Online Video Understanding
Streaming Video Understanding
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: The increasing demand for real-time interaction in online video scenarios necessitates a new class of efficient streaming video understanding models. However, existing approaches often rely on a query-agnostic ''change-is-important'' assumption, which conflates visual dynamics with semantic relevance, leading to computational redundancy and mistimed responses. To address this, we propose QueryStream, a novel framework that integrates query-awareness into the core of video processing and response scheduling. QueryStream features two synergistic components: (1) Query-Aware Differential Pruning (QDP), a policy that filters the token stream by jointly assessing semantic relevance to the query and temporal novelty against a dynamically smoothed history; and (2) Relevance-Triggered Active Response (RTAR), a dual-gated mechanism that schedules responses based on both high query relevance and significant information density. As a lightweight, training-free module, QueryStream achieves state-of-the-art performance on benchmarks such as StreamingBench and OVO-Bench under moderate pruning, and matches full-token baselines while pruning over 70\% of visual tokens. Notably, our pruning mechanism generalizes to offline tasks, where it serves as a context-denoising module that benefits long-form video understanding. This work not only reveals the vast semantic redundancy in video streams relative to user intent but also establishes a promising, intent-driven direction for efficient and robust online video understanding. Code is available at: https://github.com/Zhangkr2003/QueryStream.

Published as a conference paper at ICLR 2026
QUERYSTREAM: ADVANCING STREAMING VIDEO UNDERSTANDING WITH QUERY-AWARE PRUNING AND
PROACTIVE RESPONSE
Kairui Zhang', Zhenyu Yang Bing Shengssennn Qian Changsheng Xu1.235 "Shangai Tech University, "MAIS, Institute of Automation, Chinese Academy of Sciences, UUniversity of Chinese Academy of Sciences, *Tianjin University of Technology.
PPeng Cheng Laboratory
2hangkk edu.en
ABSTRACT
The increasing demand for real-time interaction in online video scenarios ne- cessitates a new class of efficient streaming video understanding models. How- ever, existing approaches often rely on query-agnostic "change-is-mmportan" assumption, which conflates visual dynamics with semantic relevance, leading computational redundancy and mistimed responses. To address this, pro- pose QueryStream, a novel framework that integrates query-awareness into the core of video processing and response scheduling. QueryStream features synergistic components: (1) Query-Aware Differential Pruning (QDP), a policy that filters the token stream by jointly assessing semantic relevance to the query and temporal novelty against dynamically smoothed history; and (2) Relevance- Triggered Active Response (RTAR), dual-gated mechanism that schedules re- sponses based on both high query relevance and significant information den- sity. As a lightweight, training-free module, QueryStream achieves state-of-the- art performance on benchmarks such as StreamingBench and OVO-Bench un- der moderate pruning, and matches full-token baselines while pruning over 70% of visual tokens. Notably, where it serves as a context-denoising module that benefits long-form video un- derstanding. This work not only reveals the vast semantic redundancy in video streams relative to user intent but also establishes a promising, intent-driven di- rection for efficient and robust online video understanding. Code is available at: /tctps://githi com http:/hhaaggk.200
pruning mechanism generalizes to offline tasks,
INTRODUCTION
The paradigm of video understanding is undergoing fundamental shift from offline, retrospective analysis to online, interactive scenarios, driven by applications such embodied AI (Duan et al. 2022]. autonomous driving (Grigorescuu et al. 2020)). live event monitoring (Chen et al.
2024a), ) and
real-time interactive editing Yang al. (022)) Recent advances in Large Lision-Laanuugge Models (LVLMss) (Li et al. 2023a] Dai et al. 2023) Li Liet al. 2024a Hurst et al. 2024 Chen et al.
2024b Bai
al. 2025 Comanici al. 2025) have catalyzed the development of powerful Video Large Lan-
guage Models (Video-LLMs) MMaaz et al. 2023 Li et al. 2023b Ataallah et al. 2024 Zhang et al.)
2025a Wang al. 20256 Li al. 2024a). However, these models are predominantly designed for offline settings, treating video as static, finite batch of frames. This paradigm is misaligned with the nature of streaming data, which requires processing continuous, unbounded video streams with minimal latency. The sheer volume and inherent temporal redundancy of streaming video render exhaustive, frame-by-frane processing computationally prohibitive and introduce unacceptable re- sponse delays. The central challenge, therefore, is to devise mechanisms for intelligent information filtering and proactive response generation, bridging the gap between the power of Vidco-LLM and the demands of real-time interaction.
"Corresoonding author: shengsheng.qian hwt.aeemmniiia@nnippaaa.....m @nlpr.ia.cccn

---

Published as a conference paper at ICLR 2026
videojiu
Trigger logic
User Query: When did the woman in the black tracksuit in the video throw the shot put?
Respoces
Query Rehecnnee
Timaal Diswily
i-hh
Query agsin
1-22
Passive Reaponse (6 g. ReKV)
poly
The event described does not appear in the provided vidro segment.
Remabus sitent until a new explicit prompt.
She threw the shot put after hendlig down and guickly twisting for hody.
Spuriously Triggered by Irvelevant Soone Change
Query-Agnostic Proactive (e.g. TimeChat-Oninen
After she war shourn pocpuring
After the womon in the white shir
p-32
Afur demomsrating the sonce and grip.
domosstrated the technique
All exccated the throwing nution.
No need to again
QueryStream (Ours)
Correctly remains silent, aecaitngg relevant event.
p-32
She threv the shat put after the wamaon the while demerstrated the technique.
Figure 1: A comparison of response paradigms in streaming video understanding. Given query about future event, QueryStream remains silent during irrelevant segments and proactively responds only when the queried event occurs, enabling efficient and precise interaction.
To this end, prior work can be broadly categorized into passive and proactive response models. Passive models (Yang al.) 2025b; Di al. 2025 Huang et al. 2025 Ning al. 2025) focus on efficient memory management for on-demand querying, but their defining characteristic is that they require user prompt to trigger response. In contrast, proactive models (Chen al. 2024a Wu et al., 2024bb Wang et al. 2024 Li et al. 2025a Wang et al. 2025a Yang et al. (025cc aim to autonomously determine when to respond. Despite their advanced interactivity, common drawback of these systems is their reliance on heavily trained, specialized modules for response scheduling, which often compromises their computational efficiency and response accuracy.
More recently, TimeChat-Online (Yao et al., 2025) introduced an elegant approach that leverages visual change detection concurrently prune redundant tokens and infer opportune moments for philosophy, however, rests a flawed premise: it conflates
response. This raw visual dynamics with true semantic relevance. As illustrated in Figure a model guided by such principle is thus prone to eтror. It can be spuriously triggered by visually salient yet semantically irrelevant changes, such as abrupt scene transitions or the actions of actors unrelated to the query, while missing the truly relevant event when it is brief visually inconspicuous. This disconnect between visual saliency and query-specific importance results in inaccurate responses and wasted computation, highlighting the need for query-informed paradigm.
To address these limitations, we propose QueryStream, a novel framework that instills query- awareness into the core of streaming video understanding for efficient processing and interactive response. As shown in Figure QueryStream is designed to overcome the pitfalls of prior paradigms by redefining information filtering and response scheduling through two synergistic components.
First, we introduce Query-Aware Differential Pruning (QDP), a token pruning strategy that moves beyond naive frame-to-frame fo-frame comparisons. QDP evaluates the importance of information from two perspectives: semantic relevance to the user's 's query and temporal novelty. Notably, temporal novelty determined not against the immediately preceding frame, but against a dynamically smoothed history (DSH) representation of recent frames. This design makes QDP robust to slow visual drifts and transient noise. As result, a token preserved only if satisfies both criteria: (i) must be semantically relevant to the user's query, and (ii) it must represent significant temporal deviation from the smoothed historical context. This selective policy ensures that the model's computational focus is directed toward sparse yet meaningful visual dynamics.
Second, we tackle the challenge of timely interaction with a Relevance-Triggered Active Response (RTAR) mechanism. Unlike methods that rely complex learned schedulers (e.g., predicting EOS token) or simple, query-agnostic change detection, RTAR dynamically determines optimal response moments by monitoring key signals. A response is triggered only when both conditions
2

---

Published as a conference paper at ICLR 2026
satisfied: (i) the current visual input is highly aligned with the query's semantics, and (ii) there a significant influx of new, query-relevant information, as reflected by an increase in the token keep rate under the QDP mechanism. This dual- poliey enables proactive and contextually appropriate interactions.
Our contributions are summarized follows:
We propose QueryStream, novel, training-free framework that establishes query-centric paradigm for efficient processing and proactive interaction streaming video understand- ing. Its modular design allows for seamless integration with off-the-shelf Video-LLMs.
We introduce Query-Aware Differential Pruning (QDP), a token pruning mechanism that jointly models semantic relevance and temporal novelty using a dynamically smoothed historical context, improving the accuracy and efficiency of token filtering.
We design the Relevance- Triiggeee Active Response (RTAR) policy, a dynamic scheduling mechanism that triggers responses based on dual criterion of semantic relevance and information density, enabling opportune and context-aware interaction.
Extensive experiments demonstrate that Query.Stream achieves state-of-the-art perfor- mance on multiple video understanding benchmarks, while substantially improving com- putational efficiency.
2 RELATED WORK
Streaming Video Understanding. Streaming video understanding seeks to process continuous video streams in real time for interactive applications. Early approaches can be broadly divided into passive and proactive models. Passive models emphasize efficient memory management for on- demand querying. typically through dynamic K KV-caches or memory banks that preserve historical context (Di et al., 2025 Ning al. 2025 Zhang et al. 2024). While computationally efficient, these models remain purely reactive, generating responses only upon explicit user prompting. Proactive models, in contrast, autonomously decide when to respond, for instance by predicting special EOS tokens (Chen et al., 2024a) or using auxiliary classification heads (Wang et al., 2024). The most relevant work, TimeChat-Online (Yao al. 2025), introduced an elegant proactive strategy that couples response triggering with visual change detection. However, such proactive methods are in- herently query-agnostic: their response policies are governed either by heavily trained, task-specifi heuristic. In contrast, QueryStream introduces
modules or by the simplistic iahange-is-importaan a lightweight, logic-driven proactive mechanism (RTAR) that intrinsically query-aware, thereby enabling more accurate and context interactions without additional training.
Visual Token Pruning. The redundancy of visual data in videos has motivated substantial research on token pruning. Early approaches compress frames or clips into fixed number of tokens (Li lt al. 2024b Ren et al. [2024] Xu et al., 2024 Yang et al., (0025)). which fails to adapt to the varying information density of video streams. More advanced methods introduce adaptive pruning strate- gies, though most remain query-agnostic. A notable example is the Differential Token Drop (DTD) from TimeChat-Onlin
(Yao et al. 2025). which preserves tokens based on inter-frame dissimilarity. While adaptive, its key limitation lies in conflating visual change with semantic importance, point highlighted in our discussion. Another line of work explores language-gul guided query-aware prun- ing (Song et al. 2024 Zhang et al.) 2025b Li et al.] 2025b). However, these methods are largely designed for offline processing and are ill-suited to streaming settings, since they typically require re-processing the entire video history for each new query. Our QDP bridges these paradigms by being adaptive to the evolving video content while remaining sensitive to user intent, and by operat- ing
manner that incrementally processes incoming frames without redundant recomputation. Moreover, its incorporation of a dynamically smoothed history for novelty detection enhances robustness beyond simple frame-to-fi oo-frem comparisons.
QUERYSTREAM
In this section, we elaborate on the proposed QuerySttraam framework. QueryStream is a lightweight, plug-and-play module designed to enhance pre-trained Video-Large Language Models
3

---

Published as a conference paper at ICLR 2026
User Query
Reqponee
mowe propanty
?
fgg
apen batis
Mary
M
New DISH
Future Frouse
Keep
M
Vision Encoder
DSHI
ISS
Putch
Ponifunns
Merge
Fentures /rrpp
Qanr
CLIP-Like Model Image Encoder
M %%
Fuskers Mark
Current
Future
D
Proned Tukien
Purchity
Language Model
Tripper Time
Full Frome
Your
Image Encoder
Fromes Festures
A
Monr D,
High
Query Features
Similarity
Sim R.
Lose
Pour she selte
Figure 2: Overview of the QueryStream framework. Acting as intelligent gateway, Query tream employs Query-Aware Differential Pruning (QDP) to filter tokens by intersecting semantic relevance (Mscm) and temporal novelty (Mumg.) Simultaneously, the Gllvance-TTrigeere Active Response (RTAR) policy monitors relevance (Ra) and information density (D,) to trigger the Video- LLM only optimal moments, ensuring efficient and timely responses.
(Vidco-LLMs) for online, interactive tasks by instilling awarenss into their core processing. We begin with high-level overview of the architecture in Section 3.1 followed by detailed descrip- tions of its two key technical components: the Query-Awar Differential Pruning (QDP) mechanism in Section 3.2 and the Selevance-Triggere Active Response (RTAR) policy in Section 3.3
3.1 ARCHITECTURAL OVERVIEW
The overall architecture of QueryStream is illustrated in Figure It operates intelligent pre-prost pre-processing gateway that sits between the raw video stream and a backbone Video-LLM (c.g., Qwen2.5-VL. (Bai al., 2025)). Its core philosophy is to align the model's computational focus with the user's intent by establishing a direct interaction between the visual stream and the query's semantics. Given a continuous video stream and user's query, its primary function twofold: to judiciously filter out semantically and temporally redundant visual tokens before they reach the com- putationally expensive Video-LLMM and (ii) dynamically identify the most opportune moments to trigger response from the model.
The framework's workflow follows two parallel paths processed by lightweight, pre-trained vision- language encoder (we use OpenCLIP (Cherti et al., (023)). The first path, QDP, generates a pruning mask for each frame. The second path, the RTAR policy, analyzes the frame's 's relevance and formation density to decide whether to activate the Video-LLM's decoder. The original, unprumed visual tokens are temporarily held in a memory buffer. Upon receiving a trigger signal from RTAR, the accumulated pruning masks are applied to this buffer of tokens in just-in-time manner. The resulting sparse token set, along with the query, is then fed into the backbone Video-LLM to gen- erate timely and contextually grounded response. This architecture ensures that the powerful but resource-intensive Video-I.LM is invoked sparingly and purposefully.
3.2
QUERY-AWARD DIFFERENTIAL PRUNING
The core of our method is the Query-Aware Differential Pruning (QDP) mechanism, a lightweight module designed to distill dense visual stream into sparse, query-relevant token sequence. QDP's principle. Instead of
philosophy is a stark departure from the conventional schang-is-impooraae treating all visual dynamics equally salient, it employs dual-criterion sieve that preserves visual token only if corresponding patch is (1) semantically aligned with the user's query and (2) temporally novel against dynamically maintained historical context.
4

---

Published as a conference paper at ICLR 2026
Formally, for given video stream V (ff....,ff fr} and query Q, first use lightweight vision- looonll encoder & (e.g., OpenCLIP) to extract a feature vector v for each patch, yielding set (v....., v } that represents the patch-level features of frame fi alongside query embedding q. The pruning process is then governed by two synergistic filtering criteria.
Semantic Relevance Filtering. To focus computation query-pertinent visual information, first criterion assesses the semantic relevance of each patch. patch is considered relevant its feature vector v has similarity the query embedding q that exceeds dynamic, frame-adaptive threshold. This threshold is the average similarity across all features in the current frame, making the filtering robust to varying scene complexities. The semantic mask Maem thus defined as:
Σ sim(q, vt) v
Msem (t, i) =I sim q, v/)
where I(() is the indicator function and sim(..) denotes cosine similarity. This ensures that the model's 's attention is focused exclusively parts of the scene pertinent the user's question.
Temporal Novelty Filtering. To identify genuine state changes while remaining insensitive to transient noise gradual environmental shifts, our second criterion evaluates the temporal novelty. We eschew naive frame-to-frame comparisons and instead assess novelty against a dynamically smoothed history (DSH). For each patch location i, we maintain a historical feature vector Vasa A patch is deemed novel if its feature vector v significantly deviates from this established context:
Memp (t, i) = II (sim(v;, v vá < Teemp)
Following this check, the historical context is updated to integrate the current visual information:
dah,t = a vf + (1 (1-o) a) Vish.--1
where the smoothing factor E [0, 1] controls the rate of adaptation. This DSH mechanism provides an adaptive reference, ensuring that only significant departures are flagged as temporally novel.
Synergistic Pruning Policy. The final pruning decision is logical conjunction of these two criteria: a visual token is preserved if and only if its corresponding patch passes both the semantic filter Msem (t, i) and the temporal filter Memp (t, i). The final QDP mask is thus computed as:
Mt i) M (t, A Memm((, (t, i).
This dual-filter approach ensures the downstream model processes stream purged of both query- irrelevant and temporally redundant information. Critically, to maintain spatio-tempora integrity, this mask governs the selection of the complete visual tokens. For each preserved patch, both its fea- ture vector and its corresponding Multi-modal Rotary Position Embedding (M-ROPE) are retained. By excising the positional embeddings of discarded patches, we ensure the remaining tokens retain their original and correct (temporal, height, width} coordinates. The output of QDP is thus highly purified, positionally coherent token stream containing only the most salient data for the given query.
3.3 RELEVANCE-TRIGGERI ACTIVE RESPONSE
Complementing the QDP's function of determining what to process, our Relevance-Triigeree Ac- tive Response (RTAR) policy addresses the equally critical question of when to respond. RTAR is dual-gated mechanism that synchronizes the model's responses with moments of high query- specific information influx. This achieved by jointly evaluating two complementary conditions relevance condition (R,) and density condition before triggering a response.
Relevance Condition. The first gate prevents the model from responding during visually active but query-irrelevat segments. To achieve this, it assesses whether the current frame is thematically aligned with the user's query. This condition is met the holistic relevance of the frame, computed by comparing the query embedding q with the mean-pooled frame feature vector V,, surpasses predefined threshold Tret. Formally:
Rs = I(sim(q. V.) > Trel).
Density Condition. While relevance is necessary, it is not sufficient. To ensure responses are triggered by new information, our second gate evaluates the frame's information density. We proxy
5

---

Published as a conference paper at ICLR 2026
this by the token keep rate from QDP mechanism, which naturally quantifies the influx of new, query-relevant information. The density condition is met this rate exceeds a threshold Tden:
N
D, =I
MoDe(t, 7den
N 2m1
Triggering Logic. A response is generated at timestep / only when both the relevance and den- sity conditions are satisfied, ensuring that the model acts on moments that are both contextually appropriate and informationally rich. The trigger signal T, is logical conjunction of the two states:
T, = Rg A D..
This dual-gated policy prevents two failure modes: avoids premature responses to irrelevant vi- sual activity while maintaining sensitivity to brief but significant events. By demanding both high relevance and significant information density, RTAR produces responses that are more meaningful, timely, and aligned with the user's interactive intent.
4
EXPERIMENTS
4.1 EXPERIMENTAL SETUP
Datasets and Metries. Our evaluation comprehensively assesses performance in both online and offline scenarios. For online streaming understanding, we employ two prominent benchmarks: StreamingBench (Lin et al., 2024). a comprehensive benchmark for real-time visual understand- ing, and DVO-Bench (Niu et al. (025), which focuses complex backward tracing and forward active responding capabilities. For offline long-form video understanding, we assess performance on Video-MME (Fu al. 2025) and LongVide BideeBench (Wu et al. 2024a). For all question-ane -nnwering tasks, we adopt accuracy as the primary performance metric. To quantify computational efficiency, we report the Token Keep Rate (%), defined as the percentage of visual tokens retained after pruning.
Baselines. We compare QueryStream against representative set of strong baselines. Our pri- mary comparison with TimeChat-Online (Yao et al. 2025). it represents the most relevant prior work based on query-agnostic differential pruning. We also include other leading streaming Video-LLMs such as Flash- Tlash-VStream/22 VStrram et al. 2024). VideoLLM-online (Chen et al.] 2024a) and Dispider (Qian et al. 2025). To provide a performance reference, we further compare against the original Qwen2.5VI. 7B (Bai al. 2025), which processes the full, unpruned stream of visual tokens.
Implementation Details. Our QueryStream model is implemented by replacing the query-agnostic pruning module in TimeChat-Online-71 with proposed query-aware mechanisms. Additionally, to evaluate the zero-shot generalization our pruning strategy, we also integrate the QDP module into the base Qwen2.5VL-7B model. For the feature extraction that underpins our pruning decisions, we utilize the publicly available OpenCLIP- ViT-LL14 (Cherti et al. 2023) as our lightweight vision- language encoder E. The DSH smoothing factor is set to novelty (Temg), relevance (Tre), and density (Taen) are determined on a small held-out validation set (see Appendix [A6] and are applied consistently across all experiments. Unless specified otherwise, our model processes video streams at I FPS. Crucially, QueryStream requires no additional fine- tuning; all results are achieved zero-shot, plug-and-play manner, underscoring its adaptability and ease of integration. All experiments are conducted on single NVIDIA 80 GB A800 GPU.
O.1. The thresholds for temporal
4.2 MAIN RESULTS ON STREAMING VIDEO BENCHMARKS
Performance on StreamingBench. The results on StreamingBench detailed in Table D clearly demonstrate the superiority of QueryStream's intent-driven filtering over the query-agnostic ap- proach of TimeChat-Online. At moderate token keep rate of 57.2% QueryStream achieves an overall score of 75.32, surpassing TimeChat-Online (74.32 with 55.8% keep rate) by significant 1.0-point margin. Notably, this score nearly matches the performance of the full-token TimeChat- Online baseline (75.36), demonstrating substantial computational savings with negligible perfor- mance impact. The advantage of query-award awareness becomes even more pronounced under aggres- sive pruning. With highly efficient token keep rate of just 29.6%, QueryStream's score of 74.04

---

Published as a conference paper at ICLR 2026
Table I: Performance comparison on StreamingBench. The table benchmarks comprehensive suite of models, including proprietary, open-source offline, and online Video-LLMs. A key compar- ison is drawn between our proposed QueryStream and the strong TimeChat-Onling baseline under varying token keep rates. The results highlight QueryStream's 's ability to achieve state-of-the-art per- formance while operating with significantly fewer visual tokens.
Model
rraess
Keep Rute
TR
SU
ACP
Human
89.47
92.00
93.60
91.47
95.65
92.52
88.00
88.75
89.74
91.30
91.46
Proprictary MLLMs
Gemini 1.5 pro
79.02
80.47
83.54
79.67
80.00
84.74
77.78
64.23
71.95
48.70
75.69
GPTT4
64
77:11
80.47
83.91
76.47
70.19
66.67
69.12
49.22
73.28
Claude 3.5 Sonnet
20
73.33
80.47
84.09
82.02
75.39
79.53
61.11
61.79
69.32
43.09
72.44
Open-souree Offline LMds
Vido-LLaMM227B
17
45.61
VIL.A-1.5-8B
14
53.68
49.22
70.98
56.86
53.42
53.89
54.63
48.
50.14
17.62
52.32
Vidco-CCAM. Ceo-CCAM-1140
96
56.40
57.81
65.30
62.75
64.60
$1.40
42.59
47.97
49.58
31.61
53.96
LongVA-7B
128
70.03
63.28
61.20
70.92
62.73
59.50
61.11
53.
51.67
31.73
59.96
InennLL-V2-8B
77.12
59.26
Kangaroo-7B
64
71.12
84.38
70.66
73.20
67.08
61.68
56.48
55.60
62.04
38.86
64.60
LLaVA-NcXT-Via Vidxo----odo-2BB
61
78,20
70.31
73.82
76.80
63.35
69.78
$7.11
56. 10
61.31
38.86
66. 66.96
MiniCPM- 6V
32
71.93
71.09
77.92
75.82
64.60
65.73
70.37
56,
62.32
53.37
67.44
LLaVA-OneVision
72
72.67
71.65
67.59
65.72
Owen2.5-VL-77
lip
78.32
80:47
78.86
80.45
76.73
78.50
79.63
63. 41
66.19
$3.19
73.68
Open-source Online V.deoL.LMs
Plash-VStream--
25.89
43.57
24.91
23.87
27.33
13.08
18.52
23.87
48.70
23.23
VideoLLM-oollnee
fps
31.05
45.96
32.40
31.48
27.89
35.99
Dispider-78
rps
74.92
75.53
74.10
73.08
74.44
59.92
76.14
62.91
62.16
45. 85.80
67.63
Time Chinnne
tps
55.8%
81.03
83.59
78.55
81.09
76.73
$0.37
75.93
63.82
68.47
47.87
74.32
TimeChut- CinCCh---nni
77.7%
66.
Time hhr-- Chur--Olinnnn7
tps
82.03
79.50
83.33
76. 10
78.50
79.
69.60
57.98
75.36
QueryStream-77
1ps
57.2%
82.38
84.38
79.18
82.37
77.99
81.31
78.70
044
69.32
47.34
75.32
QueryStream-78
29.6%
82.11
83.59
78.23
82.69
75.47
80.06
79.63
63.00
67.90
42.55
74.04
still outperforms TimeChat-Online (72.96 with a 33.0% keep rate) by 1.08 points, despite process- ing even fewer tokens. This consistently superior performance validates that query-aware pruning acts as an effective context-denoising mechanism. By filtering out semantically irrelevant visual provides the model with a cleaner context. This benefit is particularly evident in reasoning- -havy sub-tasks; for instance, at the 60% keep rate level, it outperforms its counterpart on Causal Reasoning (CR) and Text-Rich Understanding (TR) by 0.79 and 0.94 points, respectively.
noise,
Performance OVO-Bench. On OVO-Bench, benchmark designed to test complex reasoning, QueryStreammss 's advantages are further pronounced (Table With a token keep rate of 52.9%, our model establishes new state-of-the-art score of 49.4 among all online models, surpassing even the full-token TimeChat-Onling (46.7) by a significant 2.7-point margin. This superior performance the contrary, under aggressive pruning regime that keeps only 20.0% of tokens, QuerySttream (47.5) still maintains substantial performance lead over both the compressed (47.6 6 at 55.4% keep rate) and full-token (46.7) versions of its counterpart. A closer inspection reveals that this performance gain is consistent across all three ma- jor categories, with the most significant improvements observed in the more challenging Backward Tracing and Forward Active Responding tasks. This suggests that our intent-driven filtering provides a more robust context for complex temporal reasoning.
is not achieved the cost of efficiency;
4.3 PERFORMANCE ON OFFLINE LONG- VIDEO TASKS
To assess the generalization of our query- pruning, we evaluate its efficacy on offline long- video benchmarks. with results in Table 3 showing compelling performance across both scenarios.
Results on VideOMME. On VideOMME, our query-aware approach demonstrates a clear advan- tage over query-agnostie methods and even full-token processing. First, when QDP is applied as a zero-shot module to the base Qwen2.5-VL-77B. it achieves a score of 63.6 with a 52.4% token keep rate, outperforming its full-token counterpart (63.2). This counter-intuitive superior performance with less data our hypothesis that QDP acts an effective context- denoising mechanism. The advantage is particularly pronounced on the challenging "long" subset, where method surpasses the baseline by substantial 2.2-poin margin (52.6 vs. 50.4). Our full Query model further confirms this superiority, scoring 63.8 and outperforming the compara- ble TimeChat- Onlline configuration (63.3) a similar efficiency level.
aedding-achieving
7

---

Published as a conference paper at ICLR 2026
Table 2: Evaluation results on OVO-Bench. OVO-Bench comprises three challenging categories: (i) Real-Time Visual Perception, (ii) Backward Tracing, and (iii) Forward Active Responding. Our proposed QueryStream is benchmarked against a comprehensive suite of models. The results high- light its state-of-the-art performance among online models, demonstrating robust capabilities on complex temporal reasoning tasks while operating with significantly fewer visual tokens.
Model
P'rumes
Real-Time Vioual Perception
Backward Tracing
Forward Active Responding
Overall
OCR
AC3
ATR
STU
FRD
OIR
FPM
ASI
HLD
RRC
SSR
CRR
Avg.
Ang.
Agenns
92.6
Progrietary Multimodal Models
CiTT-4o
Open-suuce Offline LidsoLLMss
LL NeXT Video 7B
64
69.8
59.6
66.4
50.6
72.3
61.4
63.3
51.2
64.2
9.7
41.7
67.6
60.8
54.2
53.1
LLaM LLVAA-OnVVsoon 76
64
67.1
58.7
69.3
71.
60.3
62.8
52.5
23.7
45.0
24.8
66.9
60.8
50.9
52.9
VL-7B
64
34.4
48.6
65.7
50.8
52.7
IntervVL
64
68.5
38.7
27.4
44.0
50.1
LongVU
55.7
49.5
57.4
9.1
39.5
60.0
48.5
Ope-soorce Online Video-LIMM
Flab-VSurus-77 TB
Upps
25.5
32.1
29.3
33.7
29.7
25.8
29.9
36.4
33.8
5.9
25.4
5.4
67.3
600
44.2
33.2
Videol M-colin-88 88
3fps
8.1
33.9
12.1
14.0
45.5
21.2
20.8
22.2
18.8
12.2
17.3
Ify (55.4%)
74.5
48.6
68.1
48.3
69.3
598
61.4
56.9
64.9
118
44.5
318
58.5
40.0
96.8
47.6
1f (15.35%)
69.8
48.6
64.7
44.9
68.3
55.4
58.6
$1.9
62.5
9.1
42.0
12.5
36.5
40.0
36.4
45.5
TineChus-Culie-77
Mi (1009)
75.2
46.8
70.7
47.8
69.3
61.4
61.9
55.9
59.5
9.7
41.7
31.6
38.5
40.6
36.7
46.7
QueryStreamm-i
Fow (52.9%))
75.2
49.5
693
50.0
713
62.5
63.
56.9
65.5
124
46.9
35.
43.3
41.7
402
49.4
QueryStrcam-771
Fig (20.99)
74.5
47.7
70.7
46.6
713
576
61.
54.2
63.5
86
42.
332
43.1
40.8
30.0
47.5
Table 3: Results on offline long-video bench- marks. We report accuracy on Long VineeBeennn and VideoMME (w/o subtitles). Our QDP module is evalu- ated as zero-shot plug-in on Qwen2.5-VL (w/ QDP), and we also report the performance of QueryStream.
Table 4: Ablation QDP components on StreamingBench. We analyze the individual and synergistic effects se- mantic and visual pruning criteria.
Pruning Method
Keep'
Score (AlI)
Long VideBeedeee
No Praming (Baseline)
100.0
75.36
everall
long
Viisual Pruming Only
63.4
74.76
Video Longri
sec--50 sinn
30--0
SSemantic Pruning Only
61.7
74.52
Open Source Offline Videol
QueryStream (Full QDP)
57.2
75.32
LLaMA VID 7B
MovieChat-ii 7T
2046
38.2
33.4
Next Valeo-7B
43.5
45.6
Table Ablation the RTAR trig-
VideoChar2
39.3
39.5
35.2
Long 7D
52.6
46.2
gering policy. Results on -VO-Bench's 's Forward Active Responding tasks, com- paring accuracy (Acc.) with the score- based metric (Score) that rewards both accuracy and timeliness.
54.2
56.0
46.6
Video CCAM-
VideoXL.
55.5
49.2
Owen2: 5-V1-78
5-V1-88------ w/
63.4
Owen25-VL-77
Open-snurce Oulline VideoL.LMSs
ValsCCha-hulnn--t
Dispider TB
$2.8
57.2
46.9
Triggering Method
Acc. (Ang.) Score (Aog.)
TumeChat-Ouliz--1
(1009)
$5.4
62.4
48.4
Raneine:
TimeCha--nnliee-
(53.79)
57.1
63.3
52.4
TimeChat-Dniice (Densiy-Oall)
36.5
29.5
TimoChan-Online-76
(15.0%)
62.5
ChersSmeon Variaus:
QueryStream-TR
493
$2.9
Relevance Only Tragger
40.3
30.2
QueryStrram-78
(16.68))
63.2
Pull RTAR (Ours)
40.3
34.6
Results on Benn The benefits of our approach are further confirmed on Long BideoBech.... At a moderate token keep rate of 52.4%, QueryStream (57.3) already outper- forms the TimeChat-Online baseline (57.1). More compellingly, under an aggressive pruning regime that retains only 16.6% of tokens, QueryStream's performance not only remains highly competitive but improves to 58.0. This suggests that for very long videos with substantial redundancy, aggres- sive, query-aware filtering is not just beneficial for efficiency but can be critical for enhancing model focus and accuracy. Collectively, these findings show that our query-aware approach is not just a streaming optimization but robust paradigm for efficient long-video understanding.
4.4 ABLATION STUDIES
To dissect the architecture of QueryStream and validate key design choices, conduct series of detailed ablation studies. We aim to quantify the individual and synergistic contributions of the components within our QDP and RTAR mechanisms.
8

---

Published as a conference paper at ICLR 2026
Effectiveness of QDP Components. To understand the interplay between the semantic and tempo- ral filters in QDP, we conduct an ablation study with results shown in Table The analysis reveals powerful synergistic effect. Applying either the Temporal Pruning Only Semantic Pruming Only filter in isolation leads to a minor but noticeable performance degradation compared to the no- pruning haseline. This suggests that while each filter reduces token count, their individual criteria are not precise enough to fully separate signal from noise. Remarkably, our full QDP method, which forms the intersection of these two criteria, resolves this trade-off. It achieves the highest efficiency with the lowest token keep rate of 57.2% while restoring performance to a level virtually identical to the full-token baseline. These results demonstrate that the two filters are complementary. Their combination yields a stricter and more precise policy that removes noise each filter alone would retain. By preserving only tokens that are both semantically relevant and temporally novel, QDP delivers a purified context that sharpens model focus and maximizes accuracy at minimal cost.
Impact of the DSH Smoothing Factor. We conduct sensitivity analysis on the DSH smoothing factor a to evaluate the benefit of a smoothed historical context over naive frame-to-fre fo-frame comparisons. As shown in Figure 3 a governs a clear trade-off be-
Impact Smoothing
Perform Parformance Efficiency
Oueel Score OVD Berch Avg
Tokee Tieep Rade 250
Dvab
tween efficiency and performance on OVO-Bench.
B. 3. 5.
A large a = 1.0 makes the model overly sensitive to noise, leading to poor performance, while decreas- ing a stabilizes the historical context and improves accuracy. Performance peaks model achieves the best balance between score and token efficiency. Further reducing a results in an overly long memory, slowing adaptation and degrad- ing performance. These results confirm the impor- tance of smoothed, medium- term history in DSH.
Daily Beaaie
0.1, where the
Naw Coathe
1.00
0.50
0.10 DSH Smoztthing Factor [o]
0.1
Figure 3: Effect of the DSH smoothing factor (a) performance (Overall Score) and effi- ciency (Token Drop Rate) on OVO-Bench.
Analysis of the RTAR Triggering Policy. To demonstrate the superiority of our dual-gated RTAR policy (R2 A D.). we conduct an ablation OVO-Bench's Forward Active Responding tasks, with results in Table 5 The analysis compares raw accuracy (Acc.) with a timeliness-awar metric (Score). The Density-Only trigger, mirroring TimeChat-Onlinee
method, yields the lowest score (29.5) because it often activates on irrelevant dynamic events. In contrast, the Relevance-Only trigger achieves the highest accuracy (40.3) but is penalized for timeliness, resulting in low score of 30.2, since it generates redundant responses for static yet relevant scenes. Our full RTAR policy strikes the optimal balance by attaining near-peak accuracy (40.2) together with a score of 34.6, which is 4.4 points higher than the next best variant. This result confirms that the synergy of the relevance and density gates is crucial for producing responses that are both contextually appropriate and informationally novel and timely. Detailed calculation methods are provided in Appendix A.4
5 CONCLUSION
In this paper, we introduced QueryStream, a novel query-centric framework for efficient and in- teractive streaming video understanding that departs from conventional query-agnostic approaches. QueryStream establishes this paradigm through two synergistic, training-free components: Query- Aware Differential Pruning (QDP) for precise token filtering and Relevance- -rrigeree Active Response (RTAR) for timely interaction. Extensive experiments demonstrate that QueryStream achieves state-of-the-an oftthe-rrtt-tttte performance on streaming benchmarks while processing significantly fewer tokens, and generalizes effectively to offline tasks as context-denoising module. Overall, this work highlights the semantic redundancy of video streams relative to user intent and suggests a more efficient, intent-driven direction for streaming video understanding.
ACKNOWLEDGMENTS
This work is supported by the National Key Research and Development Program of China N0.20233FF33 the Beijing Natural Science Foundation (JQ23018, L252032), and the Na- tional Natural Science Foundation of China (No.U25B2072 62522218, 62276257).

---

Published as a conference paper at ICLR 2026
REFERENCES
Kirolos Ataallah, Xiaoqian Shen, Eslam Abdelrahman, Essam Sleiman, Deyao Zhu, Jian Ding, and Mohamed Elhoseiny. Minigpt4-video: Advancing multimodal Ilms for video understanding with interleaved visual-textual tokens. arXiv preprint arXiv:2404 03413, 2024.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2.5-vl technical report. arXiv preprint arXiv:2502. 2025.
Joya Chen, Zhaoyang Lv, Shiwei Wu, Kevin Qinghong Lin, Chenan Song, Difei Gao, Jia- Jia-Wei Liu, Ziteng Gao, Dongxing Mao, and Mike Zheng Shou. Videollm-onling Online video large lan- guage model for streaming video. In Proceedings of the IEEE/CVE Conference Computer Vision and Puttern Recognition, pp. 18407-18418, 2024a.
Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 24185 14198, 2024b.
Mehdi Cherti, Romain Beaumont, Ross Wightman, Mitchell Wortsman, Gabriel Ilharco, Cade Gor- don, Christoph Schuhmann, Ludwig Schmidt, and Jenia Jitsev. Reproducible scaling laws for contrastive language-iman learning. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 28 2818-2829, 2023.
Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capa- bilities. arxiv preprint arXiv:2507.0626/ 2025.
Wenliang Dai. Junnan Li, Dongxu Li, Anthony Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale N Fung, and Steven Hoi. Instructblip: Towards sencral-purpost vision-language models with instruction tuning. Advances in neural information processing systems, (6.49250-49220 2023.
Shangzhe Di, Zhelun Yu, Guanghao Zhang, Haoyuan Li, Tao Zhong, Hao Cheng, Bolin Li, Wanggui He, Fangxun Shu, and Hao Jiang. Streaming video question- answering with in-context video kv- cache retrieval. arXiv preprint 20X1v:2503.00540, 2025.
Jiafei Duan, Samson Yu, Hui Li Tan, Hongyuan Zhu, and Cheston Tan. A survey of embodied ai: From simulators research tasks. IEEE Transactions Emerging Topics in Computational Intelligence, 6(2):230-24 224000224222200 2022.
Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Video-mme: The first-ever comprehensive evalu- ation benchmark of multi-modal Ilms video analysis. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 24 24108-241 18, 2025.
Sorin Grigorescu, Bogdan Trasnea, Tiberiu Cocias, and Gigel Macesanu. A survey of deep learning techniques for autonomous driving. Journal of field robotics, 37(3)362-3886 2020.
Zhenpeng Huang, Xinhao Li, Jiaqi Li, Jing Wang, Xiangyu Zeng, Cheng Liang, Tao Wu, Xi Chen, Liang Li, and Limin Wang. Online video understanding. Ovbench and videochat-online. In Proceedings of the Computer Vision and Paltern Recognition Conference, pp. 3328-3338, 2025.
Aaron Hurst, Adam Lerer, Adam Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21277 2024.
Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li. Zejun Ma, and Chunyuan Li. iexttieeeenn Tackling multi- imatt video, and 3d large multimodal models. arXiv preprint arXiv:2407.07895 2024a.
10

---

Published as a conference paper at ICLR 2026
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image image pre-training with frozen image encoders and large language models. In International conference machine learning. pp. 19730-19742. PMLR, 2023a.
KunChang Li, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang, Limin Wang, and Yu Qiao. Videochat: Chat-centric video understanding. arXiv preprint arXiv:2305.0633 2023b.
Wei Li, Bing Hu, Rui Shao, Leyang Shen, and Liqiang Nie. Lion-fs: Fast & slow video-language thinker as online video assistant. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 3240-3251, 2025a.
Yanwei Li, Chengyao Wang. and Jiaya Jia. Llama-vid: An image is worth 2 tokens in large language models. In European Conference on Computer Vision, pp. 323-340, Springer, 2024b.
Zhangbin Li, Jinxing Zhou, Jing Zhang, Shengeng Tang, Kun Li, and Dan Guo. Patch-level sounding object tracking for audio-visual question answering. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 5075-5083, 2025b.
Junming Lin, Zheng Fang, Chi Chen, Zihao Wan, Fuwen Luo, Peng Li, Yang Liu, and Maosong Sun. Streamingbench: Assessing the gap for milms achieve streaming video understanding. arXiv preprini S0div:2411.066622 2024.
Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Shahbaz Khan. Video-chatgpt: Towards detailed video understanding via large vision and language models. arXiv preprint arXiv:2306.05424, 2023.
Zhenyu Ning, Guangda Liu, Qihao Jin, Wenchao Ding, Minyi Guo, and Jieru Zhao. Livevlm: Efficient online video understanding via streaming-orienter kv cache and retrieval. arXiv preprint arXiv:2505.15266 2025.
Junbo Niu, Yifei Li, Ziyang Miao, Chunjiang Ge, Yuanhang Zhou, Qihao He, Xiaoyi Dong, Haodong Duan, Shuangrui Ding, Rui Qian, al. Ovo-bench: How far your video-Ilms from real-world online video understanding? In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 18902-18913 2025.
Rui Qian, Shuangrui Ding, Xiaoyi Dong, Pan Zhang, Yuhang Zang. Yuhang Cao, Dahua Lin, and Jiaqi Wang. Dispider: Enabling video lims with active real-time interaction via disentangled per- ception, decision, and reaction. In Proceedings of the Computer Vision and Pattern Recognition 2025.
Conference, pp.
Shuhuai Ren, Linli Yao. Shicheng Li, Xu Sun, and Lu Hou. Timechat: A time-sensitive multimodal large language model for long video understanding. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pp. 14313-14333 2024.
Enxin Song, Wenhao Chai, Tian Ye. Jenq-Neng Hwang, Xi Li, and Gaoang Wang- Moviechat+: Question-aware sparse memory for long video question answering. arXiv preprint 11Xi::2404.17176, 2024.
Haibo Wang, Bo Feng, Zhengfeng Lai, Mingze Xu, Shiyu Li, Weifeng Ge, Afshin Dehghan, Meng Cao, and Ping Huang. Streambridge: Turning your offline video large language model into proactive streaming assistant. arXiv preprint 1tt.v::20000044665 2025a.
Yi Wang, Xinhao Li, Ziang Yan, Yinan He, Jiashuo Yu, Xiangyu Zeng, Chenting Wang, Changlian Ma, Haian Huang, Jianfei Gao, et al. Internvide22. 5: Empowering video milms with long and 2025b.
rich context modeling. arXiv preprint arXiv:2501.
Yueqian Wang, Xiaojun Meng, Yuxuan Wang, Jianxin Liang, Jiansheng Wei, Huishuai Zhang, and Dongyan Zhao. Vidcollm knows when to speak: Enhancing time-sensitive video comprehension with video-text duet interaction format. arXiv preprint arXiv:241 1.7991, 2024.
Haoning Wu, Dongxu Li, Bei Chen, and Junnan Li. Longvideobench: A benchmark for long-conter. interleaved language understanding. Advances in Neural Information Processing Systems, 0228888-288577 2024a.
11

---

Published as a conference paper at ICLR 2026
Shiwei Wu, Joya Chen, Kevin Qinghong Lin, Qimeng Wang. Yan Gao, Qianli Xu, Tong Xu, Yao Hu, Enhong Chen, and Mike Zheng Shou. Videollm-mod Efficient video-language streaming with -ixtuse-of-depths vision computation. Advances in Neural Information Processing Systems, 37:109922-109447, 2024b.
Mingze Xu, Mingfei Gao, Zhe Gan, Hong-You You Chen, Zhengfeng Lai, Haiming Gang, Kai Kang, and Afshin Dehghan. Slowfast-llava: A strong training-free baseline for video large language models. arXiv preprint arXiv:2407.15841, 2024.
Chenyu Yang, Xuan Dong, Xizhou Zhu, Weijie Su, Jiahao Wang, Hao Tian, Zhe Chen, Wenhai Wang, Lewei Lu, and Jifeng Dai. Pve: Progressive visual token compression for unified image and video processing in large vision-language models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 24939-249444 2025a.
Zhenyu Yang, Dizhan Xue, Shengsheng Qian, Weiming Dong, and Changsheng Xu. Ldre: Llm- based divergent reasoning and ensemble for zero-shot composed image retrieval. In Proceedings of the 47th International ACM SIGIR conference on research and development in information retrieval, pp. 80-90, 2024.
Zhenyu Yang, Yuhang Hu, Zemin Du, Dizhan Xue, Shengsheng Qian, Jiahong Wu, Fan Yang. Weiming Dong, and Changsheng Xu. Svbench: A benchmark with temporal multi-turn dialogues for streaming video understanding. arXiv preprint 2xXX::25022.08100, 2025b.
Zhenyu Yang, Kairui Zhang, Yuhang Hu, Bing Wang, Shengsheng Qian, Bin Wen, Fan Yang, Tingt- ing Gao. Weiming Dong, and Changsheng Xu. Livestar: Live streaming assistant for real-world 2025c
online video understanding. arXiv preprini arXiv:251
Linli Yao. Yicheng Li, Yuancheng Wei, Lei Li, Shuhuai Ren, Yuanxin Liu, Kun Ouyang, Lean Wang, Shicheng Li, Sida Li, et al. Timechat-online: 80% visual tokens are naturally redundant in streaming videos. arXiv preprint arXiv:2504. 17343, 2025.
Boqiang Zhang, Kehan Li, Zesen Cheng, Zhiqiang Hu, Yuqian Yuan, Guanzheng Chen, Sicong Leng, Yuming Jiang, Hang Zhang, Xin Li, et al. Videollama 3: Frontier multimodal foundation models for image and video understanding. arXiv preprint arXiv:2501.13106, 2025a.
Haoji Zhang, Yiqin Wang, Yansong Tang, Yong Liu, Jiashi Feng, Jifeng Dai, and Xiaojie Jin. Flash-vstream: Memory-based real-time understanding for long video streams. arXiv preprint arXiv:2406.0800 08085, 2024.
Shaojie Zhang, Jiahui Yang. Jianqin Yin, Zhenbo Luo, and Jian Luan. Q-frame: Query-aware frame selection and multi-resolution adaptation for video-Ilms. arXiv preprint ar0ivv2506.2211111 2025b.
12

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 1 STATEMENT ON THE USE OF LARGE LANGUAGE MODELS
In line with the conference policy, we disclose that Large Language Models (LLMs) were used solely writing aids. Their involvement was limited improving grammar, refining sentence structure, and enhancing readability. All scientific contributions, including the development of ideas, methodology, experiments, and conclusions, were made exclusively by the authors, who take full responsibility for the content of this paper.
A.2 ETHICS STATEMENT
We have read and adhere to the ICLR Code of Ethics. Our research is conducted solely on publicly available benchmarks for video understanding, and all datasets are used in accordance with their licenses, Our framework leverages large pre-trained models (e.g., Qwen2.5-VL. TimeChat-Online OpenCLIP), which, like others of this type, may reflect limitations their training data. While our method does not directly address such issues, it does not introduce additional risks. The intended use of QueryStream is improve the efficiency and responsiveness interactive video understanding systems, such assistive technologies monitoring tools. A positive ethical aspect is its contri- bution to sustainability: by reducing processed tokens, our method lowers computational cost and energy consumption. We declare no competing interests.
A.3 REPRODUCIBILITY STATEMENT
We have made every effort ensure the reproducibility of our work. The source code for QueryS- tream, including the implementation our Query-Aware Differential Pruning and Relevance- Triggered Active Response methods, will be made publicly available upon publication. Our frame- work is built upon publicly available models. For the base Video-LLMM, our experiments utilize both Qwen2.5-VL-7B and AimeChat-Online-
The feature encoder used is OpenCLIP-viT-L/Ie All relevant citations for these models are provided in the main text. All datasets used in our exper- iments, including StreamingBench, OVO-Bench, VideoMME, and LongVideoBent are standard and publicly available benchmarks. Critical hyperparameters and detailed experimental settings are documented in Section [.1] Furthermore, the Appendix provides a comprehensive description of our simulated evaluation protocol for the active response tasks (Appendix A.4) and analysis of our key component choices (Appendix A.5). further aiding the reproducibility of our results.
A.4 SIMULATED EVALUATION PROTOCOL FOR THE RTAR ABLATION STUDY
This section details the simulated evaluation protocol used for the RTAR ablation study presented in Table[5] which focuses the Forward Active Responding category of OVO-Bench As the official online evaluation code for OVO-Bench and the real-time inference code for TimeChat-Onling were not publicly available the time of our experiments, devised a simulated evaluation methodology designed to fairly approximate the benchmark's 's intended real-time assessment.
Our simulation proceeds follows. For a given video, first let both models process the entire stream to identify all potential response trigger points according to their respective mechanisms:
For QueryStream, a trigger is registered at any timestep t where our RTAR policy (T)) fires.
For HimeChat-Online, we simulate its density-bart based trigger by identifying timesteps where its token keep rate (the inverse of the drop rate) is significantly higher than baseline threshold, indicating a moment of high visual change.
From the sequence of trigger timestamps generated by each model, we identify distinct event in- tervals. For each interval, select the first timestamp the definitive response point, t This simulates a model making its first response upon detecting a new, relevant event and prevents dupli- cate evaluations for single continuous event.
Finally, to generate the actual response R,,', we feed only the video frames up to and including the trigger timestamp i into the respective model and perform inference. The resulting response Rm
13

---

Published as a conference paper at ICLR 2026
is then compared against the ground-truth answer Am to calculate correctness using the function F F( Am). Based on this, we compute the two final metrics:
Accuracy (Acc.): The average correctness across all responses, providing a direct measure of response quality.
Acc
N Σ ((mme: Am)
Score: A metric that jointly rewards accuracy and timeliness. It penalizes the temporal deviation of the response from the ideal moment im using an absolute difference 1t,, t. This design ensures that both premature and delayed responses are penalized, encouraging the model to act precisely when sufficient evidence becomes available.
N
Score
F(Rm's Am) 2- t'm -mm1-p
This simulated protocol ensures a fair and consistent comparison, as each model's performance is evaluated based on the context available only up to the point where its own internal logic decided to act. While the initial identification of trigger points leverages the full video stream- necessary simplification to enable offline evaluation the subsequent response generation strictly adheres to temporal constraints, thus closely approximating real-world online scenario.
A.5 ANALYSIS OF COMPONENT SELECTION FOR QUERY
The modular design of our QueryStream frame- work allows the QDP mechanism be inte- grated with various feature encoders and base Video-LLMs. This section details the empirical analysis conducted to justify our final selection of OpenCLIP. OppenLLI--VTT-L14 the feature encoder and D0L-2..5-VL-7B as the base model.
Table 6: Component selection for QueryS- tream. Performance (OVO-Bench Overall Score) of our QDP module with different feature coders and base Video-LLMs on a validation sub-
Our final choice, highlighted in gray, balances performance, efficiency, and fairness.
For an efficient yet representative analysis, we first created validation subset by ran- domly sampling approximately 10% of the data (around 250 samples) from each task category in OVO-Bench. We then evaluated the perfor- mance of our QDP module when paired with different combinations of popular OpenCLIP variants and state-of-the-an art Video-LLMs. The results, measured by the OVO-Bench Overall Score, are presented Table
Owen2.5VVL -7B
Intern VL2.5 -SB
InternVidest
Feature Encoder
-8B
OpenCI VIT-B/22/3
47.2
46.9
OpenCCLIP-ViILLLL
48.5
48.2
OpenC1 MPT--1-
48.9
48.4
49.2
The results yield two key insights. First, for any given Video-LLMM, using a more powerful CLIP encoder (from ViT-B ViT-H) generally leads to improved performance. This confirms the im- portance of high-quality feature representations for effective pruning. However, this performance gain comes the cost of significant computational overhead and latency, particularly with larger encoders like 171-H/14, which is prohibitive for real-time system. Second, the results demonstrate the versatility of our QDP module, which successfully enhances the performance of various leading Video-LLMs, underscoring its plug-and-pe nature.
Based on this analysis, our final component selection was guided by three core principles: (i) Perfor- mance, the combination must deliver strong results on the target benchmark; (ii) Efficiency, the fea- ture encoder must lightweight enough to support real-time operation; and (iii) Fair Comparison, the chosen base Video-LLM should align with our primary baseline for equitable comparison.
Consequently, OpenCLIP-ViT-LA 14 was chosen as offers the best trade-off between feature quality and computational efficiency. Owen2.5-VL-771 was selected as the base model because it is not only a strong and representative open-source model but, critically, it also serves as the foundation for our main baseline, ConeChat-Online This choice ensures that observed performance gains can be more directly attributed to our proposed quere- mechanisms rather than differences in the underlying model architectures.
14

---

Published as a conference paper at ICLR 2026
A.6 HYPERPARAMETER SELECTION
The logic-based gates in our QueryStteam framework rely on three key thresholds: the temporal nov- elty threshold (Tiem) for QDP, and the relevance (Trel and density (Tien) thresholds for RTAR. These values were determined empirically on the same held-out validation set described in Appendix A.5 Our goal was find robust set of parameters that balances performance and efficiency.
Table 7: Impact of the temporal novelty thresh- old (Temp) on token keep rate and overall per- formance on the OVO-Bench validation subset. The selected value is highlighted.
Table 8: Impact of RTAR thresholds (Trel, Tien) on the average Score on the OVO-Bench For- ward Active Responding validation subset. The selected values are highlighted.
Ttemp
Keep Rate (%)
Overall Score
Trel
Tden
Average Score
0.75
21.6
46.5
0.50
0.15
33.1
0.85
35.1
48.2
0.70
0.15
33.8
0.90
52.9
49.4
0.60
0.10
32.5
0.95
68.3
49.1
0.60
0.20
33.2
0.98
78.5
49.0
0.60
0.15
34.6
Determining the Temporal Novelty Threshold (Ttemp)- ) The threshold Tremp directly controls the filtering aggressiveness of our QDP module. A lower value leads to more aggressive pruning (lower keep rate). We performed a sweep over a range of values for Tiemp and evaluated its impact on the overall performance (Score) on the OVO-Bench validation subset.
As shown in Table a value of Tremp = 0.90 achieved the best overall score. While a more aggres- sive threshold of 0.85 offered lower keep rate, began to degrade performance, suggesting that critical temporal information was being erroneously pruned. Conversely, a more lenient threshold of 0.95 retained more tokens without providing commensurate performance gain, indicating that it allowed too much redundancy. Therefore, we selected Temp = 0.90 for all our experiments as it strikes the optimal balance between efficiency and accuracy.
Determining the RTAR Thresholds (Trel and Tden). The RTAR thresholds govern the active re- sponse policy and were tuned maximize the timeliness-aware Score metrie the Forward Active Responding tasks from the same OVO-Bench validation subset. We performed a grid search to analyze the interplay between the relevance and density gates.
The results. summarized in Table [8, indicate that combination of Tee = 0.60 and Tden = 0.15 yields the highest score. Deviating from these values hurts performance: a lower relevance threshold (Te 0.50) caused erroneous triggers on irrelevant scenes, while higher one (Trel = 0.70) missed some valid response opportunities. Similarly, a lower density threshold (Táen = 0.10) was overly sensitive to minor visual noise, whereas higher one (Táen = 0.20) failed trigger more subtle events. The chosen values represent the most robust configuration for triggering responses that are both accurate and timely.
A.7 CASE STUDY
To provide an intuitive understanding of QueryStream's operational advantages, we present single, illustrative case study in Figure The video is specifically engineered probe two common failure modes of aunos-agnostic systems: false trigger from irrelevant visual shocks and miss trigger from subtle, relevant events. To simulate these challenges, we insert several black frames to represent transient shock and include a visually subtle but query-relevant action. The figure visualizes the models' token drop patterns and their resulting responses, offering a direct comparison between QueryStream and TimeChat-Online.
Robustness to Irrelevant Shocks. The first challenge tests the models' robustness to irrelevant motion. As depicted, the query-agnostic TimeChat-Onling falters when confronted with the inserted black frames. Guided by its Shhange-is-importan philosophy, it perceives the abrupt transition major visual event, causing its token drop rate plummet. This leads to spurious trigger, prompting an erroncous and unhelpful response about the screen going black. In stark contrast, QueryStream's QDP mechanism recognizes that the black frames are semantically
False Trigger:
15

---

Published as a conference paper at ICLR 2026
Case Study
Response
[T+0s]
What persen slowing las beno??
[T-10s]
[T=12s]
[T=15s]
Ile tbance enoripteer.
Silence
orartpheee.
QuerySiream
TimeChah Online
He skews bask with sellen and black cakes
[T=31s]
Hl shana snartpheee.
slore vith yeller
[T=366)
[T+37%]
[T=45s]
sleees look with a yelirw end black ener,
duanc unartpheer.
Beskens sare iphoos,
thanca yulioe rowe,
[T=48a]
[T=57s]
shovs baok with tlack cever.
[T#63e]
shows beok with a yelore and back rover.
baak with tack caves.
gray shadlng indicatus patches pruned by GDP.
pea.
[T=80s]
Trigger
False Trigger
Figure 4: Qualitative comparison of QueryStream and TimeChat-Online challenging case study.
irrelevant to the user's query. Consequently, it continues to prune these tokens, maintaining a high drop rate and correctly remaining silent, thus demonstrating its ability to distinguish visual dynamics from semantic importance.
Sensitivity to Subtle Events. The case study also includes slow but crucial action relevant to the query, testing the models' sensitivity. TimeChat-Onlin myopic frame-to-frame comparison fails to register the small inter-frame differences of this subtle action, thus missing the event entirely and resulting in a missed trigger. QueryStream, however, excels in this scenario. Its DSH-based novelty detector registers the persistent, cumulative deviation from the established historical norm. Because this slow change is also highly relevant to the query, both of RTAR's 's gates are satisfied, leading to timely and accurate response. This case highlights the synergistic power of our dynamically smoothed history and query-aware triggering, enabling far more nuanced and intelligent interaction.
Miss Trigger:
A.8 LIMITATIONS
Despite its strong performance and efficiency, QueryStream has several limitations that highlight promising avenues for future research. Firstly, the efficacy of our QDP mechanism is fundamentally bound by the representational quality of the pre-trained OpenCLIP encoder. Its inherent constraints in discerning fine-grained details or abstract relationships may challenge the pruning precision in semantically nuanced scenarios, potentially causing critical but subtle events be missed. Sec- ondly, the current framework is designed around single, static user query and does not explicitly handle dynamic conversational contexts where user intent might evolve over multiple turns. Extend- ing the model to manage continuously updated query state or dialogue history is crucial next step. Finally, our mechanism relies on a set of fixed hyperparameters (a, Tiemp Trel. Tden) for its logic-based gates. While effective, these static thresholds may not be universally optimal. Future work could explore adaptive thresholding mechanisms or even learned, soft-gating policies to en- able more nuanced, data-driven decision-making and enhance the model's robustness across diverse video domains and query types.
16


Reviews:
[{"review_id": "7A5ecxnc9V", "reviewer": "Reviewer_6ZdJ", "summary": "This paper presents QueryStream, a framework for streaming video understanding that builds query-aware temporal representations to improve performance. The method dynamically fuses incoming frame features with past context, guided by query-conditioned similarity maps, allowing the model to emphasize temporally relevant segments as the video unfolds. The paper reports improvements on multiple benchmarks (streming + long video benchmarks) compared with existing streaming or offline VLMs.", "questions": "- Why was OpenCLIP chosen as the encoder instead of more recent options like SigLIP that are already aligned with VLMs?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 4, "contribution": 1, "strengths": "- The introduction is very well written. It clearly defines the gap between offline video-language models and real-time streaming setups, motivating the need for a query-aware temporal design.\n\n- The method is well organized, with a clear hierarchy between query-aware temporal modules, memory updates, and frame-level fusion.", "weaknesses": "- The method is presented as a new query-aware temporal representation, but in practice it relies on several manually tuned parameters and heuristic weighting functions between query and frame embeddings. The current mothod design appears quite sensitive to these hyperparameters even though they have discussed in A.4. Despite the hierarchical structure, the approach still feels heuristic and lacks a clear principle.\n\n- The proposed framework mainly reorganizes existing components such as query-conditioned similarity and temporal fusion into a clean, well-structured pipeline. In the end, it looks more like a refined arrangement of standard similarity-based scoring rather than a fundamentally new approach, so the novelty feels limited.\n\n- The paper emphasizes low-latency and lightweight design, but there is no analysis of runtime, throughput, or resource usage. In streaming scenarios, maintaining query-aware similarity maps and recurrent updates for each frame can be computationally heavy. Without latency or efficiency analysis, it is unclear whether the method is actually suitable for real-time use.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1TO1oaa1iD", "reviewer": "Reviewer_vYGP", "summary": "The paper introduces QueryStream, a novel framework designed to enhance the efficiency and interactivity of streaming video understanding models by incorporating query-awareness into the core processing loop. This work addresses the limitations of existing approaches, which typically rely on a flawed, query-agnostic “change-is-important” principle that conflates raw visual dynamics with true semantic relevance, leading to computational waste and interaction errors in real-time online video scenarios.", "questions": "See weakness above, please.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Query-Aware Differential Pruning (QDP): QDP is a novel token pruning mechanism that is unique because it employs a dual criterion that jointly assesses semantic relevance to the user's query and temporal novelty. Previous token pruning methods were often query-agnostic.\n2. Dynamically Smoothed History (DSH): Within QDP, temporal novelty is assessed not against the immediately preceding frame, but against a Dynamically Smoothed History (DSH). This adaptive historical context ensures the pruning is robust to transient noise and slow visual drifts.\n3. Relevance-Triggered Active Response (RTAR): RTAR is a novel, logic-driven dual-gated mechanism that schedules responses based on a specific confluence of two signals: high query relevance and significant information density. This departs from prior reactive (passive) models or proactive models relying on heavily trained, specialized modules.\n4. Lightweight and Training-Free: QueryStream is designed as a lightweight, training-free module that operates as an intelligent pre-processing gateway, allowing for seamless, zero-shot integration with existing off-the-shelf Video-LLMs.", "weaknesses": "1. The effectiveness of the core Query-Aware Differential Pruning (QDP) mechanism is \"fundamentally bound by the representational quality of the pre-trained OpenCLIP encoder\". The authors utilized OpenCLIP-ViT-L/14, which was chosen as a pragmatic trade-off between feature quality and real-time computational efficiency. However, the inherent constraints of this encoder in discerning \"fine-grained details or abstract relationships\" may challenge the \"pruning precision in semantically nuanced scenarios\". This reliance risks causing critical but subtle events to be erroneously pruned and missed.\n2. The logic-based gating mechanisms (QDP and RTAR) rely on a \"set of fixed hyperparameters\". These values were determined empirically via sweeps and grid searches on a held-out validation set to find an optimal balance. However, these static thresholds, while effective in the evaluated benchmarks, \"may not be universally optimal\" and could reduce the model's robustness across diverse video domains, streaming qualities, or different types of user queries.\n3. The crucial ablation study for the Relevance-Triggered Active Response (RTAR) policy (Table 5), which demonstrates its superiority in timeliness-aware scoring, was conducted using a simulated evaluation protocol. This simulation was necessary because the official real-time evaluation infrastructure for the OVO-Bench benchmark and the TimeChat-Online code were unavailable at the time of experiments. While the simulation aimed to be fair by adhering to temporal constraints during response generation, the initial identification of trigger points leveraged the full video stream.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "J4AX235OHb", "reviewer": "Reviewer_UzVR", "summary": "This paper addresses the inefficiencies of existing streaming video understanding models, where many rely on a flawed \"change-is-important\" principle that confuses visual dynamics with semantic relevance, leading to computational waste and interaction errors. To solve this, the authors propose QueryStream, a lightweight, training-free framework that embeds query-awareness into video processing and response scheduling. It integrates two main contributions: 1. Query-Aware Differential Pruning, which filters visual tokens by jointly evaluating semantic relevance to the user’s query and temporal novelty. 2. Relevance-Triggered Active Response, which dynamically determines optimal response moments by monitoring two key signals.", "questions": "See weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Query-Awareness Solves Core Inefficiencies: Unlike query-agnostic methods, it avoids spurious triggers from irrelevant visual changes and captures subtle but query-relevant events, reducing computational waste and improving response accuracy.\n2. Requires no additional fine-tuning, enabling plug-and-play integration with pre-trained Video-LLMs. \n3. Clear presentation and illustrations, making the paper easy to follow.\n4. Sufficient experiments, establishing the robustness of the proposed method.", "weaknesses": "1. QDP’s performance is limited by the representational capacity of the pretrained encoder. It may struggle with fine-grained details or abstract semantic relationships, leading to missed subtle events in nuanced scenarios.\n2. The framework assumes a single, fixed user query and cannot handle dynamic conversational contexts where user intent evolves over multiple turns\n3. RTAR and QDP rely on static thresholds tuned on a validation set. These thresholds may not be optimal across all video domains, reducing the adaptability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9rRNe2es87", "reviewer": "Reviewer_ioxx", "summary": "This paper introduces QueryStream, a training-free, plug-and-play framework that improves streaming video understanding by aligning video token processing and response timing with the semantic intent of a user’s query. Traditional streaming models follow a “change-is-important” rule that triggers responses based on visual variation, leading to irrelevant activations and inefficiency. QStream aims to solve this.", "questions": "* Why do you need the semantic relevante filtering and the relevance condition? Aren't they doing the same? It seems unclear to me and not well explained the difference. I'd provide more details.\n* Could you report latency and throughput improvements to validate real-time claims?\n* How easy would it be to implement an adaptive threshold tuning approach?\n* Could you evaluate on other models besides QwenVL2.5 (eg. VideoLLaMA3, InternVL3)?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "* Shifts from the long-standing “visual-change” heuristic to a query-centric paradigm for efficient video understanding.\n* Requires only a small OpenCLIP encoder and simple logic-based gating, making it suitable for real-time applications.\n* Works with any existing Video-LLM (e.g., Qwen2.5-VL) without retraining.\n* The combination of semantic relevance and temporal novelty is intuitive, interpretable, and effective.\n* Sets new SOTA on StreamingBench and OVO-Bench, surpassing trained baselines with far fewer tokens.", "weaknesses": "* Despite strong framing, the core mechanisms (cosine similarity, thresholding, dual gating) are algorithmically simple and largely heuristic.\n* Several thresholds may need tuning for each setup; no adaptive or learned strategy is explored.\n* Tests are limited to mid-scale models (7B) and benchmarks where scalability to larger multimodal LLMs or real-time deployment remains unclear.\n* While efficiency is discussed, actual wall-clock or energy improvements are not quantified.\n* The novelty primarily lies in combining existing ideas (query relevance + temporal novelty + dual gating) rather than developing a fundamentally new principle.\n* Table 1 bolds the results for QStream but there are baselines with better performance even though using more frames.\n* Table 3 could benefit from stronger baselines.\n\nMinor comments:\n* Authors use \\citet instead of \\citep.\n* No need to repeat QDP many times.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)