StreamBridge: Turning Your Offline Video Large Language Model into a Proactive Streaming Assistant
Authors: Haibo Wang
Bo Feng
Zhengfeng Lai
Mingze Xu
Shiyu Li
Weifeng Ge
Afshin Dehghan
Meng Cao
Ping Huang
Venue: NeurIPS 2025 poster ()
Keywords: Video Large Language Model; Streaming Video Understanding
Primary area: applications
TL;DR: We present StreamBridge, a framework that transforms offline Video-LLMs into streaming models.

Abstract: We present StreamBridge, a simple yet effective framework that seamlessly transforms offline Video-LLMs into streaming-capable models. It addresses two fundamental challenges in adapting existing models into online scenarios: (1) limited capability for multi-turn real-time understanding, and (2) lack of proactive response mechanisms. Specifically, StreamBridge incorporates (1) a memory buffer combined with a round-decayed compression strategy, supporting long-context multi-turn interactions, and (2) a decoupled, lightweight activation model that can be effortlessly integrated into existing Video-LLMs, enabling continuous proactive responses. To further support StreamBridge, we construct Stream-IT, a large-scale dataset tailored for streaming video understanding, featuring interleaved video-text sequences and diverse instruction formats. Extensive experiments show that StreamBridge significantly improves the streaming understanding capabilities of offline Video-LLMs across various tasks, outperforming even proprietary models such as GPT-4o and Gemini 1.5 Pro. Simultaneously, it achieves competitive or superior performance on standard video understanding benchmarks.

StreamBridge Turning Your Offline Video Large Language Model into a Proactive Streaming Assistant
Haibo Wang 2: Bo Feng'o, Zhengfeng Lai'o, Mingze Xu's Shiyu Li
Weifeng Ge?, Afshin Dehghan' 1 Meng Cao ff Ping Huang"
'Apple Fudan University
hibwangeucdavis.cm
fbfeng2, jeff_lal mingze_xuu xu2, shiyu_li] fadehghan, mengcao, (haangping)@appe *First author; "Core contributors; Senior authors
Abstract
We present StreamBridge, simple yet effective framework that seamlessly trans- forms offline Video-LLMs into streaming-capile models. It addresses two funda- mental challenges in adapting existing models into online scenarios: (1) limited capability for multi-turn real-time understanding, and (2) lack of proactive re- sponse mechanisms. Specifically, StreamBridge incorporates (1) memory buffer combined with round-decayed compression strategy, supporting long-context multi-turn interactions, and (2) decoupled, lightweight activation model that can effortlessly integrated into existing Video-LLMs, enabling continuous proactive responses. To further support StreamBridge, construct Stream-IT, large-scale dataset tailored for streaming video understanding, featuring interleaved video- text sequences and diverse instruction formats. Extensive experiments show that StreamBridge significantly improves the streaming understanding capabilities offline Video-LL.Ms across various tasks, outperforming even proprietary models such as GPT-4o and Gemini 1.5 Pro. Simultaneously. it achieves competitive or superior performance on standard video understanding benchmarks.
Introduction
Video Large Language Models (Video-LLMs) [1; 2; 3; 5] typically process entire pre-recorded videos at once. However, emerging applications, such as robotics [6; 7] and autonomous driving [8; 9], require causal perception and interpretation of visual information online. This fundamental mismatch highlights a critical limitation of current Video-LLMs, as they are not inherently equipped to operate in streaming scenarios where timely understanding and responsiveness are paramount.
Figure highlights two representative patterns in streaming video understanding, which also cor- respond to the key challenges adapting Video-LLMs from offline to streaming scenarios: (I) multi-turn real-time understanding and (2) proactive response generation. The first pattern involves multi-turn interactions, where the assistant receives user queries different timestamps. In each turn, while keeping accumulated visual and conversational context as historical information, the model should focus on the most recent video segment. The second pattern emphasizes more human- like, proactive behaviors. Rather than passively waiting for user prompts, the model actively monitors the visual stream and generates timely outputs based on unfolding content. For instance, Figure (bottom), the assistant provides step-by-step guidance as the drawing progresses without being explicitly asked, simulating continuous support in dynamic environments.
Work done during Haibo's internship Apple.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

What going an?
Where is this?
What is written on the sign?
A helicopter is flying mountain.
You are an airplane with rows seats.
"Change the direction of escolators and travelators'.
How drow this picture?
USER
Incoming Frames
Draw a house
BER
Drow a sun and clouds.
Color the sun blue and clouds green.
Figure 1: Illustration of streaming scenarios. Top: Multi-turn interactions. User issues queries at different timestamps, with each turn involving new video segment along with accumulated visual and text history. Bottom: Proactive responses. The assistant actively delivers timely feedback or guidance based on the incoming visual stream, without requiring explicit user prompts.
To bridge the gap between offline and streaming video understanding, introduce StreamBridge. a simple yet effective framework that seamlessly transforms pre-trained offline Video-LLMs into streaming-capate
models. In contrast to prior efforts [10; 11; 12; 13), which train streaming models from scratch but fall behind on offline video tasks, StreamBridge leverages the strong generaliza- tion capabilities of existing Video-LLMs without requiring full retraining. This approach allows developers to directly benefit from the rich world knowledge and linguistic fluency of large-scale pre-trained models, while incurring only minimal additional computational cost and data require- ments. Concretely, StreamBridge introduces memory buffer to manage incoming video frames, coupled with a round-decyyed compression strategy that merges earlier frame tokens while preserving recent ones, enabling the model to support long- long-context, multi-modal, and multi-turn interactions in streaming scenarios. For proactive capabilities, instead of modifying the base model architecture [14] or introducing streaming-speciffe objectives [12], both of which can lead to optimization conflicts and issues like probability correction [10], Stream adopts a modular design, by decoupling the proactive capability from the main Video-LLM, via compact activation model. This souu-and-play component operates in parallel with the main Video-LLM, enabling proactive behavior a flexible and non-intrusive manner while fully preserving the main Video-LLM's language fluency and general video understanding capabilities.
To further support StreamBridge, we construct Stream-IT, large-scale dataset tailored for streaming scenarios. Stream-I7 captures diverse real-time questions and proactive responses embedded within multi-turn video interactions, featuring interleaved video-text sequences. While existing datasets primarily focus on single-turn question answering [15; 16] or short-form video captioning [17: 18; 19], Stream-IT fills critical gap by enabling temporally extended, interactive video understanding. It is constructed by concatenating semantically related short clips from large-scale video-caption corpora, followed by the generation of multi-turn QA sequences that simulate realistic, time-sensitive user interactions. Moreover, Stream-IT incorporates broad spectrum task formats sourced from public datasets, thereby boosting task diversity and promoting model generalization in streaming settings.
By integrating our StreamBridge framework and fine-tume tne-ttunng on Stream-IT, we successfully convert several leading offline Video-LLMs, including LLaVA-OV [3], Oryx-1.5 [1], and Qwen2-VL. [2], into sreaming-capablle assistants. Extensive experiments demonstrate that our models achieve state-of- the-art performance streaming benchmarks such as OVO-Bench [20] and Streaming- -ench [21], outperforming even proprietary models like GPT-4o [22] and Gemini 1.5 Pro [23], while retaining or exceeding performance on conventional offfine video understanding tasks [24; 25; 26; 27; 28; 29].

---

2
Related Work
Video Large Language Models. With the rapid advancement of Multimodal Large Language Models (MLLMs) [30; 3; 31; 32; 2], Video-LLMs [33; 34; 35; 36; 15; 37; 38] have gained increasing attention for general video understanding. Typically, these models comprise a visual encoder [39; 40; for extracting frame-level representations, a modality projector (e.g., MLP [41] and Q-former [30]) to map visual features into the language space, and an LLM [42; 43] to generate contextual responses. While achieving strong results on standard video benchmarks [25; 29; 27], these models are inherently designed for static, offline settings where the entire video is pre-recorded and fully accessible at inference time. As result, they struggle in streaming environments, where video frames arrive sequentially and require real-time, temporally coherent, or even proactive responses. Our work aims to bridge this gap by augmenting offline Video-LLMs with streaming capabilities.
Streaming Video Understanding. Typical tasks in streaming video understanding. such as action recognition [44; 45; 46; 47] and anticipation [48; 491, causally process video inputs using only past and current observations. Recent efforts [50; 12; 13; 51] focus on building Video-LLMs capable of real-time conversation, generating timely responses throughout live video stream. VideoLLM- Online [10] and Flash VStream [1 introduce specialized online objectives and memory architectures to handle sequential inputs. MMDuet [14] and ViSpeak [52] add dedicated heads to facilitate proactive response generation. To benchmark streaming video capabilities, several evaluation suites have been proposed, including StreamingBench [21]. StreamBench [53]. SVBench [54], OmniMMI [55], and OVO-Bench [20]. In contrast to previous approaches that retrain models or tightly couple proactive mechanisms within the backbone, our work leverages the strong generalization abilities of pre-trained offline Video-LLMs [1; 3; 2]. We propose an efficient adaptation framework, combined with a dedicated fine-tuning dataset, to endow these models with streaming capabilities. Furthermore, observing that embedding the activation function into the main model often lead to optimization conflicts and performance degradation [10; 14), we advocate modular, decoupled design. Our method introduces a compact, plug-and-play activation model that enables proactive behaviors efficiently and non-intrusively. We also provide additional discussions how our work relates the ReKV [56], VideoStreaming [57]. and StreamChat [53] in the Appendix F.
3 Methodology
3.1 Preliminary Analysis
Streaming video understanding involves interleaved video-text inputs. From an input perspective, streaming scenarios can broadly categorized into two representative formats:
Multi-turn dialogue with interleaved video-text. In this setting, the input sequence is in the form of '<V>> <Q1> <Ai>, <V>> <Q²> <A2>, ...", where <V/>, <Q,>, and <A,> denote the video clip, user query, and assistant answer in the i-th round. Crucially, there is no delay between <Qi> and <A,>, reflecting the need for immediate responses. This format closely resembles the live interaction in dynamic environments, as shown Figure (Top).
Proactive output. The assistant answers after watching an incoming video stream, often without an explicit user query at the response time. The input can be structured as '<Q> <Vi> <A,> <V,> <Ap>.... where <Q> represents an initial prompt (e.g.. "Guide me through the task"), and the model must proactively determine when and how to respond based on the incoming video contents. This scenario requires the ability to continuously monitor evolving context and trigger responses at appropriate moments. Figure 1 (Bottom) is an example of proactive responses.
Recent benchmarks such as OVO-Bench [20] and grrcaming-Bendhe [21] attempt to evaluate these capabilities by constructing multi-turn interleaved video-text dialogues. However, due the limited input length and the lack of streaming support in current Video-LLM, these benchmarks necessarily simplify the problem. Specifically, they segment a complete long video into multiple isolated clips aligned with each query timestamp. For query <Q,> at time t,, the visual input is restricted to the uniformly sampled frames under segment Vio.... and prior dialogue history is completely discarded. As result, the multi-turn streaming scenario is reduced to a series of independent, single-turn offline tasks. To address these limitations, we propose StreamBridge, a general framework designed to introduce the actual streaming setup to existing offline Video-LLMs.
3

---

Observed Frames
Incoming Frames
86
query Q posed at #3
(RI, #2)
(84,
Frame Encoder
Activation
Encoder
Memory Buffer
True
ASSISTANT:
D
at #6
Gound-Decayee Compression (optional)
Large Language Model
Figure 2: Overview of StreamBridge. (Q: Incoming frames are encoded and stored into the memory buffer one by one, @: query Q is posed. @: The activation model monitors incoming frames and returns a binary signal D, indicating whether LLM should start answering. means concatenation.
3.2 StreamBridge
As shown in Figure 2 and Algorithm in addition to the frame encoder I(-) and the large language model LCM(.), StreamBridge proposes three key components to enable streaming capabilities: (1) memory buffer responsible for storing and retrieving frame tokens over time, (2) nound-decaye compression strategy COM(() that efficiently prunes redundant tokens from earlier rounds while preserving the most recent context, and (3) compact activation model ACT(.) that enables proactive responses by making frame-level decisions on when to generate outputs.
3.2.1 Memory Buffer
In streaming scenarios where video frames arrive sequentially, adopt memory buffer MB to store both visual and textual embeddings. As illustrated Figure 2, each incoming frame is independently encoded and appended to the buffer alongside any associated query embeddings. Conceptually, MB operates under a producer-consumer paradigm: the encoder I(.) functions as the producer, continuously generating frame-level features, while the language model LCM(() serves as the consumer, retrieving the accumulated embeddings to generate a response upon receiving user query. Formally, as detailed in Algorithm 1, at each time step t, the incoming frame F; first processed by I(-), and the resulting embeddings are stored in the memory buffer MB (Algorithm 1, line 4). Upon the arrival of a user query and a positive activation decision D, the buffer content, including both visual and textual embeddings, is flattened into a single sequence of input embeddings, which is then fed into (.M(() for response generation (Algorithm 1, line 13-16). Once response R is produced, it is also appended to the memory buffer (Algorithm 1, line 17). enabling the model preserve temporal continuity and maintain a complete history of multi-tum video-text interactions.
3.2.2 Round-Decayed Compression
Online scenarios often involve long, even infinite video streaming, which can lead to significant memory usage and inference latency. Therefore, we propose round-decayed token compression strategy tailored for multi-turn streaming settings. Specifically, we pre-define maximum allowable embedding length MaxLen for the model input. Before each response generation, the model checks whether the current input embedding exceeds MaxLen. If so, we apply round-decayed token merging strategy: starting from the earliest dialogue rounds, visual tokens are progressively merged frame-by name-by-fiie baree--b-fraeee,.. until the total length falls below MaxLen. The merging is implemented via average

---

Algorithm 1: StreamBridge Framework
Inputs: incoming frames [F1, P2...., F
2 Initializations: I(-), LCM(C.MMM.)).. ACT, MB = [-]. MaxLen, to=None;
3 while P: do
4
MB -((F):
// store the frame feature ((F)) into the Memory Buffer
if Q al limestamp then
.MB Q
te tt;
t is the timestamp when @ is posed
if t is not None then - (gtt-,,
|| 'D denotes whether response or not at timestamp
10
else
11
L D + False;
// not response if there is no
D then 12
I/ "D is true at timestamp t, and should return response R Input.Emeeds +- Flatten(MMB) if (Tee((nnpuEEmmeeds)) MaxLen then
13
15
Input Embeds +- (IapattEmbeeedd) R < LCMM(npu(Embeud)
ff compress redundant visual tokens
16
// return response R // update MB
17
MB R;
18
1 1+=1;
I/ receive subsequent frames
pooling [58] over adjacent frame tokens. This strategy ensures that the most recent visual context is retained with minimal distortion, thus maintaining the precision of real-time responses while not fully discarding historical visual contexts, At the same time, it significantly improves memory efficiency and reduces inference overhead as Figure 4. This process is encapsulated the compression function COM(()) in Algorithm 1 (line 15). The detailed pseudo codes be found in Appendix I.
3.2.3 A Plug-and-play Activation Model
To enable proactive responses in streaming Video-LLMs, decouple the activation function from the main Video-LLM. Un- like prior methods that tightly integrate activation mechanisms into the LLM [10; 12; 14; 52], our framework avoids potential interference with the language modeling capacity of the main Video-LLM. Specifically, we propose a parallel pipeline, where compact external MLLM (e.g., LLaVA-OV-0.5B [3]) is used as an independent activation model, denoted ACT(-)) As shown in Algorithm (line 9), upon receiving each new frame, the framework simultaneously forwards the current frame (along with the user query and optionally previous frames) to ACT((- de- termine whether response should be generated. If the activation signal D is positive, the buffered embeddings are sent the LLM for decoding. This design ensures high flexibility and compatibility. Furthermore, in real-time deployment, the ACT(-), the frame encoder I(.). and the main LLM()) can run concurrently in parallel threads, enabling more efficient inference.
A
.40
Activatton Model [LLaVA-Ov
<AC7 token
Flrst 2-P% 2-P% frames, keep stient
Lost PX frames, activated
Keep sifent
Activated
Figure 3: An overview of the proposed activation model. We label the last P% of frames of each video clip to be true during training.
To train the activation model (illustrated in Figure 3), we modify the architecture by replacing the standard language modeling head with score head for binary classification, and introduce learnable activation token <ACT> which is appended to the visual embeddings of each frame. After processing through the final layer, we extract the latest frame's 's activation token and feed its hidden representation into the score head to predict whether the model should respond at that time. During inference, only when predicted score greater than the activation threshold a, the main Video-LLLM can be triggered to give response. Since ACT(-) performs only binary classification (i.e., to respond or not), we aggressively pool its visual tokens for efficiency. The input sequence to the model follows the format: '<Q>< <V,> <Ap> <V₂> <Ag> ...,, where the question Q is prepended to the sequence,
5

---

and visual frames and corresponding responses are interleaved. This design enables the model to learn temporal dependencies and identify appropriate response moments throughout the video stream.
For training data, we collect a diverse set of temporally annotated video datasets across multiple tasks, including dense video captioning [59; 60), sequential step recognition [61; 62), grounded video question answering [63; 64; 65], temporal video grounding [66], and temporal action detection [67; 68]. For each task, we design specific prompt templates and randomly select among them as Q during training (details in Appendix A). To supervise activation timing, we insert the response <A,> at the end of its corresponding annotated timestamp (during inference, the response <A,> is generated by the larger main Video-LLM). Besides, only the last P% of frames of each video segment Vi are labeled as positive (i.e., response- -oothhy,,,,, while earlier frames are treated as negatives. P is dynamically sampled between 0% and 50% for each training instance, simulating a variety of activation patterns and enhancing the model's robustness to temporal variations.
4 Stream-IT Dataset
As analyzed in Section 3. streaming scenarios are primarily characterized by multi-turn real-time understanding and proactive responses. However, existing datasets and video sources fall short of fully supporting these requirements [69; 70J. To fill this gap and further enhance the streaming interaction capability of StreamBridge, we introduce Stream-IT-4 video-text dataset specifically designed for streaming instruction tuning with an interleaved multi-turn dialogue format.
Datasets for Proactive Understanding, We collect a set of public datasets enriched with timestamp annotations, spanning a wide range of tasks including: (i) Dense Video Captioning [59; 60; 71]; (ti) Sequential Step Recognition [61; 62; 72]; (iii) Grounded VideoQA [63; 73; 74; 75]. All datasets are reformatted into a proactive-style interleaved format: '<Q> <V1> <A1>, <V>><A²>; ...', where Q may be an open-ended query (e.g., "Who is the man going to find?") or goal- orientd instruction (e.g., "Show me all the steps for "ooking."). Unlike traditional single-turn datasets where a question is immediately followed by an answer [69; 70J, our structure introduces temporal delay between <Q> and <A> through the inserted video segments <V>, simulating proactive response scenarios.
Streaming PtreamnngA--12 QA-120K: Multi-Turn, Long-Form QA Construction. To further support long-context, multi-turn real-time understanding, we introduce StreaningQA-120K, a large-scale synthetic dataset constructed by composing long-form videos from existing short video clips. Labeling ong-duration videos with dense multi-turn QA pairs prohibitively expensive. To address this, we leverage short clips from large-scale rideo-caption datasets, including Web Vid-Vid11e 10M [19], Panda-70M [18], and Intern 10M [17]. We filter approximately 1.28 million clips using semantic similarity between video and caption to ensure alignment, with each clip being around 12 seconds long. With these short clips, to form coherent long-form videos, we then iteratively compute pairwise similarity between videos and concatenate highly similar clips. Each constructed video contains roughly 10 clips, with an average length exceeding 150 seconds. Captions for each clip are preserved with natural timestamps. Using these captions, employ GPT-4o [22] to generate diverse question-anss answer pairs spanning 8 task types. By default, each QA pair <Q <A is inserted immediately after its corresponding clip <Vi>, forming sequences like '<V>> <Q1> <A1>, <V2> <Q2> <A²>, ..... Here, we introduce two augmentation strategies during sequence construction:
Random QA Drop: randomly drops some QA pairs by transforming '<V,> <Q <A,>' to '<V/>' with a probability of Parop. to prevent overfitting to fixed QA positions and enhance the model's robustness in temporal variations. We set Parop to be 0.55 by default.
with probability Pahin, transforms sequences from '<V,> <Q,> <A,''' to '<Q,> <Vi> <A,>' where the visual content VÚ serves as the temporal delay between question and response for proactive scenarios. Paitt is set to 0.1 here.
QA Interval Shift:
Together, these strategies ensure that the Stream-IT dataset supports rich and varied streaming interaction formats, enabling both multi-turn real-time dialogue and proactive response capabilities across a wide range of tasks and timescales. More details on data statisties, concatenation strategy of StreamingQA- 120K, and prompts for QA generation are provided in Appendix B.

---

Method
Rff Frames
OVC-Bench Real-Time
Streuming Bench Real-Time
OCR
ACR
ATR
STU FPD
OIR
AVG.
CR
ATP
EU
PR
SU
AVG
Humann
Humun
93.96 92.57 94.83 92.70 91.09 94.02 93.20 $9.44 92.00 93.60 91:47 95.65 92.52 88.00 88.75 89.74
91.30
91.46
Proprictary Model (Omiine) Single- Turn Evaluntion
Gemini 1.5 pro [23]
IPPS
98
66.97 79.31 58.43 63.37 61 96 69.52 72.02 80.47 83.54 79.67 80.00 84.74
77.78
64,25
71.95
48.70
75,69
GPT-40 (22)
699
64,22
71.55 51.12 70.3 59.78 64.46 77.11 80.47 83,91 76.47 70.19
83,80
66.67
62.19
69.12
49.22
73.288
Open-SSuuree Models (Oftiin). Single-Turm Evaluation
Qwen2-VI--288 [2]
65.77
69,83 51.09
54.35
61.92
LLdVA-Viddd-788 [15)
09.13
69.83 49.44
59.78
LLaYA-OV-7B [3]
64332
06.44
57.80
73.28 53.37 71,29
61.96 64.02
50.38
74.22
76.03
80.72
72.67
71.65
67.59
65.45
65.72
45.08
71.12
Qwen2 VL-B2- (2)
64/1 FPS
00.40
50,46
56.03 47.19 66.34
55.43 55.95
75.20
82.81
73.19
77.45
68.52
71.03
72:22
61.19
61.77
16.11
69.04
Iniem/VL-V2-8B [76]
64/16
67..11
60.55
63.79 46.07 68.32
36.52
60.39
65.12
60.94
69.10 77.12
67.70
62.93
59
26
53.25
54.96
56.48
63.72
Opea-Source Models (Streaming),
Single Turn
Evaluation
Flash-VSiramms [1]]
28.45 33.71 25.74
43.57
27.53
58
70
23.23
Videol MDDallnn-- [10]
FPS
8.05
23,85 12.07
30.07
40.06 34.49
45.96
34.16
42.49
99
Dispide [13]
FPS
57.72
49.54 62.07 44.94 61.39 51.63 54.55
74.92
75.53 74.10 73.08 74.44 59.92
76.
62.91
62.16
16
45.80
67.63
Modells
Straofrridge (Oflline
Streamings Mulli- Turn Evalustion
0.57-115--BB [1]
HPS
(0.40
52.29 69.83 50.00 65.35 57.61
59.25 78.47
77.17 83.86 50 71.07
66.98
79.03
61.38
06.29
40.93
70.59
+ Stroow-F7
1PP
70.69 50.56 74.25 71.74 71.17
77.95 97.98
77.99 81.31 76.85
69.92
71.96
35.23
74.79
LLVA--VV-7 []]
58.
82 44. 76.25 41
76.84
77.17
75.25 64.15
75.00
+ Sircom777
FPS
77.06
70.69
73.27
69.93
82.29
72
92.09
80 86
71.07
74.46
75.00
62:20
28,50
70.92
Qwen2-VL:7Bl [2]
IPS
64.22
(00.88
78.74
77.78
63.41
72.00
+ Stroow-fll
1FP
$4.56 71.56
74.14 49.44 75.25 72.83
71.30 84.74 82.68 88.92
89.73 77.35
85.36 84.26
69.92
71.67 35.75
77.04
Table 1: Results on real-time understanding tasks on OVO-Bench and Streaming- Bench. Stremming-Benc.
memas
models under StreamBridge framework, and + Stream-I7 means finetuned on Stream-IT.
Experiments
5.1 Settings
Models and Datasets. We evaluate StreamBridge framework using three mainstream offline Video- LLMs to show its generalizability: LLaVA-OV-7B [3], Qwen2-VL.-7B [2], and Oryx-1.5-7B [1]. To preserve their general video understanding capabilities during streaming adaptation, we supplement Stream-IT with approximately 600K samples from the VL [15], VCG-Plus [35] and ShareGPT4Video [16]. For the activation model, fine-tune LLaVA-OV-0.5 [3] our collected activation datasets as described in See. 3.2.3. The videos are sampled at FPS. In Section 5.3, we use Qwen2-VL-7B as the default model unless otherwise specified. See the Appendix C for more details.
Benchmarks. For multi-turn real-time understanding, we choose OVO-Bench [20] and Streaming- Bench [21]. We primarily focus on their real-time tasks. For general video understand- ing, we evaluate our method across 7 video benchmarks, including 3 short-video benchmarks: MVBench [24], Perception Test [26], TempCompass [77], and 4 long-video benchmarks: EgoSchema [28], Long VideoBench [29], MLVU [27], and VideoMME [25]. To evaluate the proactive capability of our method, we use subtasks from ET-Bench [66] following previous works. See Appendix D for more benchmark details and evaluation metrics.
5.2 Main Results
Multi-Turn Real-Time Understanding. As discussed in Section 3.1, the results reported in the original paper [20; 21] in Table 1, marked as " (Offline), Single-Turn Evaluation ", do not reflect performance in real streaming scenarios. They segment a complete video into sev- eral individual clips, discarding historical visual and dialogue contexts, thereby limiting the upper bound of the performance. In contrast, with the StreamBridge framework, denoted as (Offline + Streaming), Multi-Tunn Evaluation " these offline models are equipped to process streaming videos at I FPS in multi-turn manner, while maintaining input length and historical contexts within a predefined maximum token budget.
Specifically, we observe that Qwen2- fLf demonstrates notable improvements in the streaming setting, with its average score on OVO- increasing from 55.98 to 63.35, and on Streaming -reaching-Bench 69.04 to 72.01. Conversely. VA-OVt shows slight performance drop when transitioning to the streaming setup: from 64.02 to 61.66 61.64 on OVO-Bench, and from 71. to 68.39 on Streaming-Bench We attribute these differences to the nature of their pretraining data, where Qwen2- VL. benefits from richer interleaved multimodal training (e.g., image/video-texx sequences), which makes it more adept at understanding interleaved video-text inputs and utilizing extended context effectively. On
Bench from
7

---

MVBench
PercetiinTest
TempCompass
Egošchema
Benc
MLVU M-Avg
VideoMME (wlo subs)
Model
Arg
Val
MC
Test
Val
Arg
Avg. Duration
16s
23s
12x
180s
473s
651s
1010s
Proprictory Models
Gemini 1.5 (23)
60.5
67.1
71.2
75.0
GPT-4o (22)
64.6
70.9
72.2
66.7
64.6
71.9
Open-Source Models
KB [78]
62.5
56.0
LosgVIL.A-7B177 [79]
67.7
57.5
LosgVU-7B [80]
66.9
67.5
65.4
60.6
Apolle-7B [1]
67.3
61.9
58.5
70.9
61.3
NVILA-SB [81]
68.1
69.7
SF-LLaYA-L.5.7BB5 [5]
69.5
63.8
62.5
71.5
63.9
Intem/VL2.5588 [82]
72.0
68.2
65.3
51.5
60.0
68.9
64,2
SigeoChat-Flash-T [83]
74.0
76.2
64.7
74.7
65.3
VideoLLaMA3-7B [37)
69.7
72.8
68.1
66.2
Oryx-1.5-78 [1]
67.6
70.0
58.8
56.3
67.5
58.8
01yx-1,5-78 (ours)
(11.0)
69.0 (110.2)
61.2
$8.9((12.6)
(14.0)
(16.7)
LLAVA-OV.7
56.7
64.8
58.2
LLaVA-Ov-78 (ours)
(1.77)
63.9 (16.8)
67.7 (12.9)
67.0
(12.0)
(13.5)
(13.0)
Owen2-VL VL-7B [2]
67.0
62.3
67.9
66.7
63.3
Qwen2-VL--7 VL-7B (ours)
64.4 (12.6)
69.9 (17.6)
71.1 (13.2)
66.9 (10.2)
59.]
69.6
(11.)
Table 2: Results on general video understanding benchmarks. Bridge framework and fine-tuned on Stream-IT.
means models under Stream-
the other hand, LLaVA-OV is trained with fewer interleaved sequences, making it less suited for multi-turn streaming inputs. When faced with long, interleaved video-text sequences in streaming scenarios, its performance tends to degrade as more historical frames accumulate. Notably, fine-tuning these models the proposed Stream-I7' leads to substantial improvements in multi-turn real-time understanding. For instance, Oryx-1.5 achieves a performance gain of +1 +11.92 on DVO-Bench and +4.2 on Brraming-Benchh Furthermore, Qwen2-VLA reaches an average score of 71.30 on OVO-Bench and 77.04 on Streaming-
outperforming proprietary models such as GPT-4o and Gemini 1.5 Pro. These results validate the effectiveness of both our StreamBridge framework and the Stream-IT dataset in enhancing multi-turn real-time understanding in streaming scenarios.
General Video Understanding, While method is designed for online scenarios, we also verify that it does not downgrade the base model's performance on standard offline video tasks. As shown in Table 2, models equipped with the StreamBridge framework and fine-tuned on Stream-I7' (denoted with 9) exhibit consistent improvements or maintain comparable performance relative to their original versions. For instance, Oryx-1.5-7B achieves 65.5 the challenging VideoMME with increase of 6.7, while LLaVA-OV-7B= outperforms its base model across nearly all benchmarks, except Long Bench Likewise, Qwen2-VL-7Bf achieves competitive results on MVBench, while surpassing its original counterpart on other benchmarks. These outcomes demonstrate that our streaming adaptation enables models to retain, or even exceed their original performance in general video understanding tasks, demonstrating the generality and non-degradability of our method.
Online Activation. We evaluate the proactive capability of our framework in Table 3. Notably, in all tasks, the question is presented at the be- ginning of the video, and the model must autonomously decide when to respond. On the ET-Bench, Stream- Bridge outperforms both VideoLL Online [10] and Dispider [13] across generation- based tasks such as DVC (Dense Video Captioning) and SLC (Step Localization and Captioning). achieving higher similarity scores of DVCssm and SLC by producing more accurate and context- aware descriptions in streaming scenarios. We attribute this to the decoupled nature of the activation model, which enables the main Video-LLM to focus solely on video understanding and language generation, free from the burden of proactive decision-making. We also observe that Qwen2-VI. achieves better text similarity scores than other Video-LLLMs, consistent with its strong real-time understanding performance presented in Table 1.
Method
ET-Bcadh
Drames
TVGri
TALei
DVCrr
DVC
SLCr:
SLCsin
VideoLLM Online Dispider
21PS 10PS
13.2
9.1
24.0
13.4
10.1
36.1
27.3
33.8
12.4
Models under StreamBrrigee Framewark
Oryx tours! LLaVA OV (ouns'
24.3
37.8
24.0
EPS
3433
24.3
37.9
26.2
22.8
16.2
Qwen2 VL. (ouzs)
JES
34.3
24.3
38.3
25.1
22.6
17.1
Table 3: Results on ET-Bench.
denotes models un- der StreamBridge framework and fine-tuned Stream- IT. TVG and TAL scores are identical across Stream- Bridge models due to sharing the same activation model.
8

---

5.3 In-Depth Analysis
Compression
0vO
Streaming
ET
LLANA 175k (803 mal))
0VD
Sueamine
MVBrrah
VileoMME
Avg.
Avg.
DVC
who SQA-
of SQA-110
Aag.
Ang.
Arg.
Osaall
Truneation
68.88
72.79
22-1
16.7
71.36
50.0
Round-Unifom
69.91
74.18
23.8
15.9
72.42
63.6
Round-Deeayed
71,30
77.04
25.1
17.1
71.30
77.04
64.4
64.4
Table 4: Ablation studies on different com- pression strategies.
Table 5: Ablation studies on Stream-IT. SQA-120K de- notes the generated StreamingQA-120K,
17500
15000
No Compression
Compression 32k
00M
35-
DVC
Lanonay Inc
12500
Compression 16k
30-
20000
Compression 8k
22-
7500
5000
2500
13
1024
Actistion Trreshels
Figure 4: Inference Latency (y-axis) vs. Frame Number (x-axis).
Figure 5: Ablation studies on the activation threshold a.
Round-Decayed Compression. We set the maximum input length MaxLen 16384, and denote the current length of the input embeddings as L. To assess the effectiveness of our round-decayed compression strategy, we compare it against two alternative methods: (1) Truncation: If > MaxLen, only keep the last L tokens in the input sequence. (2) reducing the number of visual tokens with a fixed ratio L--MxxLr per round, to keep the total length within MaxLen. The results are reported in Table 4. We observe that Truncation yields the worst performance, as it indiscriminately removes both visual and textual history tokens, severely weakening multi-tum reasoning. The Round- Uniform strategy performs slightly better, but still underperforms our method. It compresses the latest visual tokens, which are critical for real-time comprehension, thus leading to degraded performance, particularly on OVO-Bench and Streaming-Bent
We treat each round equally by
Inference Latency. We also evaluate the inference latency on a single A100-80G GPU with different MaxLen (8k, 16k, 32k), as shown in Figure 4. Our results show that our compression method maintains near-constant latency when the number of input tokens exceeds MaxLen, whereas models without compression suffer from sharply increasing delays and eventually trigger out-of- out-of-mennt memory (OOM) errors with 2048 frames. This highlights the necessity of effective compression to balance inference efficiency and memory usage in streaming settings.
Impact of Stream-IT. Table 5 ablates effectiveness of Stream-IT. Training on LLaVA-1788 alone causes marked drop on both OVO-Bench and Streaming-Bent as it lacks interlcaved vidco-text samples necessary for multi-turn interactions. Conversely, using only Stream-IT without LLaVA- 178K leads to declines in general video understanding (MVBench, VideoMME), indicating that the larger offline data corpus still contributes valuable world knowledge. Finally, removing the synthetic StreamingQA- 120K benchmarks, underscoring the crucial role of StreamingQA 120K in boosting both streaming and offline video understanding capabilities.
120 subset from Stream-IT degrades performance across both streaming and offline
Impact of MaxLen. To better understand the impact of MaxLen, we conducted ablation studies using the Qwen2- VL-StreamBridge model with I FPS sampling, varying MaxLen from 4k to 32k. From Table 6, we observe the
OVO-Bench
VideoMME
MaxLen
(Real-Time) Avg.
Avg.
4k
70.49
61.7
following: (1) For streaming tasks (c.g., OVO-Bench Real-
70.89
63.6
16k
71.30
64.4
Time): Model performance remains relatively stable across varying MaxLen values, ranging from 70.49% to 71.30%; Accuracy peaks at 16k and slightly declines at 32k, sug- gesting that further increasing the memory budget yields diminishing returns. This supports our design assumption: in streaming scenarios, models primarily rely on recent context, and older frames can be compressed without significant performance loss. (2) For offline tasks (c.g., VidcoMME): Accuracy improves consistently MaxLen increases, from
32k
71.16
64.7
Table 6: Ablation studies on MaxLen

---

61.7% at 4k to 64.7% at 32k. This means that offline tasks benefit more from retaining the full tem- poral context and more uncompressed video tokens, especially for long videos that require detailed long-range understanding. StreamBridge can flexibly balance efficiency and performance across both streaming and offline settings by adjusting the memory budget accordingly, and we set MaxTen 16k to strike good balance between them across most tasks.
Activation Threshold. The compact activation model makes a per-frame decision to trigger responses, with frequency determined by the activation threshold a (see score head in Figure 3), We adopt a default a of 0.35, following common practice [52; 14]. Figure 5 illustrates the impact of varying this threshold: both excessively low and high values of a decrease Fl scores (DVCF1 and SLCF1 on ET-Bench). A low threshold triggers overly frequent responses, while a high threshold suppresses them excessively, both of which hurt performance. Nonetheless, this syper-parameter allows users to flexibly control response frequency through o, adapting to different practical scenarios.
Conclusion
We present StreamBridge, a novel framework that transforms offline Video-LLMs into streaming- capable models. StreamBridge introduces memory buffer paired with round-decayed compression strategy, and decouples the activation function with compact activation model. We also construct Stream-IT,, a dataset with interleaved video-text sequences further support StreamBridge. Extensive experiments on diverse benchmarks demonstrate that our method not only preserves the strengths of the base models but also equips them with the ability to make timely, proactive responses across multi-turn, long-context streaming scenarios. We believe StreamBridge offers general solution for bridging the gap between offline Video-LLMs and real-world, interactive streaming applications.
Acknowledgment. We thank Yu Liu, Haiming Gang, and Mingfei Gao for their kind help.
References
[1] Zuyan Liu, Yuhao Dong. Ziwei Liu, Winston Hu, Jiwen Lu, and Yongming Rao. Oryx milm: On-deman spatial-temporal understanding at arbitrary resolution. arX9v:2409. /2961, 2024. 1, 2, 7, 8, 27
[2] Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: vl: Enhancing vision- language model's perception of the world at any resolution. arXiv:2409.10 121912 2024. 1, 2, 3, 27
[3] Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. 2xXi::2488.03326, 2024. 1, 2, 3, 5, 7, 8, 27
[4] Orr Zohar, Xiaohan Wang, Yann Dubois, Nikhil Mehta, Tong Xiao, Philippe Hansen-Estruch, Licheng Yu, Xiaofang Wang, Felix Juefei-Xu, Ning Zhang, et al. Apollo: An exploration of video understanding in large multimodal models. arXiv:2412. 10360, 2024. 1,8
[5] Mingze Xu, Mingfei Gao, Shiyu Li, Jiasen Ln, Zhe Gan, Zhengfeng Lai, Meng Cao, Kai Kang, Yinfei Yang, and Afshin Dehghan. Slowfast-llava-11 1.: A family of token-efficient video large language models for long-form video understanding. arXiv:2503. 18943, 2025. 1,8
[6] Moo Jin Kim. Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. Openvla: An open-source lasion-language- action model. (ixiv:2406002444 2024.
[7] Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research, 2023.
[8] Hao Shao, Yuxuan Hu, Letian Wang. Guanglu Song, Steven L Waslander, Yu Liu, and Hongsheng Li. Lmdrive: Closed-loop end-to-end driving with large language models. In CVPR, 2024.
[9] Yihan Hu, Jiazhi Yang, Li Chen, Keyu Li, Chonghao Sima, Xizhou Zhu, Siqi Chai, Senyao Du, Tianwei Lin, Wenhai Wang, et al. Planning-oricnted autonomous driving. In CVPR, 2023. 1
[10] Joya Chen, Zhaoyang Lv, Shiwei Wu, Kevin Qinghong Lin, Chenan Song, Difei Gao, Jia-Wei Liu, Ziteng Gao, Dongxing Mao, and Mike Zheng Shou. Videollm-online online: Online video large language model for streaming video. In CVPR, 2024. 3. 5,
10

---

[11] Haoji Zhang. Yiqin Wang, Yansong Tang, Yong Liu, Jiashi Feng, Jifeng Dai, and Xiaojie Jin. Flash-vstream: Memory-based real-time understanding for long video streams. arXiv:2406. 08088, 2024. 2, 3, 2,
[12] Wei Li, Bing Hu, Rui Shao, Leyang Shen, and Liqiang Nie. Lion-fs: Fast & slow video-language thinker as online video assistant. wrXi:2503300663, 2025. 3,5
[13] Rui Qian, Shuangrui Ding, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Dahua Lin, and Jiaqi Wang. Dispider: Enabling video iiss with active real-time interaction via disentangled perception, decision, and reaction. arXiv:250J.00 0032/8, 2025. 2,3, 7, 7, 26,27 27
[14] Yueqian Wang, Xiaojun Meng, Yuxuan Wang, Jianxin Liang, Jiansheng Wei, Huishuai Zhang, and Dongyan Zhao. Videollm knows when speak: Enhancing time-sensitive video comprehension with video-text duet interaction format. arXiv:2411.1799, 2024. 2, 3, 5, 10
[15] Yuanhan Zhang, Jinming Wu, Wei Li. Bo Li, Zejun Ma. Ziwei Liu, and Chunyuan Li. Video instruction tuning with synthetic data. arXiv:2410.02113, 2024. 2, 7.
[16] Lin Chen, Xilin Wei, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang. Zehui Chen, Haodong Duan, Zhenyu Tang, Li Yuan, et SharegptAvideo: Improving video understanding and generation with better captions. NeurIPS, 2024. 2,7 7
[17] Yi Wang, Yinan He, Yizhuo Li, Kunchang Li, Jiashuo Yu, Xin Ma, Xinhao Li, Guo Chen, Xinyuan Chen, Yaohui Wang, et al. Internvid: A largo-scale video-text dataset for multimodal understanding and generation. arXiv:2307.0690
2023. 2, 6, 23, 24
[18] Tsai-Shien Chen, Aliaksandr Siarohin, Willi Menapace, Ekaterina Deyneka, Hsiang-wei Chao. Byung Eun Jeon, Yuwei Fang, Hsin- Hsin-Ying Lee, Jian Ren, Ming-Hsuan Yang, et al. Panda-70m: Captioning 70m videos with multiple cross-modality teachers. In CVPR, 2024. 2, 6, 23, 24
[19] Max Bain, Arsha Nagrani, GUI Varol. and Andrew Zisserman. Frozen in time: A joint video and image encoder for end-to-end retrieval. In ICCV, 2021. 2, 6, 23,24
[20] Yifci Li, Junbo Niu, Ziyang Miao, Chunjiang Ge, Yuanhang Zhou, Qihao Hc, Xiaoyi Dong, Haodong Duan, Shuangrui Ding, Rui Qian, et al. Ovo-bench: How far is your video-Ilms from real-world online video understanding? arXiv:2501 20000:0000.00500, 2025. 2, 3, 24, 25
[21] Junming Lin, Zheng Fang, Chi Chen, Zihao Wan, Fuwen Luo, Peng Li, Yang Liu, and Maosong Sun. Streamingbench: Assessing the gap for milms to achieve streaming video understanding, arXiv:2411.0366 2024. 2, 7,
[22] Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, al. Gpt-4o system card. arXiv:2410.21276, 2024. 2, 6, 7, 8. 8.27
[23] Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv:2403.055300 2024. 2, 7,8, 8, 27
[24] Kunchang Li, Yali Wang, Yinan He. Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, et al. Mybench: A comprehensive molti- video understanding benchmark. CVPR, 2024. 2,
2,
[25] Chaoyou Fu, Yuhan Dai. Yondong Luo, Lei Li. Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal lims in video analysis. arXiv:2405.21075, 2024. 2, 3, 7, 25
[26] Viorica Patraucean, Lucas Smaira, Ankush Gupta, Adria Recasens, Larisa Markeeva, Dylan Banarse, Skanda Koppula, Mateusz Malinowski, Yi Yang, Carl Doersch, et al. Perception test: A diagnostic benchmark for multimodal video models. NeurlPS, 2023. 2, 2,7, 25
[27] Junjie Zhou, Yan Shu, Bo Zhao, Boya Wu, Shitao Xiao, Xi Yang, Yongping Xiong, Bo Zhang, Tiejun Huang, and Zheng Liu. Mlvu: A comprehensive benchmark for multi-task long video understanding. arXiv:2406.044 04264, 2024. 7,22
[28] Karttikeya Mangalam, Raiymbek Akshulakov. and Jitendra Malik. Egoschema: A diagnostic benchmark for very long-form video language understanding. NewrIPS, 2023. 2, 7, 25
[29] Haoning Wu. Dongxu Li, Bei Chen, and Junnan Li. Longvideobench: A benchmark for long-context interleaved video- languae understanding. NeurlPS, 2024. 7,
11

---

[30] Junnan Li. Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image wre-training with frozen image encoders and large language models. In ICML, 2023. 3
[31] Haotian Zhang, Mingfei Gao, Zhe Gan, Philipp Dufter, Nina Wenzel, Forrest Huang, Dhruti Shah, Xianzhi Du, Bowen Zhang. Yanghao Li, et al. Mm1. 5: Methods, analysis insights from multimodal Ilm fine-tuning. ICI.R, 2025. 3
[32] Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang, Jac Sung Park, Mohammadreza Salchi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, et al. Molmo and pixmo: Open weights and open data for otate-of-the-ari multimodal models, arXiv:2409.1711 2024."
[33] Hang Zhang, Xin and Lidong Bing. Iddoo-llam:: instruction tuned audio-visual language model for video understanding. (rXiv:3306.02886 2023, 3
[34] Bin Lin, Bin Zhu, Yang Ye, Munan Ning, Peng Jin, and Li Yuan. Video-Ilava: Learning united visual representation by alignment before projection. vrXiv:2311.00122 2023. 3
[35] Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Khan. Videogpt+: Integrating image and video encoders for enhanced video understanding. arXiv:2406.. 09441, 2024, 3,7
[36] Mingze Xu, Mingfei Gao, Zhe Gan, Hong-You You Chen, Zhengfeng Lai, Haiming Gang, Kai Kang, and Afshin Dehghan. Slowfast-llaw A strong training-fit free baseline for video large language models. 1iXi::2407118411 2024, 3
[37] Boqiang Zhang. Kehan Li. Zesen Cheng, Zhiqiang Hu, Yuqian Yuan, Guanzheng Chen, Sicong Leng, Yuming Jiang, Hang Zhang, Xin Li, et al. VideoLLaMA 3: Frontier multimodal foundation models for image and video understanding. wxXiv2250.13100 2025. 8.8
[38] Xinhao Li, Wang, Jiashuo Yu, Xiangyu Zeng, Yuhan Zhu, Haian Huang, Jianfei Gao, Kunchang Li, Yinan Hc, Chenting Wang, et al. Videochat-flsh: Hierarchical compression for long-context video modeling, arXiv:2501.005: 00574, 2024. 3
[39] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In ICML, 2021. 3
[40] Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image praitngi In ICCV, 2023. 3
[41] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. NeurlPS, 2023. 3
[42] An Yang. Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li. Dayiheng Liu, Fei Huang. Haoran Wei, et Qwen2. technical report. arXiv:2412.15115. 2024.
[43] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anna I Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. 23X.v:2302.1397,, 2023. 3
[44] Rocland De Geest, Efstratios Gavves, Amir Ghodrati, Zhenyang Li, Cees Snock, and Tinne Tuytelaars. Online action detection. In ECCV, 2016. 3
[45] Mingze Xu, Mingfei Gao, Yi-Ting Chen, Larry Davis, and David Crandall. Temporal recurrent networks for online action detection. In ICCV, 2019. 3
[46] Mingze Xu, Yuanjun Xiong, Hao Chen, Xinyu Li, Wei Xia, Zhuowen Tu, and Stefano Soatto. Long short-term transformer for online action detection. NeurIPS, 2021. 3
[47] Yue Zhao and Philipp Krähenbühl. Real-time online video detection with temporal smoothing transformers. In ECCV, 2022. 3
[48] Kris M Kitani, Brian D Ziebart, James Andrew Bagnell, and Martial Hebert. Activity forecasting. In ECCV, 2012. 3
[49] Rohit Girdhar and Kristen Grauman, Anticipative video transformer. In CVPR, 2021. 3
[50] Joya Chen, Ziyun Zeng, Yiqi Lin, Wei Li, Zejun Ma, and Mike Zheng Shou. Livecc: Learning video Ilm with streaming speech transcription at scale. arXiv:2504. 1600000000.60 2025.
12

---

[51] Linli Yao. Yicheng Li, Yuancheng Wei, Lei Li. Shuhuai Ren, Yuanxin Liu, Kun Ouyang, Lean Wang. Shicheng Li, Sida Li, Lingpeng Kong, Qi Liu. Yuanxing Zhang, and Xu Sun. Timechat-online: 80% visual tokens are naturally redundant in streaming videos. arXiv:2504.177 173443,, 2025. 3
[52] Shenghao Fu, Qize Yang, Yuan-Ming Li, Yi-Xing Peng, Kun-Yu Yu Lin, Xihan Wei, Jian-Fang Hu, Xiaohua Xie, and Wei-Shi Zheng. Vispeak: Visual instruction feedback in streaming videos. arXiv:2503. 12769, 2025. 5, 10
[53] Haomiao Xiong, Zongxin Yang. Jiazuo Yu, Yunzhi Zhuge, Lu Zhang, Jiawen Zhu, and Huchuan Lu. Stream- ing video understanding and multi rould interaction with memory- enhaaned knowledge. arXiv:2501.
2025. 3, 26
[54] Zhenyu Yang. Yuhang Hu, Zemin Du, Dizhan Xue, Shengsheng Qian, Jiahong Wu, Fan Yang. Weiming Dong, and Changsheng Xu. Svbench: benchmark with temporal multi-turn dialogues for streaming video understanding. arXiv:2502. 10810, 2025. 3
[55] Yuxuan Wang, Yueqian Wang, Bo Chen, Tong Wu, Dongyan Zhao, and Zilong Zheng. Omnimmi: comprehensive multi-modal interaction benchmark streaming video contexts. arXiv:2503. 2025.
[56] Shangzhe Di, Zhelun Yu, Guanghao Zhang, Haoyuan Li, Tao Zhong, Hao Cheng, Bolin Li, Wanggui He, Fangxun Shu, and Hao Jiang. Streaming video -answiinnnn with in-context video kv-cache retrieval. arXiv preprint 2r3:0:2503.00540, 2025. 3, 26
[57] Rui Qian, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Shuangrui Ding, Dahua Lin, and Jiaqi Wang. Streaming long video understanding with large language models. Advances in Neural Information Processing Systems, 37:1 17336- 193600 2024. 3, 26
[58] Linli Yao, Lei Li, Shuhuai Ren, Lean Wang, Yuanxin Liu, Xu Sun, and Lu Hou. Deco: Decoupling token compression from semantic abstraction in multimodal large language models. 20Xi::2405.00883 2024. 5
[59] Fabian Caba Heilbron, Victor Escorcia, Bemard Ghanem, and Juan Carlos Niebles. Activitynet: large-scale video benchmark for human activity understanding. In CVPR, 2015. 6, 22, 23, 24
[60] Mingfei Han, Linjie Yang. Xiaojun Chang. and Heng Wang. Shot2story20k: A new benchmark for comprehensive understanding of multi-shot videos. arXiv:2312.10300 2023. 6, 22, 23.24
[61] Yansong Tang, Dajun Ding, Yongming Rao, Yu Zheng, Danyang Zhang, Lili Zhao, Jiwen Lu, and Jie Zhou. Coin: A large-scale dataset for comprehensive instructional video analysis. In CVPR, 2019. 6, 22, 24
[62] Luowei Zhou, Chenliang Xu, and Jason Corso. Towards automatic learning of procedures from web instructional videos. In AAAI, 2018. 6, 22, 24
[63] Lconard Bärmann and Alex Waibel. Where did leave my kays2-episodii--emory-basss question answering on egocentric vidcos. In CVPR, 2022. 6, 24
[64] Qirui Chen, Shangzhe Di, and Weidi Xie. Grounded multi-hop videoqa in long-form egocentric videos. arXiv:2408. 14469, 2024, 6, 22
[65] Haibo Wang, Zhiyang Xu, Cheng, Shizhe Diao, Yufan Zhou, Yixin Cao, Qifan Wang, Weifeng Ge, and Lifu Huang. Grounded-videelll:: Sharpening fine-grained temporal grounding in video large language models. arXin:2410.03290, 2024. 6
[66] Ye Liu, Zongyang Ma, Zhongang Qi, Yang Wu, Ying Shan, and Chang W Chen. Et bench: Towards open-ended event-level video-language understanding. NewrIPS, 2024, 6, 7, 22, 25, 26
[67] Liu, Limin Wang, Yali Wang, Xiao Ma. and Yu Qiao. Fineaction: fine-grained video dataset for temporal action localization. 2022. 6, 22
[68] Hang Zhao, Antonio Torralba, Lorenzo Torresani, and Zhicheng Yan. Hacs: Human action clips and segments dataset for recognition and temporal localization. ICCV, 2019. 6,22
[69] Shuhuai Ren, Linli Yao, Shicheng Li, Xu Sun, and Lu Hou. Timechat: A time-sensitive multimodal large language model for long vidco understanding. CVPR, 2024.
[70] Bin Huang, Xin Wang, Hong Chen, Zihan Song, and Wenwu Zhu. Vtimellm: Empower Ilm to grasp video moments. In CVPR, 2024.
[71] Gabriel Huang, Bo Pang. Zhenhai Zhu, Clara Rivera, and Radu Soricut. Multimodal pretraining for dense video captioning. 1rX11:2011.1776, 2020. 6, 24
13

---

[72] Zeqian Li, Qirui Chen, Tengda Han. Ya Zhang, Yanfeng Wang, and Weidi Xie. Multi-sentence grounding for long-term instructional video. In ECCV. Springer, 2024. 24
[73] Shangzhe Di and Weidi Xie. Grounded question-answering in long egocentric videos. In CVPR, 2024. 6, 22, 24
[74] Enxin Song, Wenhao Chai, Guanhong Wang, Yucheng Zhang, Haoyang Zhou, Feiyang Wu, Haozhe Chi, Xun Guo, Tian Ye, Yanting Zhang, al. Moviechat: From dense token sparse memory for long video understanding. CVPR, 2024. 24
[75] Miquel Farré, Andi Marafioti, Lewis Tunstall, Leandro Von Werra, and Thomas Wolf. Finevideo. https: //tuggingface.eeec//////// ct htdd//aattt//uuiiaaaae /aggg//ggggggggggggggeeeee wwwe////eeeeiioo...oooo 2024. 6,
[76] Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong. Qinglong Zhang, Xizhou Zhu, Lewei Lu, et InternvI: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In CVPR, 2024. 7.27
[77] Yuanxin Liu, Shicheng Li, Yi Liu, Yuxiang Wang, Shuhuai Ren, Lei Li, Sishuo Chen, Sun, and Lu Hou. Tempcompass: Do video Iims really understand videos? arXiv:2403.00476, 2024. 7,25
[78] Jajjun Liu, Yibing Wang, Hanghang Ma, Xiaoping Wu, Xiaoqi Ma. Xiaoming Wei, Jianbin Jiao, Enhua Wu, and Jie Hu. Kangaroo: A powerful video-language model supporting long-context video input. arXiv preprint arXiv:2408. 1554,, 2024. 8
[79] Yukang Chen, Fuzhao Xue, Dacheng Li, Qinghao Hu, Ligeng Zhu, Xiuyu Li, Yunhao Fang, Haotian Tang, Shang Yang. Zhijian Liu, et al. Longvila: Scaling long- contect visual language models for long videos. arXiv preprint arXiv:2408. 10188 2024. 8
[80] Xiaoqian Shen, Yunyang Xiong, Changsheng Zhao, Lemeng Wu, Jun Chen, Chenchen Zhu, Zechun Liu, Fanyi Xiao, Balakrishnan Varadarajan, Florian Bordes, et al. Longvu; Spatiotemporal adaptive compression for long addoo-language understanding. arXiv preprint arXiv:2410. 17434, 2024. 8
[81] Zhijian Liu, Ligeng Zhu, Baifeng Shi, Zhuoyang Zhang, Yuming Lou, Shang Yang. Haocheng Xi. Shiyi Cao, Yuxian Gu, Dacheng Li. et al. Nvila: Efficient frontier visual language models. arXiv preprint arXiv:24/2.04466. 2024, 8
[82] Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open open-source multimodal models with model, data, and test-time scaling. arXiv:2412.05277 2024. 8
[83] Xinhao Li, Yi Wang, Jiashuo Yu, Xiangyu Zeng, Yuhan Zhu, Haian Huang, Jianfei Gao, Kunchang Li, Yinan Hc, Chenting Wang, et al. Vidcochat-flass Hierarchical compression for long- long-context vidco modeling, arXiv preprint rrXiv:2501.0057 2024. 8
[84] Jiyang Gao, Chen Sun, Zhenheng Yang, and Ram Nevatia. Tall: Temporal activity localization via language query. In ICCV, 2017. 22
[85] Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert- nett-networks. arXiv: 1908. 10084, 2019. 26
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction have accurately claimed the contributions of this work, including the streaming framework and datasets.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
Limitations have been discussed in the appendix.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
15

---

Justification:
This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The paper fully discloses the information needed to reproduce the main experimental results in both the main paper and the appendix.
Guidelines:
The answer NA means that the paper does include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
16

---

Answer:[ Yes]
Justification:
The paper open-sources the code and data after internal review.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: /Codeeuomiisseeeeeeee /nips. wwtpccccpppcccaeccccccccccccctlt / sssssssoiccyyyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification:
The paper has specified the training and test details.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Following common practice in the multimodal learning literature, do not report error bars in this paper because of the heavy computation overheads.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
17

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The paper has provided the computation information.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question:
Does the research conducted in the paper conform, in every respect, with the ie ines?
NeurIPS Code of Ethics https: /ttp///nurissppeeeeeeee cc htcc/public/Eee csGuidel
Answer:[Yer [Ys]
Justification:
The paper is under the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer:(Yod Yes]
Justification: Broader impacts have been discussed in the appendix.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
18

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We properly cite the original assets in the paper.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/dataseta has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
19

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The details of the newly contributed dataset/code/moddl have been discussed in the paper.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodll as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
20

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [Yes]
Justification:
The LLM is used to generate the contributed data.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to L.LM policy (https: httperrpps.ccccoofeeeeeeee/////e /LLM) for what should or should not be described.
21

---

Prompts for Dense Video Captioning:
(I) "Identify and describe all activity events in the video.".
(2) "List every event happening in the video with descriptions.""
"Detect and summarize each event sequence in the video.",
"Extract and explain all notable activities in the video.". ".
(5) "Find all significant events in the video and describe them.".
Prompts for Sequential Step Recognition:
(1) "Identify key action steps in the video and provide a brief description of each.",
(2) "Defect and outline sequence actions steps laking place in the video.".
(3) Analyze the video to determine distinct actions or steps.",
"Break down the video into meaningful steps, describing each one concisely.",
(5) "Recognize and highlight specific sequences actions within video.".
Prompts for Temporal Action Detection:
(1) "When the action <action label> happens, output <action label>.",
(2) "When the event <action label> occurs, output <action label>.",
"If you see the actuon <action labelz, return <action label>.". ",
"Upon detecting <action label>, generate the message <action labels.",
(5) "When <action label> arises, immediately output <action label>.".
Prompts for Grounded VideoQA:
(1) <Question>. Answer only when you get enough information for answering the question.",
(2) Question>. Respond only if you have sufficient details to provide a complete answer.". "
(3) Question>.. Provide answer only when you have gathered enough relevant information.",
(4) QQusttoon> Ensure you
necessary context before attempting to answer.". (5) <Qustion>>. Only reply when you are confident that your answer is accurate and well-informed.",
Prompts for Temporal Video Grounding:
(1) "Localize the visual content described by the given textual query <query> in the video.".
(2) "Detect the video segment that semantically matches the given textual query <query>.",
(3) "Give you a textual query: <query>. When does the described content occur in the video?",
(4) "Locate the visual content mentioned in the text query <query> within the vidco.".
(5) "Find the video segment that corresponds the given textual query <query>.".
Table Prompts used for datasets to train the activation model.
A Datasets Used to Train the Activation Model
To train the activation model ACT()), we compile a diverse collection of video datasets spanning five distinct tasks:
Dense Video Captioning: ActivityNet Captions [59], Shot2Story [60].
Sequential Step Recognition: YouCook2 [62], COIN [61].
Temporal Action Detection: FineAction [67], HACS [68].
Grounded VideoQA: Multihop-EgoQAA [64], EgoTimmeQ [73].
Temporal Video Grounding: Charades [84], and the TVG subset from ET-Instruct [66].
In total, our training set contains approximately 180k video samples. For each sample, we construct an input prompt using task-specific templates. A prompt randomly sampled from a predefined pool for the corresponding task to ensure stylistic diversity and improve generalization across video domains. The full list of prompt templates is provided Table 7. During training, the prompt inserted at the beginning of the input sequence in Figure 3.
22

---

System:
You good question generalor. need your help in generating high quality question-answer pairs pertaining to the video clip descriptions. Follow these instructions:
(1) Ensure the questions and answers are highly relevant to the captions and DO NOT INCLUDE TOPICS NOT MENTIONED the captions.
(2) IGNORE CONTRADICTORY OR UNREASONABLE PARTS of the captions. Do not base questions on them.
(3) hope your questions feature different causal and temporal reasoning. Questions should be diverse and be related to different aspects of the described events.
(4) Ensure that the questions in the QA chain clear and precise, directly corresponding specific
information or events in the video, and can be answered by watching the video content without the need for video description inference, avoiding questions that require assumptions.
(5) Pay attention to grammar. Avoid grammar mistakes, especially with person and tense.
(6) Ensure questions are reasonable and challenging, requiring thoughtful consideration to answer correctly.
(7) The question should not contain phrases like 'In the beginning of the clips' or 'at the beginning of the video' or in the video' or in this clips'; it can include expressions of the present recent past such as 'just now' or 'right now.
(8) Please pay attention to the tense of the sentences.
(9) Never mention the sentence like 'according to the caption' in your question, you should assume that you can really watch the video instead of reading caption.
(10) Ensure there are no references to the source of information in the QA, avoiding expressions like 'from the image', 'sequence of pictures', 'which frame', or 'which photo'; you should understand the input as a video and describe it using video footage.
Understand the following different task descriptions:
<task descriptions>
USER:
Now, please carefully review the following video caption: <caption>
According to the given caption, please select ONE task type that is best suitable to generate the QA pair, and output your question, answer and task type following the SAME FORMAT as the examples above. Remember, just generate only ONE QA pair.
Table 8: Prompts used to generate QA pairs with GPT-4o.
B Stream-IT Construction
B.1 Statisties of Stream-IT
We provide detailed statistics of the Stream-IT dataset in Table 9, including the number samples, average video duration, and the corresponding source datasets used for each task. Notably, during the construction of the dense video captioning tasks, including ActivityNet[59] and Shot2Story [60], we only arrange 20% of the sequences with the proactive format of '<Q> <Vi> <A1>, <V>> <A²>, <A2>, while the 80% of the sequences with the multi-turn format of '<V> <Q;><<,>, <V>><<<>>>>>>>>>>> <Q²><A>>,
...', where <Q,> is the question asking about current situations like 'What is happening now?'.
B.2 Concatenation Strategy for Constructing StreamingQA-
Starting from a pool of 1.28 million filtered short videos sourced from WebVid- [19], Panda- 70M [18], and Intern 10M [17], our goal iteratively merge semantically similar clips to form long-form video samples. Let V denote the entire set of filtered videas. We initiate the process by randomly sampling one clip V1 from V the anchor. We then compute pairwise semantic similarity (based on the middle frame) between V1 and all other videos in V V1. A new clip V2 is sampled according to the similarity distribution (without replacement). The procedure repeated using V/2 as the new anchor, generating V/3 from V \ V1, V2, and so on. This results in a similarity-ordere list of videos Vi, V22 V3. ... We formulate this process in Algorithm This approach allows for flcxible concatenation of any number & of clips to construct long-form sample, by directly selecting
23

---

ol Samples
Datasels
Average duration
Netivin
Dense Captioning
Shat2Soory [60] (mx66))
VITT (-1kk)))))
~2210s
(=113k))
~-317%
Sequemial Step Recogsition
m/22
COIN [6i][=]1kk]
~-1456
How ToStap [72] (~110)
~-190s
MucieChat [74] (-0.8k)
~<10k frumes
Grounded Video Question Answering
~-69)
EgoTimeQA (o~1kk)
QAFgo44 [63] (-11k)
~~150s
<-405
FineVideo (>5](k)9k)
~-200s
Multi-tun Real-rime Question Anvwering
e-120k
(Sourced from Webvidd 10M) Panda- 70M] 18), Insern (iboraidd-00M177)
StreaaningQA-122 (~11)
~-150s
Table 9; Involved tasks and datasets in Stream-IT.
Algorithm 2: Constructing Similarity-Orderred Video Clip Sequence
Inputs:
Pool of filtered short video clips Vpool {m, 12:-..,
Initializations: Vordered
Vanchor None;
3 Define: Sim( Uanchar, Crandidatoa:: function that returns the clip from Crandidates most similar to Vanchor- // Randomly select the first anchor clip I/ Add Vanchor to V Vordered Remove Vanchor from V Vpool
Vanchor <- RandonSaunple(Pooo))
Vordered Vanchor
Pooo + Vpooo Vanchoor
7 while Vpool is not empty do
8 Unext <- Siimi Canchor, Vpool)
// Find the clip in pool most similar to Vanchor
Vordered Unext
Vpool Vpool (Unext
Vanchor f- Unext:
Update anchor to the newly added clip [Vi, V2...., Vvl.
12 Output: oirdeed list of video clips Vordered
continuous span M[s+k). without re-computing similarity each time. We also prepare hallucination questions irrelevant to existing video inputs following [20] with a ratio 0.01%.
B.3 Prompt Templates for Generating QA Pairs
To generate question-answer pairs based on clip-level captions, we design diverse prompt templates for 8 distinct reasoning tasks. Table 8 provides examples of these templates. Below, we summarize the <lask descriptions> associated with each QA category:
[OP] Object Perception: Detect and identify objects, focusing on recognizing their attributes in real time.
[AR] Action Recognition: Identify human actions and interactions occurring in the current moment.
Understand spatial relationships among objects and events; reason about location, orientation, and distance.
[SA] Spatial Awareness:
[SR] Sequential Relationship: Identify the temporal order of events and actions, especially those involving "before" and "after" cues.
[CR] Causal Reasoning: Analyze cause-and-effect relationships between actions and outcomes.
[OCR] Optical Character Recognition: Recognize and interpret textual content in scenes (e.g.. subtitles, signs, charts).
[UEH] Unexpected Event Handling: Detect and react to anomalies sudden changes in the environment.
[EU] Event Understanding: Summarize and reason about sequences of temporally linked events.
These diverse prompts ensure broad task coverage and help enhance the model's generalization across different temporal and semantic understanding challenges.

---

C More Implementation Details
For the main VideoLLMs,, we use the following configurations for each model:
LLaVA-OV-7:: We apply center cropping with resolution of 384 x 3884 and use x4 down sampler (bilinear interpolation) with the frame features, resulting in 49 tokens per frame.
Oryx-1.5-7B: We use the model's s default dynamic resolution, ranging from 288 to 480 pixels. With a x4 down sampler on the frame features (average pooling), the resulting token count per frame varies between 33 and 59.
Qwen2-VL-7B: The model uses a dynamic resolution between 224 and 448, with x4 down sampling (average pooling) on the frame features, resulting in 36-64 tokens per frame.
All models are fine-tuned for one epoch using a learning rate of 2e-5 with a cosine annealing scheduler and AdamW optimizer. The image encoder remains frozen, while the visual projector and the LLM are fully trainable. The maximum length La of input embeddings is set to 16384 for the round-decayed compression.
For the activation model, we adopt I.LVA--V-0.55B as the base model. To improve efficiency, we aggressively pool the frame representations to 16 tokens per frame. During training, only the LoRA adapters, the projector, the score head, and the learnable activation token are trainable. The model is trained for 5 epochs using fixed learning rate of 2e-5 for the projector, while 2e-4 for the LoRA adapters, score head, and the learnable activation token, with Adam optimizer.
Notably, for both the main VideoLLM and the activation model, we sample frames at FPS to better simulate world frame rates. For videos longer than 256 seconds, we uniformly sam- ple 256 frames to fit within the maximum input length constraint. Experiments are conducted on 100 GPUs. During inference, we sample videos at FPS for short video bench- marks like M MVBench, Perpecption and TempCompass, while FPS for multi-turn real-time understanding benchmarks and long video benchmarks including OVO-Bench, Streaming- Bench, MLVU, Bece VidcoMME, and EgoSchema.
D Benchmarks and Metrics
Multi-turn Real-time Understanding Benchmarks. We evaluate our method on two recently proposed large-scale streaming video benchmarks: OVO-Bench [20] and Frreaming-Bende [21]. Both benchmarks are designed to assess streaming video comprehension under long-context, multi- turn settings. Our evaluation primarily focuses on their real-time understanding tasks. OVO-Bench contains 512 videos with average length of 435 seconds and approximately 1,600 questions. The evaluated tasks include: (1) Spatial Understanding (STU), (2) Object Recognition (OJR), (3) Attribute Recognition (ATR), (4) Action Recognition (ACR), (5) Optical Character Recognition (OCR), and (6) Future Prediction (FPD). Streaming-Benee consists of 500 videos with an average length of 606 seconds and approximately 2.500 questions. It includes the following tasks: (1) Object Perception (OP), (2) Causal Reasoning (CR), (3) Clip Summarization (CS), (4) Attribute Perception (ATP), (5) Event Understanding (EU), (6) Text-Rich Understanding (TR), (7) Prospective Reasoning (PR), (8)
Spatial Understanding (SU), (9) Action Perception (ACP), (10) Counting (CT). Both benchmarks are structured multiple-choice question -answernn tasks, and we report the accuracy.
General Video Understanding Benchmarks. evaluate general video comprehension ability, we test our models on seven widely used benchmarks. This includes three short-video benchmarks: MVBench [24], Perception Test [26], and TempCompass [77], and four long-video benchmarks:
EgoSchema [28]. Long [29], MLVU [27], and VideoMME [25]. These datasets span a broad range of video durations, from a few minutes to several hours. All are evaluated in a multiple-choice format, and accuracy is reported.
Online Activation Benchmarks. To assess the proactive capabilities of our framework, we evaluate performance on a subset of ET-Bench [66], including Temporal Video Grounding (TVG), Temporal Action Localization (TAL), Dense Video Captioning (DVC), and Sequential Localization Captioning (SLC). These tasks emphasize a shift from passive to active perception, requiring the model to determine when to respond based on upcoming visual inputs, rather than reacting immediately. For example, the Sequential Localization Captioning (SLC) task requires the model to both determine
25

---

the precise timing of a certain step and output its content. For evaluation metrics, we compute
the average Fl score across multiple IoU thresholds (loU {0.1, 0.3, 0.7.) for localization- based tasks. For tasks involving text generation, we adopt sentence-level similarity metrics [85] to measure the semantic alignment between model outputs and ground-truth responses, following prior works [66; 13]. Specifically, the -i--Mii-------- model in Sentence-Transforee Transformers library is used as the embedding model. Notably, in all these tasks, the question presented at the beginning of the video, and the model must autonomously decide when to respond. Moreover, the results of TVGFI, TALFI. are the same for our method with different main Video-LLMs, since they use the same activation model and will not be affected by the generated response.
E
Broader Impacts
There are many real-world applications of streaming Video-LLMs, such as patient or elderly health monitoring, autonomous driving, and collaborative robots. However, there could be unintended usages and we advocate responsible usage complying with applicable laws and regulations.
More Related Works
To address the challenge of long-context understanding in streaming video, several memory and retrieval mechanisms have been proposed. For instance, ReKV [56] introduces training-fre free frame- work that stores and retrieves the Key-Value (KV) caches of processed frames, enabling offline models to answer user queries efficiently by reloading only the most relevant context. Besides, VideoStream- ing [57] employs a meeory--rropagtee encoding architecture where a condensed representation of the preceding clip serves historical context for encoding the next, combined with an adaptive selection of memories for question-answering Moreover, StreamChat [53] proposes hierarchical memory system comprising short-term, long-term, and dialogue components to facilitate complex streaming interactions, and also contributes the StreamBench benchmark for evaluating diverse streaming scenarios. While these methods effectively advance long-com long-context retention for reactive question-ane canttionnnnswering... StreamBridge differs by introducing a cound-decayer compression strategy specifically tailored for multi-turn real-time interactions, which efficiently prunes redundant historical tokens while preserving recent context with high fidelity. Moreover, StreamBridge introduces a decoupled, lightweight activation model. This plug-and-playy component operates in parallel with the main Video-LLM, enabling continuous proactive responses. These designs, supported by our dedicated Stream-IT dataset, effectively transform general-pus purppse offline models into versatile and proactive streaming assistants without compromising their core performance.
G Limitations
Although our proposed framework and dataset significantly enhance the streaming capabilities of existing offline Video-LLMs, there still limitations worth noting. First, while Stream-IT provides large-scale multi-turn, interleaved training data, construction relies partially on synthetic QA generation and clip concatenation, which, despite careful filtering, may introduce domain shift compared truly continuous, real-world video streams. Future work could benefit from curating more organically collected long-form streaming videos with naturally evolving events and dialogues. Second, StreamBridge currently focuses on canne-bb-framn streaming under relatively low sampling rates (c.g.. FPS). Extending the framework to handle denser frame rates or multi-modal streaming inputs (e.g., audio-visum visual-text -tett) in real-time remains an important direction for future research.
26

---

H Full Results on OVO-Bench and Streaming-Bench
Method
*of
Real-Time Visual Perception
Backward Tracing
Forward Active Responding
Overall
Frames
OCR
ACR
ATR
STU FPD
OJR AVG.
EPM
ASI
HLD AVG
REC
SSR
CRRR
AVG.
Overall AVG.
Human
Homan
93.96 92.57 94.83 92,70 91.09 04.02 93 20 (92.59 93/02 91.37 99.33 (95.48 89.67 93.56
92.90
92.81
Proprictary Models (Oftline), Single- Turn Evaluation
pro (23)
LFS
[85.9]
79.31 58.43 63.37
69.32
52.64
62.54
61.67
57.15
GPT-4o (22)
64
69,88
64.22
71.55 51.12 70.30 59.78 64.46 (57.91 75.68 48.66 60.75
27.55
73.21
59,40
53.40
59,54
Open-Source
(Oftlin)).
Turg Evaluation
Qwen2-V VL-728 Qwen2-VL-72BD [2]
64
65.77
60..5
69.83 51.64 69.31 54.35 61.92
52.53
60.81
57.53
56.95
58.83
64.07
45.00
49.30
$6.27
11 [15]
14.26
63.52
57.43
40.4
34:10
60.42
LLaVA-OV:78 [3]
66.44
57,80
73.28
53.37
71.29 61.96 61.02
51,21
55.41
21.51
13.71
25.64
67.09
58.75
50.50
52.74
Owen2-VL-7B [2)
64
60.40
50.46
56.03 47.19 66.34 55.43 55.98
47.81
35.48
56.08
46.46
31,66 65.82
48.75
48.74
50.39
Inlem/VL-V--BB [76]
67.11
60.55
63.79 46.07
68.32 56.52 60.39
48.11
57,43
24.73
13:44
26.5
59.14
54.14
46.60
50.15
Open-Source Modelt (Streaming))
Single-Tur
Evaluation
Flash VStrcam-22 7B (11)
1 FPS
24.16 29.36 28:15 33.71 25.74 28.80 28.37
39.06 37.16 5.91
27.38
8.02
67.25
60,00
45.09
33.61
Vileod.MM-Online 88 [10)
2 FPS
K.05 23.85 12.07 14.04 45.54 21.20
20.79
22:22
18.80
12.18
17.73
Dispider [13]
FPS
57.72 49.54 62.07 44.94 61.39 51.63
54.55
48,46
55,41
4.30
36.06
18.05
37.36
48.75
34.72
41.78
Models under StreamBridge Bridge (Ollinn - Streomingh Multi-Turn
Evalusation
Ohya-1.5-7B []]
FFS
60:40
52.29 69.83 50.00 65.35 57.61 59.25
54.21
55.41
5.40
38.33
20:65
37.56
40.00
32.74
43:44
-Srvom-JT
FPS
84.56 75,23 70.69 50,56 74.76 71.74 71.17
69.02 59.50
79.03
69.17
20:51
66.89
60.41
49.27
63.21
LLVAA-OV7BB [3]
FPS
58.39
59.63
69.82
44.38
76.23
61.41
61.64
53.87
54,72
30.64
46.41
14.41
51.25
43.35
36.33
4K.13
-Sreame-1 IT
74.50
70.69
73.27
69.93
71.21
17.53
48.52
63.22
Owen2-VL-7D VL (2]
64.66
74.26
60.14
59.53
49.58
+ Streom--I
FFS
84.56
71.56
74.14
49.44
75.25
72.83 71.30
67.68
57.43
79.03
68.05
19.17
64.25
61.67
48.36
62.57
Table 10: Full results on OVO-Bench. means models under StreamBridge framework
Methed
Real Time Viusi Undestanding
Oreri Seerce Undentraiing
Contectial Untereseding
Ovenll
MCU
Buman
Haraa
(9:A) 92.0 9160 91A77 Vott V1S2 S0ER Whots 91.30 18000 S0.1 VANU 9022 (ML.2) $5.50
90.00
9500 100 [R535]
91.66
Prepristury (Ofliin Single Evaiiatton
Gemin 1.5mmp
2007
8151 TRAT 20100 66-76 77 25 6U.3 Ms 4s
Fo 16:00 10:6 10.00 2010
STAT
40.73
$1.90
6F:6
GPS-46 [o]]
64
71.11
RM47 8191 7644
$3.90 66.7
40:31
71.25
(1.30 17.30 43.00
56,00
16.00
41.10
13.40
(8:70)
60.15
Spen-Soure Middel (Olllin) Singl:
Ture Exaloation
IL CV-TB [3]
33
19:38
74.32
76603 80:32 73 67 11.05 07 59 65 45 65.72 45.04
71 12 I0.00 37 33,00 41.90
18
(0
35.00
36
10:00
21:07
26.55
13.74
56.76
Quent V1.78 11
0.2-
75.20
81
7319 77.45 08
61 11.19 46.11
4310
11.30
39:00
29.73
51.66
3nomVL (70)
16
60.94
09.40
67.00 62.93
56.45
37.50
12:42
$1.40
Open-Sourre Madek (Ner aorminn Singh-Tuum Turm Prallotioon
Fla VS [II)
1395
1535 4537
23.0 25.37
13.06 18.5 25.20
45.00
15:01 SLWW 2550 23,07
2135
25:30
26.50
1.56
15.13
14.04
14 (10)
139%
40.0
21:00
under Struasrridgg (Oilline Strooming
Eraluntion
Ory-1 5-30 U1]
177
76407
17.17 8138 $1..0 71 66.96 79.63 51-75 66.20 40:01
20:50
10:00
15:30
15,50 41.20
(1.50)
10.40
21:00
19.50
54.00
14.95
55.76
SSress IT
FPS
83.29 77:05 87.08 86.47 77.99 $1.31 76.85 65:92 71.96 35.23
14.70
19.20
14.00
52.00
20.30
(5.70
14.40
14:00
51.30
43.14
30.50
54.19
LL [i]
1395
76284
71.17 62.60 75.25 64.15 64.11 75.00 51.36
61.19 46.1
69.39
34.40
12:00
12:40
37.00
26.60
2000
19:00
34.40
52.04
31.74
50.96
-Srawwr IT
196
07.39 72.44 97 86 71 I77 74.46 75 (8) 03.30 70.30
76,
70
50
13.00 43.60
77 (1
56.10
10:40
17
07 41.50
17
70.71
51.75
VL-78
19P5
78.74 19.86
77.78 63.41
52.00
15:20
31.30
21 20
36.57
1995
[8424 #2.66 92 89.7 77.36 84.26 69:92 71.07 35.75
18:0
13.30
45.80
21.00
(4.10) 14.00
17.30 46.00
90.96
12.35
55.39
Table 11: Full results on Streaming- Bench..
meaas models under StreamBridge framework
27

---

I Pseudo Code of the Round-Decayed Compression in a PyTorch-like Style
def Round, Decandd_eeeyedd..m Comppessiinn Staputs__mbedd, max_len, token_ framm))
[epuuss [1, seq_len, dim], interleaved embeddings of video text; max len: the predefined maximum sequence length of inputs_embeds; token_ frame: the number of tokens per frame;
Lareeess_trret__tt is the number of tokens that need be compressed, # should be integer multiples of laren_per_fram redudant_ frrttt_frmm__________ num int sanpputs_embddeeeeeeeeeee [tteeeeeeeeeee(ee1 max_ (ax__len//ee wewee///eeeeeerree.e... prame) + compress_targe___t
coken_per_free
redudant_frame
split inputs_embeds embedd into image_embeds embeds and text_embeds by round, # e.g., mmbee_ [i] is the visual tokens of the i-th round, # and len (embadd == len (text, (ettext__mbbes) == number of rounds; image, embeds, text_ text_embeds embeds split_ image, and_text (inputse_ embeds)
new www_iinutt__emt embeds []
compress visual tokens round round for round_ids in range (len (isiiaaeemeees))__ current__imge__nntee embeeds current_teed serrent_text__mbede text_embede [round_ids [dx]
[round_idxs
if compress saarresstaaret_nue > 0 and current embede [1] >
token_per_frane frame*2:
un
compress current_ parrent_image..eo. image_embeds embeds into [1, coken_per_frame, dim];
if current iwriiiiaageeeeeeeee shape [1]
compress_target_nus
token_per_frane frame:
current_frame,_t num current_imgs iwwgememeeds.....eeeeemmeeeeeee.
embeds. e [1] // token_per, frame
perrent_image_enie = current_ image_ eatreee_mbeds..o embeds. reshape
current_frame __m,, token_per_frame,
current_image embeds shape
current_jmage__mbeee sumpress_target_.g
purrent__mage__niiee mean (dim=1)
Coureett_frame_ee num- wtweeeteeeereeeeeeee praaa frame
compress current_image embeds' first taapress_target... num token_per. beiken_per_fratte frame tokens into [1, feken__er_ffam, dim], and reserve the rest tokens;
else:
pre_ pre_image_enie embeeds current_image_c _emeeds [:,
:compress_tare
:]
pre_frame_nut pre_ ore_image_ embeds. shape [t]///toe//oe_ee prame
iiaimmeemeeeeseeeeeeeeeemeeeeeee = pre_image__m _eteeee_mbbee.eeeeeeeeee reshape
compress_targee
tat_tttrrge___uute citi_juum/ttoken frrm per_frame-
token, fer_e__er_framm
serrent_image_eeee emweds.eee...e..........e...eee. shape[-1])
pre_image___ embeds = pre pre__meee pep__mmae____ sarraageeembedeeeeeeeeee mwan (dim=1)
se_jmagee_mbbdds
post_image,
tos__maaee_mmbeedd embeds current_imms catpttt__maee__mmeeeeeee
compress_tare barget_mum-token_pert
:]
taarressttaatt....e num (pre_free frame mwm.mnmmmeeeee
frame_num-1) per_frame
current. current_image embeds = torch. cat [[pre_ image_mamiaan embedd,
image_embeds
post_image___beel
dim=1)
new_inputs_embut beabbeddd..ppppdedde append (current. saattt_mmmeeeeeeeeeee
new new_inputs_embedt eeaittss_embdds...eee append wtw.rrueeettteeeeeeereeeeeeeeeee text_embeds)
return torch. cat (new thew_inputs_embeee
dim=1)
28


Reviews:
[{"review_id": "iCBmapjpgE", "reviewer": "Reviewer_Qdek", "summary": "This paper introduces StreamBridge, a framework designed to adapt existing offline Video-LLMs for proactive, real-time streaming video understanding. The authors identify two primary challenges in this domain: the lack of multi-turn real-time understanding and the absence of proactive response mechanisms in current models. To address these, StreamBridge proposes three key components: 1. A memory buffer to manage the history of interleaved video and text inputs. 2. A round-decayed compression strategy to efficiently handle long-context inputs by merging older visual tokens while preserving recent ones. 3. A decoupled, plug-and-play activation model that runs in parallel to the main Video-LLM to decide when a response should be generated, enabling proactive behavior without interfering with the main model's architecture. \n\nFurthermore, the authors contribute Stream-IT, a large-scale instruction-tuning dataset specifically created for streaming scenarios, featuring interleaved video-text sequences and a novel synthetic component, StreamingQA-120K. The extensive experiments demonstrate that models adapted with StreamBridge and fine-tuned on Stream-IT not only achieve state-of-the-art performance on streaming benchmarks, outperforming proprietary models like GPT-4o and Gemini 1.5 Pro, but also maintain or improve their capabilities on standard offline video understanding tasks.", "questions": "See Weaknesses", "limitations": "See Weaknesses", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "1. Strengths\n\n1.1 Effective framework design\n\nThe proposed StreamBridge framework is simple yet powerful. The decision to use a decoupled activation model is particularly elegant. It avoids the optimization conflicts and performance degradation that can arise from tightly integrating activation mechanisms into the main model's architecture, a known issue in prior work. This modular, plug-and-play approach enhances flexibility and preserves the base model's core capabilities.\n\n1.2 Resource contribution (Stream-IT Dataset)\n\nThe creation and release of the Stream-IT dataset is a major contribution to the community. The methodology for constructing the long-form, multi-turn StreamingQA-120K portion by concatenating semantically related clips and using a powerful LLM (GPT-4o) for QA generation is sound and addresses a clear data gap for research in this area. The data augmentation strategies (Random QA Drop and QA Interval Shift) are thoughtful additions that increase the dataset's diversity and utility for training robust streaming models.\n\n1.3 Comprehensive empirical evaluation\n\nThe experiments are thorough and the results are compelling. The authors demonstrate the framework's generalizability by applying it to three different base models (Oryx-1.5, LLaVA-OV, Qwen2-VL). The fact that their fine-tuned models outperform even powerful proprietary systems on challenging streaming benchmarks (OVO-Bench, Streaming-Bench) is a strong result. Crucially, they also validate that this improvement does not come at the cost of performance on general offline tasks, showcasing the non-destructive nature of their adaptation method. The ablation studies effectively justify their design choices, particularly the round-decayed compression strategy and the components of the Stream-IT dataset.\n\n2. Weaknesses\n\n2.1 Generalization of the activation model:\n\nThe activation model is trained on a collection of datasets from five specific tasks using predefined prompt templates. How well would this activation model generalize to completely unseen tasks or scenarios where the need for a proactive response is more subtle and not guided by an initial, task-specific prompt? The reliance on templates might limit its true zero-shot proactive capabilities.\n\n2.2 Lack of direct efficiency comparison\n\nThe paper reports the operational FPS for StreamBridge and other streaming architectures in the results tables, but it lacks a direct, controlled comparison of end-to-end latency and computational efficiency on the same hardware. Reported FPS from different papers can be influenced by numerous factors and is not a substitute for a rigorous head-to-head benchmark. Such an analysis would provide a much clearer understanding of StreamBridge's practical efficiency relative to methods like VideoLLM-Online or Flash-VStream.", "was_revised": true, "final_justification": "My concerns have been addressed. I would like to maintain my scores (Borderline accept).", "rebuttal": "We sincerely thank Reviewer Qdek for the insightful and constructive feedback. We are particularly encouraged by the recognition of our modular and elegant StreamBridge framework, the contribution of the Stream-IT dataset, and the comprehensive empirical evaluation. We address your concerns below.\n\n**(1) Generalization of the activation model** \n\nWe agree that generalization is a critical aspect of the activation model. To encourage robustness, our model is trained on five diverse tasks (e.g., dense captioning, step localization, action detection, etc.) using a variety of prompt templates and a dynamic P% sampling strategy, which enables the model to learn activation across varied temporal patterns.\n\nTo further evaluate its generalization ability, we report results on **NExT-GQA** [1], a grounded VideoQA benchmark characterized by diverse, natural question prompts with annotated timestamps. Unlike template-based tasks like dense video captioning, NExT-GQA requires the model to trigger a response precisely when the visual evidence for answering the question appears, presenting a strong challenge for generalization.\n\n**Table: Activation Model Generalization on NExT-GQA**\n\n|               | Absolute (s) | Relative (%) |\n|---------------|---------------|----------------|\n| Activation Delay | 3.24        | 7.7            |\n\nHere in the table, **Absolute** denotes the average time gap (in seconds) between the activation timestamp and the annotated ground-truth response timestamp. **Relative** refers to this gap as a percentage of the total video duration. These results demonstrate that the activation model can proactively answer the question within a delay of 3.24 seconds or 7.7% of the video duration on average, and generalize well to unseen videos and diverse, open-ended questions, even in scenarios not encountered during training.\n\n[1] Xiao, Junbin, et al. *\"Can I trust your answer? Visually grounded video question answering.\"* CVPR 2024\n\n---\n\n**(2) Direct efficiency comparison**\n\nWe appreciate the reviewer’s suggestion and now provide a direct inference latency comparison between **Qwen2-VL-StreamBridge (ours)**, **VideoLLM-Online**, **Flash-VStream**, and **Dispider**. All models are evaluated on the ActivityNet dense video captioning task using the same hardware (a single NVIDIA A6000-48G GPU) with 1 FPS video sampling:\n\n**Table: Latency and Accuracy Comparison**\n\n| Models                  | ANet-Caption Latency (s) | OVO-Bench Real-Time Acc (%) |\n|-------------------------|---------------------------|-----------------------------|\n| Qwen2-VL-StreamBridge (ours) | 1.52                      | 71.30                       |\n| Dispider                | 2.47                      | 54.55                       |\n| VideoLLM-Online         | 0.89                      | 20.79                       |\n| Flash-VStream           | 0.95                      | 28.37                       |\n\nFrom the results, we observe that our framework is more efficient than Dispider, while slower than VideoLLM-Online and Flash-VStream. This trade-off is expected: both VideoLLM-Online and Flash-VStream rely on highly aggressive compression schemes **(1–10 tokens per frame)**, which significantly reduce latency but also lead to substantial information loss. In contrast, StreamBridge maintains a richer representation per frame and achieves significantly stronger results on challenging benchmarks such as **OVO-Bench** and **VideoMME**, highlighting a better balance between efficiency and accuracy."}, {"review_id": "1jN1mMHC1r", "reviewer": "Reviewer_ioHp", "summary": "This paper introduces StreamBridge, a framework designed to transform offline Video-LLMs into streaming-capable models. The key challenges addressed are (1) enabling multi-turn real-time understanding and (2) implementing proactive response mechanisms. StreamBridge introduces a memory buffer and a round-decayed compression strategy to manage streaming inputs efficiently. It also constructs a large-scale dataset for streaming video understanding. The approach is evaluated on benchmarks like Streaming-Bench and OVO-Bench, demonstrating competitive performance against existing models.", "questions": "See the weaknesses part above.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "### Strengths\n1. StreamBridge addresses a critical gap in adapting offline Video-LLMs to real-time streaming scenarios, which has broad applications in video analysis and interactive applications.\n2. The paper presents extensive experiments across multiple streaming benchmarks (e.g., OVO-Bench, Streaming-Bench) and ablation studies.\n3. The newly introduced Stream-IT dataset provides valuable support for training streaming video understanding models, promoting future research in this emerging area.\n\n### Weaknesses\n1. While effective, the Memory Buffer and Round-Decayed Compression strategies have been explored in prior work (e.g., LSTR [1]) in different contexts. This diminishes the perceived technical originality of the approach.\n2. The compression strategy assumes that recent frames are more relevant to incoming queries. However, this assumption may not hold universally and could be influenced by benchmark biases. A deeper investigation is needed to validate its general effectiveness. For example, conducting an ablation study where the question is intentionally delayed by varying time intervals, simulating scenarios in which a query relates to a past segment rather than the most recent visual input.\n3. The paper refers to `Qwen2-VL†`’s +2.97 improvement on Streaming-Bench as a “notable improvement,” while characterizing `LLaVA-OV†`’s -2.38 and -2.73 drops on OVO-Bench and Streaming-Bench as “slight performance drops” (L249-252). This language appears inconsistent and subjective.\n4. The use of a lightweight model (`LLaVA-OV-0.5B`) and aggressive token compression in the Activation Model raises concerns about its accuracy. However, no quantitative evaluation or ablation study is provided to validate its effectiveness. An ablation study would strengthen confidence in this component.\n5. The related work section would benefit from additional discussions with more relevant prior works, such as [2–4], to better situate the proposed approach within the context of streaming video understanding.\n\n---\n[1] Xu, Mingze, et al. \"Long short-term transformer for online action detection.\" Advances in Neural Information Processing Systems 34 (2021): 1086-1099.\n\n[2] Qian, Rui, et al. \"Streaming long video understanding with large language models.\" Advances in Neural Information Processing Systems 37 (2024): 119336-119360.\n\n[3] Di, Shangzhe, et al. \"Streaming video question-answering with in-context video kv-cache retrieval.\" arXiv preprint arXiv:2503.00540 (2025).\n\n[4] Xiong, Haomiao, et al. \"Streaming Video Understanding and Multi-round Interaction with Memory-enhanced Knowledge.\" arXiv preprint arXiv:2501.13468 (2025).", "was_revised": true, "final_justification": "The rebuttal has adequately addressed my concerns. While the paper is technically sound, its novelty remains limited (as noted in Q1). Nevertheless, I am raising my score and am now inclined toward a \"Borderline accept\" recommendation.", "rebuttal": "We thank Reviewer ioHp for the thoughtful review and valuable comments. We are glad that the reviewer acknowledged StreamBridge's effort to fill a critical gap in adapting offline Video-LLMs to real-time streaming scenarios, and appreciated our extensive benchmark coverage and the introduction of the Stream-IT dataset. We address your suggestions below.\n\n**(1) Novelty of memory buffer and round-decayed compression** \n\nWe clarify that while memory mechanisms are not entirely new (e.g., LSTR), our method is specifically tailored to **multi-turn streaming** scenarios with a fixed token budget. Unlike LSTR, which maintains separate long- and short-term memories limited to only visual tokens, our memory buffer stores **interleaved video-text** tokens, enabling richer and more coherent contextual understanding and multi-turn dialogues for multimodal LLMs.\n\nMoreover, our round-decayed token compression is a lightweight and **training-free** strategy for inference only. In contrast, LSTR introduces an additional LSTR Decoder that requires training to extract information from long-term memories via cross-attention. Our method avoids such complexity, instead merging older visual tokens through simple average pooling while preserving recent context, offering both efficiency and adaptability without additional training overhead.\n\n**(2) Assumption that recent frames are more relevant** \n\nWe acknowledge that this assumption may not always hold in **non-streaming** scenarios where earlier frames can contain critical information. To address this, our round-decayed compression strategy does **not discard older content**; instead, it **pools** earlier video tokens to retain coarse-grained historical context. This design is validated in **Table 4** and discussed in Lines 294–303. Furthermore, the memory budget can be adjusted via the **MaxLen** parameter (defined in Line 137). For instance, when **MaxLen = 32k**, the model can store up to **15 minutes** of video at 1 FPS without requiring additional compression, ensuring full retention of video information in low-latency or short-form scenarios.\n\nMoreover, standard offline video benchmarks such as **VideoMME** and **LongVideoBench**, whose questions are posed at the end of the video, inherently include scenarios where \"the question is intentionally delayed by varying time intervals, simulating scenarios in which a query relates to a past segment rather than the most recent visual input.\" As shown in **Table 2**, our approach maintains comparable performance on these benchmarks, demonstrating StreamBridge's generalization that retains relevant historical information and is not biased toward only recent context.\n\n**(3) Inconsistent language in result interpretation** \n\nThank you for pointing this out. We will revise the phrasing to ensure consistency in describing all performance changes across models.\n\n**(4) Activation model's ablation** \n\nWe agree and appreciate the suggestion. In our current setting, we use **16 tokens per frame** for the activation model. Below, we provide additional quantitative results on **ET-Bench**, exploring the impact of token granularity. We report only the **F1 score**, which reflects the precision of the activation model’s response timing:\n\n**Table: Activation Token Granularity on ET-Bench**\n\n| Tokens per Frame | TVG-F1| TAL-F1| DVC-F1| SLC-F1 |\n|------------------|------------------|------------------|------------------|------------------|\n| 8                | 30.1             | 19.3             | 33.8             | 19.1             |\n| 16               | 34.3             | 24.3             | 37.8             | 22.5             |\n| 32               | 35.1             | 23.1             | 38.3             | 21.9             |\n| 64               | 34.9             | 25.3             | 38.8             | 22.8             |\n\nAs shown, increasing the number of tokens per frame does not significantly improve performance. This is expected, as the activation model is designed to perform **binary classification** (i.e., whether to trigger a response), which requires less fine-grained semantic detail than generative tasks. Our default setting of a small MLLM and 16 tokens per frame achieves a good balance between efficiency and accuracy.\n\n**(5) Related work coverage** \n\nThank you for the references. We will expand the related work section to include [1, 2, 3], and incorporate a discussion of them to provide a more complete context for our work.\n\n[1] Qian, Rui, et al. *\"Streaming long video understanding with large language models.\"* NeurIPS 2024.\n\n[2] Di, Shangzhe, et al. *\"Streaming video question-answering with in-context video kv-cache retrieval.\"* arXiv:2503.00540, 2025.\n\n[3] Xiong, Haomiao, et al. *\"Streaming Video Understanding and Multi-round Interaction with Memory-enhanced Knowledge.\"* arXiv:2501.13468, 2025."}, {"review_id": "DFO8Tu575g", "reviewer": "Reviewer_ySNS", "summary": "This paper proposed a StreamBridge framework to transform offline video LLMs into streaming-capable models. A memory buffer and a decoupled, lightweight modules are incorporated into existing video LLMs to address fundamental challenges. Besides, a new benchmark is proposed for streaming video understanding tasks. Experiments demonstrate the effectiveness of the proposed method in various video understanding tasks.", "questions": "Overall, this paper exhibits high-quality writing, solid methodology, and sufficient experiments. I have some minor concerns listed in **weaknesses**.", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n- The paper writing is good. The motivations are clearly explained, and all the modules are analysed sufficiently.\n\n- The experimental results look good. In most cases, the introduction of StreamBridge brings consistent improvement to various baseline video LLMs.\n\n- A new benchmark is proposed that contains long-context, interleaved video-text instruction tuning, and includes diverse tasks and realistic multi-turn formats.\n\n**Weaknesses**\n\n- The overall performance is easily influenced by the choice of activation models and activation threshold $\\alpha$. The authors chose the LLaVA-OV-0.5B as this activation model. Please explain the reason and discuss its influence. From Figure 5, it appears that the variation of $\\alpha$ has a significant impact on the overall performance. Does this effect also happen in other benchmarks?\n\n- In Table 2, two results show performance degradation. These results don't match the statement ''we also verify\nthat it does not downgrade the base model’s performance on the standard offline video task in L256. \n\n- In Section 3.2.3, the authors used a heuristic sampling strategy that assigns the last P% frames as positive. It is necessary to discuss the effect of P% under different settings (e.g., set a fixed P% for each sample).", "was_revised": true, "final_justification": "The authors rebuttal has addressed my initial concerns. I will keep positive score for this paper.", "rebuttal": "We thank Reviewer ySNS for the encouraging feedback. We are pleased that the reviewer found the writing clear, the experimental results looking good, and the effectiveness of our constructed dataset. Below, we address your comments.\n\n**(1) Choice of LLaVA-OV-0.5B as the activation model.**  \nWe chose LLaVA-OV-0.5B for its **lightweight** architecture and **fast** inference speed, making it well-suited as a concurrent decision module that introduces minimal computational overhead. We agree with the reviewer that the activation threshold $\\alpha$ plays a role in response frequency. In practice, we empirically set $\\alpha$ within the range of **[0.3, 0.4]**, which we found to strike a good balance between precision and responsiveness across most tasks. This design also enables flexible control depending on downstream application requirements.\n\n**(2) Performance drop in Table 2 (Offline Tasks).**  \nThank you for pointing this out. The observed performance drops (e.g., -2.0 on LongVideoBench and -2.6 on MVBench) are isolated cases out of nearly 21 benchmark-model combinations. We will clarify these exceptions and emphasize the broader non-destructive trend. We also commit to using more accurate and objective language in the final version of the paper.\n\n**(3) Impact of the P% heuristic in activation model.**  \nWe appreciate the reviewer’s thoughtful observation. In our design, we dynamically sample P% between **0–50%** to simulate varied activation delays. This range encourages the model to avoid generating premature responses in the very early portion of a video, where insufficient context would make meaningful responses infeasible. The model is thus trained to wait until it has observed at least **(1–P)%** of the video before considering a response. On average, this means the model typically responds after viewing around **75%** of the video, which aligns with the intuition that robust understanding often requires seeing most of the content. Moreover, dynamic sampling introduces beneficial randomness, helping the model **avoid learning shortcut patterns**, such as always responding at fixed positions and promotes stronger generalization across diverse scenarios.\n\nTo further validate this, we compare two settings on ET-Bench: one with **a fixed P = 25%** and another with **P uniformly sampled from 0–50%**:\n\n| Range of P                  | TVG$_{F1}$ | TAL$_{F1}$ | DVC$_{F1}$ | SLC$_{F1}$ |\n|-----------------------------------------|------------|------------|------------|------------|\n| P = 25%                               | 34.1       | 21.6       | 33.9       | 18.4       |\n| P ~ $\\mathcal{U}$(0, 50\\%) | 34.3       | 24.3       | 37.8       | 22.5       |\n\nAs shown, dynamic sampling of P% leads to improved performance and generalization across tasks, supporting the effectiveness of this design choice."}, {"review_id": "EmuLYA1kSx", "reviewer": "Reviewer_neQ8", "summary": "This paper proposes StreamBridge that transforms offline Video-LLMs into streaming-capable models. StreamBridge designs a memory buffer to efficiently process video inputs and utilizes a lightweight activation model for proactive model responses. Additionally, proposed Stream-IT improves further models for streaming settings. StreamBridge demonstrates its effectiveness on both streaming and offline settings on various benchmarks.", "questions": "See the Weaknesses.", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n- The motivation is clear and straightforward; Existing Video-LLMs mainly explore offline video understanding settings and may not be capable of streaming settings. StreamBride is proposed to fill this gap.\n\n- The modeling is reasonable; A memory buffer with round-decayed compression is able to process video inputs efficiently, and the activation model is highly flexible.\n\n- The paper provides an extensive experiment; StreamBridge has been evaluated on both streaming and general video understanding tasks. With detailed analyses, StreamBridge demonstrates noticeable improvements.\n\n\n**Weaknesses**\n\n- When the embedding length of the memory buffer exceeds MaxLen, how exactly are earlier frame embeddings pooled? The paper mentions 1 FPS video sampling, but it's unclear whether mean pooling is applied only to the first two frames or recursively to earlier segments. A more detailed explanation or visualization would clarify the compression behavior over long sequences.\n- Line 141 suggests the model retains recent frames preferentially. What empirical evidence supports the assumption that recent visual context is more informative than earlier frames?\n- I have some concerns about the activation model. This module resembles a frame sampling strategy that selects the most likely informative frames [1] for a given question and answer. Given that it relies on representations rather than generative capabilities, is an MLLM necessary for this module? Additionally, how do the authors decide the range of P, i.e., 0 ~ 50%? Are all answers in the dataset truly answerable with the last P% of video frames?\n- While Stream-IT improves performance across tasks, it significantly underperforms the base models on the Counting (CT) task in Streaming-Bench Real-Time. What might explain this drop? Does the memory update mechanism or frame prioritization bias affect temporal accumulation required for counting?\n\n**References**\n\n[1] Revisiting the “Video” in Video-Language Understanding, CVPR 2022", "was_revised": true, "final_justification": "The authors have addressed the concerns. I will keep a positive score for the paper.", "rebuttal": "We thank Reviewer neQ8 for the positive evaluation and detailed feedback. We are encouraged by the acknowledgment of our clear motivation, the reasonableness of our modeling choices, and the extensive experimental validation. We address your concerns below.\n\n**(1) Clarification on how early frames are compressed.**  \nWe appreciate the opportunity to clarify the round-decayed compression mechanism. When the total length of visual tokens exceeds the predefined **$\\textbf{MaxLen}$**, we apply recursive average pooling starting from the earliest round. This process ensures that compression proceeds round-by-round, prioritizing the preservation of more recent frames.\n\nTo illustrate this, consider a simple example where **$\\textbf{MaxLen}$ = 200**, each frame contributes **64** tokens (i.e., **tokens-per-frame = 64**), and the current memory buffer consists of the following sequence:\n\n**Memory buffer**: frame$_1$, frame$_2$, frame$_3$, Q$_1$, A$_1$, frame$_4$, frame$_5$, Q$_2$\n\nHere, frame$_1$–frame$_3$ belong to the **first round**, and frame$_4$–frame$_5$ belong to the **second round**. The total number of visual tokens is **64 × 5 = 320**, which exceeds **$\\textbf{MaxLen}$ = 200**. Thus, at least **320 - 200 = 120** tokens must be removed. Since our average pooling reduces frame tokens in units of **tokens-per-frame = 64**, we round up the target to **128** tokens. To achieve this, we apply **average pooling** over the first three frames in the **first round**, collapsing **64 × 3 = 192** tokens into **64** tokens by summing them point-wise and dividing by 3. This reduces the total by **192 - 64 = 128** tokens, bringing the sequence within the **$\\textbf{MaxLen}$** constraint, without compressing visual tokens in the 2nd round.\n\nNow, consider a more constrained case with **$\\textbf{MaxLen}$ = 150**. We would then need to remove **320 - 150 = 170** tokens, rounded up to **192**. As before, pooling the **first round (frame$_1$–frame$_3$)** reduces **128** tokens. However, this is insufficient, so we continue to the second round, pooling frame$_4$ and frame$_5$ into a single frame representation, further reducing **64** tokens. This results in a total reduction of **128 + 64 = 192** tokens, satisfying the compression requirement.\n\nThis **round-wise, progressive** pooling strategy ensures that older rounds are compressed more aggressively while preserving recent, fine-grained visual details, aligning with our round-decay design philosophy. This process is also described with the **pseudo code in Appendix H**, and we will add more explanations in the final version.\n\n**(2) Justification for \"recent frames are more informative\"**  \nWe appreciate the reviewer’s insightful question. The design choice to preferentially retain recent frames is primarily motivated by the nature of streaming video understanding, where decisions are made **incrementally and based on the most recent context**. In such scenarios, recent visual information is often more indicative of the current event or interaction state. For instance, in tasks like action detection or dense captioning, models must react to what is happening **now** rather than rely on distant historical context.\n\nTo validate this empirically, we conducted compression ablation studies as reported in **Table 4**, where we compared our round-decayed compression with alternative strategies, including uniform pooling and FIFO truncation. The round-decay method, which progressively compresses earlier rounds while retaining fine-grained recent inputs, consistently outperformed other approaches across all streaming tasks. This confirms that recent frames indeed carry more task-relevant information in the streaming setup.\n\n**(3) Necessity of Using an MLLM for the Activation Model and the Choice of P%.**  \nOur goal is not merely to identify informative frames, but to decide **when** the model has accumulated **enough multimodal evidence** to confidently produce a response. This is inherently a **causal, streaming process**, where the model **incrementally** observes interleaved video-text inputs over time and must decide whether sufficient understanding has been achieved at each moment. An MLLM is well-suited for this role, as it is trained to perform **causal prediction** (i.e., next token prediction) over sequences of image/video and text. This makes it fundamentally different from offline frame importance estimators (e.g., [1]), which do not model sequential decision-making in a streaming context. Additionally, MLLM is pre-trained on large-scale multimodal datasets and thus benefits from strong **generalization** across diverse visual concepts and task formats. This enables the model to adapt to a wide variety of query types and video domains, including those unseen during activation model fine-tuning.\n\nRegarding the range of P%, we dynamically sample values from **0–50%** to determine which portion of the video contains answer-relevant evidence. This strategy encourages the model to generate responses only **after observing sufficient context**(i.e., **the first (1–P)%** of the video). On average, this means the model responds after seeing around **75%** of the video content. This reflects a realistic streaming behavior, where answers are typically derived from progressively revealed context, and discourages too early activations.\n\n**(4) Counting task underperformance.**  \nWe agree with the reviewer that the performance drop likely stems from the fact that counting tasks inherently require high temporal fidelity and full preservation of fine-grained visual details across time, while our round-decayed compression strategy is designed to prioritize temporal efficiency and compact historical context, which can occasionally lead to information loss in dense, detail-sensitive tasks like counting. To mitigate this, we plan to augment Stream-IT with more counting-style tasks in future iterations, encouraging the model to retain finer temporal granularity when necessary.\n\n[1] Revisiting the “Video” in Video-Language Understanding, CVPR 2022"}]

Author rebuttal:

Meta-review: The authors propose StreamBridge, a framework to convert offline Video-LLMs into streaming-capable models and Stream-IT, a large-scale dataset for streaming video understanding. The contributions of this paper include:
1.  Adapting offline Video-LLMs to real-time streaming scenarios is an important research redirection with broad applications.
2. The authors propose a StreamBridge pipeline based on a memory buffer and a decoupled activation model, which is simple yet powerful on various streaming video understanding benchmarks.
3. The collected StreamingQA-120K dataset is beneficial for the research community for training streaming models.

Below are some of the salient discussion points during the reviewer-author discussion period:
1. Reviewers have concerns about the generalization ability of the activation model, since the activation model is trained on specific data. The authors provide additional results on the NExT-GQA benchmark to demonstrate that the activation model generalizes well.
2. Since the topic of this paper is about streaming video understanding, the authors provide additional metrics (e.g., latency) to demonstrate the computational efficiency.
3. Reviewers have concerns that the authors miss some relevant related work (Reviewer ioHp Q5). The authors promise to add the discussion of these papers in the revised version.
4. The authors provide more implementation details and ablation studies of StreamBridge to address the reviewers' concerns.

Overall, the reviews have been positive, and after rebuttal, all reviewers recommend acceptance, acknowledging that it fills an important gap in the streaming video understanding. The authors are encouraged to incorporate these discussions (related works, implementation details) in the final version.
Decision: Accept (poster)