MMDuet2: Enhancing Proactive Interaction of Video MLLMs with Multi-Turn Reinforcement Learning
Authors: Yueqian Wang
Songxiang Liu
Disong Wang
Nuo Xu
Wan Guanglu
Huishuai Zhang
Dongyan Zhao
Venue: ICLR 2026 Poster (2026)
Keywords: Proactive Interaction
Video Dialogue
Video MLLM
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: Dataset and SFT+RL training framework to enhance response content and timing in proactive interaction for Video MLLMs

Abstract: Recent advances in video multimodal large language models (Video MLLMs) have significantly enhanced video understanding and multi-modal interaction capabilities. While most existing systems operate in a turn-based manner where the model can only reply after user turns, proactively deciding when to reply during video playback presents a promising yet challenging direction for real-time applications. In this work, we propose a novel text-to-text approach to proactive interaction, where the model autonomously determines whether to respond or remain silent at each turn based on dialogue history and visual context up to current frame of an streaming video. To overcome difficulties in previous methods such as manually tuning response decision thresholds and annotating precise reply times, we introduce a multi-turn RL based training method that encourages timely and accurate responses without requiring precise response time annotations. We train our model MMDuet2 on a dataset of 52k videos with two types of dialogues via SFT and RL. Experimental results demonstrate that MMDuet2 outperforms existing proactive Video MLLM baselines in response timing and quality, achieving state-of-the-art performance on the ProactiveVideoQA benchmark.

Published as a conference paper at ICLR 2026
MMDUET2: ENHANCING PROACTIVE INTERACTION OF VIDEO MLLMS WITH MULTI-TURN REINFORCE- MENT LEARNING
Yueqian Wang
Wangxuan Institute of Computer Technology, Peking University wangyseqian@pku. edu. cn
Songxiang Liu & Disong Wang & Nuo Xu & Guanglu Wan
Meituan
fliusongxiang, xunuol9, wangdisong, wanguangjujemeituan. com
Huishuai Zhang & Dongyan Zhao
Wangxuan Institute of Computer Technology, Peking University
National Engineering Research Center of New Electronic Publishing Technologies Maaanghusshaii. zhaodongyan @pku. edu. cn
ABSTRACT
Recent advances video multimodal large language models (Video MI.MMs) have significantly enhanced video understanding and multi-modal interaction ca- pabilities. While most existing systems operate in a turn-based manner where the model can only reply after user turns, proactively deciding when reply during video playback presents promising yet challenging direction for real-time appli- cations. In this work, we propose novel text-to-text approach to proactive inter- action, where the model autonomously determines whether to respond or remain silent at each turn based on dialogue history and visual context up to the current frame of streaming video. To overcome difficulties in previous methods such manually tuning response decision thresholds and annotating precise reply times, we introduce multi-turn RL-based training method that encourages timely and accurate responses without requiring precise response time annotations. We train model MMDuet2 dataset of 52k videos with two types of dialogues via SFT and RL. Experimental results demonstrate that MMDuet2 outperforms exist- ing proactive Video MILM baselines in response timing and quality, achieving state-of-ther the-ahh-rrtt art performance on the Proactive VideoQ benchmark.
INTRODUCTION
In recent years, video multimodal large language models (Video MI.LMs) have advanced rapidly. With increasingly sophisticated video understanding abilities and support for diverse input modali- ties (Li et al. 2024 Zhang et al. 2022bb Bai et al., 2025) Chen et al., 2024b] Zhang et al. 2024c Xu al. 2025). these models being deployed across expanding range of real-world applications.
Besides turn-base based interaction where the model can only reply after the user's turn, proactive in- teraction has emerged
promising and actively studied paradigm recently (Chen al.) 2024a Wang et al. 2024; Qian et al. 2025 Yao et al.] 2025). Proactive interaction is a more advanced requirement than online video conversation: it requires the model to not only understand interleaved visual and dialogue content, but also to determine on its own when to answer with appropriate con- tent during the video playback. Achieving this requires continuous monitoring of visual and textual streams, real-time detection of salient events, and the ability to deliver timely, contextually relevant responses. Such proactive video MILMs hold strong potential for real- time applications, including
"Corresponding Authors

---

Published as a conference paper at ICLR 2026
live-streamt analysis, intelligent surveillance, egocentric assistance agents, and socially interactive Al agents.
In previous works of proactive interaction (Chen et al. 2024a Wang et al. 2024 Qian et al. 2025), a video MLLM determines whether it should respond after a certain frame by predicting response probability scores, such as using additional modules, the probability of special token, or the visual token drop ratio, and compares the scores with a pre-defined threshold. However, there are two issues that remain:
(1) A threshold must be manually set during inference, and the model may never reply or often reply with duplicated content if this threshold is not properly. To alleviate this problem, we an entirely text-based approach to solve the problem of reply timing prediction: in each user turn, the user provides an optional textual content along with small amount of visual information frames from the online video), after which the assistant automatically initiates its own turn, The assistant can choose to output either textual response or "NO REPLY" to indicate it does not want to reply right after this frame.
(2) Existing methods use supervised fine-tuning to train proactive interaction models, where exact reply timestamps for each model reply are required to construct training data, which is difficult to acquire as discussed later Section 4.3.T Prior studies typically insert responses either at the end of scene at a random position in the latter half of the scene construct video-text interleaved dialogue data, followed by supervised fine-tuning during post-training. Due to difficulties in compu- tational resources and data processing pipelines, scene segmentation is usually not too fine- -raiinadd,,, making it difficult to insert responses after the exact frame where the relevant visual information appears, which hinders the timeliness of model responses.
In this work, we leverage reinforcement learning to address this issue: with a niche reward design, encourage the model to generate correct responses as early as possible while penalizing it for producing incorrect or excessively delayed responses. In this way we can significantly enhance the model's 's response timing without the need annotate the precise timestamp of each model reply in the training data.
Using a carefully crafted proactive dialogue construction pipeline, we construct around 52k videos from YouTube and Ego-Centtr Centric videos, and two types of dialogue data: one question, multiple answers (IQnA) and multiple questions, multiple answers (nQnA). Based on this dialogue data, we trained MMDuet2 using SFT+RL, resulting in a state-of-the-art proactive video MLLM that achieves significant improvements over existing proactive model baselines in both response timing VideoO..
and response quality, and has state-of-the-ar performance
In summary, the contributions of this work include: (1) An RL-based training method that can significantly improve proactive interaction experience without requiring precise reply timestamp in training data, (2) A pipeline for constructing proactive dialogue from videos, and a dataset consisting of 52k nigh-quality and diverse proactive dialogues, and (3) MMDuet2, video MLLM that has state-of-the-aat performance on proactive video QA benchmarks and provides a better interaction experience.
RELATED WORKS
2.1
PROACTIVE INTERACTION WITH VIDEOLLM
-idelLeM-Online (Chnn et al. 2024a is among the earliest efforts to adapt video-text MLLMs for proactive question answering. MMDuet (Wang et al. 2024) is trained on wider range of tasks and datasets, achieving much better experimental results, but still struggles with issues such as inaccurate response timing and redundant outputs. Dispider (Qian et al. 2025) proposes a disentangled frame- work for proactive interaction separating perception, decision, and reaction, and TimeChat-Online Yao et al. (025) emphasizes token compression when processing input video streams. ProactiveV. ideoQA (Wang et al. 2025) proposes comprehensive benchmark for proactive question answering evaluation, and proposes PAUC, an evaluation metric that is specially optimized for the fact that the model may give different responses at different times during proactive interaction.
2

---

Published as a conference paper at ICLR 2026
Pj
Pg
P2
video socnes
IQnA q¹
a]
42
nQnA
gh
a4
ai...
ag
10s
20s
30s
video time
Figure 1: A conceptual demonstration of the proactive dialogues in the proposed dataset.
Many works focus on proactive reply in other tasks besides video question answering. LiveCC (Chen al. 2025a) generates real-time video commentaries, Ego-Speak (Kim et al. 2025) studies speech initialization in face-to-face conversations. ViSpeak (Fu et al., (222) focuses on recognizing body movements in the video to trigger specified responses, and (Panchal et al. 2024) studies providing timely feedback to fitness exercisers.
2.2 REINFORCEMENT LEARNING ON VIDEOLLM
Reinforcement learning has begun to play transformative role post-training video-text multi- modal language models, moving beyond purely supervised strategies. One of the prominent meth- ods, VLM-RLAIE (Ahn et al.] 2024) uses Reinforcement Learning from AI Feedback to align video and text representations by automatically generating self-preference feedback, bolstered by context- aware reward modeling that improves video grounding during instruction tuning. Another pioneer approach, Video-R1 (Feng et al. 2025), introduces a temporal extension to rule-based reinforce- ment learning T-GRPO, which explicitly incentivizes models to leverage correct frame order, help- ing better capture the temporal dynamics of video data. VideoChat-RI (Li et al., 2025) further advanced this area by using Reinforcement Fine-Tuning with GRPO to boost spatio-temporal per- ception. Long VILA-R1-7B (Chen al.) 2025b) employs two-stage pipeline, chain-of-thoughh SET followed by RL, and uses sequence parallelism to extend RL training on long videos. R1-Omni (Zhao et al. 2025)) integrates vision, audio, and language, adopts Reinforcement Learning with Veri- fiable Rewards (RLVR) and GRPO to train models that can recognize emotion in multimodal inputs. However, existing RL-enhanced VideoMLLMs have not explored real-time interaction or multi-tury dialogue, limiting their applicability in more interactive scenarios.
DATASET CONSTRUCTION
The videos of our proposed dataset contain two major categories: web videos and ego-centric videos. A dataset statistics is shown in Table I We use the following process construct proactive QAs and their corresponding timespans in the video:
(1) Scene segmentation and captioning. Each video V is first divided into list of scenes [V1, U2, ................................ ",), ,',], and we get a detailed scene caption for each scene: [c1, ... ...... [.]] We use differ- ent methods to obtain high-quality scene boundaries and captions for different categories of videos. For web videos from Live-Whispa WhisperX, as this dataset has good correspondence between video con- tent and subtitles, thanks to its elaborate data cleaning process, use the temporal boundaries of sentences in the subtitle as the boundary of the video scenes, and use frames sampled from this scene along with its subtitle as input to an MLLM acquire detailed caption. For ego-centric videos, as these datasets have detailed segment-level annotations, we directly use these annotations. We aim to segment videos into scenes that have relatively independent and clear video content, and these scenes occupy the vast majority of the time in the video, though they may not be connected end to end in time.
(2) QA generation. We use all scene captions as input, and instruct an LLM to generate a question q and list of n answers [a..... an]. each a; corresponds to an scene v, and is derived from a scene caption e that can answer the question q. If the information in e; can not answer 9. then a, is set to "NO REPLY". We generate 2 to -nnsstion-answer lists for each video according to its video length and use superscripts q², 2 at, of to distinguish them in the rest of this paper.
3

---

Published as a conference paper at ICLR 2026
Video length
#Ques
#Answer turns
Type
#Videos
Video source
-tons
Web Videos
50228
92.7
2.0
6.7
Live- Fre-WhisperX (Chen et al. 2025a
Ego-Exo4D (Grauman al.
2023)
Ego Centic
2543
164.4
2.1
5.6
EgoExoLearn (Huang el al.
2024
Table Dataset Statisties.
(3) Proactive dialogue construction. We prepare 2 different types of proactive dialogues: "one question, multiple answers" (IQnA) and "multiple questions, multiple answers" (nQnA), each type covers half the number of all videos. A conceptual demonstration of the proactive dialogues is shown in Figure In IQnA dialogues, construct one training example. The user asks the question q¹ at the beginning of the video, and the model should reply with an answer a² within the timespan of its corresponding video scene v;. In nQnA dialogues, all of the 2-4 questions and answer lists used construct one dialogue. The user can ask any question q¹ at anytime. We use an LLM to summarize all answers of scenes that ends before the question time, i.c., [ai, , into one "immediate answer" a (1,..., ,t-1)" and the model should reply with this immediate answer at the time when the question is raised by the The answers of the following spans [a/ I they are still required to be replied within the corresponding scene until the next question is raised, then the model should then start to reply with the answers for the next question.
question q¹ and one answer list [a o² is used to
4 TRAINING PROCESS
4.1 FORMULATING PROACTIVE DIALOGUE WITH CHAT TEMPLATE
< imm_start/ Sospartt/>systmmmmYo are helpful assistant. Your task to answer questions based continuously incoming video frames. Your responses should include information from the video since your last reply (if any) If the information in this segment of the video cannot answer the question, output "NO REPLY", in]__edd)
<
are people doing in
officce2//i_end]
< / R
< ||ingttarttjaassssttttttttoooop are working at desks with
annputers_and..mm monitors..moomomommmoooo.oo engaged in various tasks. im >
llim_startt wwssssstnttsttt......... h is speaking, people are busy < at their desks with computers and monitors. end
Chat template of MMDuct2. User turns are marked in orange, assistant turns are marked in blue, and the textual contents of the dialogue between the two roles are underlined for the conve- nience of reading.
Figure 2:
The chat template of proactive interaction we use is shown in Fig. 2 It proceeds in the following process:
1. First, we use a customized system message to indicate a proactive dialogue. This not only pro- vides the model with the rules for its future responding, but also distinguishes proactive and offline video tasks with different contexts to reduce the catastrophic forgetting offline under- standing tasks during proactive training.
4

---

Published as a conference paper at ICLR 2026
Figure 3: An example of a typical video snippet in dataset processing. Video frames circled by the green polygon constitutes a video scene.
2. The user inputs a message, which includes few (1 or 2 in this paper) frames from the video, or a text input, or both frame and text.
3. In the assistant's 's turn, the model can choose to generate some text content as reply, or generate "NO REPLY" indicate that does not want to reply this round.
When the assistant's 's turn ends, the user retakes the floor and inputs a message containing frames or text. This loop continues until all sampled frames from the video have been input.
Within this chat template, the timestamp for each user obtained by multiplying the number of frames preceding this turn by the time interval between consecutive frames. For instance, with frame sample rate of frame per second, the conversation in Figure denotes the user says "What are the people doing in office?" at the 2nd second, the model replies "People are working..." at the 4th second and "A reporter is speaking..." the Sth second.
assistant turn in the video can be
Different from previous works like (Chen et al. 2024a Wang et al. 2024), major advantage of the chat template used in MMDuet2 is that it formats the entire interaction process, including video input, user input, reply time decision, and reply content generation, into messages from the user or the assistant and therefore compatible with almost all popular post-training and inference frame- works. We know that there are more efficient strategies for reply time decision, but these methods require extensive modifications to model architecture and code frameworks, which would take substantial labor to implement. We leave these discussions of potential methods for reply timing decision in the appendix.
4.2 SUPERVISED FINE-TUNING
We use Qwen2.5-VL. 3B Bai et al. 2025) as initialization. We hold out a few (1500 web and 400 ego-centric) videos for RL training, and use the rest of the dataset in the SFT training phase. The input frames are sampled at an interval of seconds from the video and we use 128 tokens per frame, frames per user turn. To build user-assistant conversations used in the SFT stage, we place model answers at the end of their reply timespans. This is to ensure that the relevant event has already occurred when making reply (which is supposed to happen within the reply timespan, i.e., the corresponding video span) to avoid introducing hallucinations. To maintain the general offline video understanding abilities, we also include 25k offline video QA data from LLLVA- Videeo (Zhang et al.) (024d) and 25k video captioning data from tarsier2 (Yuan et al., 2025) in the SFT stage. These training examples are formatted using Qwen2.5-Ve VL's default chat template and system prompt. This training process is conducted on 16 H800 GPUs and takes about 8 hours.
4.3
RL TRAINING
4.3.1
MOTIVATION USING RL
Although the model trained only with SFT has gained the ability for proactive responses, its per- formance is still unsatisfactory. We identified two main issues: First, the frequency of responses is relatively low. This may be because in the supervised training data, most of the turns are NO RE- PLY, causing the model to learn a bias towards this distribution. Second, the model often generates a response several seconds later than the key information appears, giving the user an experience of a long system delay. Automatically annotating ground-truth response time has been unsolved chal- lenge. For example, as shown in Figure 3 the caption for the scene circled by the green polygon is "Tamarind, fish sauce and sugar are added heated pan and mixed using spatula." ", which only
5

---

Published as a conference paper at ICLR 2026
provides coarse-grained scene-level annotation. It still remains unclear at which specific frame each ingredient appears so the model can generate a reply about this specific item in time. Pursu- ing overly ine-grained event timestamp would otherwise bring challenges in scene segmentation techniques and dataset construction cost.
Although providing accurate ground truth reply times is difficult, it is much easier to determine which of the two given proactive interaction outputs is better. An ideal proactive interaction system should generate replies both correctly (quantified by the text similarity between the model-generated answer and the ground truth answer) and early. More specifically, within each ground truth span, the preferences of the two responses should meet the following requirements: (1) With the same response time, the reply with higher correctness is more favorable; and (2) With the same increase in correctness induced by a new reply, the reply that comes earlier is more favorable. Therefore, an intuitive solution is to encourage the model to generate more favorable interactions through GRPO training with targeting rewards, circumventing the need to set ground truth response times. In this way, we can train the model to find the appropriate earliest response time by itself, since the model cannot generate a response about an event before actually observing it.
4.3.2 REWARD MODELING
Formally, let video contain G turns of ground-truth replies, where each turn consists of a textual response gold, and corresponding reply timespan (tstart, (end) for 1,2...., G. This means that during the interval (tstart, ,k ).nd), ), the user expects to receive the information conveyed by goldg- Within each reply timespan (tstart, gend) (here we omit the subscript "g" for simplicity). model M generates P model responses, each associated with text predp and a timestamp Tp. where = 11,..... and sstart < Ti < T2 < can be calculated upon each time the model generates new reply (usually calculated by an LLM using the ground truth text and model response text as input), resulting in list of correctness scores 81, 82, 8pp 8p, where 5 is predefined max score.
TP < gend A correctness score Sp E [0, S]
The reward is inspired by the PAUC (Proactive Area Under Curve) (Wang et al. 2025) metric. A brief demonstration of the PAUC metric is shown in Figure 4 We plot the change in the model's response score over time as polyline with add a small score of 0.5 as the initial score at timestamp gstart, the reason behind this is that subsequent output gets minimum score of = 0, it will result in worse PAUC metric than outputting nothing at all. PAUC is computed as the ratio between the area under this polyline and the maximum possible area:
the x-axis and on the y-axis. In particular, we
PAUC ( - (stort) 0.5 Eper (Tp+1 Tp) 8p + (gend Tp) sr)
(I)
(tend tstart) x S
PAUC satisfies both of the abboe-mentioned requirements. As illustrated in Figure 4 increasing the
score of a reply ((72, 82) - (72, 86)) raises the height of the polyline on the y-axis, thereby yielding
larger area under the curve. If reply can achieve a higher score than the previous one, the earlier this reply is made ((Tg,s2) )) + (T2. $2)), the earlier the polyline rises on the y-axis, which also results in larger area under the curve.
To better reflect the differences between responses in the reward and amplify the advantage of differ- ent rollouts for easier training, we made two minor modifications from the original implementation: we use max score S = 4 instead of 2, and each time when calculate the similarity between model reply and the ground truth, we only use the current turn of reply instead of all previous turns in the ground truth reply span in the original implentation of the PAUC metric. The modified PAUC reward is denoted as TPAUC- Due to space limitations, for more details about PAUC please refer to its original paper.
Besides TPAUC, we also use some additional reward to punish unwanted behaviors, mainly related to generating redundant and duplicate replies: (I) Replication reward (Trep): To prevent the model from generating replicated replies as reported as a series problem in (Wang et al. 2024 2025), ) and encourage the model to focus new information in incoming videos, for each model reply, an LLM to judge whether all information in this reply is already covered in previous replies. We use the inverse of the ratio between the number of already-covered reply entries and the total number of

---

Published as a conference paper at ICLR 2026
[WEB]
[EGO]
[TV]
[VAD]
MieooLLM-Onlie
25.9/
25.0/-
18.3/53.9
25.0
MMDuet
31.3
46.0 49.0
21.1/92.8
27.4/99; 99.2
MMDuet2 (Ours)
/
26.4 4.4 4.4
2..6/2.2
26.3 0.0
MMDuet2 (Ours)
53.3/
33.6/ 8.1
43.4/ 1.0
28.9 / 15.2
Table 2: Performance on ProactiveVideooQA Metrics reported are PAUC (w = 0.5) // reply dupli- cate proportion J, as defined in (Wang al. 2025). t; Videollm-online generated more than 1 reply for only less than 10 answer turns on the [WEB], [EGO], and [VAD] datasets. Since the sample size small, we reporting this result they have overly-large variance.
reply entries as Trep. (2) In-span reward (nin.span): To prevent the model from generating replies during video spans unrelated to the question, we use the inverse the ratio between the number of replies that do not fall in any ground truth reply span and the total number of reply entries as Tin (3) Prefix reward (Tpfz): We found that when generating new replies, sometimes the model may repeat the previous replies before adding new content, making the replies more verbose than necessary. To prevent this issue, for each turn of reply we calculate the longest common prefix between this reply and all previous replies, and mark the replies with the longest common prefix larger than threshold as "verbose prefix reply". We use the inverse of the ratio between the number of verbose prefix replies and the total number of reply entries as Tpfz.
As shown in Eq. 2 weighted sum of the rewards as the overall reward:
we use WPAUC, Wreps Win.spans and Wpfz to control the weights and use the
== WPAUC X TPAUC + Wrep X Trep + Win.span X Tin.span + Wpfz X Tpfæ
(2)
The first term WPAUC X TPAUG more likely to assign higher reward to samples with more reply turns, while the latter terms Wrep Trep + Win.spon Tin_span + Wpfz 2 X Tpfæ are more likely to assign higher reward to samples with less rewards, forming a tradeoff between informativeness and simplicity. In practice, former slightly stronger than that of the latter. This allows the model to gradually shift from the low-frequency, high-latency replies after solely training with SFT, toward generating more frequent and timely proactive replies, while without resorting to reward hacking TPAUE by producing a large number of redundant replies. After some hyperparameter search we find that wPAUC 3, Wrep 2, Win.spon = 0.5, Wpfa = 2 is good and use these hyper-parameter parameters in the subsequent experiments.
found that good weighting scheme should make the influence of the
4.3.3 TRAINING DETAILS
Since a complete video corresponds to multiple ground truth reply spans, performing rollout over the entire video and providing only single averaged reward would result very sparse rewards. Moreover, as the rewards for different ground truth reply spans are computed independently. using an average reward will introduce a temporal credit assignment problem (Sutton) 1984), making difficult to attribute the final reward to specific ground truth reply spans. To alleviate this problem, in each step we only select a short span (from 20 to 60 seconds) from the video for training and provide ground truth model replics for the dialogue turns that happen before the selected span. We sample frames with an interval of 2 seconds from the video and use 128 tokens per frame, 2 frames per user turn. We use GRPO (Shao et al.] 2024) with number of rollouts 4, implemented with SGLang Zheng et al. 2024) and verl (Sheng et al. 2025) framework. This training process is conducted on 8 H800 GPUs and takes about 20 hours.
EXPERIMENTS
5.1 EXPERIMENTS ON PROACTIVE BENCHMARKS
Performance existing benchmarks, ProactiveVideoQA WWan et al. 2025) and StreamingBench proactive output (Lin et al.) 2024) task (PO), are listed in Table 2 and S Given the deployment of
proactive interaction can be very complex, reproducing without the official proactive inference code
7

---

Published as a conference paper at ICLR 2026
Video-MME
MVBe
Long Vid
# Reply Turns 5.7 (3.4)
Wall
(w/wo sub) 67.6/61.5
-nch
-eoennch
Time
Qwen2.5-VL 3B
67.0
54.2
MMDuet
2m27s
Qwen2.5-VL. 3B
66.3/57.3
65.6
53.1
MMDuet2
3.3 (1.9)
2m52s
MMDuet2 sft (Ours)
67.1/57.7
65.3
53.3
MMDuet2 (Ours)
67.5/58.1
66.4
52.7
Table 3: Inference Wall Time on [WEB].
Table 4: Performance on several popular offline video under- standing benchmarks. 1; Our implementations.
[WEB]
[EGO]
MMDuet2
53.3/4.2/3.3
33.6/8. 1/3.5
Acc
VideoLLM-OOnlnne
1.96
-Tep
55.5/17.3/4.9
35.6/31 19800
Dispider
25.34
Tpfz
53.0/4.3/3.1
27.5/2.3/0.6
MMDuet
29.44
Tin.span
62.7/9.6/8.4
FAIL"
MMDuet2 */t (Ours)
19.59
MMDuet2 ri (Ours)
34.69
Table 6: Ablation studies on each reward item. Metrics re-
ported are PAUC1/ reply duplicate proportion J/num reply turns. Adverse consequences caused by removing a reward item are marked in red. "FAIL: The model generates response almost every turn, regardless of whether truly relevant to the question. Evaluation this task is unfeasible inference on a single data point can take more than 20 minutes.
Table 5: Performance on Proac- tive Output task of Streaming- Bench.
could lead to suboptimal results, potentially leading to misunderstandings about the capabilities of those models. So we compare with MMDuet (Wang et al., (024) and JideoLLM-Onlingg (Chen al., 2024a) as only these two models have open-sourced code for proactive evaluation. As the videos in [WEB] of ProactiveVideoO2 and PO of SteamingBench are relatively short in length (16.59 and 13.14 secs respectively), we use frame per user turn during inference. For the other three tasks ([EGO), [TV), and [VAD]) with longer videos, we use frames per user turn.
Results show that MMDuet2 outperforms existing proactive interaction models by large margin. Although MMDuet achieves good performance tasks such [ECO], it achieves so by generating a large number of unnecessary and repetitive replies to fulfill the goal of "conveying useful informa- tion early possible". However, all models perform relatively poor on [VAD], demonstrating that current models still struggle to understand surveillance videos. Some real proactive dialogue case videos can be found in the supplementary material.
Inference Speed. Here we report the actual inference speed (wall time) of the [WEB] task for MMDuet and MMDuet2. We take the following measures to ensure the comparison is as fair as possible: We use the same computational node with an H100 GPU while maintaining GPU utiliza- tion at 100%, select 64 samples from the ProactiveVideode time. Results shown in Table demonstrate that inference time is comparable though performing complete generate operation to produce "NO REPLY" at every decision.
[WEB] task and test inference wall
2010 Pock
(
Eiot spenton
(g)-
N0 0.5)
(6.5)
ptart
Ty
Figure 4: An illustration of the cal- culation of the PAUC metric (Wang
Figure 5: Dynamics of key metrics of model behavior dur- ing RL training.
al. 2025)
8

---

Published as a conference paper at ICLR 2026
[WEB]
[EGO]
SFT frame interval
1 sec
2 secs
2 secs
RL frame interval
sec
2 secs
sec
2 secs
Inference
sec
FAIL FAIL
47.0/1.0
53.3/4.2
34.7/3.7
33.6/8.1 33.5/7.5
frame interval
2 secs
39.4/0.0
44.2/1.7
30.6/1.8
Table 7: Performance of using different frame interval for SFT, RL and inference.
5.2 EXPERIMENTS ON OFFLINE VIDEO- TEXT BENCHMARKS
To verify whether our training introduces any negative impact on offline video understanding tasks, we also report performance on several widely used offline video understanding benchmarks: Video- MME (Fu al. 2024), MVBench (Li et al. 2023), and Long VideoBench (Wu et al., (024). When testing on these offline video understanding benchmarks, we use the default system prompt of Qwen2.5-VL ("You are a helpful assistant.") instead of our customized system prompt stated in fig. 2. We also report the evaluation results of our implementations, which use Imms-eval frame- work (Zhang et al.) 2024a) for evaluation and set max tokens per frame as 256. This is to ensure a fair evaluation of the impact of the post-training process proposed in this paper, it has been ported that there are big gaps between the reproduced results using Imms-eval and the performance reported in the original paper (Bai al. 2025). Results are shown in 4 After fine-tuning and inforcement learning for enhancing proactive interaction, MMDuet2's performance on offline video understanding benchmarks remains almost the same as the checkpoint before our post-training.
5.3 ABLATION STUDIES
Impact of Each Reward Item. We demonstrate the necessity of Trep, Tin_span and Tpfæ in Table 6 Results show that Trep and rin. are indispensable: without any of these 2 rewards, the model generates more duplicated responses to achieve an unreasonably high PAUC metric. Moreover, without rin_spans the model significantly increases the response density, which makes unfeasible to evaluate on long videos in [EGO].
The impact of Ppfz f is more complicated. During training, we observed an undesirable pattern where the model would first generate verbatim repetitions of the previous response before continuing to generate new content. Experimental results also show that without this reward, the model may fail on the more difficult [EGO] test set. Therefore, we add this reward penalty.
Impact of frame sample density in training and inference is different. The frame sampling density during training and inference can be a potentially critical factor influencing the interactive experience. We experimented with different frame intervals during SFT, RL, and inference phases, and the experimental results are shown in Table 7 In SFT phase, when frame interval is set to second, the model will collapse always generating "NO REPLY" in every reply turn, as the training data is overly biased towards not replying, so mark these results as "FAIL" in Table 7 and use 2 sec frame interval for SFT in subsequent experiments. In the RL phase, we found that setting different frame intervals does not have a significant impact on performance. However, in the inference phase, we found that reducing the frame interval from seconds to 1 second leads significant performance improvement. The underlying reason is that a lower frame interval leads to a higher decision rate, allowing the model to perceive the appropriate response timing earlier, which is more favorable both for user experience and for the PAUC metric. This also demonstrates the robustness of MMDuet2 to different frame sampling strategies.
We observe that the proactive interaction training framework proposed in this work demonstrates strong generalization performance with respect to the frame sampling interval. Therefore, we rec- ommend using relatively frequengycy training frame rate (2 seconds per frame) to save training costs, and a relatively high-frequency inference frame rate (1 second per frame) to achieve better interaction performance.

---

Published as a conference paper at ICLR 2026
5.4 TRAINING DYNAMICS OF THE RL PROCESS
In this subsection, aim to present the changes in model behavior during the RI. training process. In Figure [S] we show line charts of several key metrics during proactive video QA testing: the model's average number of response turns, PAUC score, and repetition. From the line chart we can clearly identify that the RL process can be divided into three stages:
Stage (step 0-180) shows a transition period, both the number of responses and the performance decline. As the training objectives and encouraged response patterns of SFT and RL differ, the model in the process of switching from the old to the new response pattern during this stage, which leads to slight drop in performance.
Stage (step 180-450) shows growth period, the model, guided by the training objective, learns to generate earlier and more accurate responses while avoiding overly frequent or repetitive replies. During this stage, the model's response frequency and PAUC performance increase rapidly, and repetition remains generally controllable.
Stage (step 450-489) shows a plateau period, the model's performance on the [WEB] network video task stabilizes. However, on the [EGO] ego-centric video task which is longer and more challenging for content understanding, the model can have some generalization issues as we observe an increase in repetition. We believe this can be alleviated by collecting more ego-centric and long- form videos as training data, which will be an important direction for future work.
6 CONCLUSION
Moving beyond conventional user-turn-initiated conversation paradigms, in this paper, we stud- ied improving proactive interaction of video multimodal large language models, which enables the model to autonomously decide when to respond during video playback. We constructed a large-scale proactive dialogue dataset comprising 52k videos with diverse question-answw structures, facilitat- ing robust training. We proposed a method to represent the proactive interaction process solely using messages between the user and the assistant without modifying the model structure, which allows our model to be directly adapted to most training and inference frameworks, facilitating hands-on usage.
By integrating reinforcement learning with specialized reward design, we train MMDuet2, a proac- tive Video MILM that significantly enhances the correctness and timeliness of proactive dialogue while reducing redundant and repetitive replies. MMDuet2 demonstrates superior performance over existing baselines on several proactive output tasks without having degraded performance on offline video understanding benchmarks.
Future research directions include: (1) Collecting more diverse data, such as teaching and learning processes, to train a more versatile proactive Video MLLM that goes beyond solely QA, (2) Inte- grating with techniques like visual token compression to cut down on computation expenses, and (3) Combining with speech understanding and generation to extend proactive interaction capabilities to more modalities.
ACKNOWLEDGMENTS
This work is supported by National Science Foundation of China (No. 62576016, 62576015) and Beijing Natural Science Foundation (L253001).
REFERENCES
Daechul Ahn, Yura Choi, Youngjae Yu, Dongyeop Kang, and Jonghyun Choi. Tuning large multi- modal models for videos using reinforcement learning from Al feedback. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 923-940, Bangkok, Thailand, Au- gust 2024. Association for Computational Linguistics. doi: 10. (0..18653/11/20 1/25////11/222./e// a.l.aallllla 1ong. URL 52.
httpss://ach anthoo http://ccllantoooooooooo////////
10

---

Published as a conference paper at ICLR 2026
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang. Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding. Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report. ArXiv, abs/2502. 13323, 2025.
Joya Chen, Zhaoyang Lv, Shiwei Wu, Kevin Qinghong Lin, Chenan Song, Difei Gao, Jia- Jei Liu, Ziteng Gao, Dongxing Mao, and Mike Zheng Shou. Videollm-onling Online video large lan- guage model for streaming vidco. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 04407-1448, 2024a.
Joya Chen, Ziyun Zeng, Yiqi Lin, Wei Li, Zejun Ma, and Mike Zheng Shou. Live: Ilm with streaming speech transcription at scale. 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 29083-29095, 2025a.
Learning video
Yukang Chen, Wei Huang, Baifeng Shi, Qinghao Hu, Hanrong Ye, Ligeng Zhu, Zhijian Liu, Pavlo Molchanov, Jan Kautz, Xiaojuan Qi. Sifei Liu. Hongxu Yin, Yao Lu, and Song Han. Scaling RL to long videos. CoRR, (00/02507..7766 2025b. doi: 10048550/ARXI..557.77799 https://don wrg/10.................o........ 10..1..85550 48550 www.v..rvvvv/..a..va 2507 07966
URL
Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shen- glong Ye, Hao Tian, Zhaoyang Liu, Lixin Gu, Xuchui Wang, Qingyun Li, Yiming Ren, Zixuan Chen, Jiapeng Luo, Jiahao Wang, Tan Jiang, Bo Wang, Conghui He. Botian Shi, Xingcheng Zhang, Han Lv, Yi Wang, Wenqi Shao, Pei Chu, Zhongying Tu, Tong He, Zhiyong Wu, Hui Deng, Jiaye Ge. Kaiming Chen, Min Dou, Lewei Lu, Xizhou Zhu, Tong Lu, Dahu Lin, Yun- feng Qiao, Jifeng Dai, and Wenhai Wang. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. ArXiv, abs/2412.052711, 2024b.
Kaituo Feng, Kaixiong Gong, Bohao Li, Zonghao Guo, Yibing Wang, Tianshuo Peng, Junfei Wu, Xiaoying Zhang, Benyou Wang. and Xiangyu Yue. Video-RI: Reinforeing video reasoning in MLLMs. arxiv preprint arXiv:2503.21776. 2025.
Chaoyou Fu, Yuhan Dai, Yondong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, Peixian Chen, Yanwei Li, Shaohui Lin, Sirui Zhao, Ke Li, Tong Xu, Xiawu Zheng, Enhong Chen, Rongrong Ji, and Xing Sun. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal Ilms in video analysis. ArXiv, abs/2405.21075, 2024.
Shenghao Fu, Qize Yang, Yunn-Ming Li, Yi-Xing Peng, Kun-Yu Lin, Xihan Wei, Jianfang Hu, Xiaohua Xie, and Wei-Shi Zheng. Vispeak: Visual instruction feedback in streaming videos. ArXiv, abs/2503. 12769, 2025.
Kristen Grauman, Andrew Westbury, Lorenzo Torresani, Kris Kitani, Jitendra Malik, Triantafyl- los Afouras, Kumar Ashutosh, Vijay Baiyya, Siddhant Bansal, Bikram Boote, Eugene Byrne, Zachary Chavis, Joya Chen, Feng Cheng, Fu-Jen Chu, Sean Crane, Avijit Dasgupta, Jing Dong, María Escobar, Cristhian Forigua, Abrham Kahsay Gebreselasie, Sanjay Haresh, Jing Huang, Md Mohaiminul Islam, Suyog Dutt Jain, Rawal Khirodkar, Devansh Kukreja, Kevin J Liang, Jia-Wet Liu, Sagnik Majumder, Yongsen Mao, Miguel Martin, Effrosyni Mavroudi, Tushar Na- garajan, Francesco Ragusa, Santhosh K. Ramakrishnan, Luigi Seminara, Arjun Somayazulu, Yale Song, Shan Su, Zihui Xue, Edward Zhang, Jinxu Zhang, Angela Castillo, Changan Chen, Xinzhu Fu, Ryosuke Furuta, Cristina Gonzalez, Prince Gupta, Jiabo Hu, Yifei Huang, Yim- ing Huang, Weslie Khoo. Anush Kumar, Robert Kuo, Sach Lakhavani. Miao Liu, Mingjing Luo, Zhengyi Luo, Brighid Meredith, Austin Miller, Oluwatumininu Oguntola, Xiaqing Pan, Penny Peng, Shraman Pramanick, Merey Ramazanova, Fiona Ryan, Wei Shan, Kiran Soma- sundaram, Chenan Song, Audrey Southerland, Masatoshi Tateno, Huiyu Wang, Yuchen Wang, Takuma Yagi. Mingfei Yan, Xitong Yang. Zecheng Yu, Shengxin Cindy Zha, Chen Zhao, Zi- wei Zhao, Zhifan Zhu, Jeff Zhuo, Pablo Arbeláez, Gedas Bertasius, David J. Crandall, Dima Damen, Jakob Julian Engel, Giovanni Maria Farinella, Antonino Furnari, Bernard Ghanem, Judy Hoffman, C. V. Jawahar, Richard A. Newcombe. Hyun Soo Park, James M. Rehg. Yoichi Sato, Manolis Savva, Jianbo Shi, Mike Zheng Shou, and Michael Wray. Ego-exo4d: Un- derstanding skilled human activity from first- and third-person perspectives. 2024 IEEE/CVF
11

---

Published as a conference paper at ICLR 2026
Conference on Computer Vision and Pattern Recognition (CVPR), pp. 19383-19400, 2023. URL https://api. semanticschooeee
Yifei Huang, Guo Chen, Jilan Xu, Mingfang Zhang, Lijin Yang, Baoqi Pei, Hongjie Zhang, Lu Dong, Yali Wang, Limin Wang, and Yu Qiao. Egoexolearn: A dataset for bridging asyn- chronous ego- and exo-centric view of procedural activities in real world. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 22072-22086, 2024. URL https://apt
Junhyeok Kim, Min Soo Kim, Jiwan Chung, Junghin Cho, Jísoo Kim, Sungwoong Kim, Gyeongho Sim, and Youngjae Yu. Egospeak: Learning when to speak for egocentric conversational agents in the wild. ArXiv, abs/2502. 14892, 2025.
Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng. Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023.
Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. Llava-onevision. Easy visual task transfer. ArXiv, abs/2408.03326,, 2024.
Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, Limin Wang, and Yu Qiao. Mybench: A comprehensive multi-modal video under- standing benchmark. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 22195-22200 2023.
Xinhao Li, Ziang Yan, Desen Meng, Lu Dong, Xiangyu Zeng, Yinan He, Yali Wang, Yu Qiao, Yi Wang, and Limin Wang. VideoChat-RI: Enhancing spatio-temporal perception via reinforce- ment fine-tuning. arXiv preprint arXiv:2504.06955 April 2025.
Junming Lin, Zheng Fang, Chi Chen, Zihao Wan, Fuwen Luo, Peng Li, Yang Liu, and Maosong Sun. Streamingbench: Assessing the gap for milms to achieve streaming video understanding. ArXiv, wabs/2411.036m 1.03228, 2024.
Sunny Panchal, Apratim Bhattacharyya, Guillaume Berger, Antoine Mercier, Cornelius Bohm, Flo- rian Dietrichkeit, Reza Pourreza, Xuanlin Li, Pulkit Madan, Mingu Lee, Mark Todorovich, Ingo Bax, and Roland Memisevic. What to say and when to say it: Live fitness coaching testbed for situated interaction. In Neural Information Processing Systems, 2024.
Rui Qian, Shuangrui Ding, Xiao wen Dong. Pan Zhang, Yuhang Zang, Yuhang Cao, Dahua Lin, and Jiaqi Wang. Dispider: Enabling video lims with active real-time interaction via disentangled perception, decision, and reaction. ArXiv, 00//2511.32111 2025.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepscekmath: Pushing the limits of mathe- matical reasoning in open language models, 2024.
Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. In Proceedings of the Twentieth European Conference on Computer Systems, pp. 1279-1297. ACM, March URL /ttp::/:/ccc.o....oo///0.11113/1
2025. doi: 10. 10.11445/68993333696677 3696075
Richard Sutton. Temporal Credit Assignment in Reinforcement Learning, PhD thesis, University of Massachusetts Amherst, 1984. URL https:ttps//.c.. hopp://weet //web//ww//....
.ebeb.c.... cs www.u. wwtp:eepppppp...........oe PhD Dissertation.
1984-994400 002. pdf
Yueqian Wang, Xiaojun Meng, Yuxuan Wang, Jianxin Liang, Jiansheng Wei, Huishuai Zhang, and Dongyan Zhao. Videollm knows when to speak: Enhancing sene-sensttve video comprehension with video-text duet interaction format. ArXiv, abs/2411.11999 2024.
12

---

Published as a conference paper at ICLR 2026
Yueqian Wang, Xiaojun Meng, Yifan Wang, Huishuai Zhang, and Dongyan Zhao. Proactivevideoqa: A comprehensive benchmark evaluating proactive interactions video large language models. ArXiv, abslla07.0033333 2025.
Haoning Wu, Dongxu Li. Bei Chen, and Junnan Li. Longvideobench: A benchmark for long-context interleaved video-language understanding. ArXiv, abs/2407.15754 2024.
Jin Xu, Zhifang Guo, Jinzheng He, Hangrui Hu, Ting He, Shuai Bai, Keqin Chen, Jialin Wang, Yang Fan, Kai Dang, Bin Zhang, Xiong Wang. Yunfei Chu, and Junyang Lin. Qwen2.5-omni technical report. ArXiv, abs/2503.202115 2025.
Linli Yao, Yicheng Li, Yuancheng Wei, Lei Li, Shuhuai Ren, Yuanxin Liu, Kun Ouyang, Lean Wang, Shicheng Li, Sida Li, Lingpeng Kong, Liu, Yuanxing Zhang, and Xu Sun. Timechat- online: 80% visual tokens are naturally redundant in streaming videos. ArXiv, abs/2504.17345. 2025.
Liping Yuan, Jiawei Wang, Haomiao Sun, Yuchen Zhang, and Yuan Lin. Tarsier2: Advancing large vision-language models from detailed video description to comprehensive video understanding. ArXiv, abs/2501.07888, 2025.
Kaichen Zhang, Bo Li, Peiyuan Zhang, Fanyi Pu, Joshua Adrian Cahyono, Kairui Hu, Yuhao Dong, Shuai Liu, Yuanhan Zhang, Jingkang Yang. Chunyuan Li, and Ziwei Liu. Lmms-eval: Real- ity check on the evaluation of large multimodal models. In North American Chapter of the Association for Computational Linguisties. 2024a.
Pan Zhang, Xiao wen Dong, Yuhang Zang. Yuhang Cao, Rui Qian, Lin Chen, Qipeng Guo, Haodong Duan, Bin Wang. Linke Ouyang, Songyang Zhang, Wenwei Zhang, Yining Li, Yang Gao, Peng Sun, Xinyue Zhang, Wei Li, Jingwen Li, Wenhai Wang, Hang Yan, Conghui He, Xingcheng Zhang, Kai Chen, Jifeng Dai, Yu Qiao, Dahua Lin, and Jiaqi Wang. Interoln-ccomposer22.55 A versatile large vision language model supporting long-contextual input and output. ArXiv, abs/2407.033220, 2024b.
Peiyuan Zhang. Kaichen Zhang, Bo Li, Guangtao Zeng, Jingkang Yang, Yuanhan Zhang, Ziyue Wang, Haoran Tan, Chunyuan Li, and Ziwei Liu. Long context transfer from language to vision. ArXiv, abs/2406. 16852, 2024c.
Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, MA Zejun, Ziwei Liu, and Chunyuan Li. Llava-video: Video instruction tuning with synthetic data. Trans. Mach. Learn. Res., 2025, 2024d.
Jiaxing Zhao, Xihan Wei, and Liefeng Bo. R1-omni: Explainable omni-multimoda. emotion recog- nition with reinforcement learming. CoRR, abw/2503.0033777 2025. doi: 10.48550/ARXIV25503.. 05379, URL http:::/t//o/// org 110.//1..000 48540 /at.S0//riivv. 2503. 05379
Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E. Gonzalez, Clark Barrett, and Ying Sheng. Sglang: Efficient execution of structured language model programs, 2024.
A DISCUSSION OF REPLY TIMING DECISION METHODS
Here we describe a more efficient implementation of reply timing instead of generating "NO RE- PLY" as described in Section 4.] In most MLLMs including Qwen2.5-VL, visual tokens are sur- rounded by special tokens like: <vis start><vis token>. Given the content up to the last <vi <vis_end>, we can train the model to predict whether the next to- ken is <vis_start>> indicating that the model wants to see one more piece of visual information, or <im_end>, indicating that the model want to stop the user's 's turn and start its own turn. In this for- mat, if the model chooses not to respond, no additional token will be added to the context, ensuring high information density. However, this requires introducing new rules into inference frameworks like SGLang (Zheng et al. 2024) or vLLM (Kwon et al., (023). which requires significant labor. Therefore, leave utilizing this more token-efficient format future work.
<vis tot.ttkkeeviv..o end>.
13

---

Published as a conference paper at ICLR 2026
B USE OF LARGE LANGUAGE MODELS (LLMS)
LLM--based tools are used to assist in writing the experimental code. LLMs are also employed to translate parts of the manuscript into English polish the text without altering its semantics.
14


Reviews:
[{"review_id": "suAR1ljGEa", "reviewer": "Reviewer_Rz9J", "summary": "This paper investigates the reward-based RL in online video LLM settings and propose MMDuet2 to autonomously determine whether to respond as soon as possible or remain silent in a proactive manner by RL training. This paper starts from curating online video llm training data, which is designed for multiturn proactivate dialogue, then design the specialized chat template for it and use SFT and RL to train the model in the off-policy and on-policy way for proactivate capabilities. Experimental results show that the proposed MMDuet2 can proactively answer user's queries in the online video streaming setting while maintain the ability to answer in offline video setting.", "questions": "See weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 2, "contribution": 3, "strengths": "1) This paper is one of the first batch to investigate RL training (esp. GRPO) for proactively answering in online streaming video settings, which is of substantial novelty. Also, the paper investigate the reward, the key component of GRPO, and model it specifically for online video settings (PAUC).\n\n2) The authors design a training dataset especially for online video streaming setting and corresponding chat template. Looking forward to the dataset open sourcing.\n\n3) The MMDuet2 trained by SFT and GRPO outperform previous online methods while maintaining offline video understanding capabilities", "weaknesses": "There are no major technical concerns about this paper, but I want to address some minor points as follows:\n\n1) As the key component to apply GRPO to online video settings, the ablations on rewards should be more addressed. Did authors try other rewards than PAUC? Please compare several reward formulations and discuss why PAUC is preferred.\n\n2) The author is encouraged to report the actual inference speed and latency of the MMDuet2 to see if it is realtime in practical scenes.\n\nAlso, the organization of this paper needs to be improved further since the current version seems too flattened with every details being straight written without a main storyline.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GO0LfkRWnA", "reviewer": "Reviewer_rgJo", "summary": "This paper proposes MMDuet2, aiming to enhance the proactive interaction capabilities of Video Multimodal Large Language Models (Video MLLMs) through multi-turn Reinforcement Learning (RL), enabling the model to autonomously decide when to respond during video playback. The authors construct a new dataset of 52k videos and present a text-to-text RL approach with a reward mechanism, including PAUC.", "questions": "1. Regarding the observed reward hacking issue, was there a deeper analysis of its root causes beyond \"early stopping\"? Were more sophisticated reward shaping, curiosity mechanisms, or RL algorithms attempted to enhance the model's robustness on complex tasks?\n\n2. Please elaborate on how the largely \"offline\" QA and captioning datasets in the SFT stage were processed and transformed to effectively support RL training for proactive interaction, rather than merely enhancing general understanding?\n\n3. The paper states that reducing the frame interval from 2 seconds to 1 second during inference significantly improves performance, even if the RL phase used a 2-second interval. Does this suggest that the model's decision-making process is highly sensitive to the temporal granularity of the input?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.The paper addresses proactive interaction, which is an important and challenging promblem for making Video MLLMs more natural and practical in real-time applications.\n\n2.  The use of RL to overcome the difficulty of precise reply time annotation is a promising avenue, and the reward mechanism design  theoretically considers timeliness, accuracy, and redundancy.\n\n3. The creation of a large-scale new dataset (52k videos with two dialogue types) provides a valuable resource for research in this field.", "weaknesses": "1. The central contribution of this paper lies in rl , which is explicitly designed to improve proactive interaction timing. However, the paper reports that during training on complex ego-centric video tasks, the model exhibited reward hacking behavior—generating large amounts of repetitive content. Although this issue is solved by early stopping, such manual action may show a instability in the reward design and optimization process.\n\n2. The occurrence of reward hacking indicates that the learned policy may not genuinely capture proactive interaction behavior but instead exploits the reward function. The paper does not provide ablations or diagnostic analyses to clarify why this failure occurs, nor does it offer evidence that the method can generalize to more complex or long-duration scenarios without collapsing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2T6456bVhO", "reviewer": "Reviewer_3Vfh", "summary": "This paper introduces a novel video multimodal large language model (Video MLLM) called MMDuet2, designed to enhance the model's proactive interaction capabilities—that is, the ability to autonomously decide when and how to respond while watching streaming videos. To address the challenges of existing methods requiring manual threshold adjustment and precise timestamp annotation of responses, the authors propose an innovative multi-round reinforcement learning (RL) training method. This method encourages the model to make timely and accurate responses at the right time through a carefully designed reward mechanism, without the need for precise timestamp annotation. Furthermore, the authors constructed a large-scale proactive dialogue dataset containing 52k videos for training. Experimental results show that MMDuet2 achieves state-of-the-art performance on benchmarks such as ProactiveVideoQA, significantly outperforming existing models.", "questions": "1. The dialogue template shown in Figure 2 of the paper appears to be standard practice for frame-by-frame streaming video understanding models. Furthermore, the core mechanism for generating \"NO REPLY\" seems functionally equivalent to predicting a specific EOS token (VideoLLM-online [1]). Therefore, aside from the specific implementation, what is the fundamental innovation of this \"text-to-text approach\"?\n2. To better contextualize the performance of MMDuet2, would the authors consider providing direct empirical comparisons against other relevant models, such as Dispider [2] and TimeChat-Online [3]? At the same time, could you please provide an ablation study analyzing the individual impact of the four reward components ($r_{\\text{PAUC}}$, $r_{\\text{rep}}$, $r_{\\text{inspan}}$, $r_{\\text{pfx}}$) on the model's behavior?\n3. The authors employed a strategy of placing the model's response at the end of its response time period during the SFT phase to avoid the model developing illusions before seeing the relevant event. However, the goal of the RL phase is to encourage the model to make the correct response as early as possible. Are these two goals contradictory during training?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. This paper innovatively introduces multi-round reinforcement learning, using a reward mechanism to teach the model to find the optimal response time, cleverly circumventing the challenge of precise time labeling.\n2. The authors constructed a large-scale dataset containing 52k videos, providing a solid data foundation for training more robust active models.\n3. Experimental results show that the MMDuet2 model trained with SFT+RL outperforms previous state-of-the-art models and our own model trained solely with SFT on authoritative active interaction benchmarks such as ProactiveVideoQA. This demonstrates the effectiveness of reinforcement learning methods.", "weaknesses": "1. Using the \"NO REPLY\" text token is a concise and universal approach, but it also means that if the model chooses not to respond, a complete generation process (generating both tokens) is still required, which limits its inference efficiency.\n2. The study on the reward component is insufficient, and related ablation experiments are lacking. The total reward is a weighted sum of four components, but the paper only mentions \"a certain hyperparameter search\" providing a set of weights without conducting ablation studies. This fails to clearly explain the specific contribution of each reward or penalty to the model's performance.\n3. The task is limited to question-and-answer type tasks, and from data construction to model evaluation, it relies heavily on the question-and-answer paradigm. Therefore, the model's generalization ability for non-question-and-answer type proactive response tasks, such as caption tasks, needs to be tested.\n4. There is insufficient comparative testing with other active response models. Related work mentions some active response models, such as Dispider [2] and TimeChat-Online[3]. However, the main experiments did not directly compare these models.\n\n[1] Joya Chen, Zhaoyang Lv, Shiwei Wu, Kevin Qinghong Lin, Chenan Song, Difei Gao, Jia-Wei Liu, Ziteng Gao, Dongxing Mao, and Mike Zheng Shou. Videollm-online: Online video large language model for streaming video. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 18407–18418, 2024a.\n\n[2] Rui Qian, Shuangrui Ding, Xiao wen Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Dahua Lin, and Jiaqi Wang. Dispider: Enabling video llms with active real-time interaction via disentangled perception, decision, and reaction. ArXiv, abs/2501.03218, 2025.\n\n[3] Linli Yao, Yicheng Li, Yuancheng Wei, Lei Li, Shuhuai Ren, Yuanxin Liu, Kun Ouyang, Lean Wang, Shicheng Li, Sida Li, Lingpeng Kong, Qi Liu, Yuanxing Zhang, and Xu Sun. Timechatonline: 80% visual tokens are naturally redundant in streaming videos. ArXiv, abs/2504.17343, 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)