When One Moment Isn't Enough: Multi-Moment Retrieval with Cross-Moment Interactions
Authors: Zhuo Cao
Heming Du
Bingqing Zhang
Xin Yu
Xue Li
Sen Wang
Venue: NeurIPS 2025 poster ()
Keywords: Moment Retrieval
Video Temporal Grounding
Primary area: applications
TL;DR: We release QV-M², the first fully human-annotated multi-moment video benchmark, and present FlashMMR that outperforms prior SOTA by up to 3 % G-mAP, laying a new baseline for multi-moment retrieval.

Abstract: Existing Moment retrieval (MR) methods focus on Single-Moment Retrieval (SMR). However, one query can correspond to multiple relevant moments in real-world applications. This makes the existing datasets and methods insufficient for video temporal grounding. 
By revisiting the gap between current MR tasks and real-world applications, we introduce a high-quality datasets called QVHighlights Multi-Moment Dataset (QV-M$^2$), along with new evaluation metrics tailored for multi-moment retrieval (MMR). QV-M$^2$ consists of 2,212 annotations covering 6,384 video segments. Building on existing efforts in MMR, we propose a framework called FlashMMR. Specifically, we propose a Multi-moment Post-verification module to refine the moment boundaries. We introduce constrained temporal adjustment and subsequently leverage a verification module to re-evaluate the candidate segments. Through this sophisticated filtering pipeline, low-confidence proposals are pruned, and robust multi-moment alignment is achieved.
We retrain and evaluate 6 existing MR methods on QV-M$^2$ and QVHighlights under both SMR and MMR settings. Results show that QV-M$^2$ serves as an effective benchmark for training and evaluating MMR models, while FlashMMR provides a strong baseline. Specifically, on QV-M$^2$, it achieves improvements over prior SOTA method by 3.00% on G-mAP, 2.70% on mAP@3+tgt, and 2.56% on mR@3. The proposed benchmark and method establish a foundation for advancing research in more realistic and challenging video temporal grounding scenarios. Code is released at https://github.com/Zhuo-Cao/QV-M2.

When One Moment Isn't Enough: Multi-Moment Retrieval with Cross-Moment Interactions
Zhuo Cao'*, Heming Du'; Bingqing Zhang', Xin Yu', Xue Li Sen Wang'
The University of Queensland, Australia
{william.cao, heming. du, tongqing.zhang xin.yu @ @uq.edu..uu xueli @elli@eesc.uue seesc.uq.edu.ae. sen. wang @ uq..edu.au
Abstract
Existing Moment retrieval (MR) methods focus on Single-Moment Retrieval (SMR). However, one query can correspond to multiple relevant moments in real-world applications. This makes the existing datasets and methods insufficient for video temporal grounding. By revisiting the gap between current MR tasks and real-world applications, we introduce high-quality datasets called QVHighlight Multi-Moment Dataset (QV-M²), along with new evaluation metrics tailored for multi-moment retrieval (MMR). QV-M² consists of 2.212 annotations covering 6,384 video segments. Building existing efforts in MMR, we propose frame- work called FlashMMR. Specifically, we propose Multi-monen module to refine the moment boundaries. We introduce constrained temporal adjustment and subsequently leverage a verification module to re-evaluate the candidate segments. Through this sophisticated filtering pipeline, low-confidence proposals are pruned, and robust multi-moment alignment achieved. We retrain and evaluate 6 existing MR methods QV-M² and QVHighlights under both SMR and MMR settings. Results show that QV-M² serves as an effective benchmark for training and evaluating MMR models, while FlashMMR provides a strong baseline. Specifically, on QV-M², it achieves improvements over prior SOTA method by 3.00% on G-mAP, 2.70% on mAP@@+tgr and 2.56% on mR @3. The proposed benchmark and method establish a foundation for advancing research in more realistic and challenging video temporal grounding scenarios. Code is released httips:/github..om//// waw.hhuubb.bom/hoocacC///....
verifieriiin
Introduction
Understanding how natural language relates visual events in videos is core problem in video- language research 33
B 37 One representative task, Moment Retrieval (MR), aims to retrieve relevant temporal segments given natural language query. Most existing MR methods [39 operate under the Single-Moment Retrieval (SMR) paradigm, assuming that each query corresponds to exactly one relevant moment within video. However, this assumption oversimplifies real-world scenarios, where a single query often aligns with multiple non-overlapping moments. For example, in instructional videos, query such "cutting vegetables" may correspond to several separate instances of chopping different ingredients throughout the video. Similarly, in sports broadcasts, "successful three-point shots" may occur multiple times within single match.
17 6 [2]
Despite the prevalence such multi-moment scenarios, existing MR methods 112 remain inherently limited to retrieving only the most relevant single segment, disregarding other
18
'Equal Contribution
Corresponding Authors
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Query: woman with short brown hair and curved bangs holding an object her hand." "
Moment
Moment
Momont
SMR Model (FlashVTG) Prediction:
(FlashMMR) Prediction:
Figure 1: Prediction in SMR vs. MMR. In multi- moment retrieval, SMR optimizes for the most probable single moment, often disregarding other valid segments. In contrast, MMR encourages comprehensive retrieval by identifying all semantically relevant moments, aligning better with real- world video understanding.
valid moments. This constraint severely limits their applicability to comprchensive video-language understanding. Addressing this fundamental gap between current MR methodologies and real- world applications requires paradigm shift from SMR to Multi- Molti-tMmenttt Retrieval (MMR).
By revisiting current video temporal grounding research 2 18 20
found that major limitation is the lack of standardized datasets and evaluation metrics tailored for MMR. To support rigorous benchmarking and further research in this area, introduce QV-M (QVHighlights 2 Moment Dataset), an enhanced dataset based on QVHighlights. QV-M² contains 2,212 high-quality
Multi-
aanaottdd queries, covering 6,384 annotated temporal segments across diverse video scenarios. Unlike previous MR datasets, which primarily support SMR evaluation, QV-M² explicitly accounts for queries with multiple relevant moments, making the first fully numan-annotater dataset dedicated to MMR benchmarking. Beyond dataset contributions, we also propose new evaluation metrics that extend standard mean Average Precision (mAP) and Intersection-over Union (IoU) to measure MMR performance. These metrics provide comprehensive evaluation considering both the SMR and the MMR settings, ensuring that future MMR research aligns with real-world requirements.
To further advance MMR, we introduce FlashMMR, a novel framework explicitly designed for MMR. A key challenge in MMR is to ensure that the retrieved moments precisely include all relevant moments and exclude false positives. To achieve this, we propose a Multi-Moment Post- Verificiiiiatioon module, which refines moment boundaries through a constrained temporal adjustment strategy and further verifies retrieved moments using semantic consistency-based re-evaluation process, This module filters out low-confidence proposals and enhances alignment with query semantics. Through this structured refinement pipeline, FlashMMR effectively reduces the presence of redundant or irrelevant moment predictions while maximizing the recall for multiple relevant instances. As show in Figure [ the objective shifts from retrieving the single most relevant moment to identifying as many relevant and semantically consistent moments as possible.
To validate the effectiveness of FlashMMR, we benchmark open-source MR models under both SMR and MMR settings QV-M² and QVHighlights. Experimental results show that the new dataset enhances the performance of all methods on both SMR and MMR task. Our comprehensive comparisons reveal that while existing methods perform well in single-moment retrieval, they struggle significantly in multi-moment scenarios due to their inherent architectural limitations. In contrast, FlashMMR consistently outperforms prior approaches across all MMR metrics, demonstrating its effectiveness, with 3.00% improvement in G-mAP, 2.70% in mAP@ @ 3+tgt, and 2.56% in mR @ on QV-M². These findings underscore the necessity of dedicated MMR frameworks advance video language understanding. FlashMMR not only enhances the performance of MMR task, but also establishes a new benchmark for future research MMR.
In summary, Our contributions are as follows:
1. We introduce FlashMMR, novel
Retrieval (MMR) framework, which incorporates a Multi-Moment Post- Verification module to refine candidate segments by enforcing temporal consistency across related moments.
2. We propose QV-M², the first fully human-annotated MMR dataset, designed to facilitate bench- marking and model development for MMR.
3. We develop a comprehensive suite of MMR evaluation metrics that extend traditional MR evalua- tion protocols. These metrics jointly assess retrieval accuracy and temporal coverage, offering a fine-grained evaluation of model performance moment retrieval.

---

Table 1: Comparison with existing moment retrieval datasets.
Avg #moment per query
Avg
Avg ratio
Fully
Dataset
Domain
#Moments/ #Videos Human-Annonn Annotated
Query Len Moment/Video
DiDeMo 1
Flickr
8.0
22.2%
41.2K / 10.6K
ANelCaptions
10
Activity
1
14.8
30.8%
72K / 15K
CharadesSTAA
Activity
1
7.2
26.5%
16.1K 6.7K
TVR 111
TV show
13,4
12.0%
109K / 21.8K
TACoS 27
Cooking
27.9
8.4%
18K O.IK
YouConk2
47
Cooking
8.7
1.9%
13.8K 2K
COIN 33
Open
4.9
9.8%
46.3K / 11 11.8K
HiREST 40
Open
4.2
55.0%
2.4K / 0.5K
NEXT-VMR
25
Open
1.5
229.5K 19K
QVHighlights
Vlog/News
1.8
11.3
16.4%
18.5K 10.2K
QV-M² (Ours)
Vlog/News
2.9
12.0
25.5%
6.4K / 1.3K
By providing a strong benchmark and a new dataset, our work lays the foundation for future research on MMR systems, paving the way for more realistic video understanding in complex environments.
2 Related Work
2.1 Single-Moment Retrieval
Singlc-Moment Retrieval (SMR) focuses the task of localizing single relevant temporal segment within a video based on a natural language query. Given the query, the model predicts the start and end timestamps of the most relevant moment. Existing SMR methods [29 can be broadly categorized into proposal-based and proposal- frree approaches.
Proposal-Based Methods. Proposal-based approaches decompose the retrieval process into two stages: candidate moment generation followed by matching and ranking. These methods can be further classified based how they generate candidate segments: Sliding Window Approaches [39
6 I systematically segment vidcos into overlapping windows and evaluate their relevance to
the query. Anchor-Ber Based Approaches [46 39 32 44] define set of predefined anchors across the
video and refine the most promising candidates. Proposal-Gese Generated Approaches 445 36 34 30
utilize deep neural networks to generate adaptive moment proposals.
Proposal-Fis Free Methods. Proposal- trope free approaches directly predict the start and end timestamps using sequence regression techniques. Instead of evaluating pre-defined candidate segments, these methods
formulate moment retrieval as an end-to-end sequence prediction problem [16 19 41
22 12 28).
Recent transformer-basee models, such as Moment-DETR [1), adopt set-based prediction paradigm to eliminate the need for non-maximum suppression (NMS) and other post-processing steps.
While proposal-free methods demonstrate higher efficiency and flexibility, they often struggle with multi-moment scenarios, they are inherently designed to retrieve only a single moment per query.
2.2 Multi-Moment Retrieval
Real-world video content often contains multiple non-overlapping moments that are semantically relevant the query. Current SMR models address this by selecting the highest loU moment as the ground truth, ignoring other valid segments
[12]. This simplification results suboptimal retrieval performance when multiple events contribute to the semantics of the query. To address these shortcomings, recent studies have explored Multi-Moment Retrieval (MMR), where query can be
mapped multiple relevant moments 25
Early Efforts in MMR. Early works attempted to adapt SMR models to MMR by modifying retrieval pipelines: Otani al. (24) identified false negatives in SMR training. Liu er al. [I7] extended proposal- based MR models to allow multiple moment predictions per query, though lacking robust mechanisms for handling dependencies among moments.
Recent Advances in MMR. Recent works have introduced dedicated architectures and datasets for MMR: SFABD S refines candidate retrieval by eliminating false negatives and improving

---

alignment with query semantics. Concurrent work by Qin et al. [2] extends moment retrieval beyond single-moment assumptions, introducing NEXT-VMR, dataset designed to support multi-moment and no-moment retrieval. However, its design and evaluation remain closely aligned with previous benchmarks, without targeted optimizations for the unique challenges of MMR. Additionally, the dataset is not yet publicly available.
Despite recent efforts in multi-monoen retrieval (MMR), a high-quality dataset built with standardized methodology is still lacking. We address this gap by introducing QV-M², a densely annotated, high- quality MMR benchmark, together with FlashMMR, which establishes a new baseline for the multi-moment retrieval task.
3
QV- Dataset
3.1 Video Collection and Annotation
Video Collection. To better reflect real-world MMR scenarios and facilitate comparison with existing SMR tasks, building on existing moment retrieval datasets is natural and effective choice for developing MMR benchmarks. Among them, QVHighlights [1] stands out as one of the most widely adopted datasets. Composed of unedited minimally edited videos with naturally rich content, it serves as an ideal foundation for extending to MMR while enabling a seamless transition and comparison with SMR.
Our dataset retains the original QVHighlights videos and adds new annotations for MMR. The videos, sourced from YouTube, include lifestyle vlogs and news footage, covering diverse scenarios such as travel, social activities, natural disasters, and protests. They vary in perspective (e.g., first-person, third-perse perdd-peroon and range from 5 to 30 minutes in length, offering both diversity and annotation feasibility. This ensures the dataset captures real-world complexity and better supports the MMR task.
Video Annotation. We design manual annotation process for the me-to-many nature of MMR. improves diversity and coverage while ensuring high quality. We define set of annotation guidelines to standardize the process, including: (i) create detailed queries that precisely capture actors, actions, and contexts; (ii) include context- context-dependent queries that require knowledge of temporal relationships within the video; and (iii) design negative (inverse) queries to mark segments where a specified action or event does not occur.
Each query must be matched with one or multiple video segments. To facilitate this, we develop an annotation interface that allows annotators to watch each video, formulate the queries, and assign start and end times for relevant segments. Details of the annotation interface are provided in supplementary material. For quality control, after every 100 videos are annotated by the primary annotator, we randomly sample 5% of these videos for re-checking by an additional annotator. If the temporal boundaries identified by the two annotators overlap by less than 90%, the batch is re-annotated by third annotator. This process ensures annotation consistency and captures the complexity of MMR. License and data usage details are provided in the supplementary material.
3.2 Dataset Statistics
Our dataset, QV-MP, consists of 2.212 new queries associated with 1,341 videos, covering total of 6,384 annotated temporal moments. As shown in Table compared to existing moment retrieval datasets, QV-M² is distinct in its MMR setting, where each query is linked to average of 2.9 moments, significantly surpassing the typical single-moment assumption in prior datasets.
To analyze the temporal properties of annotated moments, we present the distribution of moment lengths and location in Figure and (c). As shown in (a), the majority of temporal windows fall within the 2 to 20-second range, with notable 1,263 instances (19.8%) extending beyond 20 seconds, highlighting the diversity in temporal granularity.
Figure ec)) compares the ground-truth moment boundaries of QV-M2 and Charades- STA S (start time on the x-axis and end time on the y-axis; both axes normalized by video duration), and reveals that QV-M² annotations are distributed more uniformly throughout videos. Lexical analysis of QV-M² (Figure 2b)) reveals a rich vocabulary of commonly occurring nouns and verbs, emphasizing key concepts in human interactions, fashion, and daily activities. Frequent words such woman, man,

---

26,4%
QV-M training
1.0
Perventage (%)
19,895
0.3
18
13,8%
0.6
End
12
10,05%
2.5%
5.7%
3.5%
2.5%
0.0
0.4
1.0
Window L Length (seconds)
(a)
Start
(C)
(b)
Charades-SD training
800
1.0
649
Frequency
600
:35
0.8
0.6
400
lind
201
0.4
200
187
0.2
woman
wear
hair
speak
man
camera
top
shirt
talk
front
0.0
0.0
0.2
0.4
0.6
0.8
1.0
Start
Figure 2: Dataset statisties for QV-M2 (a) and (c) show the distributions of moment lengths and
temporal locations, respectively; (b) reports the top- 10 most frequent words in the annotations.
speak, wear, and scene suggest broad coverage of people-centric activities, aligning well with the dataset's focus on vlog and news content.
3.3 Evaluation Metrics For MMR
We develop new evaluation metrics by smoothly adapting SMR metrics to handle multiple moments. Specifically, label prediction true positive if its Intersection-over-umii (IoU) with any unmatched ground truth meets a threshold (e.g., 0.5); otherwise, deemed a false positive.
Generalized mAP. The generalized mean Average Precision (G-mAP) is computed by averaging the AP scores over multiple IoU thresholds:
G-mAP
Σ AP(T), TET
IT
where T denotes the set of loU thresholds (e.g., {0.5, 0.55. 0.9)), and AP(r) represents the average precision computed at threshold т.
To further capture performance nuances, categorize queries by their number of ground-truth moments and report mAP under each category (e.g., mAP@l_Ighl mAP@2_lgg, mAP@3 -!gg). Averaging these scores over multiple loU thresholds also yields the G-mAP, ensuring robust, unified metric that can evaluate both single- and multi-target scenarios.
Mean IoU@k. The mean Intersection-over Union at top-k predictions is defined as:
mloU@k
max (oUdpred, gE
QEQ
where 2,3 denotes the rank, Q is the set of all queries, and G represents the ground-truth moments associated with query g. The function [prep gt) computes the Deersection-over-nniie between the i-th prediction and one the ground-truth moment. It is worth noting that Mean IoU @+ is computed only on queries with at least ground-truth moments.
5

---

Video
Multi-Scale Temporal Prediction
Video Ensader
Feature Fusion
Temporal Feature Layering Module
ASR Module
Boundary Prediction
Dummy
Adapttive Cross-Attention
Goafileaee Sooe (olosss
Feature Encoder
(5
Dummy Encoder
Fusion Feature Selection
7
1
12
Text Query
Guide
Frature Rufinemennt Unit
The Novmwwok
Encodar
S
Pb Ps.
Temporal loU
Guidl
Extraction
Post-Verification module
Ground Momants
Figure 3: Overview of the FlashMMR Framework. In Feature Extraction, video and query features extracted via frozen encoders. The textual feature, combined with encoded dummy tokens, forms the key, while the video feature serves as the query and the text feature as the value in Feature Fusion Module. This produces fused features {fi) semantic relevance. During inference, the fused features directly passed into the Multi-Scale Temporal Prediction Module to generate the final prediction {(st, éi, [=]]== During training, the Post- Vest-ieiiiiation Module further refines the initial prediction. Specifically, the fused features aligned with the prediction to obtain a refined confidence score p4 and self-similarity matrix S., both of which are supervised using the ground truth moments.
;=1 (in purple), where color intensity indicates
Mean Recall@k. The recall at top-k predictions is defined as:
1 2
mR @k
Σ BEE
max loUdp gt) 2 isk
((4)
QEQ
where is E (1,2,3) denotes the rank, 1[-] is the indicator function, and T is the IoU threshold (e.g., (0.53, 0.35..... ...95)))))))) for determining whether prediction is considered a match to a ground truth. Similar Mean IoU@k, k Mean Recall @ is also computed only on queries with at least & ground- truth moments.
Taken together, these new metrics form comprehensive and scalable evaluation framework that effectively measures the MMR task while maintaining strong compatibility with the SMR criteria. In particular, G-mAP, mloU@ and mR @1 remain fully consistent with the standard SMR metrics, ensuring direct comparability of performance in both single- and multi-moment retrieval settings.
4 Methodology
As shown in Figure 3 the proposed FlashMMR model extends traditional SMR pipeline by intro- ducing novel Post- Verification Module. This module effectively adapts the existing framework for addressing MMR tasks. The FlashMMR consists of three key components: Feature Extraction and Fusion, Multi-Scale Temporal Processing, and the Post- Dost-Verification.
4.1 Feature Extraction and Multi-Scale Temporal Processing
Different with Single- Moment Retrieval, the goal is locate multiple video segment that best matches textual query. Given video V with clip-level features V (u, Vm and query with word features Q = {q: fe, the model predicts set of temporal spans (s,, c, o)) where (s;, <4) defines the i-th predicted moment and e; its confidence score.
we extract video features V using frozen SlowFast 4 and CLIP 26) encoders, while text features Q are derived from CLIP.
Feature Extraction. Consistent with previous research

---

The input video is segmented into clips at predefined frame rate (e.g.. 0.5 FPS), and each clip is transformed into a feature representation {"; E |2 while each query word is encoded as {q ] Both modalities are projected into shared space of dimension d via MLPs. Ly and Lg denote the video clip number and query word count, respectively. The dummy token and dummy encoder used here are identical to those in 2 21), and act as explicit sinks that absorb semantics irrelevant to the query. Further implementation details are provided in the supplementary material.
Cross-Feature Alignment. To enhance video-text feature alignment, we adopt Adaptive Cross Attention (ACA) module [21]. which integrates learnable dummy token to encode contextual infor- mation beyond explicit query representations. After we get the fused feature F it is further refined using a Transformer Encoder for long-range dependencies.
R RL_xe xd from ACA,
Temporal Feature Layering. To capture temporal variations across different moment durations, employ a multi-scale temporal processing. We construct a temporal feature pyramid by applying a series of 1D convolutions to F:
F,
if 1, (F = 2)), if 2,3 2......,
Fp
(I)
This results in set of downsampled fused feature maps {Fplp 1. P). capturing temporal dependencies at different granularities. Moment boundary predictions are computed at each scale
using shared convolutional head: Bp = a [Conv1D (a (ConvID(Fp)))
Cp. where Bp
R-p-1 x2 represents boundary predictions, Cp is a learnable scaling parameter, and is the activation function.
Adaptive Score Refinement. To improve moment retrieval confidence, we refine moment scores using both intra-scale and inter-scale scoring: C, ScoreHead, (F²), P 1, 2, ..... P. Cintra
Concat(c), ......, CP), Cinter = ScoreHead, (Concat(Fi, Fi, F2,.. F.)). The final confidence score computed as: Cinal x Gata + z) Canter where x is learnable weighting factor.
We first obtain initial moment predictions. To better address the multi-moment setting, we introduce post-verification module that enforces consistency across semantically related moments. This refinement leads to more diverse predictions and broader coverage in the multi-moment setting.
4.2 Post Verification Module
Post Verification Module refines initial predictions and improves alignment with the ground truth. It consists of two key components: Prot-Processing with Feature Refinement and Semantic Consistency Control. We describe each component in detail below.
Post-P Post-Processing with Feature Refinement. Given the initial boundary predictions á e 3 where each predicted moment b, (s;, es, C:) consists of start time s;, end time e;, and confidence score 6, we apply a structured post-processing strategy inspired by prior refinement techniques [2 21 This step ensures that the predicted windows adhere to temporal constraints and enhances their interpretability. We employ a post-processing function F(.) parameterized by set of structured constraints, including minimum and maximum window lengths, temporal clipping, and rounding heuristics:
B = F(B, Adlip Aronaa,
where Aetip and Around control boundary clipping and discretization granularity. This operation prevent excessively short or long predictions and align segments with predefined frame sampling rates. Following this step, each predicted interval is used to extract its corresponding multi-modal feature representation from the fused video embeddings F E R. xd, L, F(s; rr: e, r, where the feature segments I, are sampled based on the refined start and end timestamp.
Post Verification via Semantic Consistency Control. To re-evaluate the quality of predicted moments, we introduce a post-verification network P(.), which models semantic consistency between retrieved intervals and their relevance to the query. This network is implemented as recurrent module (-) [ capturing contextual dependencies across extracted moment representations: Y; = ((PRRU( where pt represents the refined confidence score assigned to each predicted moment, and (.)) the activation function. The refined score p E T provides an confidence adjustment that used to mitigate errors in the initial moment retrieval process.
7

---

Table 2: Cross-Dataset Performance Comparison of SMR and MMR QVHighlights and QV-M². Experimental results for each method under three settings: (i) trained and evaluated on QVHighlights. (ii) trained QVHighlights and evaluated QV-M', and (iii) trained QV-M² and evaluated on QVHighlights. The table reports the performance gains on both SMR and MMR tasks introduced by the new QV-M² dataset, and highlights the increased challenge that MMR poses to methods originally designed for SMR.
mAP
mloU@@
mR@k
Method
G-mAP
@01 @1gggt
@2_
W+igt
601
62
69.
681
602
M-DETR 12
32.79
42.02
19.45
3.67
48.81
32.75
28.54
40.19
24.56
19.55
w/ QV-M- Val
30.26
40.43
4.26
46.97
31.65
27.90
23.78
-V-M Train
34.70
43.81
20.75
5.35
51.71
34.48
31.22
43.44
27.23
23.67
EATR 9
35.96
44.15
23.80
7.70
50.91
36.44
34.00
42.85
30.14
27.65
n/ QV Val
35.42
45.59
23.71
7.19
50.30
36.42
33.10
42.37
29.87
26.92
- QV-M Train
38.65
46.90
26.80
8.82
53.26
39.71
35.25
45.64
33.77
29.99
UVCOM[33]
42,83
51.71
29.96
11.02
57.79
40.92
38.79
51.01
35.26
32.97
QV-M Vul
50.74
28.82
9.07
40.61
48.67
34.61
31.50
QQ-M Train
43.68
51.71
31.56
13.88
58.96
43,96
42.10
52.06
37.88
36.77
QD-DETR 20
38.90
48.18
24.55
7.47
54.48
38.63
36.14
46.80
31.93
29.49
w/ QQ-M" Val
36.32
46.62
24.82
6.97
52.75
37.53
33.81
45.01
30.70
27.21
- QV-M Train
40.63
49.94
26.91
8.69
56.39
39.76
36.09
48.72
32.96
29.42
TR-DETR BI
36.86
46,20
5.18
53.86
36.59
31.18
45.33
28.91
24.13
w/ QV-M Val
34.14
44,96
22.64
5.44
53.14
34.58
28.73
44.49
27.53
23.37
-V-M Train
44.49
54.70
29.68
8.60
60.73
41.85
38.10
$3,33
35.12
32.30
FlashVTG 2
48.02
57.31
35.08
13.85
61.45
43.80
39.37
53,92
38.98
35.17
w/ VQVVM Val
40.28
50.21
29.93
9.37
$7.4
44.03
39.42
49.92
37.24
32.91
- -V-M Train
48.35
57.37
35.40
15.71
62.61
44.86
41.83
55.67
40.03
37.77
FlashMMR (Ours)
56.95
15.15
45.32
40.63
w/ QV-M Va!
44.81
55.29
34.31
13.58
59.95
43.99
38.21
52.55
38.96
34.94
QV-M Train
48.42
57.46
37.37
16.41
62.40
47.52
43.13
55.38
42.40
39.29
We supervise the refined score p by leveraging the temporal Intersection-overt Union (tloU) between predicted segments and ground-truth moments. Given the ground truth annotations 3 = {(}}, compute the tloU matrix and select the highest overlap score for each prediction:
IOU max B"), dim -1).
By incorporating this post-verification module, our method effectively re-evaluates and refines initial moment predictions, leading to more accurate and coherent grounding in the multi-moment setting.
4.3 Training Objectives
Follow the previous work (2). we employ Focal Loss [14] LI Loss, and Clip-Aware Score Loss to optimize classification labels, temporal boundaries, and clip-level confidence scores, respectively. Additionally, we introduce a post-verification loss to refine moment predictions through a combination of mean squared error (MSH) loss and contrastive representation (Cross-Entropyy) loss: Lpv |lp 10 + Crepe-
The representation loss Laer enforces feature similarity consistency by encouraging temporally close frames to maintain high semantic coherence: Lrepe Σ, CE(S, TT), where S, is a cosine similarity matrix computed over fusion features, and T, represents the pairwise segment agreement derived from ground truth moment labels.
5 Experiments
5.1
Implementation Details
For fair comparison, we use SlowFast 4 and CLIP 26 as the video and text encoders, respectively. following configurations in [23] FlashMMR and FTG share identical parameter settings for common components. The post-verification loss terms Lev and Lrepe are weighted at 9 and 7. We use AdamW as the optimizer and set the NMS threshold to 0.7 during inference. SMR verification experiments QVHighlight are conducted on the validation set due to the unavailability of test set annotations. All experiments are conducted on single RTX 4090 GPU. Additional implementation details can be found in the supplementary material.
8

---

Table Comparison of performance on QV-M² test with previous state-of-the-art methods. The best results are highlighted in bold, and the second-best are underlined.
mAP
mloU@k
mR @
Method
G-mAP
@ Lgt
@ @2_1g 2_t
3-tgt
@
@2
@
@
@
@
M-DETR
12
20.65
33.71
25.85
10.95
44.14
38.98
34.34
34.81
30.95
26. 24
EATR 9 KCV
27.32
38.26
33.25
19.46
47.16
42.62
39.41
39.30
36.05
33.56
QD-DETR 20
CVPR22
28.95
39.69
37.26
18.30
50.46
46.79
40.50
42.35
40.58
36.05
TR-DETR
31.23
44.12
39.17
19.64
55.13
48.13
42.52
47.21
41.24
35.82
CG-DETR
Amiir2
28.87
43.74
32.35
18.44
52.01
47.98
43.27
43.26
40.80
36,69
FlashVTG 2
RACY125
32.14
47.16
39.48
20.19
54.49
47.85
40.92
46.64
41,30
35.94
FlashMMR (Ours)
35.14
52,59
42.52
22.89
56.29
49.64
42.92
48.81
44.33
38.50
Table 4: Comparison of performance on QVHighlights validation set with previous state-of-the- methods. The best results are highlighted in bold, and the second-best are underlined.
Method
mAP
mloU@t
mR@k
G-mAP
[_ggt
@2_igt
3++gg
@-2
M-DETR
21
32.79
42.02
19.45
3.67
48.81
32.75
28.54
40.19
24.56
19.55
EATR 9
35.96
44.15
23.80
7.70
50.91
36.44
34.00
42.85
30.14
27.65
QD-DETR 20
CVPR22
38.90
48.18
24.55
7.47
54.48
38.63
36.14
46.80
31.93
29.49
TR-DETR
AMAR2S
36.86
46,20
24.63
5.18
53.86
36.59
31.18
45.33
28.91
24.13
CG-DETR
Annr2e
43.69
52.70
30.12
10.02
60.32
45.04
42.21
52.85
38.17
35,46
FlashVTG
48.02
57.31
35.08
13.85
61.45
43.80
39.37
53.92
38.98
35.17
FlashMMR (Ours)
48.07
56.95
35.78
15.15
62.09
45.32
40.32
55.02
40.63
36.68
5.2
Comparison Results
We retrain and evaluate methods on QV-M² and QVHighlights [12 dataset under both SMR and MMR settings. The experimental results, presented in Tables demonstrate the effectiveness of QV-M2 for MMR and further show that FlashMMR consistently outperforms previous methods.
Table 2 presents Cross-Datased Performance Comparison of SMR and MMR on QVHigllights and QV-M Notably, models trained with QV-M2 consistently exhibit improved performance compared to their counterparts trained only HVHighlights, validating the effectiveness of QV-M² on both SMR and MMR supervision. FlashMMR achieves the highest overall G-mAP (48. (8.422%) and superior performance across all mloU@ @+ and mR @k metrics. We also observe performance drop across all methods when using QV-M² for evaluation, due to the increased number of one-to- one-to-many many moment queries. This further demonstrates the effectiveness of our dataset in evaluating the MMR task.
To validate the effectiveness of our proposed FlashMMR, we compare it with state-of-the-art methods on the QV-M2 and QVHighlights, as shown in Table 3 FlashMMR achieves notable improve- ments over previous methods most evaluation metrics, achieving a significant improvement over VTG (2 in G-mAP (+3.00%) mAP@ 3+tgt (+2.70%) and mR @3 (+2.56% on QV-M2. Similar results can be observed Table These results highlight the superiority of our approach in localizing multiple relevant moments.
5.3 Ablation Study
We conduct an ablation study on the two MMR datasets- QV-M- and QVHighlights to evaluate the effectiveness of the Post- Verification (PV) module in FlashMMR. As shown in Table incorporating the PV module leads to consistent performance improvements across both datasets. On QV-M the PV module brings notable gain of 3.00% in G-mAP and 3.04% in mAP@2 Igt. Similar improvements are observed across other metrics, including mAP@ 3+tgt (+2.70%), mloU@2 (+1.79%) and mR @2 (+3.03%), demonstrating its effectiveness in handling dense sae-to-many queries.
On QVHighlights, which comparatively less challenging, the PV module still yields consistent gains, including 0.70% improvement in mAP@2 2ggtttttttaate and a 1.52% increase in mloU@) @ These results validate the robustness of the module and highlight its role enhancing temporal consistency and filtering low-confidence predictions, ultimately improving multi- retrieval performance.

---

Table Ablation study of the Post Verification (PV) module on MMR task.
Dataset
FlashMMR
G-mAP
mAP@2 tgt
3+t@
mloU@2
mloU@3
mR
mR
QV-M²
w/o PV
32. 14
39.48
20.19
47.85
40.92
41.30
35.94
35.14
42.52
22.89
49.64
42.92
44.33
38.50
QVIligllighis
Wo PV
48.02
35.08
13.85
43.80
39.37
38.98
35.17
PV
48.07
35.78
15.15
45.32
40.32
40.63
36.68
Discussion and Limitation
Experimental results show that while existing SMR models struggle generalize the MMR setting, FlashMMR improves overall performance and establishes a strong baseline, with QV-M² serving as reliable testbed for advancing MMR research.
Despite these improvements, several challenges remain. First, our verification module remains in an early stage. Future work could explore more strategies, such as reinforcement learning contrastive learning for better moment discrimination, to further enhance model performance. Secondly, one limitation in MMR research is the relatively limited size of high-quality annotated datasets. Although QV-M² is sufficient for current models, its limited scale may constrain future progress as models become more advanced.
7 Conclusion
In this paper, we revisited the gap between existing single-moment retrieval methodologies and the practical complexities inherent in real-world video understanding tasks. We introduce QV-M2, the first fully human-annotated dataset for multi-moment retrieval, along with new evaluation metrics to benchmark the task. To address the limitations of traditional SMR frameworks, we proposed FlashMMR, a dedicated multi-moment retrieval model equipped with a Post-verification module. Comprehensive experiments demonstrate that FlashMMR effectively surpasses existing state-of- the-art moment retrieval methods, emphasizing the necessity and potential of future multi-moment retrieval frameworks. The proposed framework and dataset lay groundwork for future research more realistic and complex video temporal grounding tasks.
Acknowledgment
This work is supported by Australian Research Council (ARC) Discovery Project DP230101753.
References
Lisa Anne Hendricks, Oliver Wang, Eli Shechtman, Josef Sivie, Trevor Darrell, and Bryan Russell. Localizing moments video with natural language. ICCV, pages 5803-5812, 2017.
[2] Zhuo Cao, Bingqing Zhang, Heming Du, Xin Yu, Xue Li, and Sen Wang. Flashvig: Feature layering and adaptive score handling network for video temporal grounding. In WACV, pages 9208 9218, 2025.
[3] Junyoung Chung. Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv: 1412.3555, 2014.
Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He. Slowfast networks for video recognition. In ICCV, pages 6202-6211, 2019.
[5] Jiyang Gao, Chen Sun, Zhenheng Yang, and Ram Nevatia. Tall: Temporal activity localization via language guery. In ICCV, pages 5267-5275, 2017.
Runzhou Ge, Jiyang Gao, Kan Chen, and Ram Nevatia. Mac: Mining activity concepts for language-based temporal localization. WACV, pages 245-253. IEEE, 2019.
[7] Soham Ghosh, Anuva Agarwal, Zarana Parekh, and Alexander Hauptmann. Excl: Extractive clip localiza- tion using natural language descriptions. arXiv preprint 12775 2019.
10

---

[8] Cheng Huang, Yi-Lun Wu, Hong- Han Shuai, and Ching- -hhn Huang. Semantic fusion augmentation and semantic boundary detection: A novel approach multi-farget video moment retrieval. In WACV, pages 6783-6792, 2024.
[9] Jinhyun Jang, Jungin Park, Jin Kim, Hyeongjun Kwon, and Kwanghoon Sohn. Knowing where to focus: Event-aware transformer for video grounding. In ICCV, pages 13846-13856, 2023.
[10] Ranjay Krishna, Kenji Hata, Frederic Ren, Li Fei-Fei, and Juan Carlos Niebles. Dense-captioning events in videos. In ICCV, pages 706-715, 2017.
[11] Jie Lei, Licheng Yu, Tamara L Berg, and Mohit Bansal. Tvr: A large-scale dataset for video-subtitle moment retrieval. ECCV, pages 447 463. Springer, 2020.
[12] Lei, Tamara I. Berg, and Mohit Bansal, Detecting moments and highlights videos via natural language queries. NeurlPS, 34:1 14:11846-11888 2021.
[13] Suyan Li, Fuxiang Huang, and Lei Zhang. A survey of multimodal composite editing and retrieval. Vis. Intell., 3(1), 2025.
[14] Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. ICCV. pages 5980-2988, 2017.
[15] Daizong Liu, Xiaoye Qu, Jianfeng Dong, Pan Zhou, Yu Cheng, Wei Wei, Zichuan Xu, and Yulai Xic. Context-aware biaffine localizing network for temporal sentence grounding. In CVPR, pages 1 11235-11244 2021.
[16] Daizong Liu, Xiaoye Qu, and Wei Hu. Reducing the vision and language bias for temporal sentence grounding. In ACM MM, pages 4092 4101, 2022.
[17] Meng Liu, Xiang Wang, Liqiang Nie, Xiangnan He, Baoquan Chen, and Tat-Seng Chua, Attentive moment retrieval in videos. In SIGIR, pages 15-24, 2018.
[18] Ye Liu, Jixuan Hc, Wanhua Li, Junsik Kim, Donglai Wei, Hanspeter Pfister, and Chang Wen Chen. -uning:: Efficient image-to-video transfer learning for video temporal grounding. In ECCV, 2024.
[19] Chujie Lu. Long Chen, Chilie Tan, Xiaolin Li, and Jun Xiao. Debug: A dense bottom-up grounding approach for natural language video localization. In EMNLP-IJCNLP, pages 5153,, 2019.
[20] WonJun Moon, Sangeek Hyun, SangUk Park, Dongehan Park, and Jae-Pil Heo. Query-dependent video representation for moment retrieval and highlight detection. In CVPR, pages 23023-23033, 2023.
2023.
[21] WonJun Moon, Sangeek Hyun, SuBeen Lee, and Lae-Pil Heo. Correlation Correlation-uuided query gueryddependyncy calibration in video representation learning for temporal grounding. arXiv preprint 1.8853231 2024.
[22] Jonghwan Mun, Minsu Cho, and Bohyung Han. Local-global video-text interactions for temporal grounding. In CVPR, pages 10810-10819 19, 2020.
[23] Taichi Nishimura, Shota Nakada, Hokuto Munakata, and Tatsuya Komatsu. Lighthouse: A user-friendly library for reproducible video moment retrieval and highlight detection. In EMNLP, 2024.
[24] Mayu Otani, Yuta Nakashima, Esa Rahtu, and Janne Heikkilä, Uncovering hidden challenges in query- based video moment retrieval. arXiv preprint 20099 00325, 2020.
[25] You Qin, Qilong Wu, Yicong Li, Wei Ji, Li Li, Pengcheng Cai, Lina Wei, and Roger Zimmermann. Generalized video moment retrieval. In ICLR, 2025.
[26] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In JCML, pages 8748 8763. PMLR, 2021.
[27] Michaela Regneri, Mareus Rohrbach, Dominikus Wetzel, Stefan Thater, Bernt Schiele, and Manfred Pinkal. Grounding action descriptions in vidcos. ACL, 1:25-36, 2013.
[28] Cristian Rodriguez, Edison Marrese- Taylor, Fatemeh Sadat Salch, Hongdong Li, and Stephen Gould. Proposal-free temporal moment localization of a natural language query in video using guided attention. In WACV, pages 2464 2473, 2020.

---

[29] Cristian Rodriguez- Opazo, Edison Marrese-Tay Basura Fernando, Hongdong Li, and Stephen Gould. Dori: Discovering object relationships for moment localization of natural language query in video. In WACV, pages 1079-1088, 2021.
[30] Dian Shao, Yu Xiong. Yue Zhao, Qinggiu Huang. Yu Qiao, and Dahua Lin. Find and focus: Retrieve and localize video events with natural language queries. In ECCV, pages 200-216, 2018.
[31] Hao Sun, Mingyao Zhou, Wenjing Chen, and Wei Xie. Tr-detr: Task-reciprocal transformer for joint moment retrieval and highlight detection. In AAAL, pages 4998 5007, 2024.
[32] Jingwen Wang. Lin Ma, and Wenhao Jiang. Temporally grounding language queries in videos by contextual boundary-aware prediction. In AAAL, pages 12168-12175. 2020.
[33] Jianlong Wu, Wei Liu, Ye Liu, Meng Liu, Liqiang Nie, Zhouchen Lin, and Chang Wen Chen. survey on video temporal grounding with multimodal large language model. TPAMI, pages 1-20. 2025.
[34] Shaoning Xiao, Long Chen, Songyang Zhang, Wei Ji. Jian Shao, Lu Ye, and Jun Xiao. Boundary proposal network for two-stage natural language video localization. In AAAI, pages 2986-2994, 2021.
[35] Yicheng Xiao, Zhuoyan Luo, Yong Liu, Yue Ma, Hengwei Bian, Yatai Ji, Yujiu Yang, and Xiu Li. Bridging the gap: A unified video comprehension framework for moment retrieval and highlight detection. In CVPR. pages 18709-18711 19, 2024.
[36] Huijuan Xu, Kun He, Bryan A Plummer, Leonid Sigal, Stan Sclaroff, and Kate Saenko. Multilevel language and vision integration for text-to-clip retrieval. In AAAI, pages 9062-9069, 2019.
[37] Yingjia Xu, Mengxia Wu, Zixin Guo, Min Cao, Mang Ye, and Jorma Laaksonen. Efficient text-to-video retrieval via multi-modal multi-tagger derived pre-screening. Vix. Intell., 3(1), 2025.
[38] Tang Yansong, Dajun Ding, Yongming Rao, Yu Zheng, Danyang Zhang, Lili Zhao, Jiwen Lu, and Jie Zhou. Coin: A large-scale dataset for comprchensive instructional video analysis. In CVPR, 2019.
[39] Yitian Yuan, Ma, Jingwen Wang, Wei Liu, and Wenwu Zhu. Semantic conditioned dynamic modulation for temporal sentence grounding in videos. NeurlPS, 32, 2019.
[40] Abhay Zala, Jaemin Cho, Satwik Kottur, Xilun Chen, Barlas Oguz, Yashar Mehdad, and Mohit Bansal. Hierarchical video-moment retrieval and step-captioning. In CVPR, 2023.
[41] Runhao Zeng, Haoming Xu, Wenbing Huang, Peihao Chen, Mingkui Tan, and Chuang Gan. Dense regression network for video grounding. In CVPR, pages 10287-10296, 2020.
[42] Bingqing Zhang, Zhuo Cao, Heming Du, Yang Li, Xue Li, Jiajun Liu, and Sen Wang. Quantifying and narrowing the unknown: Interactive text-to-video retrieval via uncertainty minimization. In ICCV2025, 2025.
[43] Bingqing Zhang, Zhuo Cao, Heming Du, Xin Yu, Xue Li, Jiajun Liu, and Sen Wang. Tokenbinder: Text-video retrieval with one-to-many alignment paradigm. In WACV, pages 4957-4967. IEFF, 2025.
[44] Da Zhang, Xiyang Dai, Xin Wang, Yuan-Fang Wang, and Larry S Davis. Man: Moment alignment network for natural language moment retrieval via iterative graph adjustment. In CVPR, pages 1247-1257, 2019.
[45] Songyang Zhang, Houwen Peng, Jianlong Fu, and Jicho Luo. Learning 2d temporal adjacent networks for moment localization with natural language. In AAAI, pages 12870-1288 12877, 2020.
[46] Zhu Zhang, Zhijie Lin, Zhou Zhao, and Zhenxin Xiao. Cross-modal interaction networks for query-based moment retrieval in videos. In SIGIR, pages 655-664, 2019.
[47] Luowei Zhou, Chenliang Xu, and Jason Corso. Towards automatic learning of procedures from web instructional videos. In AAAI, pages
2018
12

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract and introduction elearly state the main contributions: FlashMMR framework, QV-M² dataset, and new evaluation metrics. Also see supplementary material for more theoretical and experimental evidence.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
Section 6 discusses limitations. We also report computational complexity
analysis in supplementary material.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
13

---

Answer: [NA]
Justification: The paper does not contain formal theoretical results or proofs; it focuses on model design, dataset construction, and empirical evaluation.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: The paper provides detailed descriptions of the dataset construction, model architecture, training settings, and evaluation protocols. We also include dataset annotations in the supplementary material support full reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data provided or not.
the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
14

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: We include the dataset annotations in the supplementary material support reproduction of our results. And we will spen-source code and checkpoint on github.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
httppp.ccppubllcguuudeeee Codeeubmii ssi criionPooiccy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Sectionn5.1]p specciiccs the training setup, including dataset, feature extractors, optimizer, loss weights, and GPU settings.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: We not report error bars or statistical tests, as all evaluation baselines follow deterministic protocols for fair comparison. This is consistent with prior work in moment retrieval, which also does not report statistical significance.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
15

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question:
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [No]
Justification: We report the GPU type used (RTX 4090) in Section 5.1 but we not provide details memory usage execution time. Our experiments follow standard settings similar to prior work, and total compute usage is moderate.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethies
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: Our research uses publicly available data under CC BY-NC-SA 4.0 license and is strictly intended for academic purposes. We follow the NeurlPS Code of Ethics and ensure compliance with responsible data use, faimess, and transparency principles.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The Broader Impact discussion is provided in the supplementary material.
16

---

Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairesss considerations (e.g., dep.oymmly of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our work does not involve high-risk models or data. The dataset is based on OVHighlights with additional human annotations, and both the data and models are intended solely for research purposes under a CC BY-NC-SA 4.0 license.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We use the QVHighlights dataset under its CC BC-NNC--S 4.0 license. We also properly cite and credit all external assets, including CLIP (6). SlowFast [4, etc., with references and usage terms clearly indicated.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
17

---

The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of in the package should be provided. For popular datasets, paperswi thwode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided. this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
Answer: [Yes]
Justification: We introduce the QV-M² dataset, which built upon QVHighlights and fol- lows the same CC BY-NC-SA 4.0 license. Documentation regarding annotation guidelines, quality control, and licensing provided in Section 3 and the annotation file attached in the supplementary material.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [Yes]
Justification: Our dataset involves manual annotation by trained annotators. Annotation guidelines, interface screenshots, and related details are provided in the supplementary material.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
18

---

Answer: [Yes]
Justification: The data annotation task was reviewed and approved by our institution's ethics committee equivalent process. The task involved no personal sensitive information and posed minimal risk to participants.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: LLMs were only used for writing and editing assistance. They are not involved in the core methodology, experiments, or scientific contributions of the paper.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should not be described.
19


Reviews:
[{"review_id": "mg8URaIHpr", "reviewer": "Reviewer_pD2D", "summary": "This paper introduces **Multi-Moment Video Retrieval (MMVR)**, a new video-text retrieval task addressing queries describing multiple distinct moments within a single video. The authors propose the **Cross-Moment Interaction (CMI)** module to explicitly model interactions between multiple moments. Additionally, they introduce datasets derived from existing temporal moment localization datasets adapted for MMVR. Experiments demonstrate substantial performance improvements over baselines, with good ablation studies validating the CMI module's effectiveness.", "questions": "1. Can you provide analyses or discussion showing that your new dataset accurately reflects natural multi-moment retrieval scenarios? Have you examined potential biases introduced by your dataset construction methodology? Given that we know that previous datasets for temporal moment localization suffer of temporal biases in their annotations, could you please use as a reference the work of https://arxiv.org/abs/2101.09028 that evaluate the bias of ActivityNet and CharadesSTA and evaluate yours. I would like to see the plot in Fig 2 for your dataset. Also, could you make the same plot for your predictions https://openaccess.thecvf.com/content/ICCV2023W/CLVL/papers/De_la_Jara_An_Empirical_Study_of_the_Effect_of_Video_Encoders_on_ICCVW_2023_paper.pdf such that we can evaluate potential  bias of the predictions of the model.\n\n2. Could you clarify the computational overhead associated with your proposed CMI module, particularly runtime and scalability with longer videos or larger candidate sets?\n\n3. Have you explored simpler or alternative interaction methods (e.g., hierarchical or sparse interactions) that might maintain performance while reducing computational overhead?\n\n4. Are there plans or initial experiments evaluating your method on other relevant datasets, such as instructional datasets (YouCookII or HowTo100M)?", "limitations": "The authors acknowledge limitations regarding dataset construction. However, additional analysis and explicit discussion on computational scalability, representativeness, and potential biases would further strengthen this section.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "### Strengths:\n-  The paper introduces an important and practical task, extending beyond traditional single-moment retrieval problems.\n-  The proposed Cross-Moment Interaction (CMI) module explicitly captures cross-moment interactions, conceptually enhancing retrieval performance.\n-  Comprehensive experiments and ablations demonstrate clear advantages over baseline methods.\n-  The manuscript is well-written, structured, and provides detailed implementation and experimental information.\n\n### Weaknesses:\n\n- The MMVR datasets are adapted from widely-used existing datasets, limiting the originality and potentially introducing biases.\n- Explicit modeling of cross-moment interactions might introduce significant computational complexity, but this is insufficiently analyzed.\n- The proposed approach heavily relies on established attention mechanisms and does not introduce substantial technical innovations beyond their application to the MMVR task.\n- Constructing MMVR scenarios by combining existing dataset moments could introduce unintended biases and may not reflect natural multi-moment scenarios.\n\n### Comments and Suggestions\n\n- **Abstract (Lines 3–7)**:  The problem of multiple occurrences of a described event within a video is known but relatively overlooked in temporal localization. Charades-STA contains instances where queries occur multiple times (see Figure 8 in supplemental material of [DORi, Rodriguez-Opazo et al., WACV 2021](https://openaccess.thecvf.com/content/WACV2021/html/Rodriguez-Opazo_DORi_Discovering_Object_Relationships_for_Moment_Localization_of_a_Natural_WACV_2021_paper.html)). Explicitly acknowledging such scenarios would strengthen your motivation and clearly position your proposed MMVR task.\n\n- **Abstract (Lines 10–12)**:  Do the SMR annotations explicitly encode temporal sequencing information (e.g., distinguishing between the first and second occurrences of similar events, such as \"the second time the person opens the door\")? Clarifying this would more effectively demonstrate the scope of your annotations.\n- **Introduction (Lines 25–27)**:  Given your mention of instructional videos, have you evaluated or considered evaluating your method on established instructional datasets like [YouCookII](http://youcook2.eecs.umich.edu/), [COIN](https://coin-dataset.github.io/),  or [HowTo100M](https://www.di.ens.fr/willow/research/howto100m/)? Such evaluations could better demonstrate your approach's practical applicability and robustness.\n\n- *Introduction (Lines 38–40)**:  While your annotation approach is good (\"gold-standard\"), it could be beneficial to consider a scalable \"silver-standard\" dataset creation method where existing other datasets queries are generalized or rephrased automatically without direct human annotation, particularly relevant for instructional scenarios (e.g., generalizing \"cutting tomato\" to \"cutting vegetables\" in [YouCookII](http://youcook2.eecs.umich.edu/),).\n\n- **Table 1**:  Important relevant datasets such as [YouCookII](http://youcook2.eecs.umich.edu/), [COIN](https://coin-dataset.github.io/), and [HiREST](https://github.com/j-min/HiREST) are missing. Including these would enhance completeness and better position your task relative to existing moment retrieval literature.\n\n- **Related Work**:   Notable related methods such as [TMLGA](https://arxiv.org/abs/1908.07236), [DORi](https://arxiv.org/abs/2010.06260), and [ExCL](https://arxiv.org/abs/1904.02755) are absent from your literature review. Including these references would significantly enhance the completeness and clarity of your work’s context.", "was_revised": true, "final_justification": "My concerns about the quality of the dataset, possible temporal bias has been correctly addressed by the authors. It would be great to see the silver dataset approach mentioned in the review process.", "rebuttal": "We thank the reviewer for highlighting our practical multi-moment task and the effectiveness of the PV module. We address the concerns below:\n\n**W1, Q1:** We follow the bias evaluation methodology proposed in [1], particularly the analysis of temporal annotation bias in Fig. 2. We replicate similar experiments on QV-M² and our FlashMMR model. \n\nThe results show that our dataset exhibits significantly lower temporal bias compared to other commonly used benchmarks such as Charades-STA[2] and TACoS[3]. As image uploads are not supported here, we report the distribution of normalized start/end positions in the training set below. The results demonstrate that QV-M² has a much more uniform temporal distribution, highlighting its advantage in mitigating temporal bias. \"Pred.\" refers to the predictions generated by FlashMMR. \"Interval\" denotes the normalized temporal interval of the video.\n\n| Interval | QV-M² Start % | QV-M² End % | Charades-STA Start % | Charades-STA End % | TaCos Start % | TaCos End % | Pred. Start % | Pred. End % |\n|----------|---------|-------|---------|-------|---------|-------|---------|-------|\n| [0.0, 0.1) |19.90% | 5.32% |31.39% | 0.42% |21.82% | 11.53% |15.01% | 7.49% |\n| [0.1, 0.2) |8.73% | 8.02% |9.85% | 6.80% |16.47% | 14.71% |9.52% | 10.47% |\n| [0.2, 0.3) |9.84% | 10.05% |8.59% | 10.54% |12.61% | 13.44% |11.60% | 11.57% |\n| [0.3, 0.4) |9.04% | 9.06% |8.51% | 11.90% |9.90% | 9.45% |9.15% | 8.96% |\n| [0.4, 0.5) |10.26% | 10.34% |8.87% | 12.82% |7.92% | 8.96% |10.63% | 10.96% |\n| [0.5, 0.6) |9.87% | 10.15% |9.59% | 9.24% |7.19% | 6.44% |10.79% | 10.53% |\n| [0.6, 0.7) |9.08% | 8.56% |9.62% | 8.30% |6.77% | 7.58% |9.56% | 9.50% |\n| [0.7, 0.8) |8.49% | 8.57% |8.03% | 8.24% |5.40% | 5.32% |9.04% | 9.11% |\n| [0.8, 0.9) |9.54% | 9.99% |4.67% | 8.44% |5.26% | 6.55% |9.71% | 10.74% |\n| [0.9, 1.0) |5.25% | 19.96% |0.89% | 23.31% |6.66% | 16.03% |5.00% | 10.66% |\n\n&nbsp;\n\n**W2, Q2:** We thank the reviewer’s question regarding the computational overhead of our proposed Post-Verification module. To clarify, we conducted thorough experiments analyzing both runtime and scalability concerning video length and the number of candidate predictions.\n\n**Computational Complexity Analysis:**\nThe theoretical complexity of the Post-Verification module are as follows:\n\n* **Inference:** Time Complexity: $O(K\\bar{L}d^2)$, Memory Complexity: $O(K\\bar{L}d)$\n\n* **Training:** Time Complexity: $O(K\\bar{L}d^2 + \\bar{V}^2d)$, Memory Complexity: $O(K\\bar{L}d + \\bar{V}^2)$\n\nWhere, $K$ is the number of candidate predictions per video, $d$ is the feature dimension, $\\bar{L}$ is the average length of candidate, $\\bar{V}$ is the average length of video.\n\n**Experimental Validation:**\nWe empirically validated the above theoretical analysis by varying video lengths and candidate counts. All experiments utilized a batch size of 32, a feature dimension of 256, and were performed on an NVIDIA RTX 4090 GPU with an Intel(R) Core(TM) i9-14900KF CPU.\n\n(1) Scalability with Video Length:\n\n| Video Length ($\\bar{V}$) | Inference Runtime (s) | Inference FLOPs (M) | Training Runtime (s) | Training FLOPs (M) |\n|-----|---------|-------|--------|----------|\n| 50 | 0.0133 | 19051.2 | 0.0225 | 19175.0 |\n| 100 | 0.0132 | 19051.2 | 0.0229 | 19546.0 |\n| 150 | 0.0131 | 19051.2 | 0.0259 | 20164.4 |\n| 200 | 0.0130 | 19051.2 | 0.0340 | 21030.2 |\n| 250 | 0.0129 | 19051.2 | 0.0459 | 22143.3 |\n| 300 | 0.0130 | 19051.2 | 0.0598 | 23503.8 |\n| 400 | 0.0130 | 19051.2 | 0.0927 | 26966.8 |\n\nThese results confirm our theoretical expectation: during inference, runtime and FLOPs remain stable irrespective of video length. In contrast, during training, runtime and FLOPs increase **linearly** with video length due to the $\\bar{V}^2d$ term. Since $\\bar{V}^2 < K\\bar{L}d$ in our setting, the $\\bar{V}^2d$ term remains a lower-order contribution, which explains why the additional FLOPs during training are moderate.\n\n(2) Scalability with Candidate Set Size:\n\n| Candidate Count ($K$) | Inference Runtime (s) | Inference FLOPS (M) | Training Runtime (s) | Training FLOPS (M) |\n|----|---------|--------|-------|-------|\n| 50 | 0.0134 | 19051.2 | 0.0261 | 20164.4 |\n| 60 | 0.0156 | 22861.5 | 0.0284 | 23974.7 |\n| 70 | 0.0181 | 26671.7 | 0.0310 | 27784.9 |\n| 80 | 0.0203 | 30482.0 | 0.0337 | 31595.2 |\n| 90 | 0.0230 | 34292.2 | 0.0364 | 35405.4 |\n\nThese experiments demonstrate a clear **linear relationship** between candidate count and computational overhead in both inference and training phases, again aligning well with our complexity analysis. \n\nIn summary, these experimental results demonstrate that, under the current setting, the computational complexity of our Post-Verification module is reasonable and acceptable.\n\n&nbsp;\n\n**W3:** The method indeed borrows several modules from previous approaches and combines them sensibly. However, these modules do not adapt well to the new MMR task. The newly proposed post verification module is a useful addition given the task setting, and is shown to work well in the ablation.\n\n&nbsp;\n\n**W4:** We would like to clarify that our MMR scenarios are not constructed by combining existing single-moment annotations. Instead, all multi-moment labels are manually annotated from scratch. Furthermore, to ensure annotation consistency and minimize potential bias, we employed a structured quality control process following the annotation phase.\n\n&nbsp;\n\n**C1:** Thank you for the suggestion. We will revise the abstract to emphasize that such cases already exist in previous datasets, in order to strengthen our motivation and clearly position the proposed MMR task.\n\n&nbsp;\n\n**C2:** Yes. We will clearly state that the new annotations explicitly include temporal sequencing information in the revised version.\n\nAs mentioned in the second point of our annotation guidelines, both SMR and MMR annotations encode temporal sequencing information. For example, in the video *1tGsRVsMtlc_660.0_810.0*, the query *\"A woman shows the clothing before pointing the camera at the mirror.\"* contains the temporal cue *\"before\"*, indicating that the event should occur prior to *\"pointing the camera at the mirror.\"* If one were to simply align the visual content with the text while ignoring this temporal cue, the subsequent action would mistakenly be included, making it impossible to correctly determine the end time of the intended event.\n\n&nbsp;\n\n**C3, Q4:** Yes. We still evaluate FlashMMR on YouCookII (an instructional-video dataset for SMR), even though newly proposed PV module is meant to address the multi-moment retrieval (MMR) task.\n\nAs a result, the single-moment retrieval (SMR) performance primarily reflects the capability of our baseline model, FlashVTG, as supported by the experimental results below. Our proposed module does not significantly affect the SMR task. Notably, videos in the YouCookII dataset are much longer than those in previously used datasets (e.g., QVHighlights and Charades-STA), while our baseline model is not well-suited for handling such long videos.\n\n|Method|G-mAP|mIoU@1|mR@1|\n|-|-|-|-|\n|FlashVTG (baseline)|12.50|17.87|13.38|\n|FlashMMR|12.45|17.57|13.04|\n\n&nbsp;\n\n**C4:** Thank you for the insightful suggestion. We agree that automatic query generalization, e.g., turning \"cutting tomato\" into \"cutting vegetables\", could help build a large \"silver-standard\" dataset.\n\nA practical caveat is that the generated queries from instructional datasets tend to stay at the action level, whereas mainstream MR benchmarks use event-level queries. Even so, the technique remains attractive for producing large-scale pre-training data. For the present work, we therefore focus on a manually curated, event-level benchmark and leave \"silver-standard\" expansion as promising future work.\n\n&nbsp;\n\n**C5:** Thank you for the suggestion. We have added additional relevant datasets (YouCookII, COIN, and HiREST) to Table 1 in the revised version.\n\n|Dataset|Domain|Avg #moment per query | Avg Query Len | Avg ratio Moment/Video | #Moments / #Videos | Fully Human-Annotated |\n|-|-|-|-|-|-|-|\n|YouCookII|Cooking|1|8.7|1.9%|13.8K/2K|✓|\n|COIN|Open|1|4.9|9.8%|46.3K/11.8K|✓|\n|HiREST|Open|1|4.2|55.0%|2.4K/526|✓|\n\n&nbsp;\n\n**C6:** Thank you for the valuable suggestion. We have included citations of TMLGA, DORi, and ExCL in the revised Related Work section, as they are highly relevant to our task.\n\nThese methods all contribute to moment retrieval:\n\n- TMGLA proposes a proposal-free, end-to-end method for natural language-based temporal localization using a guided attention mechanism and soft labels to improve efficiency and accuracy.\n\n- DORi proposes a language-conditioned graph-based method for temporal moment localization. It models human-object-activity interactions via spatio-temporal graphs and uses iterative message passing to align video content with natural language queries.\n\n- ExCL proposes a modular framework that fuses video and text features via cross-modal LSTMs and directly predicts clip boundaries using span predictors.\n\n&nbsp;\n\n**Q3:** Yes. We agree that hierarchical or sparse interaction mechanisms may provide a more efficient way to model semantic consistency in the post-verification stage, especially by focusing on high-confidence segments.\n\nWe adopt a GRU-based design with $L_{repr}$ to model local and global temporal consistency between predicted moments and the query. In contrast, purely hierarchical or sparse interaction mechanisms may lack fine-grained global context, and their supervision signals can become sparse. As noted in the computational complexity analysis, the design remains efficient in scenarios where $\\bar{V}^2 < K\\bar{L}d$. Nonetheless, we acknowledge the potential benefits of alternative interaction strategies, and we consider this an important direction for future exploration.\n\n&nbsp;\n\n[1] https://arxiv.org/abs/2101.09028\n\n[2] Tall: Temporal activity localization via language query.\n\n[3] Grounding action descriptions in videos."}, {"review_id": "cIkfXkkR1k", "reviewer": "Reviewer_T6Ks", "summary": "This paper tackles the topic of Multi Moment Retrieval, whereby the model aims to retrieve all relevant moments within a given video, rather than being restricted to only retrieving one. It introduces the QV-M^2 dataset, which is an adaptation of QVHighlights with newly generated sentence queries and corresponding moments. It introduces the FlashMMR model which is optimised for MMR rather than SMR. It incorporates several ideas from previous SOTA moment retrieval models, and includes a post verification module which tries to ensure semantic consistency across the predicted moments. It is shown to have strong performance on the MMR task, largely outperforming SOTA SMR models.", "questions": "- What are the key differences between QV-M^2 and QVHighlights? I think that providing some example comparisons on how the sentence queries/moments are different could be useful. Perhaps it could be included in the supplementary. Table 1 provides some statistical comparisons, which give some insight, but I still find it difficult to really understand the specific ways in which it is different. \nIs the increased number of moments per video the main difference? While the moments/video is higher in QV-M^2, the dataset is also quite a bit smaller, so I could imagine that it may be possible to take a subset of QVHighlights that is about the same size and produce a similar moments/video value from the more densely annotated videos. So I imagine there is more to it than just that. \nAs QV-M^2 is one of the major contributions of the paper, I feel that it could be stressed/made clearer why it is better for benchmarking this task than QVHighlights. \n\n- I'm also not fully clear on what separates QV-M^2 from the NExT-VMR dataset. It is stated that for NExT-VMR \"its design and evaluation remain closely aligned with previous benchmarks, without targeted optimizations for the unique challenges of MMR\". It's not clear to me what these optimisations are, and how they are addressed by QV-M^2. A small elaboration would be helpful. As this is a concurrent work I don't have an expectation of it being necessarily better than NExT-VMR, I am just curious as to the meaning of the statement. \n\n- I would like to clarify that my understanding is correct - the approach always generates n moment candidates and ranks them based on confidence score. It then uses the knowledge of how many ground truth moments there are and selects the top k predictions based on that, and compares them with the ground truth moments? Therefore this does not predict the number of ground truth moments. Rather it is assumed that there is oracle knowledge of how many ground truth moments there are in a given video?\nIf the above is the case, then while I think this approach shows a good improvement at the task that it is performing, in order for this type of approach to be truly useful, I think it would have to also be able to predict the number of ground truth moments. This would complicate the metrics used, as it would have to account for cases where the model overpredicts the number of moments also. This is not to say that this paper is not a step in the right direction, as I think it is, but it is unlikely in practice that you will know how many ground truth moments exist in a given video. \n\n- I am leaning positive on this paper, but I would like to see some more information on the QV-M^2 dataset itself.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n- I think that taking steps towards multi-moment retrieval is good, as it reflects much more the real-life intricacies of moment retrieval, where there often is not a simple one-to-one correspondence between query and moment.\n- The QV-M^2 dataset appears to be a good contribution for the MMR task.\n- The FlashMMR approach shows strong performance compared with current SOTA for SMR\n- The method borrows several modules and ideas from previous approaches (e.g. ACA, dummy encoder, feature pyramid) and combines them in a sensible way\n- The post verification via semantic consistency is a nice addition given the task setting, and is shown to work well in the ablation.\n\nWeaknesses\n- It seems that there is some information missing. I don't believe the train/val/test sizes for QV-M^2 are included.\n- I also don't see the threshold for the IoU score in the mR@k metric listed.\n- I think there perhaps could be more of a comparison made between QV-M^2 and QVHighlights, as it is not immediately clear to me what the differences are.\n- Minor point - the repeated use of the letter k in notation is a bit confusing, e.g. it is used in the metrics and then for the layers in the feature pyramid", "was_revised": true, "final_justification": "I feel the rebuttal has addressed most of my concerns. The task is useful from the perspective of making the Moment Retrieval task more practical in real-world settings, and the approach seems to work well. The QV-M$^2$ dataset also seems useful. The one concern I have with the dataset is that the test set only contains about 300 queries, which is on the smaller side as the QV-Highlights test data is not freely available and can't be merged with it. Overall, I will retain my rating of borderline accept.", "rebuttal": "We appreciate the recognition of the new QV-M² dataset and FlashMMR’s state-of-the-art performance. We address the concerns below:\n\n**W1:** Thank you for pointing out this omission. Here we provide the Train/Val/Test split size.\n\nQV‑M² is an incremental dataset, and therefore use the full set of annotations in our experiments, including the original QVHighlights labels. The exact split statistics are now included in the revised manuscript:\n\n| Dataset           | Train | Val   | Test  |\n| ----------------- | ----- | ----- | ----- |\n| TACoS             | 9790  | 4,436  | 4,001|\n| Charades-STA      | 12,404  | -  | 3,720|\n| QVHighlights     | 7218 | 1550 | 1542 |\n| **QV‑M²**         | 8878 | 1779 | 1865 |\n\n&nbsp;\n\n**W2:** We stated the IoU threshold for mR\\@k in line 178 of the paper. And we will add the implementation details in supplementary.\n\nIn our implementation, we average results over multiple thresholds: \\[0.3, 0.35, 0.4, 0.45, 0.5, 0.55, 0.6, 0.65, 0.7, 0.75, 0.8, 0.85, 0.9, 0.95]. Here, we give the definition of **Mean Recall\\@k**:\n$$\n\\text{mR}@k = \\frac{1}{|\\mathcal{Q}|} \\sum_{q \\in \\mathcal{Q}} \\frac{1}{|\\mathcal{G}(q)|} \\sum_{\\text{gt} \\in \\mathcal{G}(q)} \\mathbf{1} \\left[ \\max_{i \\leq k} \\text{IoU}(\\text{pred}_i, \\text{gt}) \\geq \\tau \\right]\n$$\nwhere $\\tau$ is the IoU threshold for determining whether a prediction is considered a match to a ground truth, $k \\in \\{1,2,3\\}$ denotes the rank, $\\mathbf{1}[\\cdot]$ is the indicator function, $\\mathcal{Q}$ is the set of all queries, and $\\mathcal{G}(q)$ represents the ground truth moments associated with query $q$.\n\n&nbsp;\n\n**W3, Q1, Q4:** To better understand the dataset itself, we present more details of QV‑M² through the following comparisons and illustrative examples. And we will add it in the revised version.\n\n**Key differences between QV‑M² and QVHighlights.** Here is the primary differences between the two datasets across different dimensions.\n\n| Dimension | QVHighlights | **QV‑M²** | Rationale |\n|-----------|--------------|-----------|-----------|\n| Avg. relevant moments / query | 1.8 | **2.5** | Better captures the one‑to‑many nature of moment retrieval. |\n| Moment density (moments / video) | 16.4 % | 25.5 % | Higher density demands more precise temporal boundaries during training and evaluation. |\n| Annotation strategy | Positive single sentence | Multiple **positive** and **negative** sentences, emphasising **context and temporal dependencies** | Encourages cross‑moment reasoning. |\n| Annotation scope | Intra‑video only | Cross‑video dense annotation: moments for the *same* query can occur in multiple videos | Increases difficulty and supports video corpus moment retrieval. |\n| Evaluation metrics | Conventional mAP / IoU | Adds **G‑mAP**, **mAP@k _tgt**, **mR@k** tailored for multi‑moment retrieval | Seamlessly evaluates MMR scenarios while remaining compatible with SMR metrics. |\n\n**Annotation example.** Although our main goal was to obtain one‑to‑many labels, we intentionally increased the semantic richness of the queries—an aspect missing from earlier MR datasets and consistent with the enhanced annotation strategy outlined above. In revised manuscript we will provide additional examples to highlight the advantages of QV‑M². Here are some representative queries from QV‑M²:\n 1. *\"Drone pans over a marina and a fountain **before** landing on a park bench where two women wave.\"* (Context and temporal dependency)\n\n 2. *\"A woman shows the clothing **before** pointing the camera at the mirror.\"* (Context and temporal dependency)\n\n 3. *\"A woman **without** sunglasses in a blue shirt talks to the camera while walking outside.\"* (Negative description)\n\n 4. *\"A reporter in a yellow shirt **without** hat is reporting on a bushfire emergency.\"* (Negative description)\n\n&nbsp;\n\n**W4:** Thank you for pointing this out. We have updated it in the revised version, using lowercase *p* to denote the layers in the feature pyramid.\n\n&nbsp;\n\n**Q2:** We appreciate the reviewer’s question and would like to clarify the statement. \n\nWhile NExT-VMR is introduced for multi-moment retrieval (MMR), its average of only 1.51 ground-truth moments per query indicates that most queries still correspond to a single ground truth moment. Furthermore, its evaluation metrics remain unchanged from single moment retrieval (SMR) setting using R1@IoU and mAP@IoU over predicted segments, without modification to account for multiple relevant moments per query.\n\nIn contrast, QV-M² provides denser multi-moment supervision with an average of 2.5 ground truth moments per query, and supports evaluation that naturally extends standard SMR metrics without assuming a single target, enabling accurate and fair assessment of multi-moment predictions. We believe this better captures the unique challenges of MMR in both data and evaluation.\n\n&nbsp;\n\n**Q3:** Thank you for your insightful comment. You are correct that our current evaluation process assumes oracle knowledge of the number of ground truth moments per query. \n\nSpecifically, model itself does not predict the number of relevant moments, but generate a fixed number of moment predictions (e.g., *n* = 10) and rank them by confidence scores, selecting the top-*k* predictions to do the evaluation. This design aligns with prior moment retrieval benchmarks (e.g., MomentDETR[1], QD-DETR[2]) and ensures fair comparison across models. Notably, our metric mR@k partially reflects the model’s ability to handle under-prediction, as it measures the recall of ground truth moments within the top-*k* predictions.\n\nWe acknowledge that this assumption limits the practical applicability of the system in open-world settings. Incorporating moment count prediction along with precision-recall-based metrics that account for over- and under-prediction, would be valuable future directions. We appreciate your recognition that our method and dataset is a step toward more realistic video understanding.\n\n&nbsp;\n\n[1] Lei, J., Berg, T. L., & Bansal, M. (2021). Detecting moments and highlights in videos via natural language queries. Advances in Neural Information Processing Systems, 34, 11846-11858\n\n[2] Moon, W., Hyun, S., Park, S., Park, D., & Heo, J. P. (2023). Query-dependent video representation for moment retrieval and highlight detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 23023-23033)."}, {"review_id": "v3Bc6MXFLO", "reviewer": "Reviewer_yR7Q", "summary": "This paper propose QVHighlights Mult-Moment, a high-quality datasets with new evaluation metrics tailored for multi-moment retrieval, which fulfill the gap between current moment retrieval tasks and real-world applications. Based on the newly proposed datasets, a novel Multi-Moment Retrieval framework FlashMMR is proposed incorporated with a Multi-Moment Post-Verification module. Experiments and evaluation are conducted under the proposed method and the baselines, providing a comprehensive assessment of model performance in MMR.", "questions": "See weaknesses.", "limitations": "The authors adequately addressed the limitations and potential negative societal impact of their work. As mentioned in weakness 2, authors could discuss some future potential methods to further expand the size of MMR dataset, based on the current experience of manual annotation.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The overall idea is novel and the task is important for real application. \n2. The experiment result is solid, with both quantitative and qualitative results of high performance of multi-moment retrieval.\n3. The proposed dataset is the first fully human-annotated MMR dataset, designed to benchmarking Multi-Moment Retrieval. New evaluation metrics are proposed for a comprehensive assessment.\n\nWeaknesses:\n1. With the rapid development of temporal-aware video large language models, I suggest authors could test few related models on the proposed benchmark, revealing and exploring the generalized video models on the performance of multi-moment retrieval.\n2. Since this paper mentioned the proposed dataset’s limited scale may constrain future progress, authors could discuss some future potential methods to further expand the size of MMR dataset, based on the current experience of manual annotation.\n3. A minor possible layout typo about ‘Post-Processing with Feature Refinement.’ In line 223.", "was_revised": true, "final_justification": "The authors adequately addressed my concern. I will maintain my score.", "rebuttal": "We thank the reviewer for noting the task’s novelty and our strong empirical results. We address the concerns below:\n\n**W1:** We compared our model with the state-of-the-art temporal-aware video large language models in a zero-shot setting.\n\nWe selected Chat-UniVi [1], a CVPR 2024 Highlight paper, as our baseline model. It can encode up to 100 frames per video, giving the temporal coverage MMR needs, whereas others like VideoLLaVA [2] and Video-ChatGPT [3] handle only around 8 frames, which is insufficient for reliable multi-moment retrieval. Below we provide the exact prompt and the corresponding zero-shot results.\n\n```txt\nThis is a {duration:.2f} second video clip. \nProvide single or multiple time intervals within the range [0, {duration:.2f}], \nformatted as '[a, b]', that corresponds to the segment of the video best matching the query: {query}.\nRespond with only the numeric interval.\n```\nThe zero-shot evaluation results are provided below, highlighting the challenges posed by QV-M².\n\n| Model | mAP@1_tgt | mAP@2_tgt | mAP@3_tgt | mIoU@1 | mR@1 |\n|-------|----------|------------|---------|---------|-------|\n| Chat-UniVi | 3.39 | 0.78 | 0.47 | 15.47 | 5.83 |\n\n&nbsp;\n\n**W2, L1:** Expanding the scale of MMR datasets is indeed crucial for driving further progress. \n\nBased on our annotation experience with QV-M², we see several promising directions to scale up:\n\n1. **Synthetic Multi-Moment Construction from other Datasets**: By transplanting existing ground-truth moments from other datasets (e.g., TACoS, Charades-STA) into unrelated regions of the same video, we can simulate multi-moment scenarios at scale. While this approach may introduce artifacts, it offers a practical way to generate large-scale pseudo-MMR data for pretraining.\n\n2. **Leveraging Video Captioning Datasets**: Since the most resource intensive aspect of MR dataset construction lies in timestamp annotation. Video captioning datasets are already temporally aligned with captions. These captions could be converted into SMR style annotations, and LLMs may be used to group semantically related but temporally disjoint segments to simulate multi-moment supervision.\n\n3. **Semi-Automatic Annotation with Vision-Language Models (VLMs)**: Another promising direction is to leverage VLMs to generate candidate temporal segments for a query, followed by human verification. This would reduce manual effort while maintaining annotation quality.\n\nWe believe these directions are promising for extending MMR datasets beyond current scale and will incorporate them into the revised version of our work.\n\n&nbsp;\n\n**W3:** Thank you for pointing this out. We will make correction in revised version, along with a careful review of similar formatting issues.\n\n&nbsp;\n\n[1] Jin, P., Takanobu, R., Zhang, W., Cao, X., & Yuan, L. (2024). Chat-univi: Unified visual representation empowers large language models with image and video understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 13700-13710).\n\n[2] Lin, B., Ye, Y., Zhu, B., Cui, J., Ning, M., Jin, P., & Yuan, L. (2024, November). Video-LLaVA: Learning United Visual Representation by Alignment Before Projection. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (pp. 5971-5984).\n\n[3] Maaz, M., Rasheed, H., Khan, S., & Khan, F. (2024, August). Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 12585-12602)."}, {"review_id": "RYVqCOMcnw", "reviewer": "Reviewer_LJnf", "summary": "This paper  addresses the limitation of existing Single-Moment Retrieval (SMR) methods that typically focus on single moments, whereas real-world queries often correspond to multiple moments in a video. To address this, the authors introduce QV-M2, a fully manually annotated dataset designed for multi-moment retrieval (MMR). They also propose new evaluation metrics tailored for MMR tasks, extending standard single-moment retrieval metrics. Additionally, this paper presents FlashMMR, a novel framework for MMR. Experiments show that FlashMMR outperforms existing methods in MMR tasks, establishing a baseline for future research.", "questions": "Figure 3, which depicts the model framework, is somewhat confusing and seems to contradict the actual code implementation. For instance, in the blue \"Feature Extraction\" section, it appears as if the dummy token is concatenated after the text token, whereas in the code, the dummy token is placed before the text token. Additionally, within the same section, the figure suggests that the concatenated dummy and text tokens are used as the key (k), while the text token alone is used as the value (v). This is illogical since the lengths of k and v would not match. Upon closer inspection of the code in transformer.py at line 280, it is evident that v is derived from the entire concatenated sequence and k is v with position encoding added. Furthermore, the module names in the figure do not fully align with the descriptions in section 4.1 \"Feature Extraction and Multi-Scale Temporal Processing\" of the paper. For example, the \"Fusion Feature Encoder\" in the figure presumably refers to the Transformer Encoder mentioned in line 204 of the paper. It is highly recommended that the authors refine the figure3 to better reflect the actual implementation. In addition, I'm curious to know if you have any measures to ensure that the predicted segments do not overlap. Is there a possibility that the top-k predictions might all be overlapping?", "limitations": "yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths: \n- The dataset is fully manually annotated, and a verification procedure is in place to ensure the accuracy of the annotations.\n- The experimental results in Table 2 demonstrate the dataset's effectiveness through cross-dataset validation.\n- The use of IoU between ground-truth and predicted temporal windows to guide the refined score p in the Post-Verification Module is an innovative approach.\n\nWeaknesses: \n- There is a basic formatting error in line 223 where a bold header was not properly line-breaked.\n- Figure 3 is confusing and contradicts the actual code implementation. The depiction of dummy tokens and text tokens in the \"Feature Extraction\" section is inconsistent with the code, and the use of k and v is illogical.\n- The paper lacks detailed discussion on the dummy encoder.", "was_revised": true, "final_justification": "The rebuttal has satisfactorily addressed most of my concerns. I would like to revise my score to BA.", "rebuttal": "We appreciate the acknowledgement of the rigorously verified dataset and the innovative IoU-guided post-verification module. We address the concerns below:\n\n**W1:** Thank you for pointing this out. We will make a correction in the revised version, along with a careful review of similar formatting issues.\n\n&nbsp;\n\n**W2:** Thank you for your detailed comments regarding Figure 3.\n\n**1. Dummy token placement:** Although this is just a schematic illustration of the model, we will revise the position of the dummy token concatenation to reduce confusion.\n\n**2. Key/Value construction in ACA:** The Adaptive Cross-Attention (ACA) [1] module’s internal attention structure is different from the classic cross-attention structure, especially in the key/value construction. Our ACA module adopts the exact same design and code implementation from CG‑DETR[1]/FlashVTG[2]. Here we clarify the exact behaviour of this module.\n\n**Formal definition**\n\n| role                | symbol                               | shape               |\n| ------------------- | ------------------------------------ | ------------------- |\n| query = video clips | $p_Q(\\mathbf V)$                     | $L_v \\times d$      |\n| key = text + dummy  | $p_K([\\mathbf Q,\\tilde{\\mathbf D}])$ | $(L_q+L_d)\\times d$ |\n| value = text only   | $p_V(\\mathbf Q)$                     | $L_q\\times d$       |\n\nThe attention weights and fused features are\n\n$$\nW_{i,j}= \\frac{\\exp\\bigl((Q_iK_j)/\\sqrt d\\bigr)}\n              {\\sum_{k=1}^{L_q+L_d} \\exp\\bigl((Q_iK_k)/\\sqrt d\\bigr)}, \\qquad\nF_i=\\sum_{j=1}^{L_q} W_{i,j}\\,V_j,\\quad F\\in\\mathbb R^{L_v\\times d}.\n$$\n\nWe denote the input video clips $V$ as a query matrix $p_Q(\\mathbf V) \\in \\mathbb{R}^{L_v \\times d}$, the key matrix as $p_K([\\mathbf Q,\\tilde{\\mathbf D}]) \\in \\mathbb{R}^{(L_q + L_d) \\times d}$ formed by concatenating the text features $\\mathbf Q$ with dummy tokens $\\tilde{\\mathbf D}$, and the value matrix as $p_V(\\mathbf Q) \\in \\mathbb{R}^{L_q \\times d}$ using only the text features. The functions $p_Q(\\cdot)$, $p_K(\\cdot)$, and $p_V(\\cdot)$ are learnable linear projections that map input features into the query, key, and value spaces, respectively.\n\n**Code‑level detail**\n\nAfter computing attention on the full key (text + dummy), we *mask out* the dummy slice of $V$:\n\n```python\n# crossattention.py (line 385)\nattn_output = torch.bmm(\n    attn_w[:, :, num_dummies:],   # weights over text positions\n    v[:, num_dummies:, :]         # value = text tokens only\n)\n```\n\nHence, dummy tokens steer the attention via the key but never inject their own content into the output—mitigating the mismatch between limited query scope and untrimmed video semantics while keeping the fused feature purely textual.\n\n**Revision actions**\n\n1. **Figure 3** will be redrawn so the Q/K/V flow and dummy masking are explicit.\n2. We will add the above equations and code pointer to the appendix for transparency.\n\nWe appreciate the reviewer’s careful reading, which helps us improve the paper’s clarity.\n\n&nbsp;\n\n**W3:** Thank you for noting that our current draft gives only a cursory treatment of the dummy encoder. We would like to clarify its role and provenance, and we will add a detailed discussion to the supplementary material:\n\n**Dummy Encoder**\n\n* **Design**: an `L`-layer Transformer encoder (`L = args.dummy_layers`) with 8-head self-attention, PReLU activation, and an optional final LayerNorm.\n\n* **Input**: `[N_dummy learnable tokens] + [text tokens]`, each with positional embeddings; standard padding mask applied.\n\n* **Mechanism**: dummy tokens attend to the full sentence, aggregating its global semantics.\n  → Output: `B × N_dummy × D`, compact text queries.\n\n* **Function.** During Adaptive Cross‑Attention, video clips (queries) can allocate weight either to true text tokens or to these dummy positions. Because the final value comes solely from text tokens, the dummy tokens function as a controllable “attention sink”: they absorb weight from video clips that are semantically irrelevant to the query, preventing spurious video‑text alignments while preserving global context modelling.\n\n* **Provenance.** The dummy encoder is inherited unchanged from previous works (e.g., CG‑DETR[1], FlashVTG[2]) and is not claimed as part of our own contribution.\n\n&nbsp;\n\n**Q1:** We appreciate the reviewer’s concern regarding potential overlap among Top‑k predictions, which is an important aspect for multi-moment retrieval.\n\n**1. Theoretical Guarantee — Dual Redundancy Mitigation**\n\n* **Temporal NMS Upper Bound.** During inference, temporal NMS with IoU threshold 0.7 is applied to all candidate segments. By definition, any two retained segments satisfy IoU ≤ 0.7, effectively limiting high-overlap cases.\n\n* **Post-Verification (PV) Consistency Constraint.** The PV module re-scores segments with tIoU supervision and contrastive representation loss $\\mathcal{L}_{\\text{repr}}$, which penalizes semantically redundant segments, promoting diversity at the objective level.\n\nConclusion:\nPV module suppresses redundancy via learning signals, while temporal NMS further provides an upper-bound on overlap, together ensuring diverse Top‑k outputs.\n\n**2. Empirical Evidence — Improved Segment Diversity**\n\nWe report average pairwise IoU among Top‑k predictions on the QV-M² val set:\n\n| Metric  | w/o PV | w/ PV     | Δ         |\n| ------- | ------ | --------- | --------- |\n| mIoU\\@2 | 47.85  | **49.64** | **+1.79** |\n| mIoU\\@3 | 40.92  | **42.92** | **+2.00** |\n\nThese consistent gains indicate that the second and third predictions better align with different ground-truth segments. If all segments were highly overlapping, mIoU\\@2 and mIoU\\@3 would not increase simultaneously.\n\n&nbsp;\n\n[1] Moon, W., Hyun, S., Lee, S., & Heo, J. P. (2023). Correlation-guided query-dependency calibration for video temporal grounding. arXiv preprint arXiv:2311.08835.\n\n[2] Cao, Z., Zhang, B., Du, H., Yu, X., Li, X., & Wang, S. (2025, February). Flashvtg: Feature layering and adaptive score handling network for video temporal grounding. In 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) (pp. 9226-9236). IEEE."}]

Author rebuttal:

Meta-review: Final rating: 4. Borderline Accept/ 4: Borderline Accept/ 5: Accept/ 4: Borderline Accept. The paper introduces QV-M, a multi-moment extension of QVHighlights (2,212 annotations, 5,522 segments) with new metrics for multi-moment retrieval, and proposes FlashMMR—a post-verification framework using constrained temporal adjustment plus verification to prune low-confidence segments. Retraining six MR methods on QV-M and QVHighlights, it reports gains over prior SOTA on QV-M (+3.00 G-mAP, +2.70 mAP@3+tgt, +2.56 mR@3), establishing a stronger benchmark and baseline for realistic temporal grounding.

Reviewers agreed that the paper introduces an important, practical task that extends beyond traditional single-moment retrieval, and that the experimental results are solid, with strong quantitative and qualitative performance for multi-moment retrieval. However, they questioned that the MMVR datasets are adapted from existing ones—limiting originality and potentially introducing biases—and that the constructed multi-moment scenes may be unnatural. They also noted that explicitly modeling cross-moment interactions adds unquantified computational cost and relies largely on standard attention with limited novelty; that evaluation breadth and scalability are insufficient (no temporal-aware video-LLM baselines, limited dataset scale without concrete expansion plans); and that there are presentation/clarity issues, including a formatting error at line 223 (bold header/line break), a possible typo in “Post-Processing with Feature Refinement,” a confusing Figure 3 that is inconsistent with the code (dummy/text tokens, k/v usage), and scant detail on the dummy encoder.

After the rebuttal, reviewers indicated their concerns were satisfactorily addressed and leaned toward acceptance. The ACs concur, agreeing that the work defines a well-motivated multi-moment retrieval task, introduces QV-M²—the first fully human-annotated, verification-backed MMR benchmark—and shows FlashMMR achieving state-of-the-art results as a strong baseline. Therefore, the area chairs agree to accept the paper and recommend that the authors refine it according to the reviewers’ suggestions for the camera-ready version
Decision: Accept (poster)