NeurIPS 2025

When One Moment Isn’t Enough: Multi-Moment
Retrieval with Cross-Moment Interactions
Zhuo Cao1∗, Heming Du1∗, Bingqing Zhang1, Xin Yu1, Xue Li1†, Sen Wang1
1 The University of Queensland, Australia
{william.cao, heming.du, bingqing.zhang, xin.yu}@uq.edu.au
xueli@eesc.uq.edu.au, sen.wang@uq.edu.au
Abstract
Existing Moment retrieval (MR) methods focus on Single-Moment Retrieval
(SMR). However, one query can correspond to multiple relevant moments in
real-world applications. This makes the existing datasets and methods insufficient
for video temporal grounding. By revisiting the gap between current MR tasks and
real-world applications, we introduce a high-quality datasets called QVHighlights
Multi-Moment Dataset (QV-M2), along with new evaluation metrics tailored for
multi-moment retrieval (MMR). QV-M2 consists of 2,212 annotations covering
6,384 video segments. Building on existing efforts in MMR, we propose a frame-
work called FlashMMR. Specifically, we propose a Multi-moment Post-verification
module to refine the moment boundaries. We introduce constrained temporal
adjustment and subsequently leverage a verification module to re-evaluate the
candidate segments. Through this sophisticated filtering pipeline, low-confidence
proposals are pruned, and robust multi-moment alignment is achieved. We retrain
and evaluate 6 existing MR methods on QV-M2 and QVHighlights under both SMR
and MMR settings. Results show that QV-M2 serves as an effective benchmark for
training and evaluating MMR models, while FlashMMR provides a strong baseline.
Specifically, on QV-M2, it achieves improvements over prior SOTA method by
3.00% on G-mAP, 2.70% on mAP@3+tgt, and 2.56% on mR@3. The proposed
benchmark and method establish a foundation for advancing research in more
realistic and challenging video temporal grounding scenarios. Code is released at
https://github.com/Zhuo-Cao/QV-M2.
1
Introduction
Understanding how natural language relates to visual events in videos is a core problem in vid...