PC-Net: Weakly Supervised Compositional Moment Retrieval via Proposal-Centric Network
Authors: Mingyao Zhou
Hao Sun
Wei Xie
Ming Dong
Chengji Wang
Mang Ye
Venue: NeurIPS 2025 poster ()
Keywords: Video understanding
multimodal alignment
contrastive learning
Primary area: applications
TL;DR: To alleviate the existing problems of precise timestamp dependence and limited query generalization, a weakly supervised compositional moment retrieval task is proposed, and a effective baseline is constructed to solve the new task challenges.

Abstract: With the exponential growth of video content, aiming at localizing relevant video moments based on natural language queries, video moment retrieval (VMR) has gained significant attention. Existing weakly supervised VMR methods focus on designing various feature modeling and modal interaction modules to alleviate the reliance on precise temporal annotations. However, these methods have poor generalization capabilities on compositional queries with novel syntactic structures or vocabulary in real-world scenarios. To this end, we propose a new task: weakly supervised compositional moment retrieval (WSCMR). This task trains models using only video-query pairs without precise temporal annotations, while enabling generalization to complex compositional queries. Furthermore, a proposal-centric network (PC-Net) is proposed to tackle this challenging task. First, video and query features are extracted through frozen feature extractors, followed by modality interaction to obtain multimodal features. Second, to handle compositional queries with explicit temporal associations, a dual-granularity proposal generator decodes multimodal global and frame-level features to obtain query-relevant proposal boundaries with fine-grained temporal perception. Third, to improve the discrimination of proposal features, a proposal feature aggregator is constructed to conduct semantic alignment of frames and queries, and employ a learnable peak-aware Gaussian distributor to fit the frame weights within the proposals to derive proposal features from the video frame features. Finally, the proposal quality is assessed based on the results of reconstructing the masked query using the obtained proposal features. To further enhance the model's ability to capture semantic associations between proposals and queries, a quality margin regularizer is constructed to dynamically stratify proposals into high and low query-relevance subsets and enhance the association between queries and common elements within proposals, and suppress spurious correlations via inter-subset contrastive learning. Notably, PC-Net achieves superior performance with 54\% fewer parameters than prior works by parameter-efficient design. Experiments on Charades-CG and ActivityNet-CG demonstrate PC-Net’s ability to generalize across diverse compositional queries. Code is available at https://github.com/mingyao1120/PC-Net.

PC-Net: Weakly Supervised Compositional Moment Retrieval via Proposal-Centric Network
Mingyao Zhou 1,2,3
Hao Su.
1,,3+
Wei Xie
Ming Dong 4.23
Chengji Wang 12,
Mang Ye
School of Computer Science, Central China Normal University
Hubei Provincial Key Laboratory of Artificial Intelligence and Smart Learning, Central China Normal University
National Language Resources Monitoring and Research Center for Network Media, Central China Normal University
Schooo of Computer Science, Wuhan University
Abstract
With the exponential growth of video content, aiming at localizing relevant video moments based natural language queries, video moment retrieval (VMR) has gained significant attention. Existing weakly supervised VMR methods focus on designing various feature modeling and modal interaction modules alleviate the reliance on precise temporal annotations. However, these methods have poor generalization capabilities on compositional queries with novel syntactic structures or vocabulary in real-world scenarios. To this end, we propose new task: weakly supervised compositional moment retrieval (WSCMR). This task trains models using only video-query pairs without precise temporal annotations, while enabling generalization to complex compositional queries. Furthermore, proposal-centric network (PC-Net) is proposed to tackle this challenging task. First, video and query features are extracted through frozen feature extractors, followed by modal- ity interaction to obtain multimodal features. Second, to handle compositional queries with explicit temporal associations, a dual- granularitt proposal genera- tor decodes multimodal global and frame-level features to obtain query-relevant proposal boundaries with fine-grained temporal perception. Third, to improve the discrimination of proposal features, proposal feature aggregator is constructed conduct semantic alignment of frames and queries, and employ a learnable peak- aware Gaussian distributor to fit the frame weights within the proposals to derive proposal features from the video frame features. Finally, the proposal quality is assessed based on the results of reconstructing the masked query using the obtained proposal features. To further enhance the model's ability to capture semantic asso- ciations between proposals and queries, a quality margin regularizer is constructed to dynamically stratify proposals into high and low query-relevanee subsets and enhance the association between queries and common elements within proposals, and suppress spurious correlations via inter-subset contrastive learning. Notably, PC-Net achieves superior performance with 54% fewer parameters than prior works by -fficiien design. Experiments on Charades CG and ActivityNet-CC demonstrate PC-Net's ability to generalize across diverse compositional queries. Code is available at /tttts://ggthhii https/ab..ccc//iiigaaoo/////-/--
Introduction
Video moment retrieval aims identify the precise timestamps in videos corresponding to user queries, which has gained significant research attention. Fully supervised methods heavily rely on
"Corresponding author: haosun@ccnu. edu. cn
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Modid
FCM Movid
Now Hirr
52,5
Query: pa sus opets close: door.
Test-Triviall pesson closet door.
43.1%
17.1%
Nove-Compaatiin pencon pat clathing on the sholf
43 ch
Inwaccaldedggg lane mec-oonamming & laberises
Sivel-Wardee penson with blanket hund clesct cloce.
44.1%
VMR Plocgnoent Sinnisage
Queryvricc pair
VMR
Trivial
WSCMR
Query-viteo pair
WSCMR
Novei-Maadd
Neve-CConpostion
i) Trainig phase
Evalustion plase
(a) The proposed WSCMR i) does not require precise timestamps for training and ii) includes generalization evaluation on compositional queries with unseen grammatical structures or words.
Parszatar DI
Pansaata P0
Pancedere (2)
(b) Performance comparison of the proposed method with the SOTAs.
Comparison of the proposed method with existing methods on paradigm and performance. Figure I:
Here, QD-DETR, is the fully supervised method, and PPS 22 is the weakly supervised method.
datasets with labor-intensive precise timestamps, limiting their scalability in practical applications 3 4 (5) as illustrated in Figure [a). Although weakly supervised methods 2 7 alleviate the annotations burden by training solely on video-query pairs, yet they fail to address a core practical challenge: generalization to queries with novel syntactic structures or unseen vocabulary
g
Recently, the compositional moment retrieval task has focused on the generalization of VMR methods across various query styles in real-world scenarios, utilizing compositional queries with novel syntactic structures and vocabulary to evaluate the generalization ability of models 1 g g.. However, this task is still fully supervised and relies on massive, precise timestamps. To this end, propose a new task, named Weakly Supervised Compositional Moment Retrieval (WSCMR), which aims to train models without requiring timestamps and enables them generalize across compositional queries. This task is fundamentally challenging due to weak pairwise supervision and limited query style in training, and the need to generalize to compositional queries with diverse temporal and semantic dependencies.
An intuitive approach to address the proposed WSCMR is to leverage existing weakly supervised models [7] However, their inherent limitations hinder them from effectively handling composi- tional queries. Specifically, these models typically follow three-stage process around the proposal: boundary generation, feature aggregation, and quality assessment. First, existing methods only gener- ate proposal boundaries through global video-query associations, which lack fine fine-grained temporal perception and are difficult to handle compositional queries with explicit temporal logic such as "the second person turns the light" [1] 12 13). In feature aggregation, existing methods employ fixed Gaussian distribution based on the proposal boundaries to aggregate proposal features from video frame features. These methods ignore the semantic gap between frame and query features [14 15). and the variability in action durations across different query compositions [8] 9. resulting in low discrimination of proposal features. Furthermore, after the quality assessment based on autore- gressively reconstructing the masked query with proposal features, existing methods [2] 6 77) [O] construct negative samples only from the proposal with the lowest reconstruction loss. This discards partially relevant proposals, preventing the model from learning to associate visual representations with specific query elements and undermining compositional generalization I
To address these challenges, a proposal-centric network (PC-Net) is constructed. First, the video and query features are extracted through frozen feature extractors, and the modal interaction based on attention mechanism is then conducted to obtain multimodal features consistent with prior works 7 Secondly, to capture explicit temporal associations in compositional queries and enhance semantic consistency, the dual-granularity proposal generator decodes frame-level and global multimodal features to produce relevant boundaries with fine-grained temporal perception. Thirdly, to alleviate the semantic gap between the frames and the query and difficulty in modeling the variability of action duration of the fixed Gaussian distribution, the proposal feature aggregator first constructs feature triplets (queries, relevant frames, and irrelevant frames) to map them to unified semantic space, and then dynamically adjusts the Gaussian peak region for ensuring discriminative proposal features that align with compositional query semantics. Finally, to assist the model in capturing the semantic associations between visual representations and query elements, a quality margin regularizer is constructed. The regularizer divides the proposals into subsets with high and low query relevance,

---

then helps the model capture the query-related visual elements that co-occur in the proposals and suppresses potential spurious associations through contrastive learning between subsets, indirectly optimizing the proposal representation to cope with potential compositional queries.
PC-Net uses generator with fewer parameters and efficient constraint losses to optimize feature aggregation and quality assessment around proposals. Experiments show a 54% parameter reduction while outperforming baselines in novel query generalization, demonstrating high efficiency and strong generalization. In summary, the contributions of this paper are as follows:
The Weakly Supervised Compositional Moment Retrieval (WSCMR) task is introduced, aiming to train moment retrieval model capable of generalizing to queries with novel syntactic structures and vocabulary while eliminating the need for precise timestamps. Furthermore, the limitations of existing methods are analyzed and validated.
PC-Net is proposed, which employs compact proposal generator, complemented by an efficient aggregator and regularizer to guide proposal feature aggregation and capture semantic associations for enhancing the model's ability of compositional generalization.
The framework achieves superior performance using 54% fewer parameters than comparable methods, demonstrating the efficiency of the design and the generalizability of the proposed framework for novel queries.
2 Related Works
2.1 Video Moment Retrieval
Video moment retrieval aims locate specific temporal segments in videos that semantically match textual queries. This field has evolved through two primary paradigms: fully supervised and weakly supervised approaches. Fully supervised methods, pioneered by Gao et al. [16], employ either
proposal-based [6 17 [8 [9] regression-based (20 21
22 23 24 strategies. Proposal-based
methods generate candidate segments using sliding windows 16 or learnable networks [T8] followed by ranking based query relevance. However, these methods face computational inefficiency due to dense proposal sampling and sensitivity to proposal quality [4). Regression based alternatives [20] 23 directly predict temporal boundaries by fusing multimodal features, enabling end-to-end training but struggling with compositional queries due to rigid annotation patterns ( Recent advances improve compositional generalization through decompose-ande -ede--nddd-rcce reconstrntt strategies [8] and saliency-aware contrastive learning with large language models [ Despite progress, fully supervised methods remain constrained by their reliance precise temporal annotations, which are labor-intensive and
prone to subjective inconsistencies 25 26). weakening their scalability.
Weakly supervised methods eliminate temporal annotations by learning from relevant video-query pairs. Multiple instance learning (MIL)-based approaches [27 28, 29 30 optimize global video- query alignment but often miss fine-grained correspondences. Masked query reconstruction methods 16,10 31| 32] enhance semantic aligmment through cross modal reconstruction. SCN (33] integrates
masked reconstruction and contrastive losses, while CNM [IO] introduces Gaussian masks and triplet learning to refine proposals. Subsequent innovations include CPL ( with learnable Gaussian distributions, counterfactual reasoning extensions 32] and mixed Gaussian frameworks 2]. Methods like PPS 34 and QMN (35) further explore iterative refinement but remain limited by fixed feature aggregation and simple contrastive objectives. Notably, weakly supervised methods ignore the generalization requirements of diverse query styles in practical applications, making it difficult to cope with queries with novel grammatical structures or words. In contrast, the proposed WSCMR task not only abandons the need for accurate timestamps but also requires the model to have good generalization capabilities for novel queries, improving practical usability.
2.2 Multi-modal Semantic Alignment
To address the inherent heterogeneity of multimodal data across temporal, spatial, and semantic dimensions [36] 37, modality alignment has evolved from shallow feature matching to deep joint feature modeling. Early approaches, such as projecting inputs into shared latent space [38], reduced intra-class variation while enhancing inter-class discrimination. Recent advances leverage semantic- to model interactions. For
level alignment (39] and contrastive learning objectives [40, 41 42
3

---

DI hogaraal Fuikar Aginggaa
140
coroed tn duar
MEN g
Iet Pripoal Quality Ancomart Draleyy Regairee
Coundreed
Hobad Frames
Propoea For are
Aciicnal Exipical
Figure 2: The overall framework of the proposed PC-Net for solving the WSCMR task consists of a dual-granulane proposal generator that uses the dual- gual-granllarty association of video-query pairs to obtain query-relevant boundaries with frame-level temporal perception, a proposal feature aggregator for modeling discriminative features, and query margin regularizer that enables the model to capture visual elements relevant to the query and suppresses potential spurious correlations.
instance, cycle-consistency loss aligns video segments with query words (36), while multi-level contrastive objectives capture hierarchical video-query relationships for improved retrieval [37]. Cheng al. [1] further refine this paradigm by generating curriculum-bast language models. Considering that the video contains large number of frames that are irrelevant to the query and relevant timestamps are not available, the alignment of the proposed method is not directly aligned at the video level, but based on the proposal frames related to the query, to perform feature alignment, which facilitates reliable proposal quality assessment.
negative queries via large
Methods
3.1 Feature Extraction and Modality Interaction
Consistent with existing weakly supervised methods 2 7 10
43 is used extract query features, and I3D [44] and C3D [5] are used to extract Charades- 16 and Activity.et-CCG (46
GLoVe
features, respectively. The word-level query features are represented as: Q (91, ....... 2v E R-xx,, where N is the number of words and dy is the hidden dimension of the query words. [U1, U2,...,, uT] E gT'xd where T is
The frame-wise video features are represented as: the number of video frames and de is the hidden dimension of the video frames. Q and V will be mapped the same latent dimension d to facilitate subsequent proposal generation, namely x V- Linear(V)) E Txd Among them, Linear(-) a linear layer. To fully explore the semantics of given query and cope with potentially novel queries, an attention mechanism is used to fuse the query's word-level features Q with the video's frame-level features V. resulting in multimodal features Hgelobal E R with the overall semantics understanding and Hlocl I 7xd with fine-grained temporal information. More details can be found in Appendix A.
Q Linear(( Q) E E. Nxx
3.2 Dual-granularity Proposal Generator
10 32 rely solely on global multimodal associations to generate proposals. This approach inherently lacks fine-grained temporal perception and ig- nores frame-level query relevance, resulting in imprecise boundaries with blurred temporal logic. Consequently, these methods exhibit limited generalization to queries involving explicit temporal de- pendencies [I] 12] To address this limitation, the dual-granularity proposal generator is constructed to capture global semantic consistency and local temporal precision jointly, for generating boundaries with both holistic scene understanding and fine- fine-grained temporal awareness.
Existing weakly supervised methods 2 7
First, the global multimodal feature Hzlobal is linearly mapped to global proposals: pplloall Linear(H(eloba E Xnammx2 pelloal = {(eglobul floball ) E [0, 1] nammx2 contains propos- als with global semantic coherence. Here, C; and w; respectively refer to the center point and duration of the i-th proposal; the same notation applies hereafter. Second, to incorporate frame-level temporal details, frame- wise multimodal features Hoocal are processed using a slot attention mechanism [47].

---

nump all-zero learnable proposal slots Pa E R are initialized and are iteratively refined via K rounds of attention interactions with H The update rule at the k-th iteration is defined as:
(Holoca)) (P
Plocal Softmax
Hoocal
+ Plan
(1)
vd
After K iterations, local proposals Pkca {[clboal w ) which involve explicit temporal boundaries and query-related frame semantics.
pump E [0, 1 1]nwm, x2 are generated, xm1
To combine the strengths of global and local proposals, the Hungarian matching [48] is employed to adaptively match and fuse their boundaries. This adaptive fusion mechanism ensures that the final nump retains both global semantic consistency and fine-grained
proposal set plused = {(efused (ffused) ) /==1
temporal perception. More details can be found in Appendix A.2
3.3 Proposal Feature Aggregator with Semantic Alignment
Existing weakly supervised methods 7 32 rely constructing fixed Gaussian distribution based on the proposal boundary to aggregate proposal features from frame features. This approach has Iwo key limitations: i) there is semantic gap between frame features and query features [14] 151: ii) the fixed distribution difficult to fit the diversity of action durations across different query combinations, which ultimately leads to a decrease in feature discrimination ability (2). To address these issues, a proposal feature aggregator with two components (Figure b)) is constructed. First, cross-modal semantic alignment constructs unified feature space via triplets of queries, related/irrelevant video frames, bridging the semantic gap between queries and frames. Second, a learnable peak-aware Gaussian distributor dynamically adjusts the Gaussian peak region simulate diverse durations, ensuring discriminative proposal features consistent with the compositional query semantics (Figure 7)
Cross-modal Semantic Alignment. To bridge the gap between videos and queries, feature triplets of queries, relevant video segments, and irrelevant video segments are constructed to map them into unified semantic space based on contrastive learning. First, the query-relevant video feature VP E 1g extraction process by a weighted average of the original video features ' as follows.
Σ ME M; = -1 fusedT LEM.
S
fusedd
T]
+ w fusedd
(2)
JM
2
Here, I.] represents a floor operation and T is the number of frames in the video. (cfosed wifised) is the center and width of the i-th proposal Pfused M, is the start and end frame sequence of the i-th proposal, and |M,] is the number frames in the video feature that fall within the i-th proposal. The query-irrelaiie video feature V E glxd; is obtained by a weighted average of the complement of the relevant area the average of the global video, follows:
T Σ v. 1 LIMIST) + D i LIMM=T 1=1
T
M,
t
Here, conditional branch 1(4) = where condition A is true otherwise 0. The semantic contrastive loss is follows. Here, 7 = 0.5 is the margin coefficient. sim(-) is the cosine similarity function.
nump
max (0, sim clobal VT) sim ( global 1p)) +7)+
Crem
(4)
TOUTT
where global contains the global context of the query from modality interaction, see Appendix A.I
10 the boundary of the i-th proposal is used to generate the corresponding initial fixed Gaussian distribution G; E !R? (Appendix A.3p. G; is the frame-level weight of the i-th proposal. However, a fixed Gaussian distribution has difficulty modeling the duration of various action compositions [9] Therefore,
Learnable Peak-aware Gaussian Distributor. Following existing works 6
5

---

propose to adaptively redistribute the peak weights of the Gaussian distribution as follows.
M())) = e-+e--00000 where ni(()= = (0o; - | 44
fused
(5)
Wa(O) - -.(()- (1- + M.()).
(6)
Here, W; E IT denotes the dynamically adjusted frame-level weight of the i-th proposal. (t()) determines whether to retain the original Gaussian value by measuring the distance between each time point I2 and the proposal center diued Specifically, the closer 2; is to the proposal center (peak area), the larger (t())) becomes (greater than 0), and thus the value of (t) approaches 1, effectively assigning weight of to the frame at time Tt. Conversely, Gaussian value is preserved. The parameter / is learnable scaling factor that adjusts the width of the peak area capture diverse action durations across compositional queries. Details of o; and Tt are provided in Appendix A.3
(t) approaches O, the original
3.4 Proposal Quality Assessment via Quality Margin Regularizer
To accurately measure the proposal quality and select the best proposals, the proposal quality assessment process follows previous work [O]. The original query is partially masked and then reconstructed from the proposal features aggregated from the original video frame features using the adjusted Gaussian distribution, and the reconstruction loss used measure the proposal quality, However, existing methods [2] 6 7 32) only construct negative proposals based on the proposal with the smallest reconstruction loss for contrastive learning, discarding partially relevant proposals, which makes it difficult for the model to capture the subtle semantic associations between relevant proposals and queries. Therefore, the quality margin regularizer is built to use the reconstruction quality of the reference proposal whole video frames) to dynamically divide the proposal set into high and low query-relevance subsets, and correlations of co-occurring visual elements related the query are established in proposals through inter-subset contrastive learning.
Proposal Quality Assessment. Consistent with previous methods 2 7 10], the reference and negative proposals are generated to constrain the contrast of the best proposals. The weight of the referenced proposal is a vector of all ones of length N, and the reconstruction loss through Formula (18) is LT more details can be found in Appendix AA.4] Suppose the proposal with the smallest reconstruction loss is denoted as Pf then the corresponding center point and duration are (chored wolueed) respectively. The reconstruction loss of Pfised through Formula (18) is Co The (c), including left negative proposal plused and right negative proposal plused Through the above proposal feature aggregation and mask query reconstruction, the corresponding two negative proposal reconstruction losses C and L obtained. The contrast loss based the optimal proposal Plused is as follows. Among them, 0, and 0g are hyperparameters, and 01 < 02.
complement of (clused wrbse)) is shown Figure 2
Loon = max (0, Co - C. + 01) + max (0, C - C, + f2) + max (0, C - Crs + 02) (7)
Quality Margin Regularizer. Previous relatively simple negative sample contrast makes it difficult for the model to identify subtle semantic differences between query-relevant proposals and avoid spurious associations. Therefore, the quality margin regularizer is built to fully utilize the given query and amplify the feature representation distinctions between high and low query relevance proposals, as shown in Figure First, the original positive proposals L Bnamp are split into two subsets according to the quality of the reference proposal LTe, and then extract the average quality of the high and low query-relevance proposal sets for contrast.
Snigh
1
Σ EE
CT = CTe
c < C,e) Liow
- fil
(8)
D
KEY
Among them, Lnigh and Liow are the average quality of high/low ruerycree proposals, respec- tively. The final quality contrastive loss between proposals is Laga = max (Ltigh Lnow + 03, 0). Here, 0g is margin hyperparameter that controls the strictness of the contrast.

---

3.5
Optimization Goals
The loss in this paper consists of five parts: proposal reconstruction loss Lo proposal contrast loss Ccon- proposal diversity constraint Lai = WWW XI from CPL 77] calculated by proposal frame-level weights W E mumm xT, semantic alignment loss Liem: and quality contrastive loss Lqua-
Lieall c Acn + Adiv + Asem + Aqun
(9)
Among them, A,, E {con, div, sem, qua} is the coefficient of the corresponding loss. During inference, the proposal with the smallest reconstruction loss L is taken as the retrieval result.
4 Experiments
Table 1: Comparative performance on the Charades-CGi dataset. Here, Rn@m denotes the Recall metric under an IoU threshold of m. Results in bold are optimal, underlined results are suboptimal. "Full Supervision" represents methods that use precise timestamp annotations during training, and "Weak Supervision" represents methods that train only on video-text pairs without timestamp annota- tions (results are reproduced based on public implementations) Same as below.
Test- Drivialivial
Novel- Cowel-Composition
Novel- Bovel-Word
Method
Params
RI 0.055
R10 mloU
RI 110.5 @007 mloU
@I0.
R1@0.7 mioU
TMN 49
18.75
8.16
19.82
8.68
4,07
10.14
9.43
4.95
11.23
TSP-PRI
22
39.86
21.07
38.41
16.30
2.04
13.52
14.83
2.61
14.03
VSLNet
5D
45.91
19.80
41.63
24.25
11.54
31.43
25.60
10.07
30.21
2D- ID-TAAN
18
48.06
27.10
43.72
32.74
15.25
31.50
37.12
18.99
35.04
Full Supervision
2D-TAN
51
53.91
31.82
46.84
35.42
17.95
33.07
43.60
25.32
39.32
LGI 52
49.45
23.80
45.01
29.42
12.73
30.09
26.48
12.47
27.62
MS-2D-IAN
53
$7.85
37.63
50.51
43.17
23.27
38.06
45,76
27.19
40.80
MS-2D-TAL -ON-TAN-30 SSL
61
37.98
40.00
50. 50.36
28.
43.15
VISA 9
53.20
26.52
47.11
45.41
22.71
42.03
42,35
20.88
40.18
Decu
58.75
28.71
Moment
DETR
54
49.48
28.04
44.82
39.42
18.62
36.61
46.76
24.75
41.70
QD-DETR 55
7.12M
59.24
33.43
50.92
42.30
21.09
38.55
46.04
26.33
42.89
QD-DETR>
7.12M
60.66
38.60
52.53
50.23
27.69
55.25
35.25
48.10
Weak Superision
WSSL 31
15.33
5.46
18.31
3,61
1,21
8.26
2.79
0.73
7.92
CNM
2.52M
31.37
CPL
3.01M
53.04
24.71
45.82
40.79
16.15
37.46
42.45
21.44
39.20
CCR
QMN 0
12.51M
51.65
22.64
45.85
40.67
15.72
37.91
46.91
21 58
41.07
2
51.74
42.01
PC-Net(OUrs)
3.34M
54.84
26.68
47.12
41.69
16.73
38.04
46.91
23.60
41.06
Table 2: Comparative performance ActivityNet-CG datasets.
Test Trivial
Novel Composition
Novel Word
Method
@10.5
RI 100.7 mloU
RI 1005
RI00.7
mloU
RI600.
R1@0.7 mloU
TSP-PRL TMN 39
34.27
18.80
37,05
14.74
1.43
12.61
16.82
7.01
17.13
8.74
4.39
10.08
9.93
5.12
11:38
Full Supervision
10.37
LGi 52
43.56
23.29
41.37
23.21
9.02
27.86
23.10
9.03
26.95
VLSNet
50
39.27
23.12
42.51
20.21
9.18
29.07
29.58
VISA
47.13
29.64
44,02
31.51
16.73
35.85
30.14
15.90
35.13
Deco 1
24.25
27.35
31.27
Moment-DFT
DETR
M4L
42.73
25.31
42.
29.29
13.71
31.65 16.63
26.84
13.34
29.95
Moment DETR
11
44.19
25.81
43.49
30.60
14.40
33.13
29.59
15,10
32.43
QD-DETR
55
7.92M
41.80
20.88
41.15
26.91
10.96
31.01
27.09
11.38
31.21
QD-DETR
7.92M
43.76
25.98
42.85
29.56
14.37
32.44
27.60
13.11
30.98
Weak Supervision
WSSL R1
11.03
4.14
15.07
2.89
0.76
7.65
3.09
1.13
7.10
CNM [0
2.38M
28.55
13.44
35.06
18.38
7.22.
28.19
21.07
29.71
CPL
4.64M
27.62
11.80
32.73
19.31
7.05
26.95
22.50
9.29
28.33
CCR 12
268.96M
27.67
12.90
33.56
19.59
7.66
27.50
21.66
28.42
QMN
272.38M
24.27
13.19
33.82
15.88
6.09
27.30
19.31
7.76
28.96
PPS 2
8.94M
30.00
15.84
32.98
20.60
9.45
26.27
22.98
11.25
27.69
PC-Net(Ours)
4.9MM
29.62
14.35
36.45
20.16
8.05
29.51
22.88
9.85
30.76
4.1 Experimental Setup
Datasets. validate the proposed PC-Net for the WSCMR task, Charades-CCC and AetivityNet-CG
applied, where accurate timestamp annotations only used for evaluating the model rather
7

---

than training, and annotations with novel queries are sourced from the literature 19. Charades- (8.312 videos) includes 8,281 training queries and three test subsets: Test-Trivial (3,096 queries with training-style phrases), Novel-Composite (3,442 queries covering verb-noun, noun-noun, verb-adverb, adjective-noun, and preposition- combinations (9), and Novel- Word (703 queries with unseen vocabulary). Aeti (20,647 videos) follows a similar split: 36,724 training queries, 15,712 Teiv-arivial, 12,028 Novel-Composition, and 3,944 Novel-Word Word.
Evaluation Metries. Two metrics are introduced. "mloU" is the mean Intersection over Union (IoU), which reflects the average overlap between predicted timestamps and ground truth, indicating overall performance. "Rn@m" measures recall at top-n predictions under an IoU threshold of m.
Implementation Details. The implementation details are presented in Appendix A.5
4.2 Comparisons with SOTAs
In addition to recent fully-supervised compositional moment retrieval methods [ 8 9 we also reproduce several recent weakly supervised methods from publicly available repositories 22 32] for comparative evaluation. To fully verify the effectiveness of the proposed method, this paper also provides comparative experiments on testing weakly supervised moment retrieval on two long video datasets (Appendix B.1 model robustness (Appendix B.2 and efficiency (Appendix B.3)
IC
The proposed PC-Net is evaluated on Charades-CG and Activity.ee--GG under weak supervision. As shown in T PC-Net achieves state-of state-of-the-art performance on Charades-CG, particularly in novel composition and unseen vocabulary scenarios. Under the Sovel-Composition split, obtains R1 @ of 41 41.699%38..... 04%. outperforming prior weakly supervised methods (CPL: 40.79% 46% QMN: 40.67% .9 %) with only 3.34M parameters- -sinnsiicanlllyy fewer than fully supervised QD-DETR (7.12M) and weakly supervised QMN (12.51M). The performance gain stems from two key innovations: (1) granularatyr proposal generator combining global and local multimodal features for robust query understanding and frame-wise temporal association, achieving 54.84% RI @0.5 on Test-Trivial (vs. 53.04% for CPL); and (2) peak-aware feature aggregation with semantic alignment via differentiable weight redistribution and contrastive learning to obtain discriminative proposal features. Notably, PC-Net achieves 89.7% mIoU (47.12%/52.53%) on Test-Trivial with 53.01% % fewer parameters than the prior SOTA QD-DETRs.
On the ActivityNet-CG dataset (Table[). PC-Net continues to outperform existing weakly supervised methods, achieving state-of-the-ar mloU scores across all splits: 36.45% (Test-Trivial), 29.51 51% Compo--oomosiiioon) and 30.76% (Novel-Word). These results highlight PC-Net's strong semantic alignment capability and query generalization. Although PC-Net's R1 @0.5 and RI @0.7 scores are occasionally lower than those other models on ActivityNet-CG. this mainly due to the dataset's long average video duration (117.6 6 seconds) and sparse feature sampling. Nevertheless, its strong mIoU performance confirms the model's capability to fully exploit the query semantics and generate high-quality proposals. Notably, PC-Net remains highly parameter-efficient, requiring only 4.97M parameters-ddramatiee dramatically fewer than CCR (268. 86M) and QMN (272.38M). The proposed semantic alignment and quality margin regularizer further enforces the semantic consistency of proposal features and the capability of capturing nuance discrepancy between queries and proposals, making PC-Net compact yet effective solution for compositional moment retrieval under weak supervision.
Table 3: Detailed ablation experiments are conducted on the Charades -Gddees-GGGee dataset.
Setting
DPG
PFA
Eaua
Test: Trivial
Novel Composition
Novel Word
LPG
Cyen
RI@05
R1@0.7 mloU
R1@0,55
RI @0.7
mloU
RI 10.5
@10007 mloU
(a)
53.04
24.71
45.82
40.79
16.15
42.45
39.20
(b)
54.39
24.87
46.89
40.97
16.40
37.77
46.06
22.32
40.98
(c)
51.87
23.19
45.41
40.38
16.56
37.36
42.16
22.30
38.98
(d)
54.43
24.06
46.42
41.61
16. 13
38.02
46.06
21.44
40.98
(e)
51.52
23.55
45.66
17.00
37.57
43.60
22.16
40.07
(f)
V
54.07
25.65
47.08
40.74
16.64
37.72
46.20
23.45
41.04
(g)
54.04
24.94
46.59
41.48
16.91
37.49
46.91
23.60
41.17
(h)
53.26
24.97
46.28
40.99
16.39
36.94
45.18
22.01
40.28
(
54.65
25.16
46.91
40.78
45.76
21.73
39.93
(
53.88
24.52
46.61
41.04
16.82
38.01
46.19
23.60
40.41
(1)
54.33
25.55
46.75
41.52
17.81
37.74
43.32
22.45
40.71
Ours
54.84
26.68
47.12
41.69
16.73
38.04
46.91
23,60
41.06
8

---

4.3 Ablations
Comprehensive ablation studies are conducted on the Charades-CG dataset (Table 3 assess the impact of each proposed module. Specifically, 'DPG' denotes the dual-granularity proposal generator, and 'PFA' refers to the proposal feature aggregator, which incorporates both cross-modal semantic contrastive loss (Cum) ) and the learnable peak-aware Gaussian distributor ( ('LPG"). The contrastive loss in quality margin regularizer is denoted as Equa- Introducing the dual-grante granularity proposal generator (DPG) (b) consistently improves performance across all metrics compared to the baseline (a). highlighting its role in enhancing temporal perception and proposal boundaries. However, the LPG and quality margin contrastive loss (c, e) slightly degrade performance, possibly due to suboptimal query semanties utilization or re-weighting side effects. Incorporating DPG settings (j. h) yields performance gains over the full model ("Ours"), underscoring the value of query query-driven proposals. Joint DPG and LPG application (f) enhances generalization, particularly on Test-Trivial and Novel-Word splits. Integrating Leem and Lqun (f&ours) significantly boosts semantic alignment and quality assessment. The full model ("Ours") achieves SOTA results, validating the necessity of each module in addressing the existing limitations in generalizing to compositional queries with diverse temporal and semantic dependencies. More ablation experiments in Appendix B.4
47.4
Heudligt, sanin Lul, onder Ditlereed Lase Caniite
46.6
04.54
46.75
-6.00
(a) Coefficient Ablation of Lee.. (b) Coefficient Ablation of Lyu. (c) Lsem and Lqua co-ablation.
Figure 3: Detailed ablation of the alignment loss and quality contrastive loss on Charades-CCS
To deeply analyze the role of the two losses of the proposed semantic alignment loss (Cyem) and the quality contrastive loss (Gqu)), the coefficients of Lxem and Lyu are independently ablated as shown in Figure It can be seen that the model performs best when the coefficient of Zeem is 0.5 and the coefficient of Lqua is 1.0. As the coefficient increases, the model's performance gradually decreases. In addition, analyze the synergistic effect of the two coefficients, the coefficient combination is also ablated here, thoroughly verifying that the model performs best when Asem IS 0.5 and Xqua is 1.0.
Query Browicg
Query persin pours sime water into the glass.
Ioas: 19:20
umadd
21 17:24
11336 3326
(a) Two Samples from the Test-Trivial split.
Query: a parten runs inte the garson
query: prewi patting on clothes
112 1.0m0
.ree
11
(b) Two Samples from the
split.
Query: pocura persos stairs Lebling soEp,
Query: o persee talking plaaso
EERING TE
11 2150
cen
(c) Two Samples from the Novel- Word split.
Figure 4: Qualitative comparison on Charades-CG dataset.

---

Query Me the eexting scrwes
fumala tediratual popated (0.v.
Figure 5: Failure case study the Test- Trivial split of ActivityNet-CG dataset.
Quary d piencc cu daseeve viening celecttion
Ovary: an indviicall tranaterrigg lioid zn croblig
Figure 6: Relevance study on the rewritten Test-Trivial split of Charades CG dataset.
4.4 Visualization
To evaluate the proposed method's retrieval performance. qualitative results on the Charades-CG dataset are presented in Figure f the test-trivial subset (matching the training query style), the method generates high-quality proposals guided by query semantics and selects retrieval results with accurate timestamp alignment
precise boundary modeling. For novel action combinations within compositional queries, Gaussian distribution peak adjustment captures various durations of diverse actions, aggregating them into discriminative features for quality assessment. In the Novel-word scenario (where unseen words are encountered during training), the model leverages deep query associations and explicit semantic alignment to demonstrate robust generalization.
These examples show that actions with long durations tend exhibit lower boundary accuracy. This issue arises from the model's inability to accurately capture the association between the start and end points of an action when generating proposals. In future work, we plan to employ more powerful feature extractors, capture query correlations, to adjust and improve proposal boundaries dynamically.
In addition, failure case study is presented in Figure 5
To thoroughly investigate the interpretability of fully supervised versus weakly supervised methods, we conducted visualizations of correlation scores between queries and video frames on the rephrased Charades Test-Trivial dataset, shown in Figure 6 The results demonstrate that, compared to the fully supervised QD-DETRs trained with precise timestamp annotations, our proposed PC-Net not only comprehensively understands unseen complex query semantics but also more accurately aligns query content with the visual features of video frames. This capability produces more rational and discriminative correlation score curves. Specifically, for the query "A person can be observed viewing television." PC-Net generates significantly higher correlation scores within the critical action interval [0.0s, 4.9s] than QD-DETRs, indicating a stronger ability to capture the semantics of "viewing a television." Similarly, in the task "An individual transferring liquid into drinking glass." although QD-DETRs exhibit responses in certain intervals, PC-Net's correlation scores are more concentrated and consistently surpass the baseline in the true action interval [7.3s, 17.3s]. This suggests a more stable alignment between query semantics and relevant visual moments.
5
Conclusion
This paper analyzes the shortcomings of existing methods, and proposes a more practical and scalable task, namely WSCMR. By fully mining the dual-granularity query semantics and temporal perception obtain query-relevant and well-bounded proposals, and improving feature discrimination through the semantic alignment and peak optimization, and the quality margin regularizer is used to establish associations between common visual elements in proposals and queries and to suppress spurious associations, proposal-centric optimization pipeline is implemented. Extensive experiments demonstrate PC-Net's excellent performance with fewer parameters. Future work will explore improving the query generalization of weakly supervised moment retrieval in long videos.
10

---

6 Acknowledgements
This work was supported in part by National Natural Science Foundation of China under Grant 62201222 and 62377026, in part by the Fundamental Research Funds for the Central Universities under Grant CCNU25ai041 and CCNU2SJC045, and part by Hubei Provincial Key Laboratory of Artificial Intelligence and Smart Learning under Grant 2025AISL011.
References
[1] Z. Cheng. Y. Pu, S. Gong, P. Kordjamshidi, and Y. Kong, "Shine: Saliency- aware hicrarchical negative ranking for compositional temporal grounding," in European Conference on Computer Vision. Springer, 2024, pp. 318-3984464
[2] S. Kim, J. Cho, J. Yu, Y. Yoo, and J. Y. Choi, "Gaussian mixture proposals with pull-push learning scheme to capture diverse events for weakly supervised temporal video grounding," " Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 3, 2024, pp. 2795-2802
[3] Z. Liu, Li, H. Xic, P. Li, Ge, S.-A. Liu, and G. Jin, "Towards balanced alignment: Modal-enhanced semantic modcling for video moment retrieval," in Proceedings of the AAAI conference on artificial intelligence, vol. 38, no. 4, 2024, pp. 3855-3863.
[4] H. Zhang. Sun, W. Jing, and T. Zhou, "Temporal sentence grounding in videos: A survey and future directions," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 8, pp. 10443 10465, 2023.
[5] Zhou, W. Chen, H. Sun, and W. Xie, "Cross-modal multiscale difference-aware network for joint moment retrieval and highlight detection," in ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). TEFE, 2024, pp. 8416-8420.
[6] M. Zhou, W. Chen, H. Sun, W. Xie, M. Dong, and X. Lu, "Query-aware multi-scale proposal network for weakly supervised temporal sentence grounding in videos," Knowledge- Systems, vol. 304, p- 12592, 2024.
[7] M. Zheng, Y. Huang, Q. Chen, Y. Peng, and Y. Liu, "Weakly supervised temporal sentence grounding with gaussian-based contrastive proposal learning." in Proceedings of the IEEE/CVF Conference on Computer Vision and Pottern Recognition, 2022. pp. 15 555- 15 564.
[8] L. Yang, Q. Kong, H.-K. Yang. W. Kehl, Y. Sato, and N. Kobori, "Deco: Decomposition and reconstruction for compositional temporal grounding via coarse-to-fine contrastive ranking," in Proceedings of the IEFF/CVF Conference Computer Vision and Pattern Recognition, 2023, pp. 23 130-23 140.
[9] J. Li, J. Xie, L. Qian, L. Zhu, S. Tang, F. Wu, Y. Yang, Y. Zhuang. and X. E. Wang. "Compositional temporal grounding with structured variational cross-graph correspondence learning." " in Proceedings of the /EEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 3032-3041.
[10] M. Zheng. Y. Huang, Q. Chen, and Y. Liu, "Weakly supervised video moment localization with contrastive negative sample mining," " in Proceedings of the AAAI Conference Artificial Intelligence, vol. 36, no. 2022, pp. 3517-3525.
[11] Y. Wang. K. Li. X. Li. J. Yu, Y. He, G. Chen, B. Pei. R. Zheng. Z. Wang. Y. Shi et al., "Internvide02: Scaling foundation models for multimodal video understanding," in European Conference on Computer Vision. Springer, 2024, pp. 396-416.
[12] A. Newman, C. Fosco, V. Casser, A. Lee, B. McNamara, and A. Oliva, "Multimodal memorability: Modeling effects semantics and decay on video memorability." in Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XVI 16. Springer, 2020, pp. 223 240.
[13] Fang, C. Ouyang, Q. Lin, and Y. Yuan, "Three heads better than one: Pure entity, relation label and adversarial training for cross-domain few-shot relation extraction," Data Intelligence, vol. 5, no. 3, pp- 807-823, 2023.
[14] F. Huo, W. Xu, J. Guo, H. Wang, and S. Guo, "C2kd: Bridging the modality gap for cross- knowledge distillation." in Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, 2024, pp. 16006-16015.
[15] V. W. Liang, Y. Zhang, Y. Kwon, S. Yeung, and J. Y. Zou, "Mind the gap: Understanding the modality gap in multi-modal contrastive representation learning." Advances in Neural Information Processing Systems, vol. 35, pp. 17 672-17 625, 2022.
[16] J. Gao. C. Sun. Z. Yang. and R. Nevatia, "Tall: Temporal activity localization via language query." in Proceedings of the IEEE international conference on computer vision, 2017, pp. 5267-5275.
11

---

[17] L. Anne Hendricks, O. Wang, E. Shechtman, J. Sivic. T. Darrell, and B. Russell, "Localizing moments in video with natural language." in Proceedings of the IEEE international conference on computer vision, 2017. pp. 5803-5812.
[18] S. Zhang, H. Peng, J. Fu, and J. Luo, "Learning 2d temporal adjacent networks for moment localization with natural language," in Proceedings of the AAAI conference on artificial intelligence, vol. 34, no. 07, 2020, pp. 12 870-72288 877.
[19] J. Gao and C. Xu, "Fast video moment retrieval," in Proceedings of the IEEF/CVF International Conference Computer Vision, 2021. pp. 1523-1532
[20] Y. Yuan, T. Mci, and W. Zhu, "To find where you talk: Temporal sentence localization in vidco with attention based location regression." in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 01, 2019. pp. 9159-9166.
[21] S. Ghosh, A. Agarwal, Z. Parekh, and A. Hauptmann, "Excl: Extractive clip localization using natural language descriptions," arXiv preprint arXiv: 1904.02755, 2019.
[22] J. Wu, G. Li, S. Liu, and L. Lin, "Tree-structured policy based progressive reinforcement learning for temporally language grounding in video," in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 07, 2020, pp. 12 103386122 393.
[23] H. Zhang, A. Sun, W. Jing. L. Zhen, J. T. Zhou, and R. S. M. Goh, "Natural language vidco localization: A revisit in span-based question answering framework," IEEE transactions on pattern analysis and machine intelligence, vol. 44, no. 8. pp. 4252 4266, 2021.
[24] H. Sun, M. Zhou, W. Chen, and W. Xie, "Tr-detr: Task-reciprocal transformer for joint moment retrieval and highlight detection," in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 5, 2024, pp. 4998-500 -0097
[25] H. Zhou, C. Zhang, Y. Luo, Y. Chen, and C. Hu, "Embracing uncertainty: Decoupling and de-bias for robust temporal grounding," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 8445-8454,
[26] K. Ma, H. Huang. J. Chen, H. Chen, P. Ji, X. Zang. H. Fang, C. Ban, H. Sun, M. Chen et al., "Beyond un- certainty: Evidential deep learning for robust video temporal grounding," arXiv preprint arXiv:2408. 16272, 2024.
[27] N. C. Mithun, S. Paul, and A. K. Noy-Chowddhury "Weakly supervised video moment retrieval from text queries." in Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, 2019, pp. 11 592-11 601.
[28] Z. Zhang, Z. Zhao, Z. Lin, X. He et al., "Counterfactual contrastive learning for weakly-supervised vision- language grounding." Advances in Neural Information Processing Syxtems, vol. 33, pp. 18 123-18 134, 2020.
[29] J. Wu, G. Li, X. Han, and I.. Lin, "Reinforcement learning for weakly supervised temporal grounding of natural language in untrimmed videos." in Proceedings of the 28th ACM International Conference on Multimedia, 2020, pp. 1283-1291.
[30] J. Huang. Y. Liu, S. Gong, and H. Jin, "Cross-sentence temporal and semantic relations in video activity localisation," in Proceedings of the IEEE/CVF international conference computer vision, 2021, pp. 7199-7208.
[31] X. Duan, W. Huang, C. Gan, J. Wang, W. Zhu, and J. Huang, "Weakly supervised dense event captioning in videos." Advances in Neural Information Processing Systems, vol. 31, 2018.
[32] Z. Lv, B. Su, and J.-R. Wen, "Counterfactual cross-modality reasoning for weakly supervised vidco moment localization," in Proceedings of the 31 ACM International Conference on Multimedia, 2023, pp. 6539-6547.
[33] Z. Lin, Z. Zhao, Z. Zhang, Q. Wang, and H. Liu, "Weakly--upervised video moment retrieval via semantic completion network." in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 07, 2020, pp. 11 539-11 546.
[34] J. Wei, Y. Xu, M. Chen, Y. Zhang, R. Feng, and S. Gao, "Conditional video-text reconstruction network with cauchy mask for weakly supervised temporal sentence grounding." in 2023 IEEE International Conference on Multimedia and Expo (ICME). IEEF, 2023, pp. 1511-1516.
[35] M. Cao, F. Wei, C. Xu, X. Geng, L. Chen, C. Zhang, Y. Zou, T. Shen, and D. Jiang, "Iterative proposal refinement for wcakly-supervised video grounding," in Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, 2023, pp. 6524 6534..
[36] S. Ging, M. Zolfaghari, H. Pirsiavash, and T. Brox, "Coot: Cooperative hierarchical transformer for video-text representation learning." Advances in neural information processing systems, vol. 33, pp. 22 605-2266666 618, 2020.
12

---

[37] B. Zhang, C. Yang. B. Jiang, and X. Zhou, "Video moment retrieval with hierarchical contrastive learning," in Proceedings of the 30th ACM International Conference on Multimedia, 2022. pp. 346-355.
[38] V. E. Liong, J. Lu, Y.P. Tan, and J. Zhou, "Deep coupled metric learning for cross-modal matching," /EEE Transactions on Multimedia, vol. 19, no. 6, pp. 1234-1244, 2016.
[39] M. 7hang, Li, H. Zhang, and I.. Liu, "Deep semantic cross modal hashing with correlation alignment," Newrocomputing, vol. 381. pp. 240-251, 2020.
[40] H. Luo, L. Ji, B. Shi, H. Huang. N. Duan, T. Li, J. Li, T. Bharti, and M. Zhou, "Univl: A unified video and language pre-training model for multimodal understanding and generation." arXiv preprint arXiv:2002.06355. 2020.
[41] C. Sun, F. Baradel, K. Murphy, and C. Schmid, "Learning video representations using contrastive bidirec- tional transformer," arXiv preprint (.XXiiII06.05707 2019.
[42] A. Miech, J.-B. Alayrac, L. Smaira, I. Laptev, J. Sivic, and A. Zisserman, "End-to-end learning of visual representations from uncurated instructional videos," in Proceedings of the IEEE/CVF conference on computer vision and pattern recogmition, 2020, pp. 9879-9889.
[43] J. Pennington, R. Socher, and C. D. Manning, "Glove: Global vectors for word representation." in Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), 2014, pp. 1532 1543.
[44] J. Carreira and A. Zisserman, "Quo vadis, action recognition? a new model and the kinetics dataset," in proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 6299-6308.
[45] D. Tran, L. Bourdev, R. Fergus, L. Torresani, and M. Paluri, "Learning spatiotemporal features with 3d convolutional networks," in Proceedings of the IEEE international conference on computer vision, 2015, pp. 4489 4497.
[46] R. Krishna, K. Hata, F. Ren, L. Fei-Fei, and J. Carlos Niebles, "Dense- captte-captioning events in videos," in Proceedings of the IEEE international conference on computer vision, 2017, pp. 706-715.
[47] F. Locatello, D. Weissenborn, Unterthiner, A. Mahennran, G. Heigold, Uszkoreit, A. Dosovitskiy, and
T. Kipf. "Object-centric learning with slot attention." Advances in neural information processing systems, vol. 33, pp. 11 525-11 538, 2020.
[48] T. Yu, R. Wang, J. Yan, and B. Li, "Learning deep graph matching with chamnet-indepeeddntt embedding and hungarian attention," in International conference on learning representations, 2019.
[49] B. Liu, S. Yeung. E. Chou, D.-A Huang. L. Fei-Fei, and C. Niebles, "Temporal modular networks for retrieving complex compositional activities in videos," in Proceedings of the European Conference on Computer Vision (ECCV), 2018. pp. 552-568.
[50] H. Zhang, A. Sun, W. Jing, and J. T. Zhou, "Span-based localizing network for natural language video localization," arXiv preprint arXiv:2004. 13931, 2020.
[51] C. Li, Z. Li, C. Jing, Y. Jia, and Y. Wu, "Exploring the effect of primitives for compositional generalization in vision-and- languaug in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023. pp. 092--99 101.
[52] J. Mun, M. Cho, and B. Han. "Local-global video-text interactions for temporal grounding." in Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, 2020, pp. 10810-1081 819.
[53] S. Zhang. H. Peng, J. Fu, Y. Lu, and J. Luo, "Multi-scar 2d temporal adjacency networks for moment localization with natural language," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 12, pp. 9073-9087 2021.
[54] J. Lei, T. L. Berg, and M. Bansal, "Detecting moments and highlights in videos via natural language queries." Advances in Neural Information Processing Systems, vol. 34, pp. 11 846-11 11 858, 2021.
[55] Moon, Hyun, Park. D. Park, and J.-P. Heo, "Query-dependent video representation for moment retrieval and highlight detection," in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 23 022-23300 23033 033.
[56] A. Vaswani, Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. "Attention is all you need," Advances in neural information processing systems, vol. 30, 2017.
[57] M. Regneri. M. Rohrbach, D. Wetzel, S. Thater, B. Schiele. and M. Pinkal, "Grounding action descriptions in videos," Transactions of the Association for Computational Linguistics, vol. 1, pp. 25-36, 2013.
[58] K. Grauman, A. Westbury, E. Byrne, Z. Chavis, A. Furnari, R. Girdhar, J. Hamburger, H. Jiang, M. Liu, X. Liu et al., "Ego4d: Around the world in 3,000 hours of egocentric video." in Proceedings of the IEEE/CVF conference on computer vision and paterrn recognition, 2022. pp. 18 19022.
[59] F. Mu, S. Mo, and Y. Li. "Snag: Scalable and accurate video grounding." in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 18 93993-15 940.
13

---

[60] M. Rohrbach, M. Regneri, M. Andriluka. S. Amin, M. Pinkal, and B. Schiele. "Script data for attribute- based recognition of composite activities." in Computer VCC 2012: 12th European Conference on Computer Vision, Florence, Itlly October 7-13, 2012, Proceedings, Part 12. Springer, 2012, pp. 144-157.
[61] C. Feichtenhofer, H. Fan, J. Malik, and K. He, "Slowfast networks for video recognition." in Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 6202-6211. 62111.
[62] A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv et al., "Qwen3 technical report," arXiv preprint arXiv:2505.092
2025.
[63] B. Liberatori, A. Conti, Rota, Y. Wang, and E. Ricci, "Test-time zero-shot temporal action localization." in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024, pp-
18 7277221877 729.
[64] Y. Tang, Wang, C. C.Zhang, J. Liu, and Zhao, "Leaannalee feature augmentation framework for temporal action localization," IEEF Transactions on Image Processing, 2024.
[65] C. Chen, Chen, W. Yang, H. Wang, J. Knittel, Zhao, S. Koch, T. Ertl, and S. Liu, "Enhancing single-frame supervision for better temporal action localization," IEEE Transactions on Visualization and Computer Graphics, 2024.
[66] Y. Jiang. W. Zhang. X. Zhang, X.-Y. Y. Wei, C. W. Chen, and Li, "Prior knowledge integration via Ilm encoding and pseudo event regulation for video moment retrieval," Proceedings of 32nd ACM International Conference on Multimedia, 2024, pp. 7249-7258.
[67] G. Medioni, I. Cohen, F. Brémond, S. Hongeng, and R. Nevatia, "Event detection and analysis from vidco streams," " IEEE Transactions on pattern analysis and machine intelligence, vol. 23, pp. 873-889, 2001.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction of this paper accurately reflect its contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Yes, the paper discusses the limitations of the work in the Appendix.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't It acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
15

---

Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
In the implementation details and ablation experiment sections, the experimen- tal environment and hyperparameter settings have been fully provided to ensure that the core results of the proof can be reproduced.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
16

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
The relevant codes and instructions have been packaged into a zip file and uploaded.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
//ips.cccppblli... wwtppbplcggguiiieecceeeeeoooeeee http:/gddess/eee/ sondeeSubmsssiooPooo
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The hyperparameters of the modules mentioned in the article have been stated in the article. If not stated explicitly (in case of any), they consistent with the baseline method described in the article.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: The paper does not provide error bars, confidence intervals, statistical significance tests for the experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
17

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
The parameter settings and experimental environments of all experiments have been published in the Experimental Details. In addition, the trainable parameters, FLOPS, and inference time of the comparison methods and the proposed method have been summarized in table.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics http:///wwaarppp..ccppppa/taaaaa
Answer: [Yes]
Justification: Coding rules followed existing baseline codes that adhered to the relevant conference ethics guidelines.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
18

---

Justification: As analyzed in the introduction. the WSCMR task proposed in this paper improves the limitations of existing weakly supervised moment retrieval tasks and makes more suitable for practical applications.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation. generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: The baseline model code and data used for training and testing in the paper have been approved by industry researchers, and its open source repository respects the license and terms of use.
Guidelines:
The answer NA means that the paper does not existing assets.
19

---

The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the datasets..paerssttt paperswi thcode. com/datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
20

---

Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question:
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our L.LM policy for what should or should not be described.
21

---

A More Technical Details
A.1 Multi-modal Feature Fusion
To effectively model the interaction between video features V and query features Ô. we propose a two- stage framework that first extracts global query semantics and then establishes both global and local multimodal associations. The pipeline is formalized as follows: the input query features 2 Nx (where N is the number of query tokens, d is the feature dimension) are augmented with a learnable CLS token 1 E IRlxd to encode global semantics. The concatenated input [4*)|[ E (R is processed through a self-attention mechanism to generate the global query feature galobal E Rixdd and an encoded query representation Qene E R Nxxx xd
This is mathematically expressed as:
galobal δene SelfAtt([("*1]]]
(10)
where the CLS token oth acts as an aggregator for global query semantics, and the self-attention mechanism computes pairwise attention scores between all tokens in the input sequence. The
output galobal captures the global context of the query, while ςemc retains refined local query features influenced by the global context. The self-attention operation follows the standard formulation:
QK
Attention(Q, K, V) Softmax
V
(11)
are linear projections of the input sequence ( Ô using learnable weight matrices. This stage ensures that the model learns a compact, global representation of the query while preserving local interactions among query tokens.
where
K, K,
In the subsequent cross-attention stage, the input video features i R (where T is the number of video frames) are also augmented with a learnable CLS token wels E Bixd to capture global association between video and query. Then, the global query feature qelobal and encoded query features Qenc are concatenated
( to attend over the video features [1]] E (T++)x) (where T the number of video frames) through the cross- attention mechanism. This process generates two outputs: Hgslohal R 1xd multimodal feature capturing the global association between the query and video, and Hooal E R frame-wise multimodal features modeling local query-video interactions. The operation is formalized as:
new query vector [q Qemqq Q
Hglobal Hoocal CrossAun([D"1111]) sloball [eme),
|| || ((0x))
[ystobull
(12)
where the cross-attention mechanism computes:
QKT Vily
CrossAttn(Q, V) Softmax
V.
(13)
with Q derived from the concatenated video features [uth [[]] and K, V as linear projections of [y Qume]. The output 7ysloball aggregates attention-weighted video features to encode holistic relationships for generation proposals with the semantic consistency of the video-query pair, while Hlocal preserves frame-level interactions for fine-bounded proposals with fine-grained temporal perception.
A.2 Details of the Fusion of Global and Local Proposals
Given that single-granularity proposals are subject to corresponding limitations, that is, global proposals pglohal have blurred boundaries due to the inability to perceive frame-level video-query correlation, while local proposals Pl are difficult to avoid interference from irrelevant frame features due to the lack of overall query semantic guidance. Therefore, we propose to use adaptive fusion alleviate the fingleggraulaan related boundaries and temporal logic. First, the Hungarian matching distance (48) is calculated for measuring the correlation between the corresponding proposals of two sets, follows:
limitation, thereby obtaining set of proposals with query-
globul
local TT())
I arg min
Σ
(14)
HEEAN
global
local w (I(s)
1=1
22

---

14
Normellane Weaghh
0.
Puk Regan (Luuradte f+1.3
0.
522
Normalized Proms Nimber (i) Gausslaa Weight: Distribution
Normalized Bromm Namber
(b) Dynamic Weight Allocation Distriaatio
Figure 7: Comparison of the original Gaussian scores and the proposed learnable peak-aware redistribution. 3 is a learnable parameter for controlling peak range to capture diverse durations of various actions in compositional queries.
where Av is the permutation and combination space between Peelobal and Pk. TI the optimal matching function. Global-local proposal boundary fusion based on the best match IT".
c = (aa) ct global
fused
+ [1 a(a]] local (T)
(15)
W
Jused = (a) w
global
local + (1 (a)) (1)
(16)
where (ar) I+e-r and is the learnable coefficient. plused = ((elused ( num; is the final proposal set injected by global semantic consistency and local temporal dynamics.
A.3 Original Gaussian Scores Vs. the Proposed Learnable Peak-aware Redistribution
This paper first obtains the corresponding frame-level weights according to the proposal boundary according to the existing work. However, the rigid Gaussian distribution is difficult to fit the duration characteristics of various actions in different compositional queries, resulting in low discriminability of the proposal features aggregated by these weights. Therefore, we propose a learnable peak-aware Gaussian distributor better cope with the diverse continuous frames of novel action compositions. Gaussian distribution first used to generate corresponding weights according to the center and duration of each proposal. For the i-th proposal in plused [[cluned w [ussed) ) muulyy and normalized frame number It, calculate the Gaussian weights:
1
(Ite
fused c.
t- 1-1
G:(()=
exp
where 34 1 (1= 1,2,.,, T),
(17)
T, 2TT
20²
where flssdd is the center point in Plused G. E R is the frame-level weight of the i-th proposal and 0 peak of the model is not single frame but more potential key frames relevant compositional
max (n (ntere 10 10-2) following existing works 10] 7.. can seen from Figure 7 the learnable
queries are included to improve the discriminability of proposal features.
A.4 Masked Query Reconstruction
To accurately evaluate the quality of proposals. we follow the principle of existing work 77] 35 the closer the proposal and query are semantically, the higher the proposal quality. Consistent with prior works [10] [6) some words in the original query will be masked and restored through the corresponding proposal features through autoregressive operations. The restored queries by responding proposals will be compared with the original query, and the negative logarithmic loss will be used to quantify the quality of the proposal. Specifically, the words in the original query = { are replaced by specific symbol [MASK] at ratio of 1/3 to obtain the query Š to be reconstructed. The word prediction is made based on before the current word to be reconstructed and the proposal features through autoregression [56]. Finally, the reconstruction loss is used to measure the proposal quality.
Σ
log P (sft) vo W., S....
(18)
j=1
23

---

Among them, @ is the Hadamard dot product for extracting proposal-specifie features based on video features V and W. is the fraee-level weights of i-th proposal from the learnble peak-aware Gaussian distributor, e is the reconstruction loss of the i-th proposal, that is, the corresponding quality.
A.5
Implementation Details
We use GloVe 43 extract textual features with hidden dimension of 300. For video features, 13D [44] is used for Charades-CG and C3D [45] for ActivityNet-CGG yielding feature dimensions of 1024 and 500, respectively. The number of proposals is set to 8, and the slot attention module is iterated 4 times. The initial value of the proposal fusion coefficient a is 0.2. The loss coefficient for cross-modal semantic alignment is 0.5, and the margin quality of contrastive loss is 0.146, consistent with 82 77. We use a batch size of 32 and train for 30 epochs. All experiments are conducted on a single NVIDIA GeForce RTX 4090 GPU. All configuration parameters have been open sourced for easy reproduction, see the supplementary materials.
B More Experiments
B.1 Performance Comparison in the Context of Weakly-Supervised Video Moment Retrieval
Table 4: Comparative performance TACoS [57] and Ego4D 15
datasets. Rn@w denotes the Recall @n metric under an IoU threshold of m. Bold results indicate the best-performing methods, while underlined results represent the second-best methods.
TACoS
Ego4D
Method
Params
RI @001
RI@0.33 R160.55 mloU
Params
RI @0.1
RI #0.3
R1@0,5 mioU
CNM 10
3.30M
27.77
7.05
2.32
2.84M
4.40
0.97
0.49
2.06
CPL 7
3.95M
30.14
9.50
9.52
3.72M
6.12
2.35
CCR
32
17.77M
31.04
7.95
9.28
33.53M
6.20
1.63
2.41
QMN
6
21.18M
32.59
9.07
9.27
36.94M
6.03
1.72
2.38
PPS
8.26M
24.44
10.07
3.87
8.08
8.03M
4.46
0.91
0.26
2.08
Ours
4.28M
30.64
10.25
4.00
9.79
4.05M
6.23
1.52
0.69
2.50
To further prove the effectiveness of the proposed method, the proposed method also experimented on two public long video positioning datasets, namely TACoS (57) and Ego4D [58] and the ex- perimental settings are consistent with previous works [6 55 The TACoS dataset derived from the MPII cooking activity dataset [60] and focuses on action recognition in laboratory kitchen environment. The challenge is accurately locate the fuzzy temporal boundaries of short-term and subtle actions (such as "cutting vegetable")) in long videos, and solve the semantic alignment problem of natural language queries and dynamic visual content. At the same time, it is limited by the high-density annotation of laboratory scenes, and the generalization ability of the model is facing test; the Ego4D-NLQ dataset contains first-person videos of multiple scenes around the world, requiring the model to handle occlusion, perspective changes, and long-term cross-modal understanding in complex dynamic environments. At the same time, needs to adapt to the open domain generalization challenges brought by multi-national and multi-cultural differences. We use C3D [45] and SlowFast [61] to extract video fcatures of TACoS and Ego4D respectively.
On TACoS, our method (Ours) achieves RI @0.1 30.64% and mloU 9.79% with 6.66M parameters, which better than high-parameter methods (such QMN's 23.55M parameters only improve R1 @0.1 to 32.59%), and significantly outperforms CNM (2.32%) and PPS (3.87%) at high loU threshold (RI @0.5). On the Ego4D dataset, our method surpasses the same-scale methods (such as CNM's 4,40% and 0.97%) in R1 @0.1 (6.23%) and RI @0.3 (1. (1.52%) with 6.42M parameters, while mloU reaches 2.50%, close to high-parameter methods (such as CCR's 2.41 1).. Experiments show that the proposed method achieves a balance between parameter efficiency and retrieval precision in long video temporal localization tasks complex scenes by optimizing the lightweight architecture design, especially in fine-grained action and cross-modal alignment.
24

---

Table 5: Comparison of Test-Trivial split rewriting queries based the Charades-CG and AetivityNet- CG datasets, using Qwen3 for query rewriting to obtain semantically consistent and diverse queries. means the fully supervised method. Bold results indicate the best-performing methods, while underlined results represent the second-best methods.
Rewritten
Charades-CG
ActivitYNet-CG
Model
Query
RI@0.3
RI@0.5 R1@0.7 mloU
R1@0,1
RI@0.3 RI@0.5 mloU
QD-DETR! PC-Net(OUrs)
60.66
38.60
52.53
43.76
25.98
42.86
70.87
54.84
26.68
47.12
52.78
29.62
14.35
36.45
QD-DETR!
65.67
53.62
32.53
46.27
53.18
35.76
20.23
38.04
CNM
V
43.86
29.46
14.02
29.49
45.86
25.23
13.43
33.01
CPL
44.64
31.88
14.50
29.85
45.77
24.00
11.60
31.26
CCR
V
59.14
45.06
21.38
39.28
44.65
26.95
10.58
33.92
QMN
55.94
40.50
18.67
37.08
45.79
24.42
13.14
33.73
PPS
59.82
43.02
20.93
39.31
48.36
26.13
13.24
31.71
PC-Net(Ours)
61.18
47.13
22.61
40.90
49.85
27.64
14.38
34.59
B.2 Model Robustness Comparison
To test the robustness of models to diverse query expressions, have additionally used Qwen3 62 to perform query rewriting on the Test-trivial set the most representative Charades CG and ActivityNet-CC datasets. We will further add these convincing experiments to the original paper to demonstrate the PC-Net's effectiveness. As shown in Table our method maintains good performance under this expression style, fully demonstrating its generalization ability different query styles. Even under the more challenging ActivityNet-CCG mloU metric, PC-Net experiences less performance degradation (-1.86%) than the fully supervised method QD-DETR, (-4.82%) when faced with queries with widely varying expression styles. This is due to the adaptation of the constructed dual-granularity proposal generator potentially complex temporal logic queries, the good proposal boundaries obtained, and the powerful proposal representation ability of the aligned proposal feature aggregator, which makes the discriminative proposal easy to select. subsequent work, we will further explore more complex forms of semantic combination.
Here the query rewriting tips queries with multiple expression styles.
allow Qwen3 62 rewrite semantically consistent
mess messages
{
"role": "system",
"content":
"You are a video content retrieval assistant specialized in query rewriting. "Your task is to generate semantically identical but linguistically diverse versions relatede queries while strictly preserving their original meaning."
}.
{
"role": "user", "content": (
f"Reurite the following video moment retrieval query into semantically equivalent f"but linguistically diverse version. Requirements:
f"1. Preserve the exact meaning of the original query. \n"
25

---

f²2. Alter phrasing (e.g., syntax, vocabulary, active/passive voice). \n"
f33. Ensure compatibility with video content retrieval (e.g., action/object/ccnn descriptions) An\n"
f"Original Query: \"(original "annn"
Reewritten Query:"
]
B.3
Model Efficiency Comparison
Table 6: Performance and efficiency comparison on Charades- CG and ActivityNet-GG Test-Trivial sets [9] FLOPs measured in giga operations (G), parameters in millions (M), and inference time in milliseconds per sample (ms). Best results are in bold.
Charades-CG
ActivityNet-CG
Method
FLOPss.
Params 4
Time
mloU f
FLOPs 4
Params +
Time 4
mIoU 1
CNM
0.30
2.52
0.13
37.88
0.29
2.38
0.13
35.06
CPL
7
1.91
3.01
0.70
45.82
2.98
4.64
0.87
32.73
CCR 32
2.36
9.01
0.83
45.62
36.17
268.96
3.15
33.56
QMN 6
10.04
12.51
1.70
45.85
43.85
272.38
3.96
33.82
PPS 2
3.28
7.31
1.27
45.63
2.94
8.94
1.15
32.98
PC-Net (Ours)
1.94
3.34
1.45
47.12
3.18
4.97
2.78
36.45
As shown in Table 6 our proposed PC-Net demonstrates superior performance across both Charades- CG and ActivityNet-CG benchmarks, achieving state-of-the-arr mloU scores of 47. 1..1%% and 36.45% respectively, while maintaining competitive efficiency. On Charades-CG, PC-Net outperforms previous best method (QMN (6) by -1.27% mloU with 80.7% fewer FLOPs (1.94G vs. 10.04G) and 73.3% fewer parameters (3.34M vs. 12.51M). For ActivityNet-CGG, our method surpasses CNM [O] by +1.39% mloU despite modest increase in computational cost (3. 18G FLOPs 0.29G FLOPs), highlighting its scalability to larger-scale datasets. Notably, PC-Net achieves these gains without excessive parameter growth, retaining model compactness (4.97M parameters) compared to resource- heavy alternatives like CCR (32) (268.96M parameters). The balance between accuracy and efficiency emphasizes the high-quality output of dual- dual-granularity proposal generation fusion and the role of semantic alignment and quality contrast loss in assisting the model to learn efficient parameters for effective cross-modal association.
B.4 More Hyperparameter Ablation Details
To further investigate the impact of the number of iterations K and the initialization value of the learnable proportional factor on the model performance, we conducted detailed ablation experiment on Charades-CG (Test-Trivial), as shown in Figure g Ablation studies reveal that the optimal number of iterations K=4 in the dual-granularity proposal generator balances global coherence and local temporal logic, achieving 47.12% Recall @ with mean IoU. For the proposal feature aggregator, a peak width coefficient 3=0.2 dynamically adjusts Gaussian distributions to suppress irrelevant frames while preserving key semantic cues, yielding 26.7% R @ with loU@0.7. These parameters outperform baselines and demonstrate the necessity of adaptive temporal modeling and feature aggregation in weakly supervised settings, enabling robust generalization diverse query styles and explicit temporal logic. The proposed framework achieves state-of-the-an performance, highlighting its potential for real-world applications in compositional moment retrieval.
To validate the efficacy of key modules PC-Net, ablation studies are conducted based on dynamic coefficients in Figure The dual-granularity proposal generator addresses coarse temporal perception by adaptively fusing global and frame-level boundaries, where the fusion coefficient exhibits an exponential growth phase (epochs 1-12) followed by stable phase (epochs 13-30),
26

---

27.0
47. $7.8%
26.5
Optimal 8-0.2. Max:
Recall@ moan lou (%)
47.00
36.1%
46.1%e
26.0-
46.75
Optimal at
Km44
(47.12%)
RB1 lol.g..
46.
46.50
46.1%
25.0
46.25
Performance gradusly decresses with
46.00
24.0
45.75
23.5
0.4
1.0
Iterations 2g
Ablation Peak Width Coeffidint Initialization
(a) Ablation the number iterations
(b) Ablation of peak width coefficieet 3 initialization.
Figure 8: Ablation of the iteration number K for generating local proposals in the hual-granularity proposal generator and the initialization of weight redistribution peak width coefficients / in the proposal feature aggregator.
0.35
0.26
Learnable Coefficient.
0.30
(epochs 1-12)
Exponential Growth Phase
Learnable Parameter
0.24
Stable Learning (epochs 13-30)
0.20
SOTA Epoch with Normalized Peak Width: 0,248
0.20
SOTA Epoch
0.18
0.15
0.16
0.10
a modulates proposal Mision
peak range
0.14
O
0.12
0
10
15
20
25
30
10
15
20
25
30
Training Epoch
Training Epoch
(a) Visualization of dynamic proposal fusion coefficients (b) Visualization of learnable peak width coefficients 8 based on the Charades CG dataset [9].
a, based on the ActivityNet CG dataset (9).
Figure 9: Visualization proposal fusion coefficients the dual-granularity proposal generator and peak width coefficients in the proposal feature aggregator.
indicating progressive adaptation to query-specific temporal logic. For feature aggregation, the learnable peak width coefficient B converges normalized width (0.248), demonstrating that the dynamic Gaussian distribution effectively mitigates semantic gaps and captures diverse durations of compositional actions by focusing on query-relevant regions. These results collectively validate that the proposed modules alleviate limitations in boundary generation, feature aggregation, and semantic association modeling under weak supervision.
C Limitations
The modular design of PC-Net, with its dynamic boundary modeling and semantic consistency modeling, demonstrates strong cross-task adaptability, enabling direct application to tasks like temporal action localization 663 promising results in weakly supervised compositional moment retrieval (WSCMR), the recall at high loU thresholds is less impressive in complex dynamic scenes with occlusion viewpoint changes (c.g., Ego4D.. To address these limitations, future work will focus on dense temporal modeling via adaptive frame sampling that prioritizes gueevarelevant keyframes in long videos, thereby reducing information loss and optimizing proposal boundaries.
64
65
or event detection 666 67
While PC-Net has achieved
27


Reviews:
[{"review_id": "uuFScIwwyt", "reviewer": "Reviewer_SEF7", "summary": "This paper introduces a new task—Weakly Supervised Compositional Moment Retrieval (WSCMR)—and proposes a novel architecture, PC-Net, to tackle it. The authors target the generalization challenge in moment retrieval, especially under weak supervision and compositional queries with novel syntactic structures or vocabulary. PC-Net incorporates a dual-granularity proposal generator, a semantic-aligned feature aggregator, and a quality margin regularizer. Experiments on Charades-CG, ActivityNet-CG, TACoS and Ego4D demonstrate strong performance and generalization with high parameter efficiency.", "questions": "1. The paper claims existing methods poorly generalize to compositional queries with novel syntactic structures or vocabulary, but the 'novel queries' in experiments only use simple combinatorial patterns. Queries with higher semantic richness (e.g., diverse sentences generated by LLMs) should be tested.\n\n2. The model structure appears over-engineered and insufficiently concise. How to validate the rationality of these structures? For example, disassemble the modules into existing frameworks to verify their generalizability.\n\n3. The performance improvement seems marginal despite lower parameters. Could it achieve greater gains at the same parameter level?", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Paper Strengths\n\nNovel Task Definition: Clearly motivated definition of WSCMR addressing practical scalability issues.\nMethodological Innovation: The integration of dual-granularity proposal generation and semantic-aware aggregation is well-executed and contributes to generalization.\n\nPerformance and Efficiency: Achieves state-of-the-art results with significantly fewer parameters than existing baselines.\n\nThorough Evaluation: Demonstrates effectiveness and generalization across multiple datasets (Charades-CG, ActivityNet-CG, TACoS, Ego4D) with thorough ablations and visualizations.\n\nClarity and Structure: The paper is well-organized with sufficient technical detail and intuitive figures.\n\nMajor Weaknesses\n\nLimited Analysis of Weak vs. Full Supervision: While the model demonstrates strong performance, the paper lacks a deeper analysis of how weak supervision compares to full supervision in terms of interpretability, robustness, and training stability. A more thorough exploration of this trade-off would strengthen the contribution.", "was_revised": true, "final_justification": "After reading the authors' rebuttal, most of my concerns have been well addressed. I am leaning to my current positive ratings.", "rebuttal": "## Weaknesses\n\n> 1. **While the model performs well, it lacks analysis of interpretability, robustness, and training stability of fully and weakly supervised methods, which contributes to this paper's contribution.**\n\n- Thank you for your valuable suggestions and recognition of the model's performance. Firstly, the core difference between fully supervised and weakly supervised methods is whether the training phase relies on accurate timestamps of the relevant moments. The reliance of fully supervised methods on cumbersome timestamp annotations leads to their low scalability in new application scenarios. \n- Secondly, we will make the following additions to strengthen the contribution as suggested: \n  - In terms of interpretability, we will add cross-modal attention visualization figures to intuitively compare and show the frame-level query relevance learned by the fully supervised and weakly supervised methods, to help illustrate that the weakly supervised method also has strong semantic learning capabilities. \n  - For robustness, we rewrite the queries in the most representative Charades-CG and ActivityNet-CG Test-trivial sets based on the latest stable version of Qwen3 to simulate different query styles. As shown in Table R1 in Q1, the performance of the proposed PC-Net under the new queries is still better than the existing weakly supervised methods, highlighting its powerful proposal representation capabilities, enabling it to better adapt to the diverse descriptions of real scenes, verifying its robustness advantage (the new query used in the test will be made public in the open source repository). \n  - Finally, to support the training stability of the proposed method, we will supplement the corresponding loss function decline curve.\n\n## Questions\n\n> 1. **The 'novel queries' in experiments only use simple compositional patterns. Queries with higher semantic richness (e.g., diverse sentences generated by LLMs) should be tested.**\n\nTable R1: Comparison of Test-Trivial split rewriting queries based on the Charades-CG and ActivityNet-CG datasets, using Qwen3 for query rewriting to obtain semantically consistent and diverse queries. 'Fully' means the fully supervised method.\n\n|Model|Rewrite queries using Qwen3|Charades-CG||||ActivityNet-CG||||\n|:-:|:-|:-|:-|:-|:-|:-|:-|:-|:-|\n|||**R1@0.3**|**R1@0.5**|**R1@0.7**|**mIoU**|**R1@0.3**|**R1@0.5**|**R1@0.7**| **mIoU**|\n|QD-DETR$_s$(Fully)|No|-|60.66|38.60|52.53|-|43.76|25.98|42.86|\n|PC-Net(Ours)|No|70.87|54.84|26.68|47.12|52.78|29.62|14.35|36.45|\n|QD-DETR$_s$(Fully)|Yes|65.67|53.62|32.53|46.27|53.18|35.76|20.23|38.04|\n|CNM|Yes|43.86|29.46|14.02|29.49|45.86|25.23|13.43|33.01|\n|CPL|Yes|44.64|31.88|14.50|29.85|45.77|24.00|11.60|31.26|\n|CCR|Yes|59.14|45.06|21.38|39.28|44.65|26.95|10.58|33.92|\n|QMN|Yes|55.94|40.50|18.67|37.08|45.79|24.42|13.14|33.73|\n|PPS|Yes|59.82|43.02|20.93|39.31|48.36|26.13|13.24|31.71|\n|PC-Net(Ours)|Yes|**61.18**|**47.13**|**22.61**|**40.90**|**49.85**|**27.64**|**14.38**|**34.59**|\n\n- Thanks for your suggestions. \n  - To make a fair comparison with existing methods, our experimental settings in the original paper follow the previous study (QD-DETR$_s$), using the same compositional mode (e.g., 'Novel-Composition' and 'Novel-Word') to verify the generalization performance of different methods. \n  - As suggested, to test the robustness of models to diverse query expressions, we have additionally used Qwen3 to perform query rewriting on the Test-trivial set of the most representative Charades-CG and ActivityNet-CG datasets. We will further add these convincing experiments to the original paper to demonstrate the PC-Net's effectiveness.\n\n- As shown in Table R1, our method maintains good performance under this new expression style, fully demonstrating its generalization ability to different query styles. Even under the more challenging ActivityNet-CG mIoU metric, PC-Net experiences less performance degradation (-1.86%) than the fully supervised method QD-DETR$_s$ (-4.82%) when faced with queries with widely varying expression styles. This is due to the adaptation of the constructed dual-granularity proposal generator to potentially complex temporal logic queries, the good proposal boundaries obtained, and the powerful proposal representation ability of the aligned proposal feature aggregator, which makes the discriminative proposal easy to select. In subsequent work, we will further explore more complex forms of semantic combination.\n- The LLM prompts and examples for rewriting queries are shown below:\n\n```python\nmessages=[\n            {\n                \"role\": \"system\",\n                \"content\": \"You are a video content retrieval assistant specialized in query rewriting. \"\n                           \"Your task is to generate semantically identical but linguistically diverse versions \"\n                           \"of video-related queries while strictly preserving their original meaning.\"\n            },\n            {\n                \"role\": \"user\",\n                \"content\": f\"Rewrite the following video moment retrieval query into a semantically equivalent \"\n                           f\"but linguistically diverse version. Requirements:\\n\"\n                           f\"1. Preserve the exact meaning of the original query.\\n\"\n                           f\"2. Alter phrasing (e.g., syntax, vocabulary, active/passive voice).\\n\"\n                           f\"3. Ensure compatibility with video content retrieval (e.g., action/object/scene descriptions).\\n\\n\"\n                           f\"Original Query: \\\"{original_query}\\\"\\n\\n\"\n                           f\"Rewritten Query:\"\n            }\n        ],\n```\n\n| Original                        | Rewritten                                                |\n| ------------------------------- | -------------------------------------------------------- |\n| person pouring it into a glass. | An individual transferring liquid into a drinking glass. |\n| person put the clothes.         | Someone places the garments.                             |\n\n> 2. **The model structure appears over-engineered and insufficiently concise. How to validate the rationality of these structures? For example, disassemble the modules into existing frameworks to verify their generalizability.**\n\n- We clarify that the proposed PC-Net is not over-designed, and its process is relatively clear, as recognized by reviewers #H739. As shown in the model overview in Figure 2, the key modules contain three parts around the proposal, which are boundary generation, feature modeling, and quality assessment. The components are designed based on the two challenges faced by WSCMR: **(1)** weak pairwise supervision and limited query style in training, and **(2)** the need to generalize to compositional queries with diverse temporal and semantic dependencies.\n- We are pleased that the reviewer raised the issue of component rationality, which is also a key consideration in model construction. In fact, we conducted detailed component combination experiments in the ablation experiments to verify the contribution of each component to generalization ability through rigorous ablation experiments based on the existing framework CPL (Setting a) , as shown in Table 3: \n  - The dual-granularity proposal generator and cross-modal semantic contrastive loss significantly alleviate the problem of insufficient fine-grained temporal perception and cross-modal gap (compare rows (b)&(d) and Ours); \n  - The learnable peak-aware Gaussian distributor enhances proposal features by modeling action duration diversity (row (j) & Ours); \n  - The quality margin regularizer effectively enhances visual-semantic association learning and suppresses the potential false correlation (row (h) & Ours). \n- In summary, the innovation of PC-Net lies in alleviating the generalization challenge of weakly supervised scenarios: dual-granularity proposal generator handles complex temporal dependencies, proposal feature aggregator narrows the semantic gap and adapts to variable action duration, and the quality margin regularizer suppresses spurious associations. These components work closely together to deal with the core challenge of WSCMR. \n\n> 3. **The performance improvement seems marginal despite lower parameters. Could it achieve greater gains at the same parameter level?**\n\n- Thank you for your insightful questions. The performance gains are not marginal, given the efficiency and inherent difficulty of the task. First, as shown in Table 1, our method achieves a 0.58% mIoU gain on the Novel-Composition split and a 1.86% mIoU gain on the Novel-Word split compared to CPL, while only increasing the number of parameters by 0.33M. Crucially, it also surpasses the fully supervised VISA (CVPR'22) method on the Test-Trivial split. In contrast, the CCR method uses three times as many parameters as CPL but performs inferior to CPL, highlighting the challenges of WSCMR.\n\n- Our core contribution is a concise and reproducible framework designed to address the unique challenges of WSCMR, such as compositional generalization and weak paired supervision. A key design principle is parameter efficiency, achieving competitive gains with minimal additional complexity. As mentioned in the conclusion, future work will explore enhancing the understanding of compositional queries in long videos, possibly through modules such as refined temporal modeling, which could increase the number of parameters and further improve performance."}, {"review_id": "iKX4GVDV6A", "reviewer": "Reviewer_H739", "summary": "The paper introduces a proposal-centric network (PC-Nets) for locating video moments in long videos according to natural language queries. For better retrieval of video moments according to queries with novel structure and vocabulary, the PC-Net features a dual-grained proposal generator, a learnable peak-aware Gaussian distributor, and a quality margin regularizer. The proposed model has been extensively evaluated on multiple benchmark datasets across various settings. It is shown that the PC-Net achieves competitive performance to existing solutions with fewer parameters.", "questions": "I am generally happy with this submission. It will be helpful if the authors can clarify why their proposed model is targeting WSCMR rather than general VMR as suggested in the weaknesses section.", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n+ The paper is well written and easy to follow\n+ The weakly-supervised compositional moment retrieval studied in this paper is challenging but practical, which requires generalisation to unseen scenarios while avoiding labour-intensive annotations of moment boundary for training\n+ The proposed method is technically solid and well justified. Various components and designs have been made to improve video moment retrieval from different perspectives, from proposal generation to cross-modal feature aggregation and proposal quality assessment.\n+ The experiments are comprehensive with detailed ablation studies.\n\nWeaknesses:\n+ About motivation, the WSCMR task is explicitly targeted in this paper but it seems like most proposed designs are also valid for general video moment retrieval (VMR). I failed to find strong connections between the designs of PC-Net and the unique challenges of WSCMR. It might be helpful if the authors can spell out and briefly summarise how the challenges unique to WSCMR are addressed by the proposed model. Otherwise, it should be ok to expand the scope to general VMR instead of a specific scenario.\n+ About clarity, given that PC-Net is composed of both common components widely adopted in the field and newly proposed ones, I found it hard to identify the latter, given all components are elaborated to a certain extent.", "was_revised": true, "final_justification": "My main concerns about the unique challenges of WSCMR and how they are addressed by the proposed designs are resolved by the authors. It would be helpful to the clarity if the authors could incorporate their clarification about which components are proposed in this work and which are borrowed in their final version.", "rebuttal": "## Weaknesses\n\n> 1. **I failed to find strong connections between the designs of PC-Net and the unique challenges of WSCMR. It might be helpful if the authors can spell out and briefly summarise how the challenges unique to WSCMR are addressed by the proposed model. Otherwise, it should be ok to expand the scope to general VMR instead of a specific scenario.**\n\n- Thank you for your suggestion. First, the relationship between the proposed PC-Net and the challenges faced by WSCMR is analyzed. The challenges of the proposed WSCMR are: **(1)** only relying on weak paired video-query supervision to model fine-grained cross-modal semantic associations; **(2)** the query styles in the training data are limited, and the model needs to be generalized to compositional queries containing novel syntactic structures, unseen vocabulary, and complex temporal semantic dependencies. PC-Net is designed to directly respond to these challenges:\n  - **Coping with diverse syntax and temporal dependencies (Challenge 1&2):**  the dual-granularity proposal generator generates proposal boundaries with fine-grained temporal awareness by jointly decoding frame-level and global multimodal features. This enables it to effectively understand the explicit temporal logic in compositional queries (such as \"the second one\", \"after...\"), overcoming the defects of existing methods that rely only on global associations and have difficulty in handling such complex structures.\n  - **Mitigating semantic gap and capturing actions with flexible durations (Challenge 1&2):**  the proposal feature aggregator with semantic alignment employs triples of query-related/unrelated frames to align cross-modal features in a unified semantic space, thereby narrowing the semantic gap. Second, it dynamically adjusts the Gaussian peak area to flexibly adapt to the variable duration of actions described by compositional queries. This ensures that the aggregated proposal features are discriminative and aligned with the query features.\n  - **Learning visual-semantic associations and suppressing spurious correlation (Challenge 1):** the quality margin regularizer helps the model capture co-occurring relevant visual elements in proposals while suppressing potential spurious associations by partitioning proposals into high/low correlation subsets and performing contrastive learning. This indirectly optimizes proposal features modeling and cross-modal fine-grained associations, alleviating the difficulty of cross-modal semantic associations in the absence of precise timestamp associations.\n- Thus, each core component of PC-Net closely corresponds to the challenges of WSCMR. Experimental results verify the effectiveness and efficiency of this design. We will outline this correspondence more clearly in the manuscript to clarify this potential confusion.\n- Besides, this paper mainly emphasizes that the WSCMR task avoids laborious timestamp annotation and has good generalization ability for diverse queries, so it has a broader research prospect than general VMR, rather than referring to \"specific scenarios\". Lastly, the proposed PC-Net does not contain modules optimized for using precise timestamps and is therefore not suitable for the general VMR. \n\n> 2. **About clarity, given that PC-Net is composed of both common components widely adopted in the field and newly proposed ones, I found it hard to identify the latter, given all components are elaborated to a certain extent.**\n\n- Thank you for your question about the clarity of the model components, which helps us to explain our work more accurately. First, PC-Net integrates common components widely used in the field for proposal quality assessment. Specifically, to compare fairly with existing weakly supervised methods, we use a standard Transformer decoder to reconstruct the masked query and use the corresponding reconstruction loss to assess the proposal quality.\n- The primary innovation of this study resides in analyzing the shortcomings of the existing moment retrieval paradigm and proposing the weakly supervised compositional moment retrieval (WSCMR) task and the construction of three novel modules designed to address the distinct challenges of WSCMR as described in the response to Weakness 1.\n- These modules distinguish our approach from prior work: \n  - The dual-granularity proposal generator models frame-level associations and captures temporal information to refine proposal boundaries. \n  - The proposal feature aggregator that enhances proposal representations through bridging the cross-modal semantic gap and modeling diverse action durations.\n  - The quality margin regularizer that emphasizes relevant visual elements co-occurring in proposals while suppressing spurious associations. \n\n## Questions\n\n> 1. **I am generally happy with this submission. It will be helpful if the authors can clarify why their proposed model is targeting WSCMR rather than general VMR as suggested in the weaknesses section.**\n\n- Thank you for recognizing the efforts made in this article. We clarify that there are two main reasons for focusing on WSCMR:\n  - General VMR relies on precise timestamp supervision during training, which limits its applicability in such real-world scenarios where annotations are scarce.\n  - Existing weakly supervised VMR methods are difficult to generalize to unseen queries with complex syntactic and temporal logics, which hinders their practical effectiveness. \n- WSCMR addresses these deficiencies by eliminating the need for timestamp supervision and explicitly enhancing the compositional generalization ability to new queries, making it a more promising research direction than VMR. Therefore, the design of the proposed PC-Net is tailored to the core challenges of WSCMR  as described in the response to Weakness 1. We will further highlight the close connection between the proposed components and the WSCMR task in the final version. Thank you again for your high recognition of this article."}, {"review_id": "58KP5U2Olo", "reviewer": "Reviewer_PDfn", "summary": "The paper proposes the new task of weakly supervised compositional moment retrieval (WSCMR), building on weakly supervised video moment retrieval but applying the models to datasets with with novel syntactic structures or vocabulary. A new model, PC-Net, is proposed to tackle this task, which utilises a number of modules. It uses a dual-granularity proposal generator to produce proposals using a combination of global and local knowledge. It has a proposal feature aggregator with semantic alignment which includes the ability to adjust the Gaussian peak to account for diverse durations. Cross-modal semantic alignment is used to relate the video and text features. The proposals are assessed using query mask reconstruction, and it includes a quality margin regularizer to identify subtle semantic differences between query-relevant proposals. Performance is measured on the Charades-CG and ActivityNet-CG datasets. It achieves broadly improved performance over existing weakly supervised methods, albeit PPS is stronger on many metrics on the ActivityNet-CG benchmark.", "questions": "- (line 215) Is the complement just the part of the video outside of the proposal? So is this why there are two parts to the complement (n1 and n2), one for either side of the proposal?\n- Overall, while I think this is a promising paper and a promising task, the main issues which I have are with readability and the difficulty in understanding parts of the paper, along with some concerns over performance on ActivityNet-CG on R1@m compared to PPS.", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n- It is beneficial to attempt the Compositional Video Moment Retrieval task in a weakly supervised manner to avoid the need for extensive annotations. Good from a real-world practicality perspective\n- The components generally make sense in accordance with the aim of improving the model's ability to handle diverse moment durations.\n- Performance on Charades-CG is strong across all metrics, while performance on ActivityNet-CG is also quite strong.\n- The ablations are thorough and display the importance of each component.\n- The model shows good efficiency compared with previous methods\n\n**Weaknesses**\n- The method contains many parts and it is a bit difficult to understand parts of the paper\n- For example, equations 5&6 for the redistribution of weights of the Gaussian distribution. I found it difficult to understand why it is formulated this way, or what the motivation for these equations having this form is. Line 194-196 does not make sense to me grammatically or semantically, \"η_i(t) through the distance discrimination mechanism, the peak area and the edge area are divided to achieve dynamic weight allocation\". It doesn't help that σ_i and x_t are also not described and only defined in the appendix. Even in the appendix, the purpose of σ_i is not really explained. \n- In line 154, c and w are not defined. They are defined much later in the paper but at this point it is potentially confusing.\n- I also found the lack of definition of the complement (line 215) and the origin of the two negative proposal reconstruction losses (line 217) to be confusing.\n- PPS consistently outperforms PC-Net on R1@m metrics on ActivityNet-CG, which perhaps indicates that PC-Net is not as good at achieving tight moment localisations on this dataset.\n- (minor) Line 223 it should say Figure 2(c) rather than Figure 2(e) I assume.", "was_revised": true, "final_justification": "The rebuttal addressed my main concerns. I feel the paper could do with some improved writing to improve the ease of understanding, and the extra information from the rebuttal should be included in a final version. The results, while still weaker on some metrics for ActivityNet-CG, are still strong enough I think, taking into consideration their reasoning for the difference in performance between PC-Net and PPS on the different metrics. As a result, I raise my rating to borderline accept.", "rebuttal": "## Weaknesses\n\n> 1. **The method contains many parts and it is a bit difficult to understand parts of the paper.**\n\n- Thank you very much for your careful review and valuable suggestions. We will make revisions based on your suggestions to improve readability and clarify the contributions. \n\n> 2. **Questions related to equations (5)-(6).**\n\n- Due to limited space, the wording in the original text may have been confusing, and the following points are clarified in detail.\n- **Motivation of equations (5) and (6):** Existing methods only use a fixed Gaussian distribution to generate proposal weights, which ignores the diverse durations of potential actions in queries, resulting in insufficient discriminative power of aggregated proposal features from video features. The core function of formulas (5) and (6) is to dynamically redistribute the standard Gaussian weights, to assign higher weights near the center of the proposal (peak area) where the action is most likely to occur, forming a \"platform\" instead of a sharp single-point peak in the standard Gaussian. Direct hard assignment to achieve peak platform effect is non-differentiable and difficult to optimize through training. Therefore, we introduce a variable $η_i(t)$ with the learnable factor $\\beta$. The original formula and explanation are as follows.\n     $$M_i(t) = \\frac{1}{1 + e^{-1000 \\cdot \\eta_i(t)}}, \\text{where}\\ \\eta_i(t) = \\beta \\sigma_i - |x_t - c_i^{\\text{fused}}| \\tag{5}$$\n     $$W_i(t) = G_i(t) \\cdot (1 - M_i(t)) + M_i(t)  \\tag{6}$$\n- **Clarification of $\\sigma_i$ and $x_t$ :** $\\sigma_i$ and $x_t$ are standard variables in the standard Gaussian distribution, with definitions consistent with prior methods like CPL [1]. $\\sigma_i  = \\frac{\\max\\left(w_i^{\\text{fused}}, 10^{-2}\\right)}{9}$ originates from the fused proposal width $w_i^{\\textrm{fused}}$ and governs the shape and width of the Gaussian curve: longer proposals (larger $w_i^{\\textrm{fused}}$) result in larger $\\sigma_i$ and flatter curves, while shorter proposals yield smaller $\\sigma_i$ and sharper curves. $x_t  = \\frac{t - 1}{T - 1}\\ (t = 1, 2, \\ldots, T)$ acts as an auxiliary variable representing positions along the frame sequence axis, used to compute the Gaussian weight at a specific time step $t$.\n\n- **Clarification of $η_i(t)$ and \"distance discrimination mechanism\":** \n    - $η_i(t) = βσ_i - |t - c^{\\textrm{fused}}_i|$ implements a temporal distance discrimination mechanism\n      - $c^{\\textrm{fused}}_i,w_i^{\\textrm{fused}}$ is the temporal center and width of the $i$-th proposal\n      - $|t - c^{\\textrm{fused}}_i|$ computes absolute temporal distance between temporal point $t$ and $c^{\\textrm{fused}}_i$\n      - $βσ_i$ sets an adaptive peak range with the learnable $\\beta$ where: $\\sigma_i = \\frac{\\max(w_i^{\\textrm{fused}}, 10^{-2})}{9}$ ensures numerical stability and scales the fused weight $w_i^{\\textrm{fused}}$\n  - **Mask dynamics ($M_i(t)$)**:  \n    $M_i(t) = \\frac{1}{1+e^{-1000\\cdotη_i(t)}}$ creates a differentiable mask:\n    - When $t$ approaches $c^{\\textrm{fused}}_i$ (small $|t - c_i^{\\textrm{fused}}|$):\n      - $η_i(t) > 0$ → $M_i(t) \\approx 1$ tends to assign the maximum weight 1\n    - When $t$ nears video edges (large $|t - c_i^{\\textrm{fused}}|$):\n      - $η_i(t) < 0$ → $M_i(t) \\approx 0$ tends to preserve the original standard Gaussian weight\n    - The scaling factor 1000 ensures sharp transitions.\n  - **Dynamic weight redistribution ($W_i(t)$)**:  \n    $W_i(t) = G_i(t) \\cdot (1 - M_i(t)) + M_i(t)$ combines:\n    - **Standard Gaussian weights** $G_i(t)$ calculation following previous methods (CPL,CCR) \n    - **Peak range controlled by $\\beta$**: \n      - Near center ($M_i(t) ≈ 1$): $W_i(t) ≈ 1$ (forms \"peak area\")\n      - At edge ($M_i(t) ≈ 0$): $W_i(t) ≈ G_i(t)$ (preserves standard Gaussian weights)\n    - As visualized in Appendix Figure 5, this mechanism transforms standard Gaussian weights into redistributed weights with enhanced central emphasis, enabling capture of diverse action durations with the learnable $\\beta$.\n\n  [1] Weakly supervised temporal sentence grounding with gaussian-based contrastive proposal learning (CVPR'22)\n\n> 3.  **In line 154, c and w are defined later in the paper and it is potentially confusing.**\n\n- Since the proposal boundary is not used immediately after generation and involves subsequent feature aggregation, the definitions of c and w are given in line 183. We will add symbolic explanations to alleviate confusion.\n\n> 4. **Lack of definition of the complement (line 215) and the origin of the two negative proposal reconstruction losses (line 217) to be confusing.** \n\n- Each proposal is defined by its center point ($c_i$) and duration ($w_i$) (line 214). As shown in Figure 2(c), the **complement** of the best proposal $\\mathcal{P}^{\\text{fused}}_O$ consists of two segments:\n  - $\\mathcal{P}^{\\text{fused} -l}_O$ (left negative proposal)\n  - $\\mathcal{P}^{\\text{fused} -r}_O$ (right negative proposal)\n\n- The **negative proposal reconstruction losses** ($\\mathcal{L} _{n_1}^{re},\\mathcal{L} _{n_2}^{re}$; line 217) are computed for these left/right complements:\n  - Features are aggregated for $\\mathcal{P}^{\\text{fused} -l}_O$ and $\\mathcal{P}^{\\text{fused} -r}_O$ using their center points and widths (durations). \n  - Reconstruction loss follows prior work (e.g., CNM, CPL; line 213) to get $\\mathcal{L} _{n_1}^{re},\\mathcal{L} _{n_2}^{re}$.\n- We will explain its specific meaning more clearly in the final version (i.e., the set of negative proposals on the left and right ends), and strengthen the correspondence with Figure 2(c).\n\n> 5. **Performance comparison of the R1@m metric on ActivityNet-CG.**\n\nTable R1: Comparison of Test-Trivial split rewriting queries based on the Charades-CG and ActivityNet-CG datasets, using Qwen3 for query rewriting to obtain semantically consistent and diverse queries. 'Fully' means the fully supervised method.\n\n|Model|Rewrite queries using Qwen3|Charades-CG||||ActivityNet-CG||||\n|:-:|:-|:-|:-|:-|:-|:-|:-|:-|:-|\n|||**R1@0.3**|**R1@0.5**|**R1@0.7**|**mIoU**|**R1@0.3**|**R1@0.5**|**R1@0.7**| **mIoU**|\n|QD-DETR$_s$(Fully)|No|-|60.66|38.60|52.53|-|43.76|25.98|42.86|\n|PC-Net(Ours)|No|70.87|54.84|26.68|47.12|52.78|29.62|14.35|36.45|\n|QD-DETR$_s$(Fully)|Yes|65.67|53.62|32.53|46.27|53.18|35.76|20.23|38.04|\n|CNM|Yes|43.86|29.46|14.02|29.49|45.86|25.23|13.43|33.01|\n|CPL|Yes|44.64|31.88|14.50|29.85|45.77|24.00|11.60|31.26|\n|CCR|Yes|59.14|45.06|21.38|39.28|44.65|26.95|10.58|33.92|\n|QMN|Yes|55.94|40.50|18.67|37.08|45.79|24.42|13.14|33.73|\n|PPS|Yes|59.82|43.02|20.93|39.31|48.36|26.13|13.24|31.71|\n|PC-Net(Ours)|Yes|**61.18**|**47.13**|**22.61**|**40.90**|**49.85**|**27.64**|**14.38**|**34.59**|\n\n- PPS employs a computationally intensive strategy based on complex, redundant boundary enumeration (predicting i+1 potential boundaries for the i-th proposal, followed by voting) to inflate R1@m. This brute-force approach captures potential target regions at considerable computational expense, and compromises the quality of individual proposals, thereby undermining semantic completeness (as evidenced by lower mIoU).\n\n- In contrast, PC-Net prioritizes semantic association between query and video frames while PPS focuses on boundary engineering. Here, PC-Net dominates: it achieves +3.07% to +3.47% higher mIoU than PPS across all ActivityNet-CG splits (Table 2), proving superior semantic completeness. Critically, PPS's redundant boundary engineering and voting backfire on other long-video benchmarks where semantic coherence matters most. Furthermore, as shown in Appendix B.1, on the TACoS and Ego4D datasets, PC-Net has only half the parameters of PPS, but its performance is slightly better (e.g., R1@0.5 is +0.43% on Ego4D) in Table 4. This efficiency and scalability demonstrate PC-Net's purer cross-modal modeling, while PPS tends to boundary engineering.\n\n- To test the robustness of models to diverse query expressions, we have additionally used the Qwen3 to perform query rewriting on the Test-trivial set of Charades-CG and ActivityNet-CG datasets. As shown in Table R1,it should be emphasized that after the queries in ActivityNet-CG were semantically rewritten by LLM, the R1@m indicator of PPS also dropped sharply, which fully proves that PPS focuses on boundary engineering rather than the more important cross-modal semantic associations, making it difficult to adapt to diverse queries. \n\n- Future work will refine boundaries only when robust query-proposal semantic alignment is first maximized, ensuring efficiency and semantic integrity without resorting to exhaustive enumeration.\n\n\n> 6. **(minor) Line 223 it should say Figure 2(c) rather than Figure 2(e) I assume.**\n\n- Thanks for pointing this out, we will make further corrections.\n\n## Questions\n\n> 1. **(line 215) Is the complement just the part of the video outside of the proposal?**\n\n- That's correct. The definition of complement is the area outside the target area (proposal) relative to the full set (video), located on both sides of the proposal.\n\n> 2. **While I think this is a promising paper and a promising task, the main issues are with readability and the difficulty in understanding parts of the paper, along with some concerns over performance on ActivityNet-CG.**\n\n- Thanks for your affirmation about research prospects and proposed methods. Due to limited space, some concepts are not introduced in detail, which makes it difficult to understand. We will add relevant explanations of formulas (5) and (6) and the clear definition of the complement of line 215.\n- For more details on the PPS experimental analysis, please see the response to Weakness 5. In short, PPS leverages boundary engineering, achieving a slight lead in R1@m over ActivityNet-CG. However, this introduces significant computational overhead and weakens cross-modal semantic learning. When faced with rewritten queries or other scenarios, PPS performs poorly, while PC-Net maintains strong generalization capabilities."}, {"review_id": "5DzekAbmnN", "reviewer": "Reviewer_CBCf", "summary": "The paper introduces a new task, Weakly Supervised Compositional Moment Retrieval (WSCMR), in which models are trained only with video–query pairs (no temporal annotations) yet are expected to generalize to queries that contain unseen grammatical structures and vocabulary. To tackle this task the authors propose PC-Net, a “proposal-centric network” that (i) fuses global and frame-level features through a dual-granularity proposal generator, (ii) aggregates frame features with a learnable peak-aware Gaussian distributor and cross-modal contrastive alignment, and (iii) applies a quality-margin regularizer that contrasts high- and low-quality proposals. Experiments on Charades-CG and ActivityNet-CG show that PC-Net attains state-of-the-art mIoU while using fewer parameters than prior work.", "questions": "1. Fig. 4 only shows successful cases. Please provide examples where PC-Net fails (e.g., long-range compositional queries) and analyse why.\n2. Why is the whole-video reconstruction loss an appropriate decision boundary?\n3. What hyper-parameter budget and feature extractors were used when re-running CPL, QMN, PPS, etc.?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths:**\n1. Solid engineering: each module is motivated by a concrete limitation in prior weak-supervised methods (coarse boundaries, fixed Gaussians, simple negatives).\n2. High-level pipeline is clearly illustrated (Fig. 2) and symbols are defined.\n3. Raises the bar for weak-supervised moment retrieval by adding compositional generalization—a realistic setting absent from prior work.\n\n**Weaknesses:**\n1. While the overall framework is new, the individual components feel like incremental improvements or combinations of existing techniques. The dual-granularity generator's use of global and local features is a common design pattern. The cross-modal semantic alignment (part of PFA) relies on a standard triplet contrastive loss. The \"learnable peak-aware Gaussian distributor\" is a minor modification to the Gaussian weighting used in prior work. The quality margin regularizer applies another layer of contrastive learning, which has become a standard tool. The novelty appears to lie more in the specific combination of these parts rather than in a truly groundbreaking new mechanism.\n2. Some design choices seem heuristic and lack strong theoretical or empirical justification. For instance, the Quality Margin Regularizer contrasts the average reconstruction losses of high- and low-relevance proposal sets. It's unclear why this is superior to methods that contrast individual proposals. Averaging could potentially wash out important learning signals from specific high-quality candidates. The paper does not provide an analysis to justify this specific design choice over simpler alternatives.\n3. The ablation study in Table 3 raises some concerns about the stability and contribution of individual modules. When added in isolation, the learnable peak-aware Gaussian (LPG) in setting (c) and the quality margin loss in setting (e) both lead to a decrease in performance on the Novel-Composition split compared to the baseline (a). The authors briefly mention this but do not provide a satisfying explanation. This suggests that the components are not robustly beneficial on their own and that the overall performance gain is the result of a carefully-tuned combination of modules, rather than a set of independently strong contributions.\n4. Gains on ActivityNet-CG are modest and sometimes below baselines on recall at stricter IoU—contradicting the “superior” claim.\n5. Writing is dense, with long sentences and inconsistent tense; several typos (“quries”) hamper readability.", "was_revised": true, "final_justification": "Thanks for the authors' detailed response. After careful reading of the rebuttal and comments from other reviewers, I decided to raise my score.", "rebuttal": "## Weaknesses\n\n> 1. **While the overall framework is new, the novelty appears to lie more in the specific combination of the proposed parts rather than in a truly groundbreaking new mechanism.**\n\n- First, we clarify that the core contribution of this paper is **the first analysis of potential shortcomings of the existing moment retrieval paradigm in real-world scenarios, proposing the WSCMR, and constructing a simple and effective baseline**. \n- To systematically addresses two core challenges in WSCMR: (1) fine-grained cross-modal alignment under weak supervision, and (2) generalization to compositional queries, the following components are constructed to form PC-Net.  \n  - The dual-granularity proposal generator explicitly models complex temporal dependencies (e.g., \"after\", \"second\") by jointly decoding frame-level and global features, overcoming existing methods' inability to understand temporal structures in queries. \n  - The proposal feature aggregator employs cross-modal triple contrastive learning to bridge frame-query semantic gaps, while its learnable peak-aware Gaussian distributor dynamically adjusts key area scopes to adapt to variable action durations. \n  - To suppress weak supervision's inherent false correlations, the quality margin regularizer compares high/low relevance proposals, forcing the model to focus on strongly co-occurring visual elements and enhancing robustness to compositional queries. \n\n> 2. **Some design choices appear heuristic and lack strong theoretical or empirical justification (the averaging operation in the quality margin regularizer).**\n\n- First, we have explained the motivation for component design in the introduction and the corresponding method description, and have demonstrated the role of components in promoting the generalization ability of the model through detailed ablation experiments in Table 3, with no lack of evidence.\n- Existing methods overlook that partially relevant proposals can guide the model to capture co-occurring visual elements while suppressing spurious associations. To address this, we introduce a quality margin regularizer ($\\mathcal{L} _{\\text{qua}}$) as a supplement to the reconstruction loss ($\\mathcal{L} _O^{re}$).\n- $\\mathcal{L} _{\\text{qua}}$ expands the reconstruction loss margin between high- and low-relevance proposal subsets, using the reference proposal as a boundary. This highlights co-occurring elements in high-correlation subsets and suppresses spurious links in low-correlation subsets. As Table 3 (row (h) vs. Ours) shows, optimizing this contrast enhances generalization, confirming $\\mathcal{L} _{\\text{qua}}$'s effectiveness.\n- The motivation for the construction of the remaining components can be found in the response to weakness 1.\n\n> 3. **The ablation study in Table 3 raises some concerns (robustness) about the stability and contribution of individual modules(setting (c)  and (e)) .**\n\n- Further analysis of the settings follows. Performance slightly declines without the dual-granularity proposal generator due to poor proposal boundaries. This occurs because: (c) Reallocated Gaussian weights cannot adapt to the optimal peak interval within low-quality boundaries, and (e) Such proposals contain irrelevant frames internally while suppressing relevant frames, confusing the model's ability to learn query-relevant moment associations.\n\n- Introducing the proposed dual-granularity proposal generator significantly improves performance (Table 3: (row (j) & Ours) and (row (h) & Ours)), effectively mitigating these limitations of existing methods. Furthermore, robustness analysis in the response to Weakness 4 confirms that PC-Net maintains leading performance even when encountering query expression styles completely dissimilar to those seen during training, demonstrating the inherent robustness of the components.\n\n> 4. **Gains on ActivityNet-CG are modest and sometimes below baselines on recall at stricter IoU—contradicting the “superior” claim.**\n\nTable R1: Comparison of Test-Trivial split rewriting queries based on the Charades-CG and ActivityNet-CG datasets, using Qwen3 for query rewriting to obtain semantically consistent and diverse queries. 'Fully' means the fully supervised method.\n\n|Model|Rewrite queries using Qwen3|Charades-CG||||ActivityNet-CG||||\n|:-:|:-|:-|:-|:-|:-|:-|:-|:-|:-|\n|||**R1@0.3**|**R1@0.5**|**R1@0.7**|**mIoU**|**R1@0.3**|**R1@0.5**|**R1@0.7**| **mIoU**|\n|QD-DETR$_s$(Fully)|No|-|60.66|38.60|52.53|-|43.76|25.98|42.86|\n|PC-Net(Ours)|No|70.87|54.84|26.68|47.12|52.78|29.62|14.35|36.45|\n|QD-DETR$_s$(Fully)|Yes|65.67|53.62|32.53|46.27|53.18|35.76|20.23|38.04|\n|CNM|Yes|43.86|29.46|14.02|29.49|45.86|25.23|13.43|33.01|\n|CPL|Yes|44.64|31.88|14.50|29.85|45.77|24.00|11.60|31.26|\n|CCR|Yes|59.14|45.06|21.38|39.28|44.65|26.95|10.58|33.92|\n|QMN|Yes|55.94|40.50|18.67|37.08|45.79|24.42|13.14|33.73|\n|PPS|Yes|59.82|43.02|20.93|39.31|48.36|26.13|13.24|31.71|\n|PC-Net(Ours)|Yes|**61.18**|**47.13**|**22.61**|**40.90**|**49.85**|**27.64**|**14.38**|**34.59**|\n\n- Thank you for your question. PPS slightly surpasses PC-Net in R1@m through boundary engineering. However, PC-Net has a significant lead in the mIoU metric, which measures the semantic completeness of proposals. Therefore, we use the term \"superior\". A detailed analysis is as follows.\n- While PPS achieves marginally higher R1@m on ActivityNet-CG, it relies on heavy temporal boundary engineering (redundant proposal enumeration and voting) to inflate recall at the cost of generalizability and efficiency. In contrast, PC-Net excels on the mIoU metric, which directly measures temporal overlap and reflects semantic completeness of retrieved moments. It outperforms PPS by +3.07% to +3.47% mIoU across all ActivityNet-CG splits, demonstrating stronger semantic integrity. Moreover, PC-Net achieves this with half the parameters of PPS while dominating long video benchmarks like TACoS and Ego4D (e.g., +0.43% R1@0.5 on Ego4D), demonstrating its overall superiority.\n- We will refine boundaries through query-guided optimization in future work. This will maximize semantic alignment while preserving efficiency and integrity, ensuring progress on all key metrics.\n- To test model robustness, we used Qwen3 for semantic query rewriting on Charades-CG and ActivityNet-CG datasets. Notably, after LLM-based rewriting of ActivityNet-CG queries, PPS's R1@m dropped sharply, demonstrating its over-reliance on boundary engineering rather than cross-modal semantic associations, which limits adaptability to varied queries.\n\n[1] Siamese learning with joint alignment and regression for weakly-supervised video paragraph grounding.\n\n[2] Query-aware multi-scale proposal network for weakly supervised temporal sentence grounding in videos.\n\n[3] Temporal sentence grounding with relevance feedback in videos.\n\n> 5. **Writing is dense, with long sentences and inconsistent tense; several typos (“quries”) hamper readability.**\n\n- We apologize for the inconvenience caused by this article. The official template is used to write this article. We will further reduce long sentences and fix potential typos to improve readability.\n\n## Questions\n\n> 1. **Please provide examples where PC-Net fails (e.g., long-range compositional queries) and analyse why.**\n\n- Thank you for your suggestion, we will add the corresponding visualization images in the final version. Here we take the prediction results of ActivityNet-CG after rewriting by Qwen3 as an example, such as the response to Weakness 4.\n\n```python\n# [vid, duration, GT, query, prediction]\n[\"v_Y_dtU10XIsg\", 66.06, [6.94, 42.61], \"A female individual is located outdoors, clearing snow with a shovel.\", [0.0, 60.48]]\n[\"v_NMBu3DIn1eg\", 202.29, [126.43, 146.66], \"The man keeps wiping the chair, making it progressively cleaner.\", [49.3, 202.29]]\n```\n\n- These examples shows that for actions with a long duration, the boundary accuracy is low. This is due to the failure to accurately model the association between the start and end of the action when generating proposals. In the future, we will further try more powerful feature extractors and fully model query correlation, and use it to dynamically adjust and improve the proposal boundary.\n\n> 2. **Why is the whole-video reconstruction loss an appropriate decision boundary?**\n\n- Consistent with previous studies (CPL, CCR, CNM), the full video reconstruction loss, as described in line 210, serves as a suitable decision boundary because it reflects moderate levels of query-correlation: it is weaker than perfectly aligned video moments in terms of semantic relevance, but stronger than completely unrelated video moments [4]. Leveraging this established property, we use the full video loss to split all proposals into high and low query relevance subsets for contrastive learning, capturing query co-occurrence related elements and suppressing potential spurious associations.\n\n[4] Weakly supervised temporal sentence grounding with gaussian-based contrastive proposal learning.\n\n> 3. **What hyper-parameter budget and feature extractors were used when re-running CPL, QMN, PPS, etc.?**\n\n- When reproducing the comparison models such as CPL, QMN, and PPS, we reproduced them completely based on their open source repositories and strictly adopted the hyperparameter settings specified in the corresponding methods to ensure the fairness of the experimental comparison. All weakly supervised methods (including the proposed PC-Net) use the same feature extractor. \n- In addition, the hyperparameter configuration of the proposed PC-Net is also detailed in the implementation details, and the specific hyperparameter settings are provided through the code in the appendix to ensure transparency and consistency. \n- Due to the limited upload capacity of the attachment, all the comparison method reproduction details and the remaining checkpoints will be made public after the paper is accepted."}]

Author rebuttal:

Meta-review: This paper receives positive scores. The reviewers raise some concerns regarding the limited analysis of weak vs. full supervision, the limited motivation for each component design, the complex model architecture, the marginal performance improvement, etc. Most of the questions are addressed in the rebuttal and recognized by reviewers. Eventually, this paper is recommended to be accepted. Authors should incorporate these review feedbacks, and polish the paper writing in the final version.
Decision: Accept (poster)