Learning Event Completeness for Weakly Supervised Video Anomaly Detection
Authors: Yu Wang
Shiwei Chen
Venue: ICML 2025 poster (2025)
Keywords: Weakly-supervised video anomaly detection
multi-modal video understanding
Primary area: applications->computer_vision
TL;DR: A novel dual-structed framework is proposed to learn event completeness for weak-supervised video anomaly detection

Abstract: Weakly supervised video anomaly detection (WS-VAD) is tasked with pinpointing temporal intervals containing anomalous events within untrimmed videos, utilizing only video-level annotations. However, a significant challenge arises due to the absence of dense frame-level annotations, often leading to incomplete localization in existing WS-VAD methods. To address this issue, we present a novel LEC-VAD, Learning Event Completeness for Weakly Supervised Video Anomaly Detection, which features a dual structure designed to encode both category-aware and category-agnostic semantics between vision and language. Within LEC-VAD, we devise semantic regularities that leverage an anomaly-aware Gaussian mixture to learn precise event boundaries, thereby yielding more complete event instances. Besides, we develop a novel memory bank-based prototype learning mechanism to enrich concise text descriptions associated with anomaly-event categories. This innovation bolsters the text's expressiveness, which is crucial for advancing WS-VAD. Our LEC-VAD demonstrates remarkable advancements over the current state-of-the-art methods on two benchmark datasets XD-Violence and UCF-Crime.

Learning Event Completeness for Weakly Supervised Video Anomaly Detection
Yu Wang' Shiwei Chen?
Abstract
of anomaly events in arbitrarily long and untrimmed videos. However, major challenge faced by VAD is its reliance on costly and dense annotations that precisely mark the start and end of each anomaly event. To mitigate this depen- dency, weakly supervised VAD (WS-VAD) is introduced. This paradigm facilitates training with only video-level an- notations, thereby streamlining the annotation process and enhancing the feasibility of VAD in practical applications.
Weakly supervised video anomaly detection (WS-VAD) is tasked with pinpointing tempo- ral intervals containing anomalous events within untrimmed videos, utilizing only video-level an- notations. However, a significant challenge arises due to the absence of dense frame-level annotations, often leading to incomplete local- ization existing WSD methods. To ad- dress this issue, we present a novel LEC-VAD, Learning Event Completeness for Weakly Su- pervised Video Anomaly Detection, which fea- tures dual structure designed to encode both category-aware and category-agnostic semantics between vision and language. Within LEC-VAD. we devise semantic regularities that leverage an anomaly-aware Gaussian mixture learn precise event boundaries, thereby yielding more complete event instances. Besides, we develop novel memory bank-based prototype learning mecha- nism to enrich concise text descriptions associ- ated with anomaly-event categories. This innova- tion bolsters the text's expressiveness, which is crucial for advancing WS- VAD. Our LEC- VAD demonstrates remarkable advancements over the current daate-of-the-att methods on two bench- mark datasets XD- VD-Vieeee and UCF-Crime.
The majority of contemporary methods (Wu et al., 2024b; Liu et al., 2024; Wu et al., 2024d; Zhou et al., 2023; Cho et al., 2023b; Luo et al., 2021; Ramachandra et al., 2020; Wang et al., 2023a; Feng et al., 2021) for WS-VAN VA-V adhere to a systematic pipeline. Initially, the process entails extract- ing frame-level features by leveraging pre-trained models (Radford et al., 2021; Jia al., 2021). Following this, these extracted features tegrated with a multiple instance learning (MIL) strategy (Paul et al., 20.8). Ultimately, the identification of abnor- mal events is achieved through the analysis of the predicted anomaly confidences. Despite achieving promising results, such classification paradigm assigns each video frame to zero more anomaly categories. During inference, it relies manually designed posteprocessing steps aggregate these frame-level predictions into consecutive anomaly snip- pets with explicit boundaries. However, this paradigm often results in incomplete and fragmented anomaly segments, as shown in Figure. 1.
input into binary classifiers and in-
To address the challenge of incomplete anomaly event detection in such paradigm, we propose LEC-VAD, Learning Event Completeness for Weakly Supervised Video Anomaly Detection, including two novel mechanisms: mem- ory bank- baakd prototype learning strategy and Gaussian mixture prior-based local consistency learning. For the for- mer, we resort to textual descriptions of anomaly-category and devise dual structure to encode both coarse -aause-graiedd and fine- aoe-grained semantics between vision and language. In this procedure, textual expressions of anomaly categories are typically concise and limited in expression, novative memory bank-based prototype learning strategy is developed to strengthen the text's entropy. Besides, we also dynamically enhance text representations by integrat- ing learnable text-condition visual prompts. For the latter, since the model trained perform frame-wise prediction, it lacks explicit understanding of anomaly event bound-
1. Introduction
In recent years, vidco anomaly detection (VAD) has gar- nered increasing interest due to its widespread applications (Carreira & Zisserman, 2017; Dosovitskiy & al, 2021; Radford et al., 2021), such intelligent surveillance (Sul- tani et al., 2018; Wang et al., 2024), multimedia content review (Zhu et al., 2023), and intelligent manufacturing (Gupta et al., 2024). VAD aims to predict temporal intervals
in-
'School of Computer Science and Technology. Tongji Uni- versity. Shanghai. China. 'Department of R&D Data, Microsoft Asia-Pacific Technology CO Ltd, Shanghai, China.. Correspon- dence to: Yu Wang <yuwangtj @yeah.net>.
Proceedings of the 4 International Conference on Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Learning Event Completeness for Weakly Supervised Video Anomaly Detection
language and vision. The mainstream approaches can be grouped into two categories, Le., joint encoder and dual en- coder. Methods with a joint encoder (Li et al., 2021; Lu et al., 2019; Zhang et al., 2021) employ a multi-modal encoder to facilitate fine-grained interactions between vision and lan- guage. However, despite their promising performance, these methods suffer from notable drawback in that they neces- sitate processing each text-image pair individually during inference, leading significant inefficiencies. In contrast, approaches employing the dual-encoder structure (Radford et al., 2021; Jia et al., 2021) utilize two separate encoders to extract visual and linguistic features. Recently, signifi- cant advances have been attributed to large-scale contrastive pre-training within this paradigm, which has dramatically enhanced the performance of numerous multi-modal tasks, including text-image retrieval (Wang et al., 2023b; Jiang & Ye, 2023), visual question answering (Sima al., 2025; Lin et al., 2023b), and video grounding (Lin et al., 2023a; Xiao et al., 2024). In this paper, we embrace CLIP (Radford et al., 2021) and transfer it for video anomaly detection.
Confideite Confilerce Confulance
Ground Truth
RP
LEC-VAD (Ours)
Figure I: Yellow denotes ground truth anomaly intervals, Blue denotes the prediction intervals, and the black lines depict the predicted confidences. We observe that ex- isting methods, exemplified by ReFLIP, typically suffer from the fragmentation of intervals, ultimately yielding in- complete anomaly instances. Instead, our LEC-VAD pro- duces smoother scoring patterns, leading to more complete anomaly intervals.
aries, thus resulting in an objective discrepancy between the classification based training and localization based infer- ence. To bridge this gap, we hypothesize that prediction re- sults should exhibit local consistency. To explicitly enforce this prior, we refine anomaly scores with learnable Gaussian mixture masks. This enables anomaly scores to incorpo- rate contextual information from nearby frames, thereby improving the smoothness of predicted anomaly snippets.
2.2. Weakly Supervised Video Anomaly Detection
Weakly Supervised video anomaly detection (WS-VAD) has attracted considerable attention in recent years due to its broad applicability (Wang et al., 2025; 2024; Zhu et al., 2023; Gupta et al., 2024; Sultani et al., 2018) and manage- able computational requirements. Sultani et al. (Sultani et al., 2018) are pioneers in this field, collecting large- scale dataset for video anomaly detection annotated at the video level and employing a multiple instance ranking strat- egy to pinpoint anomalous events. Subsequent research endeavors have focused on various aspects of enhancing WS- VAD performance. One line of research (Zhong et al., 2019; Tian et al., 2021; Zhong et al., 2024; Li al., 2022: Huang et al., 2022; Zhou et al., 2023) has explored cap- turing the temporal relationships among video segments. This has been achieved through the use of graph structures (Zhong et al., 2019), self-attention strategies (Tian et al., 2021; Zhou et al., 2023), and transformers (Lv et al., 2023; Huang et al., 2022; Li et al., 2022). These methods provide a deeper understanding of how different parts of video interact, which is crucial for accurately detecting anoma- lies. Another paradigm has been to investigate self-training schemes (Yang et al., 2024; Rai et al., 2024; Zhang et al., 2023; Feng et al., 2021). These schemes generate snippet- level pseudo-labels, which are then used to iteratively refine the anomaly scores. This process helps in improving the accuracy of anomaly detection by providing more granular feedback. Besides, Sapkota et al. (Sapkota & Yu, 2022) and Zaheer et al. (Zaheer et al., 2024) seek solutions to alleviate the impact of false positives. Recently, there has been surge in efforts to leverage multi-modal knowledge to boost WS-VAD performance. For example, PEL4VAD
The main contributions of this paper are five-fold: 1) We propose LEC-VAD, a novel method learn event com- pleteness for video anomaly detection with only video-level annotations. 2) A dual structure is devised to mine both category-aware and category-agnostic semantics between vision and language. 3) Since text descriptions of anomaly categories are typically concise and limited in expression, we develop a memory bank-based prototype learning mech- anism to enrich textual representations. 4) Based on our hypothesis that prediction results should exhibit local consis- tency, we propose prior-driven event completeness learn- ing paradigm to refine anomaly scores equipped with leam- able Gaussian mixture masks. 5) Extensive evaluations on the XD-Violence and UCF-Crime datasets have shown that our LEC- VAD achieves state-of -he--artt performance. No- tably, it demonstrates a significant advantage over existing methods for finer-grained anomaly-event detection, outper- forming them by substantial margin.
2. Related Work
2. Vision-Language Pre-training
Cross-modal lasion-language understanding (Jian et al., 2024; Wu et al., 2023; Wei et al., 2024) is a fundamental task that necessitates precise representation alignment between
2

---

Learning Event Completeness for Weakly Supervised Video Anomaly Detection
GMP-based Local
Linear Layer
Consistency Learning
e
Constraint
Video Features
Text Features
Video Features
Category-agnostic Classifier
CLIP Image Encoder
Tranffrmer
Top-K
Global GCN
Category-aware
Classifier
Vision-only Anomaly-aware Branch
The vidco clip contains!
Self- Aten-tnt
M
The video clip contains la fighting event.
CLIP Encoder
MIL
x
'The vidco clip contains! bunglary event.
The video clip contains lan explosion event.
Memory Bank
Text Features
Protolype-based Attention Block
Cross-modal Anomaly-aware Branch
Update
Figure 2: Overview of the proposed LEC-VAD, We develop a dual-structured framework to perceive both category-agnostic and vategory-award anomaly clues. We incorporate textual expressions of anomaly categories into WS-VAD, and develop novel memory bank-based prototype learning mechanism to enrich concise textual descriptions associated with anomaly- event categories. Besides, Gaussian mixture prior- based semantic regularities are devised to learn complete event boundaries.
(Pu et al., 2024), VadCLIP (Wu al., 2024d), and ITC (Liu al., 2024) inject text clues of anomaly-event categories for WS-VAD, while PE-MIL (Chen et al., 2024) and MACIL- SD (Yu et al., 2022) introduce audio elements to enhance their capabilities. These innovations have yielded promising results. Wu al. (Wu et al., 2024a) have gone further by developing framework that can retrieve video anomalies based on language queries or synchronous audio. They (Wu al., 2024b) have also proposed open-vocabulary video anomaly detection paradigm to identify both known and novel anomalies in open-world settings. Besides, the use of pre-trained vision-language models, i.e., CLIP (Radford et al., 2021), has also emerged as powerful tool for extract- ing robust representations (Zhu Pang. 2024; Wu al., 2024d; Dev et al., 2024; Yang al., 2024). For example, Yang et al. (Yang et al., 2024) utilize pre-trained CLIP to generate reliable pscudo-labels Lv et al. (Lv et al., 2023) devise an unbiased multiple-instance strategy to learn in- variant representations. STPrompt (Wu et al., 2024c) learns spatio-temporal prompt embeddings based on pre- trained
vision-language models.
Despite significant advances, these existing works fail to harness the principle that predictions should exhibit local consistency. In contrast, our proposed LEC-VAL learns Gaussian mixture masks alongside multi-granullan seman- ties to ensure the completeness of anomaly event detection.
3. Methodology
In this section, we first introduce the proposed dual- structured framework LEC-VAD. Then, we elaborate on the proposed memory bank- based prototype learning mech- anism and Gaussian mixture prior-based local consistency learning strategy in Section 3.2 and Section 3.3 respectively.
3.1. Overview
Problem Definition. Given dataset consisting videos
V (v; ) with different annotation granularities, each video v; is annotated with coarse-grained and fine-grained
3

---

Learning Event Completeness for Weakly Supervised Vidco Anomaly Detection
video-level labels y/ E {0, and di E {O..., C} respec- tively, where C' is the number of anomaly categories. If oy is flagged as normal, y/ 0 and I = 0, otherwise 9: = 1 and 9: is assigned a specific category. Coarse- grained WS- needs to predict the likelihood of each snippet containing an anomaly. Fine-grained WS-VAD de- mands the prediction of specific anomaly event instances, denoted as {s,. cy. I5, wy}. Here, 8; and ej denote the start and end timestamps of the j-th anomaly instance, 9j repre- sents the anomaly category, and wy signifies the prediction confidence for the j-th instance.
category (belongs to (0, 13) category, follows:
6 Pe
1 K
Σ JEH
max MCO 2
Se
(2)
where H denotes the index set of K snippets, and pe is the category-agnostic predictions. Then, a binary cross-entropy loss is imposed on pe and ground truth y:
Lagnostic
Σ Ye log(p
(3)
nE (0.1)
This paper proposes a novel dual-structured framework, LEC-VAD, for VSD, as illustrated in Fig- ure. 2. The core idea to learn anomaly event complete- ness in the absence of dense annotations. To accomplish this goal, we endeavor to capture both category-aware and category-agnoste anomaly knowledge through dual struc- ture, avoiding subtle clues being overwhelmed by salient features. This allows our model to grasp a comprehen- sive understanding of event concepts. Besides, to ensure local consistency of predictions, learnable Gaussian mix- ture masks are developed to improve the smoothness of predicted anomaly snippets. Specifically, we initially em- ploy pre-trained image encoders to extract visual features Xvideo E R'xx where T denotes the number of snippets and d is the feature dimension. As in (Wu et al., 2024d), in order to capture local temporal dependencies, we divide the frame-level features into overlapping windows of equal length. Subsequently, we apply a local transformer layer with constrained attention scopes (Liu al., 2021), ensur- ing no information exchange between these windows. This process results in the enhanced feature X₁ E Rxx Then, to further capture global temporal dependencies, we apply a GCN module on X,- Specifically, following the setup in (Wu et al., 2024d), global temporal associations of features are modeled based on their similarity. This process can be summarized as follows:
Framework.
Furthermore, although 5T reflects category-awars anomaly information, it lacks associations with textual descriptions. To this end, we model cross-modal interactions through a cross-attention operation. Here the text feature Freat serves as the query, while the video feature F Fvideo acts as the key and value, resulting in an enhanced representation Fiv RR. (C+1)xd The computation is outlined as follows:
Q = -Fiext Wg. K = FFeidoo WK, V = Frideo W
Q KTT
(4)
F.. stmaa
-.V, Vd
where W q, We. and W. are learnable projection matrices in R Furthermore, given the concise and limited ex- pressive nature of textual descriptions for anomaly-event categories, we develop a memory bank- based mechanism to store category prototypes. These prototypes are leveraged to acquire enhanced text descriptions Faug. The ultimate text representation, Fatg: is fusion of Fiv and Faug. for- mulated as Fetg Fiv + Faug. Furthermore. we utilize an external memory bank to strengthen text expressiveness, acquiring an enhanced feature Faug. After that, we perform aware detection through a dot product of Fetg and video features Fuideos which presented as follows:
5 norm(Fouded) norm FT Fdg)
(5)
F Frideo CELU(Softman (Softmar(AAX)W))
X,XI
where norm denotes the & normalization. Finally, we obtain the category-aware anomaly detection logit by inte- grating sh" and s" formulated as awaare s + s*" This logit is then utilized via the MIL principle to yield video- level classification outcomes. More precisely, for the c-th anomaly category, the logit p is calculated as follows:
A
| 1Xll22-11X1112
where A is the adjacency matrix, the Softmax operation is used to ensure that the sum of each row equals one, W is the learnable weight, and Foideo is the enhanced video feature.
Besides, pre-trained text encoder is adopted to extract linguistic features of all anomaly-event categories Fiess E R(C+1)xd Then, we apply a fully-connected layer, fol- lowed by softmax operation, on Foideo to perform bi- nary classification for category-agnostic anomaly detection and multivariate classification for category-aware anomaly detection, yielding prediction results 5 E Rx2 and ST E R TxC For s², we compute the average of the top-K scores over the temporal dimension, specifically for the e-th
P
max #C (.r..T) K
1
Σ JEH
aware
je
(6)
[HIK
Subsequently, a cross-entropy constraint is imposed to su- pervise the process of fine-grained anomaly classification:
Lamare
Σ go log(p"")
(7)
cm0

---

Learning Event Completeness for Weakly Supervised Video Anomaly Detection
3.3. Gaussian Mixture Prior-hased (GMP-based) Local Consistency Learning
To learn complete anomaly events and improve the smooth- ness of snippets, we hypothesize that prediction scores should be locally consistent. Based on this hypothesis, we propose Gaussian mixture prior-based local consis- tency learning mechanism, encoding multiple anomaly cate- gories GMM components, thereby generating anomaly constraint scores 8gmm E R².. Sgmm is then adopted to regularize the predicted anomaly scores s as follows:
This paper proposes the hypothesis that prediction scores ought to exhibit local consistency. To explicitly enforce this prior and learn complete instances, we model anomaly scores with learnable Gaussian Mixture Models (GMMs) for each temporal position t, where each component of GMM is tailored to encode category-specift anomaly mask. In par- ticular, we first concatenate Faug with the visual feature of each snippet Feideo by broadcast mechanism in Python get integrated multi-modal representation I'm E RTxCxd Based on Fm. we utilize shared fully- fully-connected layer to predict Gaussian kernels fo of Gaussian masks for each category. Finally, we generate anomaly scores $qause (t) at t-th temporal position from the t-th Gaussian mixture mask. The detailed procedure is as follows:
Cymm
pmmm
[[] (1),
(8)
Besides, to guarantee the consistency of the category- agnostic anomaly score, i.e., 1 - 8 87" 0), and category- aware anomaly score, i.e. st[t, 1]. for each snippet, we introduce I regularization loss, formulated as follows:
c
G² G'= a (t)),, where a = s [t, c).
c=0
Creg
1 1 T
-
L1
S [t,
[t,,, 1|I.
G¹ = expl B(/TT - -111)2
)
(12)
(ot)
8 9gmm! (1) = (G
Overall, the optimization loss of the proposed LEC-VAD is summarized as follows:
where of indicates the probability each abnormal event occurring, assigned s [t. c]. controls the variance of the Gaussian mask G In this manner, these masks exhibit local smoothness, rendering them suitable for constraining anomaly scores S² as defined in Eq. 8.
Call = Lagnostic + Loware + Xmmmm + Cregs
(10)
where \ and yy are weights that balance different terms.
3.4. Inference
3.2. Memory Bank-based Prototype Learning
During inference, for the coarse-grained WS-VAD, cal- culate the average value of 1 1- """(t [t. 0] and s [t, 1] as the I-th frame's 's anomaly confidence. For fine- fine-grained WS-VAD, we adopt a two-step thresholding strategy to generate anomaly instances. Specifically, we first retain these anomaly cate- gories with video-level activations exceeding a predefined threshold Tela- Then, for each retained anomaly category, select snippets with fine-grained matching scores in s exceeding the threshold l'nno as candidates. These tempo- rally consecutive candidates are merged to form anomaly instances. Following AutoLoc (Shou et al., 2018), the outer- inner score of each instance based on s is regarded as the confidence score of each instance. Based on these confi- dence scores, we apply a non-maximal suppression (NMS) to avoid redundant proposals.
Since text descriptions of anomaly-event categories are gen- erally concise and expressive-limited we construct an ternal memory bank to retain group of textual prototypes, denoted as MM (C+1)xd These prototypes are sub- sequently leveraged to strengthen textual representations. In particular, we initialize M with the original CLIP fea- tures corresponding to the anomaly-even categories. Dur- ing training, the enhanced text features Fto are concate- nated with the contents of M to produce context-aware features Fate (2C+2)x Subsequently, apply m self- attention blocks on Fate to absorb semantic knowledge from prototype features M, thereby acquiring the augmented rep- resentation Frre E (22C++))xd For F'te. we extract the first C + representations along the first dimension, de- noted as Fang E (T+x)) as the refined text representa- tions. Meanwhile, prototype features in M are updated momentum-based fashion, specifically as follows:
4. Experiments
4.1. Experimental Settings
M = 1 x M + -1) x Faug:
(11)
Datasets.
UCF-Crime (Sultani et al., 2018) consists of 1900 untrimmed videos, spanning across 13 categories of anomalous events. These videos are collected from surveil-
where 7) denotes the momentum coefficient for an update.
5

---

Learning Event Completeness for Weakly Supervised Vidco Anomaly Detection
mAP6
Supervision
Modality
Mothods
Feature
APr
Methods
0.1
0.2
0.3
0.4
0.5
AVG
Unspervised
RGB+-Audi
LTR (Hasam al,, 2016)
BGD+
30.77
RAD (Sultani et al., 2018)
5.73
4.41
2.69
1.93
1.44
3.24
CTRED (Wu & Lau, 2021)
1SD4VOGish
75.90
AVVD (Wu et al., 2022)
10:27
7.01
6.25
3.42
3.29
6.05
WS-AVVD
et 2022)
BD+VGGish
RGBio
DMU (Zhon al., 2023)
BCU (Zhing et al., 2023)
RDaVGGish
BD+VOGish
K1.43
81.77
ITC (Liu et al., 2024)
VadCLIP (Wu et al., 2024d)
13:54
11.72
7.83
9.24
6.40
7.45
4.53
5.46
2.93
3.79
7.90
6.68
MACIL-SD (Na sl., 2022)
BD+VGGish
83,40
ReFLIP (Dev et al., 2024)
14.23
10.34
9.32
7.54
6.81
9.62
PE-MIL et al. 3024)
ISD+ VGGish
88.21
LEC VAD (Ours)
19.65
17.17
14.37
9.45
7.18
13.56
(Sahinn
2018]
73.20
RIFML (Tian ot al., 2021)
BD
77.81
ST-MSL (1. al., 2022)
78.28
Table 4: Fine- Fine-grained comparisons on UCF-Crime.
LA-Net & Wu, 2022)
BDD
80.72
Weakly Supervised
CoMo (Cho al. 2023s)
E3D
81,30
DMU (Zhou et 2023)
$2.41
PEL/VAD (Pu et al., 202())
BDD
85,59
at 2024)
RGR
TRC- (Curs)
B3D
88.47
OV at. 202/1b
CLIP
66.33
CLIP-TSA (Jo0 al. 2023)
CLJP
82.19
IPS- (2hong et al. 2024)
CLIP
83.14
TPWNG (Yang 2024)
VadCLIP (Wu et ol., 20244)
CLIP
$4.51
ITC (Liu et 2024)
CLIP
85.45
ReFLIP (Dev et al., 2024))
CLIP
85.81
LEEC VAD (Ours)
CLIP
86.56
Table 1: Coarse- Coarse-grained grained comparisons on XD-ViD-iee Violence.
(a) CLIP
(b) LEC-VAD
Supervision
Methods
Feature
AUC(%)
Figure 3: The representation visualization for 7 categories in XD-Violence testing set by using the t-SNE method (Van der Maaten & Hinton, 2008).
LTR (Hasan et 2016)
13D+VGGish
50.60
Unsupervised
GODS (Wang& & Cherian, 2019)
T3D
70.46
GCL (Zahcer al., 2022)
B3D
71.04
TCN-CIBL (Zhang al., 2019)
C3D
78.66
GCN GCN-Anomaly (Zhong et al., 2019)
C3D
81.08
GLAWS (Zaheer al.,
83.03
LEC-VAD (Ours)
C3D
84,75
lance footage captured in various indoor settings and street scenarios. We adhere to standard data split, where the training set and testing set comprise 1610 and 290 videos, respectively. XD-Violence (Wu et al., 2020) is larger-scale benchmark comprising 4754 untrimmed videos sourced from movies and YouTube, encompassing violence event categories. 3954 videos and 800 videos are employed for training and testing respectively.
RAD (Sultani et al., 2018)
13D
77.92
WS-AVVD (Wo et 2022)
B3D
82.44
RTFML. (Tian et al., 2021)
130
84,30
CTRFD (Wa & Liu, 2021)
B3D
84.89
LA-Net (Pu & Wo, 2022)
13D
85.12
ST-MSL (Li et al., 2022)
13
85.30
IFD- (Zhong et 2024)
85.47
NG-MIL (Park al. 2023)
13D
85,63
CLAV (Cho et al., 20233)
BD
86.10
Weakly Supervised
BCU (Zhang ct 2023)
86.22
DMU (Zhou nl., 2023)
B3D
86.75
PE-MIL (Chen et al., 2024)
13D
86.83
MGFN (Chen
2023)
BD
86.98
Evaluation Protocol. For coarse-grained WS- VAD, we adopt frame-level Average Prevision (AP) for Vi-Venlence and frame-level AUC for UCF-Crime. For fine- grainnd WS-VAD, we follow previous works (Wu et al., 2024d; Liu et al., 2024) and compute the mean Average Precision (mAP) across IoU thresholds from 0.1 to 0.5 in increments of 0.1. Besides, average of mAP (AVG) is also reported for more comprehensive evaluation.
LEC/VAD (Ouns)
130
88,21
er (Ju et al., 2022)
CLIP
84.72
OVVAD (Wu et al., 2024b)
CLIP
86.40
IFS-VAD (Zhong ct 2024)
CLIP
86.37
UM.I. al., 2023)
CLIP
86.25
CLIP-TSA (Joo 2023)
CLIP
87,58
TPWNG (Yang et al., 2024)
CL.IP
87.79
VadCLIP (Wa et 2024d)
STProompt (Wu et 2024:)
CLJIP
88.08
ReFLIP (Dev et al., 2024)
CLIP 88.57
ITC (Liu al., 2024)
CLIP 89.04
LEC-VAD (Ouns
CLIP 89.97
Implementation Details. The pre-trained text encoder of CLIP (VIT-B/16) is adopted and multiple vision encoders including 13D (Carreira & Zisserman, 2017), C3D (Tran et al., 2015), and the CLIP (ViT-B/16) are explored to extract frame features. The value of K is determined as K max (LT/166) (16],,),, 1), and the momentum coefficient 1) is to 0.99. We adopt the AdamW optimizer and train our LEC- VAD with a batch size of 64. The learning rate is set to 3e-5 and the model is trained for 10 epochs. We apply NMS with an IoU threshold of 0.5, and set the threshold Fels- and Tano 3, X, 7, and m are explored in the experimental sections (Figure 5).
Table 2: Coarse-graines comparisons on UCF-Crime
mAP@
Methods
0.1
0.2
0.3
0.4
0.5
AVG
RAD (Sultani et 2018)
22.72
15.57
9.98
6.20
3.78
11.65
AVVD (Wi et al,, 2022)
30.51
25.75
20.18
14.83
9.79
20.21
VadCLIP (Wuce al., 2024d)
37.03
30.84
23.38
17.90
14.31
24.70
ITC 2024)
40.83
32.80
25.42
19.65
1.5.77
26.83
RI (Dev 2024)
39.24
33.45
27.71
20.
86
17.22
27.36
LEC-VAD (Ours)
48.78
40.94
34.28
28.02
23.68
35.14
to 0.1 and 0.2. The
Table 3: Fine- fine-grained comparisons XD- Violence
6

---

ble 4, our method consistently exhibits remarkable improve- ments across all evaluation metries, achieving an amazing Learning Event Completeness for Weakly Supervised Vidco Anomaly Detection
mAP@IoU
mAP@IoU
VOB
CMB
VAP
PMB
0.1
0.2
0.3
0.4
0.5
AVG
0.1
0.2
0.3
0.4
0.5
AVG
42.86
33.82
30.58
24.66
18.92
30.17
42.17
34.66
27.98
22.47
17.24
28.90
46.03
38.87
32.18
27.02
22.04
33.23
46.24
38.75
31.97
26.63
20.05
32.73
48.78
40.94
34.28
28.02
22.68
34.94
46.01
38.24
31.80
26.56
20.04
32.53
Table 5: Explorations of the model structure. "VOB" de- notes the vision-only anomaly-aware branch, and "CMB" denotes the cross- moos- anomaly-aware branch.
48.78
40.94
34.28
28.02
22.68
34.94
Table 7:
Ablation studies of text enhancement. "VAP" de- notes the vision-aware prompt strategy and "PMB" denotes the prototype-based memory bank mechanism.
mAP@IoU
Crey Cymmn
0.1
0.2
0.3
0.4
0.5
AVG
44,59
36.34
28.97
22.83
17.45
30.04
47.04
38.52
31.23
25.38
20.26
32.49
46.69
39.51
32.17
27.16
22.01
33.51
AVG of 13.56. This represents a substantial improvement of
48.78
40.94
34.28
28.02
22.68
34.94
40.96% over the ReFLIP- VAD method. Notably, our LEC- VAD brings more gains when using more stringent evalu- ation criteria (larger IoU). For instance, on XD- XD-Violence, LEC-VAD gets a 37.51% 1% relative boost when IoU is 0.5 and a 19.47% relative boost when IoU is 0.1. This reveals that LEC-VAD can learn more complete instances. A similar phenomenon occurs in UCF-Crime. These observations re- veal the superiority of our proposed algorithm in discerning subtle distinctions among diverse anomaly events.
Table 6: Ablation studies of loss function.
4.2. Main Results
We undertake a comprehensive evaluation of LEC-VAD's anomaly detection capabilities by benchmarking it against widely-used approaches across two datasets. The resultant findings underscore that our LEC- VAD achieves state-of- the-art performance across all evaluation metrics and at various granularity levels. We will elaborate on it below.
To further investigate our model's capability in learning discriminative visual representations for different anomaly categories, we visualize the feature distributions extracted by the original CLIP and our model on the XD- Violence test set, as shown in Figure 3. From the visualization, we observe that the distribution of original CLIP features ex- hibits significant overlap and confusion among different categories. In contrast, our model's 's extracted features for each category form more concentrated clusters with well- defined boundaries. This reveals that our model possesses the capability to learn discriminative features despite the absence of explicit snippet-level supervision. This property is advantageous for our LEC-VAD, enabling it to detect anomaly events with greater completeness and confidence.
First, we assess coarse- grained anomaly detection on XD- Violence and UCF-Crime, with the respective results pre- sented in Table 1 and Table 2. For XD-Violence, we employ 13D and CLIP features of videos, achieving tfate-of-the-ar performance. Specifically, our method achieves an impres- sive 88.47 AP using I3D features, marking a 0.42 absolute gain compared to PB-MIL. Similarly, utilizing CLIP features results in 86.56 AP, which acquires a 0.75 absolute gain over ReFLIP. Notably, our approach even surpasses methods that incorporate additional audio features. For UCF--rime we adopt C3D, 13D, and CLIP features of videos, all achiev- ing remarkable results. Specifically, our method achieves an impressive 84.75 AP when employing C3D features, making significant 1.72 absolute improvement compared to CLAWS. Similarly, our method demonstrated 88.21 AP with I3D features and even higher 89.97 AP with CLIP features, translating to 1.23 and 0.93 absolute gains re- spectively, compared to the best practices. These presented results demonstrate our model's 's superiority and robustness when utilizing diverse visual features.
4.3. Ablation Study
We conduct thorough ablation studies XD-Violence and systematically report fine-grained anomaly detection results across different evaluation metrics to dissect the contribu- tions of diverse factors to the overall performance.
Model components. In Table 5, we conduct an in-depth analysis to examine the impact of various model compo- nents on anomaly detection performance. Specifically, we investigate the utilities of the vision-only anomaly-aware branch (VOB) and the cross-modal anomaly-aware branch (CMB), aiming to gain insights into how these components contribute to the overall performance of our model. Over- all, the fully intact model demonstrates superiority over its castrated counterparts across all evaluated metrics. This ob-
Furthermore, our method manifests more powerful advan- tages for fine-grained anomaly detection. As shown in Table 3. our method exhibits significant improvements across all evaluation metrics on the XD- Vi-violen,,, culminating in an outstanding AVG of 35.14 This represents a considerable improvement of 28.44% over the ReFLIP- VAD method. Also, on the more challenging UCF-Crime, as shown in Ta-
7

---

Learning Event Completeness for Weakly Supervised Video Anomaly Detection
Our wife GMP
Ground Truth
ReFLIP
500 2004
2000
150
LEC-VAD (Our))
Our w/ GMP
Cesecten
423
300
1497
(4) Sheoting
Fighting
Figure 4: Visualization about the utility of our GMP.
Ground Truth
ReFLIP
LEC-VAD (Ours)
Visualization of coarse-graie -oauee- anomaly detection results compared with ReFLIP.
Figure 6:
(a) The hyper- parameter 0
(b) The hyper- parameter X.
differences in outcomes when incorporating Gaussian-based scores versus not using them. From the results, we observe that employing GMP yields more comprehensive and sound outcomes.
Text enhancement. This paper introduces textual descrip- tions of anomaly categories to facilitate cross- cross-modal seman- tic interactions. Consequently, we explore the proposed en- hancement mechanism for concise text descriptions, as illus- trated in Table 7. We observe that employing the developed vision-aware prompt strategy (VAP) and prototype-bar based memory bank mechanism significantly improves anomaly detection performance compared the variants lacking these components. The integration of these two mechanisms further obtains impressive AVG of 34.94, achieving an improvement of 20.90% over the variant without them.
(c) The hyper-parameter: 7
(d) The hyper-parameter m
Figure 5: Effects of hyper-parameters.
servation indicates that both VOB and CMB are beneficial WS-VAD. Notably, CMB brings more gains compared to VOB, with AVG of 33.23 versus 30.17. This reflects the importance of cross-modal interactions for WS-VAD.
Loss functions. In Table 6, we delve into the impact of Gaussian mixture prior and score regularization by carefully controlling the loss terms Greg and Lymm within the loss function. Undoubtedly, the overall loss achieves optimal performance compared to the castrated counterparts. Be- sides, observe that apply more rigorous constraint evaluation criteria (characterized by larger loU values), us- ing Cymm can bring more gains. This phenomenon suggests that introducing the Gaussian mixture prior can help detect more complete abnormal events. We also provide some visualized comparisons in Figure 4, which illustrates the
Hyper- prrper-prraeters. We conduct an extensive exploration of the hyrrr-parrmeter configuration for 3, X, 7, and m, as illustrated in Figure 5. For B, we incrementally adjust it from 0.1 to 0.9 increments of 0.2, and the optimal is obtained / 0.7, demonstrating that relatively larger variances of Gaussian masks are beneficial to guarantee local consistency. The dlype--paaamteen À and are metic- ulously tuned to ascertain their optimal values, ultimately determining À = 0.3 and ạ =le-4 as the most effective configuration. Besides, our investigation into the number of
8

---

Learning Event Completeness for Weakly Supervised Video Anomaly Detection
anomaly event displays coarse-grained detection results the bottom. We observe that ReFLIP incorrectly identifies segments featuring people (such pedestrians) as clues to anomaly events in this example, failing to discern be- havioral differences. In contrast, our LEC- VAD accurately focuses on the behavioral characteristics associated with an "arrest", demonstrating its sensitivity to specific contextual and behavioral clues.
Drtettee Sorre
Shooting
Accident
Furthermore, Figure presents some fine- fineggriined visual- ization results to investigate the inter-relationships between predictive scores across multiple anomaly classes. Instead of a blended outcome from multiple categories, we observed that the true anomaly class has been accurately identified.
Ditecton
5. Conclusion
This paper proposed novel LEC-VAD for WS-VAD, gineered to encode category-awan and category-agnoste semantics of anomaly events. We hypothesize local con- sistency in predictions and develop a prior-driven learning mechanism with learnable Gaussian masks. Besides, a mem- ory bank-based prototype learning mechanism was proposed to enrich textual features. Overall, LEC-VAD achieved re- markable advances in XD-Violence and UCF-Crime.
Explasion
lighting
Impact Statement
180 198
2.00
1080 13:00
20.0 J55
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Normal
Figure 7: Visualization fine-grained predictions.
prototype-basee based attention blocks m, revealed that m is sufficient for achieving optimal performance. Increasing m observed to potentially induce overfitting.
A Acknowledgements
This work was supported in part by the National Natural Science Foundation of China under Grant 62406226, in part sponsored by Shanghai Sailing Program under Grant 24YF2748700, and in part sponsored by Tongji Univer- sity Independent Original Cultivation Project under Grant 22120240326.
4.4. Qualitative Results
To further demonstrate the superiority the proposed LEC- VAD for WS- S-D, we have visualized several examples in Figure 6 and Figure 7. Figure 6 compares the anomaly de- tection outcomes of our LEC-VAD with the state-of-the-art ReFLIP method. In the upper section of the figure, an ex- ample depicting the "cxplosion" anomaly event showcases the coarse-grained detection results of both methods. Upon examination, becomes evident that our LEC-VAD is capa- ble of detecting more complete anomaly events with greater confidence than the highly competitive ReFLIP method. It appears that ReFLIP primarily relies the visual pres- ence of fire to judge explosion anomalies in this scenario, whereas our method possesses deeper understanding of the semantic content and context associated with explosions. Consequently, our LEC-VAD can detect more complete instances. An additional example depicting the "arrest"
References
Carreira, J. and Zisserman, A. Quo vadis, action recogni- tion? new model and the kinetics dataset. In Proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6299-6308, 2017.
Chen, J., L., Su, I.., Zha, Z.-j., and Huang, Q. Prompt- enhanced multiple instance learning for weakly super- vised video anomaly detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recog- nition, pp. 18319-18329, 2024.

---

Learning Event Completeness for Weakly Supervised Video Anomaly Detection
Chen, Y., Liu, Z., Zhang, B., Fok, W., Qi X., and Wu, Y--C. Mgfn: Magnituue glance-and-foeus network for weakly-supervised video anomaly detection. In Pro- ceedings of the AAAI conference on artificial intelligence, pp. 387-395, 2023.
Jian, Y., Gao, C., and Vosoughi, S. Bootstrapping vision- language learning with decoupled language pre-training. In Proceedings of the Neural Information Processing Systems, pp. 57-72, 2024.
Jiang, D. and Ye, M. Cross-modal implicit relation reason- ing and aligning for text-to-image person retrieval. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2787-2797, 2023.
Cho, M., Kim, M., Hwang, S., Park, C., Lee, K., and Lee, S. Look around for anomalies: supervised anomaly detection via context-motion relational learning. Pro- ceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 12137-12146, 2023a.
Joo, H. K. Vo, K., Yamazaki, K., and Le, N. Clip-tsa: Clip- assisted temporal self-attention for weakly-supervised video anomaly detection. In Proceedings of the IEEE International Conference on Image Processing, pp. 3230- 3234, 2023.
Cho, M., Kim, M., Hwang, S., Park, C., Lee, K., and Lee, S. Look around for anomalies: Weakky-supervine anomaly detection via context- motion relational learning. In Pro- ceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pp. 12137-12146, 2023b.
Ju, C., Han, T., Zheng, K., Zhang, Y., and Xie, W. Prompting visual- language models for efficient video understanding. In Proceedings of the European Conference on Computer Vision, pp. 105-124, 2022.
Dev, P. P., Hazari, R., and Das, P. Reflip-vad: Towards weakly supervised video anomaly detection via vision- language model. IEEE Transactions on Circuits and Systems for Video Technology, 2024.
Li, J., Selvaraju, R., Gotmare, A., Joty, S., Xiong, C., and Hoi, S. C. H. Align before fuse: Vision and language representation learning with momentum distillation. In Proceedings of the Neural Information Processing Sys- tems, pp. 9605-9705, 2021.
Dosovitskiy, A. and et al. An image is worth 16x16 words: Transformers for image recognition at scale. In Pro- ceedings of the International Conference on Learning Representations, 2021.
Li, S., Liu, F., and Jiao, L. Self- Self-training multi-sequence learning with transformer for weakly supervised video anomaly detection. In Proceedings of the AAAI Confer- ence on Artificial Intelligence, pp. 1395-1403, 2022.
Feng, J., Hong, F.-T., and Zheng, W.-S. Mist: Multiple instance self-training framework for video anomaly detec- tion. In Proceedings of the IEEF Conference on Computer Vision and Pattern Recognition, pp. 14009-14018, 2021.
Lin, K. Q., Zhang, P., Chen, J., Pramanick, S., Gao, D., Wang, A. J., Yan, R., and Shou, M. Z. Univtg: Towards unified video-language temporal grounding. In Proceed- ings of the IEEE International Conference on Computer Vision, pp. 2904,,204,22 2023a.
Gupta, H., Sharma, S., and Agrawal, S. Artificial antelligence-base anomalies detection scheme for identi- fying cyber threat on iot-based transport network. JEEE Transactions on Consumer Electronics, 101) 2024.
Lin, W., Chen, J., Mei, J., Coca, A., and Byrne, B. Fine- grained inteiitteratton multi- modal retrieval for retrieval augmented visual question answering. In Proceedings of the Neural Information Processing Systems, pp. 22820- 22840, 2023b.
Hasan, M., Choi, J., Neumann, J., Roy-Chowdhury, A. K., and Davis, L. S. Learning temporal regularity in video sequences. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 733-742, 2016.
Liu, T. Lam, K.-M., and Bao, B.-K. Injecting text clues for improving anomalous event detection from weakly labeled videos. JEEE Transactions on Image Processing, 33:5907-5920, 2024.
Huang, C., Liu, C., Wen, J., Wu, L., Xu, Y., Jiang, Q., and Wang, Y. Weakly supervised video anomaly detection via self-guided temporal discriminative transformer. IEEE Transactions Cybernetics, 54(5):3197-3210, 2022.
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEBE International Conference on Computer Vision pp. 10012-10022, 2021.
Ja, C., Yang, Y., Xia, Y., Chen, Y.-T., Parekh, Z., Pham, H., Le, Q. Sung, Y.-H., Li, Z.., and Duerig, T. Scaling up visual and vision-language representation learning with noisy text supervision. In Proceedings of the Interna- tional Conference on Machine Learning, pp. 4904-4916 2021.
Lu, J., Batra, D., Parikh, D.,, and Lee, S. Vilbert: Pre- training task-agnostic visiolinguistic representations for ussion-add-laggun tasks. In Proceedings of the Neural Information Processing Systems, pp. 13 13-23, 2019.
10

---

Learning Event Completeness for Weakly Supervised Video Anomaly Detection
Luo, W., Liu, W., Lian, D., and Gao, S. Future frame predic- tion network for video anomaly detection. IEEE Transac- tions Pattern Analysis and Machine Intelligence, 44 (11):750557522) 2021.
Shou, Z., Gao, H., Zhang. L.,, Miyazawa, K., and Chang, Weakly-supervised temporal action lo- calization in untrimmed videos. In Proceedings of the European Conference on Computer Vision, pp. 154-171, 2018.
S.-F. Autoloc:
Lv, H., Yue, Z., Sun, Q. Luo, B., Cui, Z., and Zhang, H. Un- biased multiple instance learning for weakly supervised video anomaly detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8022-8031, 2023.
Sima, C.,, Renz, K., Chitta, K., Chen, I.,, Zhang, H., Xie, C., BeiBwwenger, J., Luo, P., Geiger, A., and Li, H. Driv- elm: Driving with graph visual question answering. In Proceedings of the European Conference on Computer Vision, pp. 256-274, 2025.
Park, S., Kim, H., Kim, M., Kim, D., and Sohn, K. Nor- mality guided multiple instance learning for weakly su- pervised video anomaly detection. In Proceedings of the IEEE Winter Conference on Applications of Computer Vision, pp. 2665-2674, 2023.
Sultani, W. Chen, C., and Shah, M. Real-world anomaly de- tection surveillance videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6479 6488, 2018.
Tian, Y., Pang, G., Chen, Y., Singh, R., Verjans, J. W., and Carneiro, G. Weakly-supervised video anomaly detec- tion with robust temporal feature magnitude learning. In Proceedings of the IEEE International Conference on Computer Vision, pp. 4975 4986, 2021.
Paul, S., Roy, S., and Roy- KoyyChhwdhury, A. K. W-tale: Weakly-supervised temporal activity localization and clas- sification. In Proceedings of the European Conference on Computer Vision, pp. 563-579, 2018.
Pu, Y. and Wu, X. Locality-aware attention network with discriminative dynamics learning for weakly supervised anomaly detection. In Proceedings of the IEEE Inter- national Conference on Multimedia and Expo, pp. 1-6, 2022.
Tran, D., Bourdev, L.,, Fergus, R., Torresani, L., and Paluri, M. Learning spatiotemporal features with 3d convolu- tional networks. In Proceedings of the IEEE International Conference on Computer Vision, pp. 1489-4449 2015.
Van der Maaten, and Hinton, G. Visualizing data using t-sne. Journal of Machine Learning Research, 9(11), 2008.
Pu. Y., Wu, X., Yang, L., and Wang, S. Learning prompt- enhanced context features for weakly-supervised video anomaly detection. IEEE Transactions on Image Process- ing, 33:4923 4936, 2024.
Wang, J. and Cherian, A. Gods: Generalized one-class dis- criminative subspaces for anomaly detection. In Proceed- ings of the IEEE International Conference on Computer Vision, pp. 8201-8211, 2019.
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. Learning transferable visual models from natural language supervision. In Proceedings of the International Conference on Machine Learning, pp. 8748-8763, 2021.
Wang. Y., Li, Y., and Wang. H. Two-stream networks for weakly temporal action localization with semantic-aware mechanisms. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 18878-18887, 2023a.
Rai, A. K., Krishna, T., Hu, F., Drimbarean, A., McGuin- ness, K., Smeaton, A. F., and O' connor, N. E. Video anomaly detection via spatio-temporal pseudo-anomaly generation: A unified approach. In Proceedings of the IEFE Conference on Computer Vision and Pattern Recog- nition, pp. 3887-3899, 2024.
Wang. Y., Zhao, S., and Chen, S. Action-semantic con- sistent knowledge for weakly-supervised action localiza- tion. IEEF Transactions Multimedia, 26: 10299-10289,, 2024.
Ramachandra, B., Jones, M. J., and Vatsavai, R. R. A survey of single-scene video anomaly detection, IEEE Transac- tions on Pattern Analysis and Machine Intelligence, 44 (5):2293-22) 2312, 2020.
Wang, Y., Zhao, S., and Chen, S. Sql-net: Semantic query
learning for superviuee temporal action localization. IEEE Transactions on Multimedia, 27:84-94, 2025.
Wang. Z., Gao, Z.,, Guo, K., Yang. Y., Wang, X., and Shen, H. T. Multilatrral semantic relations modeling for image text retrieval. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2830- 2839, 2023b.
Sapkota, H. and Yu, Q Bayesian nonparametric submodular video partition for robust anomaly detection. In Proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3212-3221, 2022.

---

Learning Event Completeness for Weakly Supervised Video Anomaly Detection
Wei, Z.,, Pan, Z., and Owens, A. Efficient vision-language pre-training by cluster masking. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recog- 2024.
Yu, J., Liu, J., Cheng, Y., Feng, R., and Zhang, Y. Modality- aware contrastive instance learning with self-distillation for weakly-supervised audio-visual violence detection. In Proceedings of the ACM International Conference on Multimedia, pp. 6278-6287, 2022.
nition, pp.
Wu, P. and Liu, J. Learning causal temporal relation and feature discrimination for anomaly detection. IEEE Trans- 2021.
Zaheer, M. Z., Mahmood, A.. Astrid, M.. and Lee, S.-I. Claws: Clustering assisted weakly supervised learning with normalcy suppression for anomalous event detection. In Proceedings of the European Conference Computer Vision, pp. 358-376, 2020.
actions on Image Processing,
Wu, P., Liu, J., Shi, Y., Sun, Y., Shao, F,, Wu, Z., and Yang, Z. Not only look, but also listen: Learning multimodal violence detection under weak supervision. In Proceed- ings of the European Conference on Computer Vision, pp. 322 339, 2020.
Zaheer, M. Z.., Mahmood, A., Khan, M. H., Segu, M., Yu, F., and Lee, S.-I. Generative cooperative learning for unsupervised video anomaly detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 14744-14754, 2022.
Wu, P., Liu, X., and Liu, J. Weakly supervised audio-visual violence detection. IEEE Transactions on Multimedia, 25: 25:1674-1685, 2022.
Zaheer, M. Z.,, Mahmood, A., Astrid, M., and Lee, S.-I. Clustering aided weakly supervised training to detect anomalous events in surveillance videos. IEEE Transac- tions Neural Networks and Learning Systems, 35(10): 14085-14098, 2024.
Wu, P., Liu, J., He, X., Peng, Y., Wang, P., and Zhang, Y. Toward video anomaly retrieval from video anomaly de- tection: New benchmarks and model. IEEE Transactions on Image Processing, 33:2213-2225, 2024a.
Zhang, C., Li, G., Qi. Y., Wang, S., Qing, L., Huang, Q., and Yang, M.-H. Exploiting completeness and uncertainty of pseudo labels for weakly supervised video anomaly detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 16271 16280, 2023.
Wu, P., Zhou, X., Pang, G., Sun, Y., Liu, J., Wang, P., and Zhang, Y. Open-vocabulary video anomaly detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 18297-18307, 2024b.
Wu, P., Zhou, X., Pang, G., Yang. Z., Yan, Q. Wang, P., and Zhang, Y. Weakly supervised video anomaly detec- tion and localization with spatio-temporal prompts. In Proceedings of the ACM International Conference Multimedia, pp. 9301-9310 2024c.
Zhang. J., Qing, L., and Miao, J. Temporal convolutional network with complementary inner bag loss for weakly supervised anomaly detection. In Proceedings of the IEEE International Conference on Image Processing, pp. 4030-4034, 2019.
Wu, P., Zhou, X., Pang. G., Zhou, L., Yan, Q. Wang, P., and Zhang, Y. Vadelip: Adapting vision-language models for weakly supervised video anomaly detection. In Proceed- ings of the AAAI Conference on Artificial Intelligence, pp. 6074-6082, 2024d.
Zhang, P., Li, X., Hu, X., Yang. J., Zhang, L.,, Wang, L., Choi, Y., and Gao, J. Vinvl: Revisiting visual represen- tations in rision-languag models. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.
2021. Zhong. J.-X., Li, N., Kong, W., Liu, Li, and Li, G. Graph convolutional label noise cleaner: Train a plug-and-play action classifier for anomaly detection. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 1237-1246, 2019.
Wu, W., Wang, X., Luo, H., Wang. J., Yang. Y., and Ouyang, W. Bidirectional cross-modal knowledge exploration for video recognition with pre-trained vision-langee languaee mod- els. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6620-6630, 2023.
Xiao, J., Yao, A., Li, Y., and Chua, T.-S. Can i trust your answer? visually grounded video question answering. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pp. 13204-13214, 2024.
Zhong, Y., Zhu, R., Yan, G., Gan, P., Shen, X., and Zhu, D. Inter-clip feature similarity based weakly supervised video anomaly detection via multi-scale temporal mip. IEEE Transactions on Circuits and Systems for Video Technology, 2024.
Yang. Z., Liu, J., and Wu, P. Text prompt with normality guidance for weakly supervised video anomaly detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 18899-18908, 2024.
Zhou, H., Yu, J., and Yang, W. Dual memory units with un- certainty regulation for weakly supervised video anomaly
12

---

Learning Event Completeness for Weakly Supervised Video Anomaly Detection
detection, In Proceedings of the AAAI Conference Artificial Intelligence, pp. 3769-3777, 2023.
Zhu, J. and Pang. G. Toward generalist anomaly detection via in-context residual learning with few-shot sample prompts. In Proceedings of the IEEE Conference on Computer Vision and Paltern Recognition, pp. 17826- 17836, 2024.
Zhu, I.., Zheng, C.,, Guan, W., Li, J., Yang, Y., and Shen, H. T. Multi-modal hashing for efficient multimedia re- trieval: A survey. IEEE Transactions Knowledge and Data Engineering, 36(1):239-22) 200399226022000 2023.
13


Reviews:
[{"review_id": "PaPIUIafhP", "reviewer": "Reviewer_QCZn", "summary": "This paper explores weakly supervised video anomaly detection, introducing a dual-structure framework that captures both category-aware and category-agnostic semantics through vision-language integration. To enhance anomaly scoring, the authors propose a learnable Gaussian mixture mask that produces smoother scoring patterns. The effectiveness of the proposed approach is validated on standard benchmark datasets, including UCF-Crime and XD-Violence.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Reviewer QCZn\n\n__Q1: The impact of the local transformer layer and the GCN module in enhancing video features.__\n\nThanks for your advice. The CLIP image encoder is primarily used to extract image features. However, it exhibits limited proficiency in modeling temporal dependencies inherent in videos. To address this limitation, we integrate a local transformer layer alongside GCN modules to augment video features. In detail, we incorporate one local Transformer layer and 4 GCN layers. In the following table, we present ablation studies that explore the rationale behind our choice and report the fine-grained AVG performance on XD-Violence.\n|Local Transformer/GCN|1|2|3|4|5|\n|:-:|:-:|:-:|:-:|:-:|:-:|\n|**1**|29.71|31.89|33.77|**34.94**|34.46|\n|**2**|28.24|30.27|32.03|33.15|32.76|\n|**3**|26.68|27.91|28.88|30.40|30.21|\n\nAlso, we report fine-grained AVG results on UCF-Crime:\n|Local Transformer/GCN|1|2|3|4|5|\n|:-:|:-:|:-:|:-:|:-:|:-:|\n|**1**|10.46|11.67| 12.94|**13.56**|13.27|\n|**2**|9.37 |10.56|11.37|11.98|11.60|\n|**3**|8.01 |9.15|10.03|10.51|10.46|\n\n__Q2: The rationale behind the choice of 𝐾 in top-𝐾 scores__\n\nIn this paper, we set $K=max(\\lfloor T/16\\rfloor$), as explained in Implementation Details. This decision stems from our utilization of multi-instance learning (MIL) to get video-level outcomes. Previous works that used MIL used the same setup [1,2], which can **adapt to the length of videos**. The denominator uses 16 because these two datasets sample 16 consecutive frames into one segment during feature preprocessing.\n\n[1] Two-Stream Networks for Weakly-Supervised Temporal Action Localization with Semantic-Aware Mechanisms, CVPR 2023\n[2] Vadclip: Adapting vision-language models for weakly supervised video anomaly detection, AAAI 2024\n\n__Q3: The inference time of the proposed method__\n\nThanks for your advice. I provide the average inference time (in seconds) of each video for fine-grained predictions on XD-Violence and UCF-Crime, and compare results with representative methods with a single Nvidia V100 GPU:\n|Methods|Inference Time on XD-Violence|Inference Time on UCF-Crime|\n|-|:-:|:-:|\n|VadCLIP|0.21|0.43|\n|ReFLIP|0.38|0.63|\n|ours|0.23|0.46|\n\nWe observe that our method outperforms the SOTA ReFLIP in terms of speed. When compared to VadCLIP, it incurs only a marginal increase in processing time, yet achieves substantial improvements in performance.\n\n**Q4: The definitions of VOB and the CMB.**\n\nThanks for your comments. VOB denotes both coarse-grained and fine-grained predictions derived exclusively from visual information, without incorporating category text descriptions. Instead, CMB combines visual inputs and text descriptions of anomaly classes.\n\n__Q5: Eq. 8 is chosen to regularize the predicted scores rather than ground truth.__\n\nThanks for your comments. Eq.8 acts as a regularizer on the prediction scores and should not be used for GT. We hope to constrain the consistency of coarse-grained and fine-grained predictions by using Equation 8 \n\n__Q6: The C3D result for LEC-VAD is missing in Table 1.__\n\nThanks for your advice. We apologize for the omission of this result in the original paper. To rectify this, we will include the C3D results for our method in Table 1. This result is also presented here:\n|Modality|Methods|Features| AP(\\%)|\n|:-:|:-:|:-:|:-:|\n|RGB|LEC-VAD|C3D|79.58|\n\n__Q7: Discussions about publications including LLM- or VLM-based approaches.__\n\nThanks for your advice. Some LLM- or VLM-based methods are used to generate abundant and diverse anomaly class descriptions. Notably, these works introduce **additional data and knowledge** provided by LLM, and the results based on them may be unfair comparisons. This introduction of external expert knowledge is not in the scope of this article. However, **we will add relevant works and discussions in the related work**.\n\n__Q8: The concept of Event Completeness and how our method addresses this aspect.__\n\nEvent completeness can be conceptualized as the proportion of predicted intervals relative to the total actual event intervals. It can be reflected when using larger tIoU values. As shown in Table 3 and Table 4, compared with ReFlip, our method achieves more **relative improvements** for larger tIoU values.\n\n__Q9: A detailed comparison between the proposed method and ReFLIP-VAD.__\n\nThe core idea of ReFLIP-VAD includes: 1) modeling the global and local temporal dependencies; and 2) designing learnable prompt templates to provide interpretable and informative class clues. However, our method focuses on **the event completeness**. While our focus is also on generating informative text descriptions, we introduce an innovative memory bank-based mechanism, eschewing the use of prompt templates generated by an additional pre-trained encoder.\n\n__Q10: Consider enlarging the text in Figure 3 for improved readability.__\n\nThanks for your advice. We will improve them in the camera-ready version.", "claims_and_evidence": "The experimental evaluation and ablation study are thorough, effectively verifying the contribution of each component and the influence of hyperparameters in the objective function.\n\nHowever, the paper could be further improved by explicitly demonstrating the impact of the local transformer layer and the GCN module in enhancing video features, which is currently not addressed. Additionally, the rationale behind the choice of 𝐾 in top-𝐾 scores and their impact remain unclear to the reviewer.", "methods_and_evaluation_criteria": "The inference time of the proposed method is not provided.\n\nAdditionally, the definitions of the vision-only anomaly-aware branch (VOB) and the cross-modal anomaly-aware branch (CMB) are unclear. Are the authors referring to VOB as the component that outputs F_{video}, or does it serve a different purpose? Please clarify.", "theoretical_claims": "This paper primarily focuses on the empirical design of improved architectural structures and learning objectives, without making any theoretical claims.\n\nHowever, Equation 8 could be further justified, particularly regarding the choice to regularize the predicted anomaly scores rather than the ground truth scores, as the latter may appear to be a more suitable option.", "experimental_designs_or_analyses": "The C3D result for LEC-VAD is missing in Table 1. Please clarify the reason.", "relation_to_broader_scientific_literature": "This study could be valuable to the video anomaly detection community. Additionally, some of the paper's empirical design choices for post-processing may also benefit the action detection community.", "essential_references_not_discussed": "Please consider including recent publications in this field, particularly those focused on LLM- or VLM-based approaches, along with relevant discussions.", "other_strengths_and_weaknesses": "Please clarify the concept of Event Completeness in the proposed approach, as it remains unclear. At the very least, the authors should justify how their method addresses this aspect.", "other_comments_or_suggestions": "Please provide a detailed comparison between the proposed method and ReFLIP-VAD.\n\nAdditionally, consider enlarging the text in Figure 3 for improved readability.", "questions_for_authors": "Please refer to the above comments.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "N/A"}}, {"review_id": "hlyreijBGu", "reviewer": "Reviewer_K2pV", "summary": "This paper proposes a new WSVAD framework, LEC-VAD, that utilizes visual and language modalities for category-agnostic and category-aware anomaly detection. The authors employ a Gaussian mixture method to guide the model in predicting more complete anomaly boundaries. Additionally, a memory bank-based prototype learning mechanism is introduced to enhance the text feature representation related to anomalies. LEC-VAD achieves state-of-the-art performance in both coarse-grained and fine-grained results on the XD-Violence and UCF-Crime datasets.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "__Q1: The fine-grained detection assumes closed-set categories. However, the reliance on category labels in VAD may limit the generalizability, since most abnormal categories are unknown/unpredictable.__\n\nThanks for your suggestion. We agree that an over-reliance on category labels could potentially limit the generalizability. However, our proposed two mechanisms can mitigate the risks, including the usage of the  CLIP text encoder and the prototype-based memory bank mechanism (PMB). First, CLIP itself can recognize open-set categories. Second, PMB can be understood to some extent as identifying general patterns and characteristics of anomalies, instead of focusing on learning specific categories. \n\nTo verify the generalization, we report the AUC on **an extra dataset UBnormal**, where only 7 abnormal categories are visible during training and 12 abnormal categories are used for testing. The results show our model can efficiently handle open-set anomaly categories, especially when using PMB.\n|Methods|AUC(%)|\n|-|:-:|\n|RAD|50.30|\n|Wu et al.[1]|53.70|\n|DMU|59.91|\n|RTFML|60.94|\n|ReFLIP|61.13|\n|**Ours w/o PMB**|60.85|\n|**Ours w/ PMB**|**61.65**|\n\n[1] Not only look, but also listen: Learning multimodal violence detection under weak supervision. ECCV\n\n__Q2: The Gaussian mixture mechanism primarily enforces local consistency rather than explicitly addressing completeness. Visualizations of Gaussian-rendered scores would strengthen this claim.__\n\nThanks for your suggestions. We define the average tIoU between predictions and GT for all classes' instances as a metric of the Event Completeness (EC) and report fine-grained results on XD-Violence:\n|Methods|EC(%)|\n|-|:-:|\n|Ours w/o GMP|12.6|\n|Ours w/ GMP|**15.5**|\n\nWe offer visual comparisons that illustrate the differences in outcomes when incorporating Gaussian-based scores versus not using them in https://anonymous.4open.science/r/icml_vis-CDD6/README.md\n\nBesides, we also provide visualizations for various categories in https://anonymous.4open.science/r/Visualization_ICML_Rebuttal-B61B/README.md We find that our model can get desirable results.\n\n__Q3: The proposed Gaussian mixture and memory-based prototype techniques.__\n\nWe appreciate the reviewer's insightful observations. First, both [1] and [2] are used in VAD instead of WS-VAD. Besides, they only model a **unimodal Gaussian model** with complex kernel function designs, but our method models prediction scores of multiple anomaly categories as **Gaussian Mixture Model** to ensure local consistency of predictions. With GMM, our model can learn the correlations between multiple anomaly categories, but the unimodal Gaussian model cannot.\n\nFor the memory-based prototype technique, [3] uses point annotations of action clips to create a memory bank storing reliable visual prototypes with threshold strategies. However, our weakly supervised scenario cannot access point annotations, so our memory-based is designed to build flexible text representations of anomaly categories. This can be also understood as general patterns of learning abnormal behaviors, as we explained in Q1. Besides, [4] learned prototype-guided discriminative embeddings to separate normal and abnormal data. It uses a different paradigm and purpose, which is essentially different from our method.\n\n__Q4: The absence of coarse-grained ablation studies weakens the evaluation.__\n\nThanks for your suggestions. Due to the page limit, we showed more challenging fine-grained results in the original paper. Here, we provide coarse-grained results using I3D features on XD-Violence\n\n[1] Ablation studies of the model structure:\n|VOB|COM|AP(%)|\n|-|:-:|:-:|\n|✅||84.85|\n||✅|87.10|\n|✅|✅|**88.47**|\n\n[2] Ablation studies of the loss function:\n|$L_{reg}$|$L_{gmm}$|AP(%)|\n|-|:-:|:-:|\n|||85.43|\n|✅||86.95|\n||✅|87.79|\n|✅|✅|**88.47**|\n\n[3] Ablation studies of text enhancement:\n|VAP|PMB|AP(%)|\n|-|:-:|:-:|\n|||82.14|\n|✅||86.69|\n||✅|86.83|\n|✅|✅|**88.47**|\n\nWe observe that coarse-grained results show consistent conclusions with fine-grained versions, which further reveals the utilities of the proposed components.\n\n__Q5: Multi-label evaluation for XD-Violence dataset.__\n\nWe use the same evaluation paradigm and codes as prior works like VadCLIP, ReCLIP, etc. Among 800 test videos, 47 samples have multiple labels. As labels for individual anomaly events aren't provided, the current evaluation paradigm only computes mAP without considering classes, which may introduce bias. However, since the proportion of such multi-label cases is low and all current methods follow this paradigm, the comparison remains fair. We also evaluated results only on 47 videos and found that its results were far lower than those on 800 videos, which reveals its results did not play a leading role.\n\n|Methods|0.1|0.2|0.3|0.4|0.5|AVG|\n|-|:-:|:-:|:-:|:-:|:-:|:-:|\n|VadCLIP|35.37|26.36|23.85|6.03|4.46|19.21|\n|ReFLIP|39.38|26.96|24.73|14.47|9.89|23.09|\n|Ours|**41.68**|**28.32**|**27.47**|**20.20**|**15.37**|**26.61**|", "claims_and_evidence": "The claims are mostly clear and convincing.", "methods_and_evaluation_criteria": "The approach is conceptually sound, and the evaluation datasets align with community standards.", "theoretical_claims": "N/A.", "experimental_designs_or_analyses": "The paper includes comprehensive experiments: comparisons across feature backbones, ablation studies, and hyperparameter analysis. However, all ablation studies focus solely on fine-grained detection results, lacking ablation experiments on coarse-grained performance.", "relation_to_broader_scientific_literature": "The key contributions include the Gaussian mixture prior for smoother local supervision and the memory-based prototype mechanism for feature enrichment. However, similar techniques (e.g., Gaussian kernels for temporal modeling [1] and smoother guidance[2], memory-based prototypes for representation enhancement [3,4]) have been explored in action/anomaly detection literature. The authors need to clarify how these adaptations differ in the context of anomaly detection or provide novel insights specific to VAD.\n\n[1]Gaussian temporal awareness networks for action localization. CVPR2019.\n[2]GlanceVAD: Exploring glance supervision for label-efficient video anomaly detection.\n[3]HR-Pro: Point-supervised temporal action localization via hierarchical reliability propagation. AAAI2024.\n[4]Anomaly Detection with Prototype-Guided Discriminative Latent Embeddings.", "essential_references_not_discussed": "The cited baselines are comprehensive and include recent WS-VAD methods.", "other_strengths_and_weaknesses": "Strengths: \n\n1.The paper is well-structured and clearly written.\n\n2.The motivation—detecting more precise anomaly boundaries—is reasonable and addresses a critical limitation in WS-VAD.\n\n3.The proposed method achieves state-of-the-art performance across multiple benchmarks and evaluation metrics.\n\nWeaknesses:\n\n1.Problem setting: The fine-grained detection assumes closed-set anomaly categories are available, this is a common practice in the Temporal Action Localization (TAL) field.  However, the reliance on category labels in Video Anomaly Detection (VAD) domain may limits the generalizability of the trained model, since most abnormal categories are unknown/unpredictable in real-world application (the key difference between VAD and TAL ).\n\n2.Motivation: While the title and motivation emphasize \"Event Completeness,\" the Gaussian mixture mechanism primarily enforces local consistency rather than explicitly addressing boundary completeness. Visualizations of Gaussian-rendered anomaly score and its impact on boundary precision would strengthen this claim.\n\n3.Method：The proposed Guassian mixture and memory-based prototype techniques has been used in other video action/anomaly detection methods[1-4]. Their adaptation to VAD lacks domain-specific innovations or insights compared to prior work in action detection.\n\n4.Experiments: As noted earlier, the absence of coarse-grained ablation studies weakens the evaluation.\n\n[1]Gaussian temporal awareness networks for action localization. CVPR2019.\n[2]GlanceVAD: Exploring glance supervision for label-efficient video anomaly detection.\n[3]HR-Pro: Point-supervised temporal action localization via hierarchical reliability propagation. AAAI2024.\n[4]Anomaly Detection with Prototype-Guided Discriminative Latent Embeddings.\n\nOverall, the contributions in the current version of paper are limited. However, if the authors address my concerns, I may consider increasing my rating.", "other_comments_or_suggestions": "None.", "questions_for_authors": "The XD-Violence dataset contains videos with multiple anomaly categories and events, and labels are not available for each anomaly event. Could the authors clarify how mAP is computed in this multi-label scenario?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "I6wwsUMWgl", "reviewer": "Reviewer_MBS1", "summary": "This paper is dedicated to improving anomaly detection performance by enhancing the completeness of predicted events. A dual-branch structure is introduced to capture both category-aware and category-agnostic semantics between vision and language. A prototype learning mechanism based on a memory bank is proposed to improve the representation of textual features. Besides, the approach utilizes learnable Gaussian masks to achieve local consistency in predictions.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Reviewer MBS1\n\n__Q1: The dual structure has already been proposed in VadCLIP, so it would be preferable not to present it as a primary contribution.__\n\nThanks for your suggestions. We will update it on the camera-ready version.\n\n__Q2: The ablation experiments use the setting where only one module is ablated independently, which is not ideal for verifying the relationships between each contribution. It is recommended to provide ablation experiments that incrementally introduce each module.__ \n\nWe appreciate the reviewer's insightful suggestion. Here, we provide an extra ablation study on XD-Violence to verify the relationships between each contribution. We have observed that the proposed modules can bring positive gains. \n\n|VOB|CMB|VAP|PMB|0.1|0.2|0.3|0.4|0.5|AVG|\n|-|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|$\\checkmark$||||33.74|28.57|22.45|17.81|13.40|23.19|\n||$\\checkmark$|||39.23|32.96|25.77|20.03|16.33|26.86|\n|$\\checkmark$|$\\checkmark$|||42.17|34.66|27.98|22.47|17.24|28.90|\n|$\\checkmark$|$\\checkmark$|$\\checkmark$||46.24|38.75|31.97|26.63|20.05|32.73|\n|$\\checkmark$|$\\checkmark$||$\\checkmark$|47.67|39.98|32.85|27.59|21.00|33.82|\n|$\\checkmark$|$\\checkmark$|$\\checkmark$|$\\checkmark$|**48.78**|**40.94**|**34.28**|**28.02**|**22.68**|**34.94**|\n\n\n__Q3: Multiple dense anomalous events may exist in an anomalous video, such as some testing videos in XD-Violence, and it is important to evaluate whether the proposed method might incorrectly classify these frequent anomalies as a single anomalous event.__\n\nWe appreciate the reviewer's insightful suggestion. For fine-grained anomaly detection, the model is asked to predict anomalous events about each category, as reported in Table 3 and Table 4. Here, we divide the original test set (800 videos) into two parts. One part contains videos with only one type of abnormal events, and the other part contains videos with multiple types of abnormal events. The former contains 753 videos, while the latter contains 47 videos and is more challenging. We report fine-grained prediction results in these 47 videos that contain multiple anomaly categories:\n\n|Methods|0.1|0.2|0.3|0.4|0.5|AVG|\n|-|:-:|:-:|:-:|:-:|:-:|:-:|\n|VadCLIP|35.37|26.36|23.85|6.03|4.46|19.21|\n|ReFLIP|39.38|26.96|24.73|14.47|9.89|23.09|\n|Ours|**41.68**|**28.32**|**27.47**|**20.20**|**15.37**|**26.61**|\n\nWe observe that our proposed method achieves significant improvement, especially when the the evaluation criteria are more stringent, namely larger tIoU values.\n\n __Q4: The visualizations in Figure 3 should provide more details, such as the source of the visualized features, particularly clarifying whether the visualized normal features include those from normal segments within anomalous videos. Meanwhile, this visualization demonstrates the discriminative visual representations for different anomaly categories. However, in the event completeness visualization presented in Figure 5, which is a binary classification result, the analysis lacks the explanation of how the visual representations for different anomaly categories contribute to enhancing event completeness.__\n\n We sincerely appreciate the reviewer's insightful comments and constructive suggestions regarding the visualizations in our paper. Below, we address the specific concerns raised about Figure 3 and Figure 5.\n\nFor Figure 3, we acknowledge the reviewer's request for clarification on the source of the visualized features. The visualized features are the enhanced feature $X_{l}$ learned by our model, and the source of them are features extracted by the CLIP image encoder. Besides, the visualized normal features include those from normal segments within anomalous videos. As shown in Figure 3, these features form clusters well, further proving that our learned features are highly discriminative. **We will revise the caption of Figure 3 for these suggestions in the camera-ready version**.\n\nAlthough Figure 5 highlights binary classification results, the model utilizes the distinctive representations learned for different anomaly categories (depicted in Figure 3) to inform its decisions. This capacity to differentiate among anomaly types indirectly helps delineate the boundaries and attributes of anomalous events. \n\nFurthermore, **In the anonymous link https://anonymous.4open.science/r/Visualization_ICML_Rebuttal-B61B/README.md, we also provide some visualization examples to analyze the influence of different anomaly categories on enhancing event completeness.**\n\n\n __Q5: Some other suggestions about expressions and the section structure.__\n\n Thanks for your valuable suggestion. We will improve the structure of Section 3 and some expressions in this paper in the revised version.", "claims_and_evidence": "The performance of the proposed method in coarse-grained and fine-grained anomaly detection has been effectively validated on two datasets. However, the mechanism of the introduced dual structure and memory bank-based prototype learning for improving the completeness of event predictions have not been clearly stated. In addition, in Figure 5, more examples are needed to demonstrate the improvement in the completeness of event predictions.", "methods_and_evaluation_criteria": "Based on the provided main results and visual experiments, the proposed method appears to be meaningful in improving the performance of WSVAD. Besides, since the dual structure has already been introduced in VadCLIP, it would not be appropriate to present it as a primary contribution.", "theoretical_claims": "No", "experimental_designs_or_analyses": "The performance of the proposed method in coarse-grained and fine-grained anomaly detection has been effectively validated on two datasets. In addition, the ablation experiments utilize the setting where only one module is ablated independently, which is not ideal for verifying the relationships between each contribution. It is recommended to provide ablation experiments that incrementally introduce each module. Moreover, the visualizations in Figure 3 could be enhanced by including additional details, such as the source of the visualized features, particularly indicating whether the visualized normal features include those from normal segments in anomalous videos. Meanwhile, this visualization demonstrates the discriminative visual representations for different anomaly categories. However, in the event completeness visualization presented in Figure 5, which is a binary classification result, the analysis lacks the explanation of how the representations for different anomaly categories contribute to enhancing event completeness.", "relation_to_broader_scientific_literature": "The dual-branch structure has already been proposed in VadCLIP. The proposed method should emphasize the difference between its dual-branch framework and that in VadCLIP, with experiments highlighting the difference contribute to performance improvements. Additionally, as the issue of event completeness is extensively studied in the Weakly Supervised Temporal Action Localization task, citing these works can convincingly introduce event completeness into WSVAD.", "essential_references_not_discussed": "Please refer to the comments on 'Relation to Broad Scientific Literature'.", "other_strengths_and_weaknesses": "Strengths: \nTo the best of our knowledge, this paper is the first to introduce the completeness of events in predictions, which has been the issue also emphasized in other temporal localization tasks, to WSVAD. The issue addressed has indeed limited the performance of WSVAD but has been overlooked. Based on the visualization experiments, the proposed method learns discriminative features and achieves more accurate and complete predictions.\n\nWeaknesses: \nMore comprehensive ablation studies should be conducted to further validate the effectiveness of the proposed modules. The mechanism of the dual structure and memory bank-based prototype learning in enhancing the completeness of event predictions is expected to be clearly explained and validated.", "other_comments_or_suggestions": "Memory bank-based prototype learning and GMP-based local consistency learning are introduced alternately in Section 3, though the structure of this section could be improved for better clarity and flow. Additionally, the tense usage should be checked, as both present and past tenses are used inconsistently in some paragraphs, such as in the conclusion.", "questions_for_authors": "Q1: The dual structure has already been proposed in VadCLIP, so it would be preferable not to present it as a primary contribution.\nQ2: The ablation experiments use the setting where only one module is ablated independently, which is not ideal for verifying the relationships between each contribution. It is recommended to provide ablation experiments that incrementally introduce each module.\nQ3: Multiple dense anomalous events may exist in an anomalous video, such as some testing videos in XD-Violence, and it is important to evaluate whether the proposed method might incorrectly classify these frequent anomalies as a single anomalous event.\nQ4: The visualizations in Figure 3 should provide more details, such as the source of the visualized features, particularly clarifying whether the visualized normal features include those from normal segments within anomalous videos. Meanwhile, this visualization demonstrates the discriminative visual representations for different anomaly categories. However, in the event completeness visualization presented in Figure 5, which is a binary classification result, the analysis lacks the explanation of how the visual representations for different anomaly categories contribute to enhancing event completeness.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "No"}}, {"review_id": "BWJbB60AhC", "reviewer": "Reviewer_zcbm", "summary": "The main challenge in video anomaly detection is the lack of dense frame-level annotations, leading to incomplete localization in existing WS-VAD methods. To tackle this,  authors introduced Learning Event Completeness for WS-VAD, featuring a dual structure that captures both category-aware and category-agnostic semantics. It uses an anomaly-aware Gaussian mixture to define precise event boundaries and includes a memory bank-based mechanism to enhance text descriptions of anomaly-event categories. The proposed method shows promising results on the XD-Violence and UCF-Crime datasets.\n\n## Update after rebuttal\n\nWith additional experiments, some of my concerns have been addressed. However, some of my questions remain, specifically regarding the loss function ablations. Therefore, I will keep my score as it is.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Reviewer zcbm\n\n__Q1: The formulation of the local transformer and global GCN needs clarification. What are the main contributions of these modules? how does it affect the proposed approach?.__\n\nWe appreciate the reviewer's insightful suggestions. The CLIP image encoder is primarily used to extract image features. But it exhibits limited proficiency in **modeling temporal dependencies inherent in videos**. To address this limitation, we integrate a local transformer layer alongside GCN modules to augment video features. We conduct ablation studies on GCN and Transformer modules and report the fine-grained AVG performance on XD-Violence:\n\n|GCN|Local Transformer|0.1|0.2|0.3|0.4|0.5|AVG|\n|-|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|$\\checkmark$||46.39|38.45|32.16|26.23|20.92|32.83|\n||$\\checkmark$|45.21|37.58|30.92|24.94|18.79|31.49|\n|$\\checkmark$|$\\checkmark$|**48.78**|**40.94**|**34.28**|**28.02**|**22.68**|**34.94**|\n\nIn detail, we incorporate one local Transformer layer and 4 GCN layers. We present ablation studies that explore the rationale behind our choice and report the fine-grained AVG performance on XD-Violence:\n|Local Transformer/GCN|1|2|3|4|5|\n|:-:|:-:|:-:|:-:|:-:|:-:|\n|**1**|29.71|31.89|33.77|**34.94**|34.46|\n|**2**|28.24|30.27|32.03|33.15|32.76|\n|**3**|26.68|27.91|28.88|30.40|30.21|\n\nAlso, we report fine-grained AVG results on UCF-Crime:\n| Local Transformer/GCN|1|2|3|4|5|\n|:-:|:-:|:-:|:-:|:-:|:-:|\n|**1**|10.46|11.67| 12.94|**13.56**|13.27|\n|**2**|9.37 |10.56|11.37|11.98|11.60|\n|**3**|8.01 |9.15|10.03|10.51|10.46|\n\n__Q2: In the loss function formulation, only two loss functions are evaluated and presented in the manuscript. What about the other loss functions?__\n\nThanks for your comments. We do some ablation studies to evaluate the utilities of $L_{gmm}$ and $L_{reg}$ in Table 6. For the $L_{agnostic}$ and $L_{aware}$, since our model is designed to predict coarse-grained and fine-grained anomalies simultaneously, $L_{agnostic}$ and $L_{aware}$ are the basic terms used to train the model. Specifically, $L_{agnostic}$ is utilized to supervise coarse-grained anomaly detection, and $L_{aware}$ is utilized to supervise fine-grained anomaly detection. If either of them is removed, the model cannot detect anomaly events of the corresponding granularity (coarse-grained and fine-grained).\n\n**Q3: In Table 5, what are the differences between the last rows and the second-to-last row in terms of experimental settings? If there are different settings, how did the authors evaluate them?**\n\nThanks for your comments. The last row of Table 5 indicates that the proposed algorithm leverages both visual information and textual descriptions of anomaly categories for both coarse-grained and fine-grained detections, while the second-to-last row only uses the visual information of the video. There is no difference in the experimental settings of other parameters.\n\n__Q4: As shown in Figure 2, there are three branches that are components of the proposed approach. However, in the ablation section, only VOB and CMB are evaluated. What about the other branch?__\n\nThanks for your suggestions. In the ablation section, we evaluated the VOB and CMB, and the other branch is the proposed Gaussian Mixture Prior-based Local Consistency Learning mechanism, which is only used to guide the model's training. We explore ablation studies about this module by controlling the loss term $\\mathcal{L}_{gmm}$ in Table 6. For clarity, we add several experiments and put them in a table for comparisons:\n\n|GMP|VOB|CMB|0.1|0.2|0.3|0.4|0.5|AVG|\n|-|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n||$\\checkmark$||37.23|30.96|23.77|18.03|14.33|24.86|\n|||$\\checkmark$|38.61|31.54|24.04|18.67|14.81|25.53|\n|$\\checkmark$|$\\checkmark$||42.86|33.82|30.58|24.66|18.92|30.17|\n|$\\checkmark$||$\\checkmark$|46.03|38.87|32.18|27.02|22.04|33.23|\n||$\\checkmark$|$\\checkmark$|46.69|39.51|32.17|27.16|22.01|33.51|\n|$\\checkmark$|$\\checkmark$|$\\checkmark$|**48.78**|**40.94**|**34.28**|**28.02**|**22.68**|**34.94**|\n\n__Q5: The authors need to discuss some of the related work in the introduction section to strengthen the motivation for the proposed approach.__\n\nWe appreciate the reviewer's insightful suggestions. **We will strengthen the discussion about our motivation in the camera-ready version**. Here, we also re-emphasize it. We observe the existing methods predict incomplete and fragmented event segments, and an example is also provided in Figure 1. The core reason for this phenomenon is the lack of frame-wise annotations in the weakly supervised learning framework, resulting in sparse semantics. For this reason, we propose a novel Gaussian mixture prior-based strategy and memory bank-based prototype mechanism to mitigate this issue.\n\n__Q6:The authors need to carefully review the figures and table captions to ensure clarity for the reader\".__\n\nThanks for your suggestions. We will modify these typos and inaccurate expressions.", "claims_and_evidence": "The proposed approach is somewhat incremental, and the results are promising. However, the problem formulation and motivation behind the approach are somewhat weak and need clearer articulation. The authors attempt to reformulate the problem in the second paragraph on page 1, but the proposed approach—which includes a dual structure  category-aware and category-agnostic semantics, an anomaly-aware Gaussian mixture to define precise event boundaries, and a memory bank-based mechanism to enhance text descriptions—requires further clarification with some of existing approaches. Additionally, comparing this approach with existing work is necessary to clearly address the challenges it aims to solve. This would provide a stronger foundation for the proposed approach and enhance its completeness.\n\nThe overall structure of the proposed approach presents some issues, as shown in Figure 2. Q represents textual features, while k and V represent video features, indicating that it involves cross-attention rather than self-attention in the Cross-Modal Aware block, which requires correction. On page 4, in the second column, the authors state \"modeling cross-modal interactions through a cross-attention operation.\" \n\nFurthermore, there is no clear formulation regarding the local transformer and global GCN. It remains unclear what the main contribution is beyond merely combining existing approaches to improve performance; the approach needs something novel to distinguish itself.\n\nThe experimental results support the contributions of the proposed approach, which outperforms existing methods by a large margin across different evaluation settings. The visualization results further demonstrate the advantages of the proposed approach.", "methods_and_evaluation_criteria": "The authors propose an approach to address the challenges in video anomaly detection by introducing Learning Event Completeness techniques. The proposed approach is evaluated in both coarse-grained and fine-grained settings, yielding very promising results, as shown in the quantitative and qualitative analyses. Furthermore, when compared to approaches from 2023 and 2024, the proposed method outperforms them by a large margin.\n\nSome of the experimental settings require clarification. For example, in Table 5 (\"Explorations of the Model Structure\"), the authors evaluate the vision-only anomaly-aware branch and the cross-modal anomaly-aware branch. What are the differences between the last  rows and the second-to-last row in terms of experimental settings? If there are different settings, how did the authors evaluate them?\n\nFurthermore, the main contribution of the paper is the dual structure that captures both category-aware and category-agnostic semantics. It is essential to clearly present how this approach is evaluated and to articulate the contributions of the proposed method more explicitly.\n\nAdditionally, in Table 6 (\"Ablation Studies of Loss Function\"), only two loss functions are presented. However, as defined in Equation 10, there are four loss functions. What are the contributions of the other loss functions, and why were they not evaluated? A clear explanation regarding this would be beneficial.", "theoretical_claims": "The proposed approach makes sense but appears somewhat incremental, primarily consisting of a combination of existing methods. However, the results in both coarse-grained and fine-grained settings are interesting.\n\nThe approach lacks some theoretical explanations, particularly regarding the problem formulation and motivation for the dual structure that captures both category-aware and category-agnostic semantics. In Section 3, the vision branch includes a local transformer and global GCN, but this needs clearer presentation. \n\nAdditionally, it is important to explain how the main contributions enhance the learning of discriminative features. While some formulations are provided, they are not entirely clear. If these aspects were supported by a more robust theoretical explanation, it would significantly enhance the paper and clarify the concepts for the reader.", "experimental_designs_or_analyses": "The experimental design is somewhat satisfactory, as the authors evaluate the approach in different settings, such as coarse-grained and fine-grained. However, there are some issues to address. For instance, in Table 1, only one unsupervised approach is presented. It is unclear whether the proposed approach is evaluated in an unsupervised manner. It appears that the evaluation is conducted with weakly supervised setting under different backbone networks, making the unsupervised row less relevant in this context.\n\nIn the fine-grained setting, it is essential to clarify the specific evaluation setting used for the proposed approach—whether it is weakly supervised, unsupervised, or something else. This needs to be explicitly stated.\n\nAdditionally, a major issue in the experimental settings is the evaluation of the proposed modules and the loss function formulation in Tables 5 and 6. These aspects need to be clearly presented for better understanding.", "relation_to_broader_scientific_literature": "The authors attempt to address the main challenges of video anomaly detection with a dual structure that captures both category-aware and category-agnostic semantics. The proposed idea is intriguing, and the experimental results are promising, particularly in two main settings: coarse-grained and fine-grained.\n\nHowever, a significant aspect of this paper is how the problem is formulated and the motivation behind it, where there appear to be gaps. As mentioned earlier, the novelty of the approach seems somewhat incremental. While the GMP-based Local Consistency Learning branch is a new contribution, its presentation is unclear, and there is a lack of strong emphasis on its significance.", "essential_references_not_discussed": "The authors have included most of the related works in the experimental comparisons published in 2023 and 2024. The existing approaches are compared across different experimental settings. However, the authors need to discuss some of the related work in the introduction section to strengthen the argument and motivation for the proposed approach.\n\nAs I mentioned, the formulation of the problem and motivation is weak, which may explain why the authors did not include a discussion of the 2023 and 2024 papers.", "other_strengths_and_weaknesses": "The results of the proposed approach are significant; however, it appears to be a combination of existing methods. There is a lack of clear presentation in some sections, particularly in the proposed approach and the experimental settings.\n\nFor instance, the formulation of the local transformer and global GCN needs clarification. What are the main contributions of these modules? Additionally, it is important to explain how these components enhance the performance of the proposed approach and the overall contributions of the proposed modules. These aspects should be emphasized for better understanding.", "other_comments_or_suggestions": "The authors need to carefully review the figures and table captions to ensure clarity for the reader. For example, in Figure 2, it should specify \"cross-attention\" rather than \"self-attention,\" based on the features of Q, K, and V.", "questions_for_authors": "Most of my concerns are outlined in the above, and the authors need to address them. Specifically, in section three regarding the proposed approach and experimental settings, I have the following questions:\n\n1.) The formulation of the local transformer and global GCN needs clarification. What are the main contributions of these modules? how it affect the proposed approach? It is important to explain how these components enhance the performance of the proposed approach and the overall contributions of the proposed modules.\n\n2). In the loss function formulation, only two loss functions are evaluated and presented in the manuscript. What about the other loss functions?\n\n3). What are the differences between the last rows and the second-to-last row in terms of experimental settings? If there are different settings, how did the authors evaluate them?\n\n4) . As shown in Figure 2, there are three branches that are components of the proposed approach. However, in the ablation section, only VOB and CMB are evaluated. What about the other branch? What is the contribution of the proposed branch?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: All four reviewers recommended weak accept. The reviewers generally appreciated the proposed idea and the experimental results, particularly in two main settings: coarse-grained and fine-grained.The reviewers also raised several questions in the initial review including,  ablation experiments on coarse-grained setting, multi-label evaluation results, ablations when incrementally introducing each module, and questions related to clarity in the presentation. Post-rebuttal, reviewers mentioned that most of their concerns are addressed. Given the manuscript, the rebuttal, and that all reviewers are generally positive, the AC agrees with the reviewers and recommend acceptance. Authors are encouraged to take into account all the suggestions of reviewers and rebuttal when preparing the revised manuscript.
Decision: Accept (poster)