Ex-VAD: Explainable Fine-grained Video Anomaly Detection Based on Visual-Language Models
Authors: Chao Huang
Yushu Shi
Jie Wen
Wei Wang
Yong Xu
Xiaochun Cao
Venue: ICML 2025 poster (2025)
Keywords: Visual Language Model; Label Enhancement; Video Anomaly Detection
Primary area: applications->computer_vision

Abstract: With advancements in visual language models (VLMs) and large language models (LLMs), video anomaly detection (VAD) has progressed beyond binary classification to fine-grained categorization and multidimensional analysis. However, existing methods focus mainly on coarse-grained detection, lacking anomaly explanations. To address these challenges, we propose Ex-VAD, an Explainable Fine-grained Video Anomaly Detection approach that combines fine-grained classification with detailed explanations of anomalies. First, we use a VLM to extract frame-level captions, and an LLM converts them to video-level explanations, enhancing the model's explainability. Second, integrating textual explanations of anomalies with visual information greatly enhances the model's anomaly detection capability. Finally, we apply label-enhanced alignment to optimize feature fusion, enabling precise fine-grained detection. Extensive experimental results on the UCF-Crime and XD-Violence datasets demonstrate that Ex-VAD significantly outperforms existing State-of-The-Art methods.

Explainable Fine-grained Video Anomaly Detection Based on Visual-Language Models
Chao Huang' Yushu Shi
Jie Wen
Wei Wang'
Yong Xu²
Xiaochun Cao !*
Abstract
Classifee
With advancements in visual language models (VLMs) and large language models (LLMs), video anomaly detection (VAD) has progressed beyond binary classification fine-graine fine-grained cate- gorization and multidimensional analysis. How- ever, existing methods focus mainly on coarse- grained detection, lacking anomaly explanations. To address these challenges, we propose Ex-VAD, an Explainable Fine- sine-grained Video Anomaly Detection approach that combines fine-grained classification with detailed explanations of anoma- lies. First, we use VLM to extract frame-level captions, and an LLM converts them to video- level explanations, enhancing the model's explain- ability. Second, integrating textual explanations of anomalies with visual information greatly en- hances the model's anomaly detection capabil- ity. Finally, we apply label-enhanced alignment to optimize feature fusion, enabling precise fine- grained detection. Extensive experimental results on the UCF-Crime and XD-Violence datasets demonstrate that Ex-VAD significantly outper- forms existing State-of-Ther Art methods.
Vosuel Encocee
Classifier
(a) Binary classification VAD
(b) Multi-classfficttion VAD
AT1IY
Visuel Encoder
Text Encodl
VLM
Asomaly Explanation
(c) Training free VAD
Our Explainable VAD
Figure 1. Recent research in VAD can be categorized into three types: a) Traditional binary classification VAD, b) Multi- classification VAD, and c) Training free VAD. Building on the optimization of these approaches, Ex VAD is presented as: d) Explainable VAD based on VLMs and LLMs.
et al., 2021; Nawaratne et al., 2020). For example, rapid detection of dangerous behaviors for timely intervention in surveillance, detection of abnormal road conditions avoid accidents in autonomous driving, and identification of ab- normal vital signs to provide timely assistance in healthcare monitoring.
1. Introduction
Traditional VADs (Huang et al., 2024; Wang et al., 2025; Huang et al., 2023; 2022; Ramachandra et al., 2022; Liu et al., 2024: Zaigham Zaheer al., 2020; Yan al., 2023) typically coarse-grained analyze videos, determining only whether a video contains abnormal behavior and catego- rizing it as normal or anomalous. However, such ap- proaches (Nguyen & Meunier, 2019) face significant limita- tions in practical applications. First, coarse-graie coarse-grained detec- tion fails to provide detailed descriptions of specific types of abnormal behavior, which is inadequate in scenarios that require tailored responses distinct anomalies, for exam- ple, addressing varying security threats in surveillance sys- tems or diagnosing multiple abnormal conditions in medical monitoring. Second, coarse-grained methods are easily in- fluenced by the complexity of video backgrounds and the diversity of scenes, making it challenging to pinpoint the time and type of anomalies accurately. This deficiency com-
Video Anomaly Detection (VAD) is an important technol- ogy with wide range of applications that cover areas such security surveillance, healthcare, autonomous driving, and content auditing (Zhao et al., 2017; Wang et al., 2019; Samaila et al., 2024). It aims to improve the safety and effi- ciency of systems by automatically identifying anomalous events or behaviors through the analysis of video data (Ren
'Shenzhen Campus of Sun Yat-Sen University, School of Cy- ber Science and Technology. Shenzhen, China "arrbi Institute of Technology, School of Computer Science and Technology, Shenzhen, China. Correspondence to: Xiaochun Cao <caoxi- aochun @ mammlllyyy www.ii..yss........ sys..edu.cn....
Proceedings of the 4 International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Ex-VAD: Explainable Fine-grained Video Anomaly Detection Based on Visual-Language Models
promises both the detection accuracy of the system and its response efficiency.
lies. Finally, we employ a Label Augment and Alignment Module (LAAM), which uses an LLM to expand anomaly category labels into phrases, selects the top-k phrases seman- tically most similar to the original labels, and aligns them with the fused multimodal features to obtain fine-grained anomaly categories. In summary, Ex- EX-VAD effectively inte- grates multimodal features, fine-grained classification, and anomaly explanations, providing comprehensive solution to video anomaly detection with enhanced interpretability and accuracy.
Fine-grained video anomaly detection becomes particularly important to distinguish different types of anomalous behav- ior further and provide more targeted and interpretative de- tection results. In recent years, visualllanguage pre-training (VLP) models such as CLIP (Radford et al., 2021) have significantly improved the semantic representation of im- ages and text through contrast learning, driving advances in visual representation. CLIP-based task-specific models have excelled in various visual tasks, achieving unprecedented performance breakthroughs. In VAD, some researchers ( (Wu et al., 2024c;a) have used CLIP's image-text alignment to achieve fine-grained anomaly detection.
Our main contributions are summarized as follows.
We develop an Anomaly Explanation Generation Mod- ule (AEGM), which utilizes a VLM and an LLM to generate explanations for video anomalies, allowing the model to detect abnormal behavior and analyze its cause, thereby enhancing its semantic interpretation.
Despite advances, existing methods still struggle to explain anomalous behavior effectively. Even when anomalous events are successfully detected, models often fail to provide clear explanations for the causes of the anomalies, posing significant challenges to decision-makenn makers. For example, in security monitoring, the detection of abnormal behavior in a specific area without a clear explanation can complicate sub- sequent response efforts, leading inefficiency and delays. Consequently, enhancing the interpretability of VAD has become a crucial focus in the field's development. Recently, the rapid progress in LLMs has introduced new possibilities for VAD. Some researchers (Zanella et al., 2024; Ye et al., 2024) have proposed training-free anomaly detection meth- ods by generating descriptive text explanations of anomalies using VLMs and LLMs. However, these methods primar- ily rely on the generated text for anomaly detection, often neglecting the full potential of the visual modality. Other researchers (Lv & Sun, 2024; Kim et al., 2023a; Tang et al., 2024b) have achieved interpretable anomaly detection by fine-tuning large models. While effective, these approaches often result in complex models that may be challenging to deploy and maintain.
We propose a Label Augment and Alignment Module (LAAM) that enhances label semantics, enabling the model to better align videos with anomaly categories, thereby improving fine Ane-grained anomaly classification, particularly for complex categories.
Extensive experimental results show that our method outperforms existing approaches in both coarse- grained and fine- tine-grineed accuracy, improving overall anomaly detection and classification precision.
2. Related Work
2.1. Video Anomaly Detection
According the output of existing VAD, it can be divided into binary-classffication VAD (Ramachandra al., 2022; Liu et al., 2024), multi-classification VAD (Sultani et al., 2019; Wu et al., 2024a;c), and interpretable VAD (Lv & Sun, 2024). Traditional VAD methods classify videos as normal or abnormal. They typically adopt a classification paradigm. Firstly, pre-trained visual models are used to extract frame-level features. Then, these features are fed into binary classifier based Multiple Instance Learning (MIL) for training. Finally, abnormal events are detected based on the predicted anomaly confidences.
To address these challenges, we propose novel method called Ex- VAD, which is designed to overcome the lim- itations of traditional VAD methods, particularly in fine- grained classification and anomaly explanation. Specifi- cally, we first propose an Anomaly Explanation Genera- tion Module (AEGM), which extracts frame-level captions from videos using VLMs, followed by cleaning step to refine the captions. The cleaned captions are then integrated by an LLM to generate video-level anomaly explanations through specific prompts, which enable the model to detect abnormal behavior in the video and analyze its cause. Sec- ond, we develop a Multimodal Anomaly Detection Module (MADM), which encodes the text from AEGM and extracts both temporal and spatial features between video frames. These features are then fed into a graise- anomaly classifier to determine whether the video contains anoma-
With the development of the CLIP model, some methods have attempted to make improvements. VadCLIP (Wu et al., 2024c) proposed fine-grained Weakly Supervised Video Anomaly Detection (WSVAD) method that can distinguish different types of abnormal frames. VadCLIP encodes text labels into class embeddings and calculates the matching similarities between class embeddings and frame-level vi- sual features to obtain an alignment map. Each input text label represents a class of abnormal events, thus achieving fine-grained WSVAD.
2

---

Ex- VAD: Explainable Fine-grained Video Anomaly Detection Based on Visual-Language Models
Interpretability is of utmost importance in VAD, especially in sensitive or high-stake applications. Early methods of- ten relied on black-box models, and their prediction results were difficult to trust. Recently, some methods have utilized Large I Language Models (L..Ms) and Vision-Languaa Mod- els (VLMs) to generate understandable reasoning through semantic insights and textual explanations. For example, VADor (Lv & Sun, 2024) fine-tunes the projection layer of VideoLLaMA to integrate anomaly detection with seman- tic reasoning. HAWK (Tang et al., 2024ajenhances inter- pretability by integrating motion-based reasoning through interactive VLMs. However, there are still challenges in balancing the granularity of explanations and computational efficiency.
time-consuming and highly sensitive to template content. To address this challenge and reduce the dependence on hand-crafted language designs, PEL4VAD (Pu et al., 2024) used ConceptNet definitions to create prompt templates and expanded class labels through a conceptual dictionary, significantly improving open- ppen--ocabulary object detection. Based on this approach, this paper uses GPT4 (OpenAI & etc, 2024) to generate rich semantics for simple labels, and uses CLIP image-text alignment to allow the VAD model to achieve better performance in fine-grained anomaly classifi- cation.
3. Approach
3.1. Architecture
2.2. Visual Language Model in VAD
As shown in Figure 2. the proposed Ex-VAD consists of three components: an Anomaly Explanation Generation Module (AEGM), Multimodal Anomaly Detection Mod- ule (MADM), and Label Augment and Alignment Module (LAAM). VAD processes input videos V by first utiliz- ing the AEGM to generate anomaly explanation text E. This text serves two purposes: providing interpretative explana- tions for video anomalies and acting as the text modality input for the MADM, where it is fused with visual features for coarse- graiiee anomaly detection. Finally, the LAAM refines the detection by expanding and aligning labels to achieve fine-grained anomaly classification, ensuring both interpretability and accuracy in video anomaly detection. The implementation details are introduced as follows.
Vision language models (VLMs) offer a new perspective for detecting anomalies video anomaly detection (VAD), especially in fine-gras ane-grained classification and explanation of anomalous behaviors. Traditional VAD methods (Tian et al., 2021a; Li et al., 2022b;a) mainly focus on identifying anomalous behaviors in videos but lack detailed classifi- cation of these behaviors. (Wu et al., 2024c) leverages the pre-trained CLIP model to align video frames with labels VAD, enabling fine-grained anomaly classifica- tion. Meanwhile, the use of LLMs VAD is still in its in- fancy(Kim et al., 2023b) and LAVAD (Zanella et al., 2024) implemented training- free VAD using pre-trained LLMs and VL.Ms. This method efficiently transforms L.MMs into video anomaly detectors by generating textual descriptions of each frame in the test video, which is combined with prompting to activate LLMs for time series aggregation and anomaly score estimation. Additionally, by referring to VLMs, we establish strong complementary relationship between visual and textual modalities. This approach not only enables the detection of anomalous behaviors but also provides clear explanations for each behavior, enhancing the explanation of anomaly detection.
3.2. Anomaly Explanation Generation Module
LAVAD (Zanella et al., 2024) demonstrated the feasibility of achieving anomaly detection by prompting VLMs and LLMS to generate text descriptions. Inspired by this ap- proach, our AEGM improves the prompting mechanism to guide LLMs in time series aggregation and the generation of anomaly explanations. This not only helps the visual module enhance the performance of VAD but also serves as an explanation for the causes of anomalies, further enhanc- ing the interpretability of detection. As shown in Figure 3, AEGM consists of two sub-modules: the Caption Extraction and Cleaning Module, and the Explainable Modules Based on LLM.
2.3. Prompt Learning
Prompt learning, technique for adapting prompt words to fit a specific task, was initially applied mainly in the field of Natural Language Processing (NLP) and has gradually been extended to the visual domain. CLIP (Radford et al., 2021) relys on fixed hesd-desig cues (e.g.. a photo of class), which are suitable for open domains but not flex- ible enough. CLIP-COOR (Zhou et al., 2022a) introduces learnable context vectors, enhancing performance with lim- ited samples but struggling with generalization. These ad- vances refine prompt adaptation, improving vasion-language models across diverse tasks. In VAD, VADCLIP leverages trainable textual templates to generate precise anomaly de- seriptions. However, manually designing prompts remains
Caption Extraction and Cleaning Module. With the rapid development of VLMs, the ability to generate captions from videos has become increasingly powerful. First, uniformly sample n frames from the video V. For each frame T, E V, we use the SOTA captioning model &c i.e. BLIP-2 (Li et al., 2023) and set appropriate prompts Pe to generate frame-level text descriptions:
T: Pc +C.()..
(1)
3

---

Ex- VAD: Explainable Fine-grained Video Anomaly Detection Based on Visual-Language Models
Input Video
Lag-er
L Label A Aumment and A Aignmentt Module
Temporal A Adagher
Local and Clobal
Vison orguage
[Abuse] [Assuilt
(LAAM)
[stealing]
[vandalism]
Anomaly Explanation G eneration Module (A AMG
MultimodalAnomaly Detection Module (MADM)
LLM
Cagture inderidual destruction obsarvad
Caption Extraction and Cleaning Model
AEGM
Image Encoder
Violusi behavior being threatened
Berng attacked Slons mairreatmnt Physical herm inflicted Entry captured
Caption Prompt
Evidence of vandellion im captured Explosion
Gunfirm procress An arest is taking place
Text Encoder
LGT-A dapter
Witnessing somoone staling valuable
Thed cunino sugport appeharded
Theft
Video Frin2 option
Aggressive confzontation progress maltrrmment
Faaed LeSSco
Foo
Text Encoder
Explatnable Model Based LLM
Coarse- Grained A Anomaly Classifler
Explanation
Top-K
Prampt
VLM
User
①
Ambormal Video
Yes explosions nost Irain tracks cause firus, smoke.
VidooM Exnianatione
Soore
workers manage calmly.
Framcy
Video1 Explasstions
Yess, In office. distrussed
pecson, restruind officess control la gninedd
1 Explesions
Frome
Prameya
Namber
Figure 2. Our Ex-VAD includes three components: an Anomaly Explanation Generation Module using VLM and LLM to generate anomaly explanation text, Multimodal Anomaly Detection Madule combining enriched visual and textual features for coarse anomaly classification, and Label Augment and Alignment Module that refines the detection by expanding and aligning labels to achieve fine- fine-grained anomaly classification.
Due to the randomness VLMs, some irrelevant captions may be generated, which may harm training. Since the scenes in the video are captured by static camera a high frame rate, the semantic content between frames overlaps to some extent. From this perspective, we alleviate the above problems by designing an image-text alignment mechanism. Specifically, we use a vision-language encoder to encode the captions of each frame. For each frame 1, E V. we calculate its closest caption:
the prompt Ps into the advanced LLM PLEM to obtain the explanation F. for video V:
E Ps (LLM(T),
where - - T,...., fx, Through the above methods, obtain an anomaly description E that is more accu- rate semantically and temporally than the captions T.,
3.3. Multi-Modal Feature Fusion
f; = arg (ax (T)}
(2)
This component primarily performs coarse- satse-grained anomaly detection by entering the fused visual and text features into an anomaly classifier. For visual features, we follow prior work (Wu et al., 2024c) to uniformly sample dense vidco frames from the input video at 16-frame intervals, obtaining video frame sequence V. The video frames are then encoded by the frozen visual encoder F, in CLIP produce frame features Fy. To bridge the gap between the image and video domains in CLIP, we adopt the approach from (Wu et al., 2024c), modeling the temporal dependencies of the video frame sequence using the Local and Global Temporal Adapter (LGT-Adapter):
TET
where {+, is the cosine similarity, Ey is the image encoder of the VLM, Er is the text encoder and 7 = (Ti..., Tx) )- This module allows us to generate fairly accurate text de- scriptions for each video frame.
Explainable Modules Based on LLM. The cleaned cap- tions can describe frame information more accurately than the initial captions, but they are only simple descriptions and cannot describe abnormal phenomena in detail. Therefore, we prompt LLM iee. LLAMA-3 (Touvron et al., 2023)to generate the required anomaly explanations. Specifically, we input the collection /' of cleaned frame captions and
Fr = L.G)(E,(V).
(4)

---

Ex- VAD: Explainable Fine-grained Video Anomaly Detection Based on Visual-Language Models
Caption Estraction and Cleaning Model...
cording the similarity score, the top-k sentences with the highest similarity are selected from the generated sentences.
Image Grcoder
Image Test Caption Clean
The features of the screened sentences are integrated with the original label embeddings to form the final enhanced label embeddings Fz.
Caption Model
Caption
Clean Caption
shovs group polirr oliass intnacinng win person wor ape-spporcto be Diutoon proffio
dusrt wards to dessiite
Please use
VLM
Encoder
Seet
po ki officr while other offcess
what hoppasiing in picture.
ceatyy drese applorcto
Fine-grained Classification. Calculate the matching simi- larity between these category embeddings Fy and Fusion features FF to obtain an alignment map M R where is the number of text labels. In this alignment map, each input text label represents a class of abnormal events. By analyzing the similarity between the video and differ- ent category labels, a more detailed classification of ab- normal events is achieved, naturally achieving the goal of fine-grained classification.
ploce pod los
Explainable Model Based on LLM
Video Explonations
Pleoe
atlice dictionsed persas instriaieed by officee after conmao is gares.
what shathappeeed...
And seower with We whither anything
LLM
Figure 3. The Anomaly Explanation model first generates video frame captions via VI.M, then cleans up the frame captions us- ing the image-text module, and finally generates a detailed video interpretation using LLM. This interpretation is later used as textual modality to enhance the performance of anomaly detection conjunction with the visual modality, addition to being used as an anomaly interpretation for the video.
3.5. Loss Function
Binary Classification Loss. We follow previous work (Wu et al., 2020) and use the Top-k mechanism to select K highest anomaly confidence levels among anomalous and normal videos as video-level predictions. The classification loss LAC is then computed using the binary cross-entropy between video-level prediction and ground truth:
We use the exception of explanation E generated from AEGM as textual information. These textual informa- tion are encoded by the frozen textual encoder Fr in CLIP to produce textual features FT (E)).. Subse- quently, the textual features and visual features are fused into Fp = Fv + Fr, which is then input into a binary clas- sifier that contains a feed-forward network (FFN) layer, an FC layer, and a Sigmoid activation to obtain the anomaly scores s E R=-1;
LECE -ly + (1 - y) log(1 s)],
where denotes the predicted score and y the true label (usually 0 represents normal, and I represents abnormal).
Multiple Class Loss. For multi-classification tasks, we propose the MIL-Align mechanism to align the frame-level fusion feature F'r and all label embeddings Fz. Specifically, for each video, we select the top-k similarity values and compute the average measure how well this video is aligned with the current class. Then, we can obtain vector V = {UI...., Um J that represents the similarity between this video and all classes. We hope the video and its paired textual label emit the highest similarity score among others. To achieve this, the multi-class prediction is first computed as follows:
s = Sigmoid(FC(FFN( + FF)).
(5)
3.4. Label Augment and Alignment Module
This part mainly includes the following two steps: label aug- mentation set construction and fine-grained classification.
Label Augmentation Set Construction. We utilize a pre- trained LLM (OpenAI& & etc, 2024) to generate m descrip- tive sentences related to each category label. To filter the sentences that are most semantically related the category labels, we calculate the semantic similarity between the category labels and the generated sentences by using sine similarity. Specifically, first, the category labels T, and the related descriptive sentences {S1,... ..., } generated by the LLM are encoded into vectors. Then, the cosine sim- ilarity between the label vector and the sentence vector is calculated as:
exp(v:/T) P: = E, exp(vs/T))
88)
where p; is the prediction with respect to the ith class, and T refers to the temperature hyper-parameter for scaling. Fi- nally, the alignment loss LMCE can be computed by the cross-entropy:
LMCE
Ya log(p:);
(9)
SL. = Vr. Vs:
where y/ is the ground truth label and m is the total number of classes.
(6)
[v
|vs.
where VL and Vs, represent the embedding vectors of the category label and the generated sentence, respectively. Ac-
Contrastive Loss. To pull apart the normal class embed- dings from the anomaly class embeddings, we introduce
5

---

Ex- VAD: Explainable Fine-grained Video Anomaly Detection Based on Visual-Languagg Models
mAP@IOU(S)
mAp@ (OU(%)
Method
Mothod
0.1
0.2
0.3
0.4
0,5
AVG
0.1
0.2
0.3
0.4
0.5
AVG
Random Baseline
0.21
0.14
0.04
0.02
0.01
0.08
Random Baseline
1.82
0.92
0.48
0.23
0.09
0.71
RealAD (2018)
5.73
4.41
2.69
1.93
1.44
3.24
RealAD (2018)
22.72
15.57
9.98
6.20
3.78
11.65
RTFM (2021)
12.59
7.54
6.44
5.42
1.54
6.71
RTFM (2021)
31.25
26.85
21.94
13.56
12.54
21.23
AVVD (2022)
10.27
7.01
6,25
3.42
3.29
6,05
AV (2022)
30.51
25.75
20.18
14.83
9.79
20.21
DMU(2023)
11.32
7.62
5.97
4.33
2.36
6.32
DMU(2023)
32.33
28.88
22.57
14.33
13.68
22.36
CIIP-TSA(2023)
12.62
8.13
6.66
4.28
1.91
6.72
CLIP-TSA(2223)
34.53
32.88
28.11
13.65
10.01
23.84
UMTL(2024)
11.84
7.85
6.52
3.97
2.84
6.60
UMIL(2024)
34.44
27.13
22.63
19.85
13.24
23.46
VadCLIP(2024)
11.72
7.83
6.40
4.53
2.93
6.68
VaUCLIP(/024)
37.03
30.84
23.38
17.09
14.31
24.70
STPromp((2024)
11.56
7.49
6.13
5.11
2.11
6.48
STPromp((202))
38.21
25.63
28.66
13.11
11.63
23.44
Ex-VAD (Ours)
16.51
12.35
9,41
7.82
4.65
10.15
Ex-VAD (Ours)
40.14
32.75
28.78
20.15
18.35
28.23
Table I. Fine-grained comparisons on UCF-Crime.
Table Fine- comparisons on XD- -iooeileee
the contrast loss. Specifically, we first calculate the cosine similarity between the normal class embedding and other ab- normal class embeddings, and then compute the contrastive loss Lets as follows:
comprehensive ablation studies to validate the effectiveness of each module in the proposed model.
4.1. Experimental Setups
LA,
LoTS= = >- max 0.
(10)
Datasets. We perform experiments the UCF-Crime and XD- Violinee datasets. UCF-Crime consists of 1,900 untrimmed surveillance videos with total duration of 128 hours, covering 13 real-world anomalies (e.g., abuse, rob- bery, explosion) and normal activities. In the WSVAD, 1,610 videos are used for training with video-level annota- tions, while 290 videos are used for testing with frame-level annotations. XD-Violence contains 4,754 untrimmed videos totaling 217 hours, making it one of the largest multimodal violence detection datasets. It includes six types of vio- lence (e.g.. abuse, car accidents, explosions) across diverse sources such as surveillance, films, and games. The dataset is divided into 3,954 training videos and 800 testing videos, with video-level labels.
II |LA,
where LN is the normal class embedding, and LA is the abnormal class embedding.
Overall, the final total objective of Ex-VAD given by:
f = LBCE + LMCE + XLeTs.
(11)
3.6. Inference
ExVAD contains three branches that enable to handle fine-grained and coarse grassee WSVAD tasks and anomaly interpretation. Regarding fine-grained WSVAD, we follow previous work (Wu et al., 2023) and use a thresholding strat- egy on the alignment graph M to predict anomalous events. For coarse -ouuse-rraieed WSVAD, we follow previous work (Wu et al., 2024c) in employing two methods to compute the frame-level anomaly degree. The first method directly uses the anomaly scores from the coarse grained classification, while the second method uses the alignment map from the fine-grained classification, i.e., the similarity between the video and the normal class minus 1 is the anomaly degree. Finally, we choose the best of these two methods to compute the frame-level anomaly degree.
Evaluation Metries. For coarse-grained WSVAD, the eval- uation uses frame-level Average Precision (AP) and frame- level AUC for Dioienle and only frame-level AUC for UCF-Crime. For fine-grained WSVAD, mean Average Precision (mAP) values are calculated under different Inter- section over Union (loU) thresholds (ranging from 0.1 to 0.5 with stride of 0.1.. The average mAP (AVG) is also reported, and mAP is computed only for abnormal videos in the test set.
Implementation Details. All experiments are conducted on a single NVIDIA RTX A100 GPU using PyTorch. The network employs frozen image and text encoders from pre- trained CLIP (ViT-B/16) with a Transformer-bis layer and GELU activation. BLIP-2 is used for caption generation, while Llama-3.1 generates anomaly explana- tions. Visual and text features are fused via concatenation. Key hyperparameters include: o = 1, 7 0.07, context length / = 20, window length in LGT-Adapter (64 for XD- Violence, 8 for UCF-Crime), and À (1 x 10 -4-4 for XD- Violence, for UCF-Crime).
4. Experiments
FFN
In this section, we perform experiments on the UCF- Crime (Sultani et al., 2019) and XD- Violence (Wu et al., 2020) datasets. Ex-VAD focuses on fine-grained anomaly detection and explainability. We compare it with other meth- ods designed for fine-grate fine-grained anomaly detection and explore novel approaches for explainable coarse- -raieeerriddd anomaly detection using L.LMs and VLMs. Furthermore, we conduct
6

---

Ex- Ex-VAD: Explainable Fine-grained Video Anomaly Detection Based on Visual-Language Models
Category
Method
Fine-grained
Explainability
XD-Violence(AP) UCF-Crime(AUC)
LAVADCZamella el al., 2024)
62.01
80.28
Training-Fres
VERA(Ye al., 2024)
X
88.2
86.6
Fine-tuning LLMs
VADOr(Lv & Sun, 2024)
X
88.1
RealAD(Sultani et al., 2019)
X
75.18
84.14
RTFM (Tian et al., 2021b)
V
X
78.27
85.66
AVVD(Zhou et al., 2022b)
V
78.10
84.57
TEVAD(Chen ct al., 2023)
X
X
79.80
84.9
Weakly
DMU(Zhou et al., 2023)
V
X
82.41
86. 75
CLIP-TSA(Ioe et al., 2023)
X
82.17
87.58
Macihenaacnn et al., 2024)
X
86.75
VADCLIP(Wu al., 2024c)
X
84.51
88.02
STPrompi(Wu et al., 2024b)
83.97
88.08
Ex- VAD (Ours)
86.52
88.29
Table 3. Coarse-grainn grained comparisons of methods on XD- Violence and UCF-C UCF-Crime datasets.
Visual
AEGM LAAM
AUC (%)
LAAM
0.1
0.2
0.3
0.4
0.5
AVG
Captionl Explainable
[CLS]
14.38
10.54
6.92
5.03
2.51
7.87
video (CLS]
14.77
10.68
6.69
4.78
3.73
8.13
86.76
Leurnable-Poon
15.18
12.03
6.65
4.96
3,20
8.40
86.33
Label-Augment Prompt
16.51
12.35
9.41
7.82
4.65
10.15
87.86
88.29
Table 6. Effectiveness of the Label Augment Alignment Module for fine-grained anomaly detection.
Table 4. Effectiveness of each module for Coarse-grained anomaly detection.
UCF-Crime, outperforming VADCLIP, STPrompt, and TCVADS by 1.32, 1.52, and 7.24, respectively. On XD> Violence, Ex- VAD achieves 28.23 AVG, exceeding these methods by 3.53, 4.79, and 11.28, respectively. Unlike meth- ods like VADCLIP, STPrompt, and TCVADS which align visual features with text embeddings from CLIP or LLLMs, Vx-VAD introduces a novel approach. Using AEGM, prompts VLMMs and LLMS generate textual information, fuses this with visual features, and aligns the representation with labels. This generated textual information enriches se- mantics and enhances detection performance. Additionally, LAAM expands label semantics by converting single labels (e.g.. " "Abuse") into descriptive phrases (e.g., "Someone being mistreated"), better aligning with visual-text features.
AEGM
0.1
0.2
0.3
0.4
0.5
AVG
Captions
17.74
13.27
10.25
7.01
6.10
10.88
Explainahle Text
16.51
12.35
9.41
7.82
4.65
10.15
Table Effectiveness of the Anomaly Explainable Generation Module for fine-grained anomaly detection.
4.2. Comparison Results
Fine-grained WSVAD Results. The fine-grained detection task is more challenging as it involves detecting the presence of anomalous events while also accurately identifying their specific categories. To demonstrate the superiority of our proposed Ex- VAD, conduct comparisons with several VAD methods, including RealAD (Sultani et al., 2019),
Coarse-grained WSVAD Results. Additionally, com- pare the results of the state-of-the-art methods for coarse- grained anomaly detection, including the training- freee methods LAVAD (Zanella et al., 2024) and VERA (Ye et al., 2024); fine-tuned models to achieve interpretable VADor (Lv & Sun, 2024), and the above for fine-grained anomaly detection methods.
RTFM (Tian et al., 2021b), AVVD (Zhou et al., 2022b), DMU (Zhou et al., 2023), CLIP-TSA (Joo et al., 2023), UMIL (Sánchez-Maciíl et al., 2024), VADCLIP (Wu et al., 2024c), and STPrompt (Wu et al., 2024b). For fairness, CLIP (ViT-B/16) is used for all feature extractors.
Tables 1 and present the fine-grained detection results on UCF-Crime and XD- Vi-Veeeee datasets, evaluated using mean average precision (mAP) and average accuracy (AVG) across IOU thresholds (0.1-0.5). Our Ex-VAI consistently achieves the best results, highlighting its superior perfor- mance. Specificallyy, Ex- VAD achieves an AVG of 9.00
Table 3 shows that while LAVAD and VERA are simple and interpretable due to their lack of training, they do not sup- port fine-grained detection. Our method, Ex- VAD, performs best on the UCF dataset and second best on the XD dataset. VADOr achieves explainability through fine-tuning but lacks fine-grained detection support. For methods supporting fine-

---

Ex- VAD: Explainable Fine-grained Video Anomaly Detection Based on Visual-Language Models
grained detection, older approaches like RealAD underper- form (75.18 AP on XD--iolence), while recent methods, including AVVD, DMU, and STPrompt, show consistent improvement. VADCLIP and TCVADS push the state of the art, with TCVADS achieving 85.58 AP on XD-Violence and 88.58 AUC on UCF-Crime. Ex- VA- uniquely com- bines fine-grained detection and interpretability, excelling in both. Although its performance on UCF-Crimm (88.29 AUC) is marginally below TCVADS (88.58), it leads to XD) Violence, highlighting its versatility. This dual capability makes -AD an optimal choice for practical applications requiring precision and insights into detection results.
10.00
0.50
8.33
12.00
180
Number the diverse templatesti)
Number the templotes7)
(a)
(3)
Figure 4. Sensitivity analysis of a different number of templates K to generalization of (a) coarsc-graine grained detection and (b) fine- grained detection.
4.3. Model Analysis
Ablation Study. To evaluate the impact of the two key components, AEGM and LAAM, we conducted ablation experiments on the UCF-Crime dataset by removing one or both components from Ex-VAD, with results summarized in Table 4. The findings reveal that generating captions solely for videos degrades performance, whereas cleaning these captions and generating anomaly explanations signif- icantly enhances it. This highlights the negative impact of low-quality captions, which often contain redundant or erroneous information, and the complementary role of high- quality anomaly explanations in improving visual perfor- mance. While AEGM is primarily designed for fine-grat fine-grained anomaly detection, it also contributes coarse-grained de- tection improvements.
Method
Trainable Params
Inference Time
MAC'
RTFM
24.72M
8.28ms
126.59G
DMU
6.49M
16.60ms
21.00G
CLIP-TSA
16.41M
18.33ms
102.63G
VADCLIP
35.17M
22.30ms
29.17G
ExVAD
9.97M
15.37ms
12.04G
Table 7. Comparison of Trainable Parameters, Inference Time, and Multiply-Ade Operations (MACs). The best and second-best val- ues are highlighted with bold text and underlined text, respectively.
the highest similarity to the original labels achieves opti- mal label enhancement for video anomaly detection. In this setting, the AUC for coarse-grained detection peaks at 88.28%, while the average mAP@ IOU for fine- -na-grained detection reaches its highest value of 10. 15%, demonstrat- ing the best detection performance. However, excessive enhancement (k: 5) may introduce noise, resulting in per- formance degradation. These results highlight that moderate label enhancement significantly enhances the model's 's over- all detection capability and anomaly localization accuracy.
Effectiveness of the AEGM. We evaluate the effectiveness of fine-grained anomaly detection for AEGM, with results shown in Table 5. The analysis shows that Captions alone outperform Explainable Text in fine-grainet anomaly de- tection, as Captions provide frame-level semantic details, while Explainable Text offers concise video-level sum- mary. However, Explainable Text enhances fine-grat aineggraiedd anomaly detection while also providing transparent, summa- rized explanations of anomalies at the video level. Therefore, we choose Explainable Text for the final model to balance performance and interpretability.
Analysis of Computational Efficiency. We evaluate the number of trainable parameters (Trainable Params), infer- ence time of a frame (Inference Time), and multiply-ade add operations (MACs). Table 7 demonstrates that method, Ex VAD, achieves a well-balanced trade-off between model complexity and size, optimizing both performance and re- source usage effectively.
Effectiveness of LAAM. We evaluate the effectiveness of LAAM in fine-grained VAD, as summarized in Table 6. The results demonstrate that LLAM-augmented labels signifi- cantly enhance detection accuracy compared to manually defined cue words and learnable prompt-based approaches. This improvement highlights the value of leveraging LLAM to generate semantically rich and contextually relevant la- bels that align more effectively with the visual and textual features used for fine-grained anomaly detection.
5. Conclusion
In this paper, we propose Ex-VAD, an explainable ap- proach for fine-grained video anomaly detection. First, the Anomaly Explanation Generation Module (AEGM) ex- tracts and refines frame-level captions using VLMs, and then generates video-level anomaly explanations with an LLM. Second, the Multimodal Anomaly Detection Module (MADM) encodes the text and extracts temporal and spatial
Effectiveness of Top-k. Figure 4 presents the impact of dif- ferent top-k values in the LAAM module on coarse- grasine and fine-grained detection results, respectively. The trend graphs reveal that selecting the top 4 phrases (A: = 4) with
8

---

Ex- Vx-VAD: Explainable Fine-grained Video Anomaly Detection Based on Visual-Language Models
features detect coarse-grained anomalies. Finally, the La- bel Augment and Alignment Module (LAAM) expands and aligns anomaly category labels with multimodal features to achieve fine-grained anomaly detection. Experiments show that Ex-VAD outperforms existing methods in fine- and coarse-graine grained anomaly detection, providing a more transparent and effective solution.
Joo, H. K., Vo, K., Yamazaki, K., and Le, N. Clip-tsa: Clip- assisted temporal self-attention for weakly-supervised video anomaly detection. In 2023 IEEE International Conference on Image Processing (ICIP), pp. 3230-3234. IERE, 2023.
Kim, J., Yoon. S., Choi, T., and Sull, S. Unsupervised video anomaly detection based on similarity with prede- fined text descriptions. Sensors (Basel, Switzerland), 23, 2023a.
Acknowledgements
This work was supported in part by the National Nat-
Kim, J., Yoon, S., Choi, T., and Sull, S. Unsupervised video anomaly detection based on similarity with predefined text descriptions. Sensors, 23(4))6225,, 2023b.
ural Science Foundation of China (No.62301621), in
part by Shenzhen Science and Technology Program (No.
20231121 No. KOTI200211000933500 in part by Shenzhen General Research Project (No.
Li, G., Cai, G., Zeng, X., and Zhao, R. Scale-aware spatio- temporal relation learning for video anomaly detection. In European Conference Computer Vision, pp. 333-350- Springer, 2022a.
(CY220241222155940007 in part by Guangdong Ba-
sic and Applied Basie Research Foundation (No. 2025A151501 1398, No.202318030300000000 in part by the CIE-Smartchip research fund (No. 2024002,, in part by the Fundamental Research Funds for the Central Universities, Sun Yan-sen University under Grants No. 23xkjc010.
Ji, Li, D., Savarese, S., and Hoi, S. Blip-2: bootstrapping language-image pre-training with frozen image encoders and large language models. In Proceedings of the 40th In- ternational Conference on Machine Learning, ICML'23. 233. JMLR.org, 2023.
Impact Statement
This paper presents work whose goal is to advance the field of Computer Vision. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here.
Li, S., Liu, F., and Jiao, L. Self-training multi-sequence learning with transformer for weakly supervised video anomaly detection. In Proceedings of the AAAI Confer- ence on Artificial Intelligence, volume 36, pp. 1395-1403, 2022b.
References
Liu, Y., Yang, D., Wang, Y., Liu, J., Liu, J., Boukerche, A., Sun, P., and Song, L. Generalized video anomaly event detection: Systematic taxonomy and comparison of deep models. 56(7), 2024. ISSN 0360-0300.
Chen, W., Ma, K. T., Jian Yew, Z., Hur, M., and Khoo, D. A.-A. Tevad: Improved video anomaly detection with captions. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 5549-5559, 2023.
Lv, H. and Sun, Q. Video anomaly detection and explanation via large language models, 2024.
Huang, C., Yang. Z., Wen, J., Xu, Y., Jiang, Q., Yang, J., deep autoen-
and Wang, Y. self-supervision-augnee coder for unsupervised visual anomaly detection. IEEE Transactions on Cybernetics, 12(12): 2022.
Nawaratne, R., Alahakoon, D., De Silva, D., and Yu, X. Spatiotemporal anomaly detection using deep learning for real-time video surveillance. IEEE Transactions on Industrial Informatics, 16(1):393-402, 2020.
Huang, C., Wen, J., Xu, Y., Jiang, Q. Yang, J., Wang, Y., and Zhang, D. Self-supervised attentive generative adversarial networks for video anomaly detection. IEEE Transactions on Neural Networks and Learning Systems, 34(11):9389-94) 2023.
Nguyen, T. N. and Meunier, J. Anomaly detection in video sequence with appearance-motion correspondence. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 1273 1283, 2019.
Huang, C., Wen, J., Liu, C., and Liu, Y. Long short-term dynamic prototype alignment learning for video anomaly detection. In Larson, K. (ed.), Proceedings of the Thirty- Third International Joint Conference on Artificial Intelli- gence, IJCAI-24, pp. 866-874. International Joint Con- ferences on Artificial Intelligence Organization, 2024. doi: 10.249633ijcai.202244 2024/99. Main Track.
OpenAl and etc. Gpt-4 technical report, 2024. URL https: 08774.
/ //arxivv
Pu, Y., Wu, X., Yang. L., and Wang. S. Learning prompt- enhanced context features for anomaly detection. JEEE Transactions on Image Process- ing, 33:4923 4936, 2024.
video

---

Ex-VAD: Explainable Fine-grained Video Anomaly Detection Based on Visual-Language Models
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8763. PMLR, 2021.
Wang, B., Huang, C., Wen, J., Wang, W., Liu, Y., and Xu, Y. Federated weakly supervised video anomaly detection with multimodal prompt. Proceedings of the AAAI Con- ference on Artificial Intelligence, 39(20):10177-2102) Apr. 2025.
Ramachandra, B., Jones, M. J., and Vatsavai, R. A survey of single-scene video anomaly detection. IEEE Transac- fions on Pattern Analysis and Machine Intelligence, 44(5): 2293- 12, 2022. doi: 10.1 10.1199/PAAM.22000000000
Wang, L., Huynh, D. Q., and Mansour, M. R. Loss switch- ing fusion with similarity search for video classification. In 2019 IEEE International Conference on Image Pro- cessing (ICIP), 2019.
Ren, J., Xia, F., Liu, Y., and Lee, I. Deep video anomaly detection: Opportunities and challenges. In 2021 Interna- tional Conference on Data Mining Workshops (ICDMW), pp. 959-95999999 90999966.22 2021.
Wu, P., Liu, J., Shi, Y., Sun, Y., Shao, F., Wu, Z., and Yang. Z. Not only look, but also listen: Learning multimodal violence detection under weak supervision. In Computer OisionEECV/ 2020: 16th European Conference, Glasgow, UK. August 23 23-28, 2020, Proceedings. Part XXX 16, pp. 322-339. Springer, 2020.
Samaila, Y. A.. Sebastian, P., Singh, N. S. S., Shuaibu, A. N., Ali, S. S. A., Amosa, T. I., Mustafa Abro, G. E., and Shuaibu, I. Video anomaly detection: A systematic review of issues and prospects. Neurocomputing. 591: 127726, 2024. ISSN 0925-2312.
Wu, P., Liu, X., and Liu, J. Weakly supervised audio- violence detection. IEEE Transactions on Multimedia, 25:1674-1685, 2023.
Wu, P., Liu, J., He, X., Peng, Y., Wang, P., and Zhang, Y. Toward video anomaly retrieval from video anomaly de- tection: New benchmarks and model. IEEE Transactions on Image Processing, 33:2213- 2024a.
Sánchez-Macián, A., Martínez, J., Reviriego, P., Liu, S., and Lombardi, F. On the privacy of the count-min sketch: Extracting the top-k elements. IEFE Transactions Emerging Topics in Computing, 2024.
Wu, P., Zhou, X., Pang, G., Yang, Z., Yan, Q., Wang. P., and Zhang. Y. Weakly supervised video anomaly de- tection and localization with spatio-temporal prompts. MM '24, New York, NY, USA, 2024b. Association for Computing Machinery. ISBN 9798400706868, doi: 10. 145/3664647.3681444
Sultani, W., Chen, C., and Shah, M. Real-world anomaly detection in surveillance videos, 2019.
Tang, J., Lu, H., Wu, R., Xu, X., Ma, K., Fang, C., Guo, B., Lu, J., Chen, Q., and Chen, Y--C. Hawk: Learning to understand open-world video anomalies, 2024a.
Wu, P., Zhou, X., Pang. G., Zhou, L., Yan, Q.. Wang, P., and Zhang, Y. Vadclip: Adapting vision-language models for weakly supervised video anomaly detection. In Proceed- ings of the AAAI Conference on Artificial Intelligence, volume 38, pp.
Tang, Y., Guo, J., Hua, H., Liang, S., Feng, M., Li, X., Mao, R., Huang, C.,, Bi, J., Zhang, Z., Fazli, P., and Xu, C. Vidcomposition: Can milmss analyze compositions in compiled videos? arXiv preprint arXiv:2411.10979, 2024b.
2024c.
Yan, C., Zhang, S., Liu, Y., Pang, G., and Wang, W. Feature prediction diffusion model for video anomaly detection. In 2023 IEFF/CVF International Conference on Com- puter Vision (ICCV), pp. 5504-5514, 2023.
Tian, Y., Pang, G., Chen, Y., Singh, R., Verjans, J. W., and Carneiro, G. Weakly-supervised video anomaly detec- tion with robust temporal feature magnitude learning. In Proceedings of the IEEE/CVF international conference computer vision, pp.
2021a.
Ye, M., Liu, W., and He, P. Vera: Explainable video anomaly detection via verbalized learning of vision-languag mod- els. arXiv preprint arXiv:2412.0109) 2024.
Tian, Y., Pang, G., Chen, Y., Singh, R., Verjans, J. W., and Carneiro, G. Weakly-superriseed video anomaly detec- tion with robust temporal feature magnitude learning. In Proceedings of the IEEE/CVF international conference on computer vision, pp. -975-4986, 2021b.
Zaigham Zaheer, M., Lee, J.-H., Astrid, M., and Lee, S.-I. Old is gold: Redefining the adversarially learned one- class classifier training paradigm. In 2020 IEEF/CV Conference on Computer Vision and Pattern Recognition (CVPR), pp. 14171-14181, 2020.
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A,, Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., Rodriguez, A., Joulin, A., Grave, E., and Lam- ple, G. Llama: Open and efficient foundation language models. ArXiv, abs/2302. 13971, 2023.
Zanella, L., Menapace, W., Mancini, M., Wang. Y., and Ricci, E. Harnessing large language models for training- free video anomaly detection. Proceedings of the
10

---

Ex-VAD: Explainable Fine-grained Video Anomaly Detection Based on Visual-Langue Models
IEEE/CVF Conference on Computer Vision and Pallern Recognition, pp. 18527-18536, 2024.
Zhao, Y. Deng, B., Shen, C., Liu, Y., Lu, H., and Hua, X.-S. Spatio-temporal autoencoder for video anomaly detection. Proceedings of the 25th ACM International Conference Multimedia, pp. 1933-1941, 2017.
Zhou, H., Yu, J., and Yang. W. Dual memory units with un- certainty regulation for weakly supervised video anomaly detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 3769-3777, 2023.
Zhou, K., Yang, J., Loy, C. C., and Liu, Z. Learning to prompt for vision-language models. International Jour- nal of Computer Vision, 130(9), 2022a. ISSN 1573-1405.
doi: 10.1000/s1 120-02211263-002-116656--
Zhou, X., Girdhar, R., Joulin, A., Krähenbühl, P., and Misra, Detecting twenty-thousan classes using image-level supervision. In ECCV, 2022b.

---

Ex- VAD: Explainable Fine-grained Video Anomaly Detection Based Visual-Language Models
A. appendix.
Please whether the video contains any unusual
Please "Yes'
whether contains any
behaviour. provide brief description of the anomaly.
behaviour. please provide brief description anomaly.
Yes, Flames appear from litter, growing intense.
Ex
Yes, explosions near train tracks cause fires, smoke: workers manage calmly.
Ex
Arson
Explosibrs
1006
2000
30k
2004
Please anewer with 'Yes' 'No" to whether the video contains any unusual
Please answer with 'Yes' or "No" to whether the video contains any unusual behaviour. so, please provide brief description of the anomaly.
behaviour. If please provide brief description of the anomaly.
Yes, explosion in control room causes chaos.
Ex
Yes, individuals and handle merchandise suspiciously.
Ex
10
Shopliting
Explosions
1000
any
Please answer with 'Yes' 'No' to whether the video contains any unusual behaviour. please provide brief description the anomaly.
please provide description anomaly.
Yes, road accidents cause chaos, people fall.
Ex
No, this is normal video.
Ex
RoadArcidents
150
255
354
180
209
40
Figure Confidence visualization on the UCF-Crime dataset.
Qualitative Analyses. Figure illustrates the qualitative visualization of Ex- VAD. The blue curve represents the anomaly prediction score, while the grey area highlights the ground truth anomaly time positions. The figure also showcases fine-grained anomaly categories and anomaly explanations, which are generated by querying the LLM. As shown, Ex- VAD effectively detects unused anomaly categories, describes anomalous phenomena, and accurately differentiates between normal and abnormal clips in anomalous videos.
12


Reviews:
[{"review_id": "19HB13vy8T", "reviewer": "Reviewer_p4Ua", "summary": "Paper proposes an explainable VAD approach which combines fine-grained classification with explanations. The approaches use pre-trained VLM and LLM to extract the relevant features. The approach employed 3 linear combination of 3 loss functions for the fine-grained classification of anomalous videos. A novel label-enhanced alignments method was used to optimize the feature fusion. Experiments on 2 popular VAD datasets show promising results against the SOTA methods.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewers for their valuable comments. We will add these valuable comments to the revised manuscript.\n\n**R1：Novelty of the Proposed Pipeline.** We apologize for failing to highlight our contributions and novelty. Different from existing coarse-grained VAD, our method is unique in using the generated anomaly text for both anomaly explanation and fine-grained anomaly detection. Compared with traditional fine-grained anomaly detection, our accuracy has a large improvement, e.g., in the UCF dataset, the average mAP(AVG) is 3.67% higher compared with SOTA. Additionally, compared with fine-tuning a large language model (LLM) or a multimodal large language model (MLLM) for anomaly detection, our approach requires significantly fewer parameters (9.19M) and achieves a faster inference speed (15.37ms). This demonstrates a good trade-off between complexity and effectiveness.\n\n**R2：Reasons for not using ShanghaiTech and UCSD-Ped2.** The key idea of our method aims at the fine-grained detection and description of the causes of anomalies, and its advantage is to generate anomaly explanation text through VLMs and LLMs, which both describe the causes of video anomalies and greatly improve the accuracy of the fine-grained anomaly classification. Therefore, we select datasets that contain fine-grained anomaly detection in existing studies, such as UCF-Crime and XD-Violence, whereas the Shanghai and UCDF datasets have not been used by researchers for fine-grained anomaly detection, and thus cannot be compared with the superiority of our method.\n\nConsidering your suggestion, we will add related experiments for the shanghaiTech and UCSD-Ped2 datasets. Due to time constraints, we only had time to add the results of the coarse-grained anomaly detection experiment on shanghaiTech, and will add the results on UCSD-Ped2 later. As shown in the below table, we performed coarse-grained anomaly detection on the ShanghaiTech dataset and compared it with other methods:\n\n|**Method**|Feature|AUC(%)|\n| - | - | - |\n|GAN-Anomaly(CVPR2019)|TSN|84\\.4|\n|RTFM(ICCV2021)|I3D|97\\.21|\n|CMRL(CVPR2023)|I3D|97\\.60|\n|TEVAD(CVPR2023)|I3D|98\\.10|\n|PE-MIL(CVPR2024)|I3D|**98.35**|\n|STPrompt(ACMMM2024)|CLIP|97\\.81|\n|Ours|CLIP|_98\\.23_|\n\nAlthough our method is not SOTA, it is only 0.12% less than the best result, 0.13% higher than TEVAD, and 0.42% higher than STPrompt, which also uses the clip to extract features. This experiment shows that our method is also applicable to the Shanghai dataset.\n\n**R3: Citation and discussion of TEVAD.** TEVAD makes full use of VLM to generate video captions as textual modalities and achieves improved accuracy in anomaly detection by fusing it with visual modalities. We will cite and discuss it in the final manuscript. \n\nBoth TEVAD and our method utilize VLMs to generate captions, which extends the traditional single visual modality to visual and textual multimodal branching, improving anomaly detection. Our advantages over TEVAD are shown below. First, our proposed method not only determines whether a video frame is abnormal but also further identifies the abnormal category (Abuse, Arrset, etc.). Second, instead of just generating captions, the video anomalies are further leveraged to describe the video anomalies through LLMs.\n\n**R4: others.** Thank you for pointing out the spelling error, we will change Table 6: 'Lable Augmnet' to 'Label Augment' in the camera-ready paper.", "claims_and_evidence": "The experimental results in Table 1 (UCF-Crime), Table 2 (XD-Violence) and Table 3 (Coarse grained, UFC-Crime, XD-Violence) shows that the proposed method outperform other SOTA methods.", "methods_and_evaluation_criteria": "The methods and evaluation criteria are sensitive. However, there are other VAD datasets which should also be included in the experiments, e.g. ShanghaiTech [1] and UCSD-Ped2 [2]\n\n\n[1] Dan Xu, Rui Song, Xinyu Wu, Nannan Li, Wei Feng, and Huihuan Qian. Video anomaly detection based on a hierarchical activity discovery within spatio-temporal contexts, Neurocomputing\n\n[2] Wen Liu, Weixin Luo, Dongze Lian, and Shenghua Gao. Future frame prediction for anomaly detection–a new baseline.\nIn Proceedings of the IEEE conference on computer vision and pattern recognition", "theoretical_claims": "No theoretical claims. Equations were read but not checked.", "experimental_designs_or_analyses": "No major issue on experimental designs and analysis.\n\nThe exclusion of other benchmarks, like ShanghaiTech and UCSD-Ped2 should be explained to avoid suspicion of cherry-picking results.", "relation_to_broader_scientific_literature": "Paper's approach is similar to other explanable AI using VLM. The application of fine-grained VLM is also not very novel. However, the experimental results are promising.", "essential_references_not_discussed": "The earlier work in 2023 using VLM for VAD is not cited and discussed.\n\nChen, W., Ma, K. T., Yew, Z. J., Hur, M., & Khoo, D. A. A. (2023). TEVAD: Improved video anomaly detection with captions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 5549-5559).", "other_strengths_and_weaknesses": "Strengths\n1. Paper's experimental results are strong for the 2 benchmarks experimental upon.\n2. The use of pretrained VLM models for explainable VAD is somewhat novel. But there are some prior works.\n3. Label Augment and Alignment Module is incrementally novel.\n\nWeaknesses\n1. Not much technical contributions. Main novelty is the pipeline of modules and the Label Augment and Alignment module.\n2. Experimental results while strong, are not particularly significant compare to SOTA.", "other_comments_or_suggestions": "Typos:\nTable 6: \"Lable Augmnet\" should be \"Label Augment\"", "questions_for_authors": "Will the paper include the two benchmarks, ShanghaiTech and UCSD-Ped2 to show the generalizability of the proposed method?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "Not applicable."}}, {"review_id": "m91G6TLFxL", "reviewer": "Reviewer_QRsp", "summary": "This paper proposes Ex-VAD, an explainable fine-grained video anomaly detection method that integrates visual-language models (VLMs) and large language models (LLMs). The approach consists of three main modules: the Anomaly Explanation Generation Module (AEGM), the Multi-modal Anomaly Detection Module (MADM), and the Label Augment and Alignment Module (LAAM). The AEGM uses VLMs to generate frame-level captions and leverages LLMs to produce video-level anomaly explanations. The MADM combines text and visual features for coarse-grained anomaly detection, while the LAAM enhances fine-grained classification by expanding and aligning labels. Experimental results demonstrate that Ex-VAD significantly outperforms existing methods on the UCF-Crime and XD-Violence datasets. ## update after rebuttal", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewers for their valuable comments. We will add these valuable comments to the revised manuscript.\n\n**R1: Resource consumption.** We appreciate your concern regarding the resource consumption of Ex-VAD due to the integration of VLMs and LLMs. We also recognize that due to the integration of multiple complex modules such as VLM and LLM, it may impose high resource requirements when deployed. Therefore, in the experimental session, we compared the training parameters and Multiply-Add Operations(MACs). Table 7 demonstrates that our method achieves a well-balanced trade-off between model complexity and size, optimizing both performance and resource usage effectively.\n\n|Method|Trainable Params|Inference Time|MACs|UCF-Crime(AUC%)|\n| :- | :-: | :-: | :-: | :-: |\n|RTFM|24\\.72M|**8.28ms**|126\\.59G|_85\\.66_|\n|DMU|**6.49M**|16\\.60ms|_21\\.00G_|86\\.75|\n|CLIP-TSA|16\\.41M|18\\.33ms|102\\.63G|87\\.58|\n|VADCLIP|35\\.17M|22\\.30ms|29\\.17G|88\\.02|\n|ExVAD|_9\\.97M_|_15\\.37ms_|**12.04G**|**88.29**|\n\n**R2: Fine-tune the model.** We appreciate your concern and suggestion about fine-tuning the model. At the beginning of our experiments, we thought that there were two ways to obtain anomaly explanations: the first way is to fine-tune the LLM to achieve the interpretability of anomaly videos, and the second way is to invoke the knowledge of the LLM by setting the cue words to obtain the desired anomaly explanations. Through the experiments, we found that the computational resources, model complexity, and reasoning speed required for fine-tuning LLM are much higher than that of the method of setting cue words when there is not much difference in accuracy. Therefore, ultimately, our approach is to efficiently utilize the general knowledge of LLM by setting cue words to achieve anomaly interpretation and improve detection accuracy.\n\n**R3: Multi-modal feature fusion during the training process.** During the training process, we optimize and select the multimodal fusion strategy in the following two steps. Firstly, we refer to the fusion strategies commonly used by previous studies, such as contact, cross-modal attention, and addition, etc. Secondly, we try the above methods in the experimental process and select the optimal fusion method by comparing the accuracy.\n\n**R4: Future work.** Thanks for your attention, our next plan is to further standardize the process of anomalous explanatory text generation, and its qualitative analysis. Specifically, 1. optimize the text generation strategy: make the generated anomaly explanations more precise and readable by adjusting the prompt engineering or introducing more reasonable templates. 2. enhance the standardization of the qualitative analysis: introduce the qualitative assessment criteria of the text, such as BLEU, CIDEr, METEOR, etc.", "claims_and_evidence": "The authors claim that Ex-VAD outperforms existing methods in both fine-grained and coarse-grained video anomaly detection tasks. The experimental results support this claim, especially in the fine-grained detection task, where Ex-VAD achieves significantly higher mean Average Precision (mAP) compared to other methods on both datasets.", "methods_and_evaluation_criteria": "The design of Ex-VAD is reasonable, integrating multi-modal feature fusion and label augmentation alignment, making it suitable for video anomaly detection tasks. The primary evaluation metrics, including mAP (mean Average Precision) and AUC (Area Under Curve), are standard practices in the field.", "theoretical_claims": "I have carefully reviewed the theoretical claims made in the paper. The authors argue that introducing the text modality can further enhance the capability of anomaly detection, especially for fine-grained anomaly detection. This reasoning is sound and aligns well with the goals of the paper, and the experimental results also validate this conclusion.", "experimental_designs_or_analyses": "The experimental design is reasonable, as it compares both coarse-grained and fine-grained metrics. Although the coarse-grained metric on the UCF dataset is only 0.21% higher than the SOTA method, there is a significant improvement in the fine-grained metric, such as a 3.67% increase on the UCF dataset. These results demonstrate the effectiveness of the proposed method.", "relation_to_broader_scientific_literature": "This method is closely related to the previous VADCLIP approach but differs in that it utilizes LLMs to generate anomaly explanations for videos as textual information rather than relying solely on visual information. Compared to previous methods, this textual information adds more detail to the videos, complementing the visual information and significantly enhancing the capability for fine-grained anomaly detection.", "essential_references_not_discussed": "In addition to the weakly supervised methods and training-free methods mentioned, the authors have not taken into account some recently published approaches, such as those based on LLMs. Discussing these methods would help to clarify whether the explainability of Ex-VAD is effective.", "other_strengths_and_weaknesses": "Strengths:\n+ Ex-VAD combines fine-grained classification with anomaly explanations, filling the gap in explainability of existing video anomaly detection methods. This is particularly important for applications requiring precise responses, such as surveillance and healthcare monitoring.\n+ By integrating text information generated by the AEGM with visual features, Ex-VAD leverages the strengths of both modalities to significantly enhance anomaly detection performance.\n+ The extensive experiments on two benchmark datasets show that Ex-VAD outperforms existing methods in both fine-grained and coarse-grained detection tasks, demonstrating its effectiveness and superiority.\nWeaknesses:\n+ Although Ex-VAD performs well in terms of computational efficiency, its reliance on multiple complex modules (e.g., VLMs and LLMs) may lead to higher resource consumption in practical deployment. The authors could further explore model lightweighting or optimization strategies.\n+ Ex-VAD relies heavily on pre-trained VLMs and LLMs, which may limit its applicability to specific domains or custom datasets. You may investigate how to fine-tune the model with limited data to improve its performance in specific tasks.", "other_comments_or_suggestions": "If possible, please include comparisons with some of the latest methods from 2025.", "questions_for_authors": "+ Could you elaborate on how you specifically selected and optimized the strategy for multi-modal feature fusion during the training process?\n+ Are there plans to extend the Ex-VAD method to other types of datasets or domains in future work, such as anomaly detection and early warning in autonomous driving?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2KTgOqaXdb", "reviewer": "Reviewer_A7T4", "summary": "This paper introduces Ex-VAD, an explainable fine-grained video anomaly detection method based on visual-language models and large language models. By integrating modules for anomaly explanation generation, multi-modal feature fusion, and label augmentation and alignment, Ex-VAD achieves both fine-grained classification and explainability. Experimental results show that Ex-VAD significantly outperforms existing methods on the UCF-Crime and XD-Violence datasets, demonstrating its potential in the field of video anomaly detection.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewers for their valuable comments. We will add these valuable comments to the revised manuscript.\n\n**R1: Robustness of the label.** We use the SOTA large language model GPT to generate M phrases for label expansion and select the top-k among them as the final labels. To ensure the generated labels remain relatively stable, k is set to be much smaller than M. Experiments demonstrate that the best performance is achieved when k=4. This approach ensures a certain level of robustness in the quality of the generated labels.\n\n**R2: Analysis of Anomaly Explanation.** Currently, evaluation methods for video-text tasks are more common in video understanding. Typical quantitative analyses compare the similarity between generated text and reference text using metrics such as BLEU, CIDEr, and METEOR. Additionally, qualitative analysis is conducted by comparing texts generated by different models, where higher relevance to the video indicates better quality. In the appendix, we provide a visualization analysis of our model, which includes interpretable text that, to some extent, reflects the model's understanding of the video. In the final manuscript, we will further supplement this aspect.\n\n**R3: More comprehensive comparison with other recent SOTA methods.** You can see the reply to Reviewer VnC3's “**R4: More methods to compare.**” for a more detailed answer.\n\n**R4: Quantitative analysis of explainable texts.** Quantitative analysis of explainable texts is our next step. Our next plan is to further standardize the process of anomalous explanatory text generation, and its qualitative analysis. Specifically, 1. optimize the text generation strategy: make the generated anomaly explanations more precise and readable by adjusting the prompt engineering or introducing more reasonable templates. 2. enhance the standardization of the qualitative analysis: introduce the qualitative assessment criteria of the text, such as BLEU, CIDEr, METEOR, etc.", "claims_and_evidence": "The authors claim that Ex-VAD not only detects abnormal behaviors but also generates detailed anomaly explanations, which is valuable for applications requiring precise decision-making. The experimental results support this claim, especially in the fine-grained detection task, where Ex-VAD achieves significantly higher mAP on both datasets compared to other methods. However, it is suggested that the authors further quantify the quality of the explanations to more comprehensively demonstrate their interpretability advantages.", "methods_and_evaluation_criteria": "The proposed Ex-VAD method and its evaluation criteria are well-suited for the problem of video anomaly detection. The use of mAP (mean Average Precision) and AUC (Area Under the Curve) as primary metrics aligns with established practices in the field, ensuring that the method is assessed using standard and widely-accepted benchmarks. However, it would be beneficial if the authors could provide a more detailed discussion on the validation of the explainability of the generated text.", "theoretical_claims": "I have reviewed the theoretical claims presented in the paper. The paper demonstrates the effectiveness of Ex-VAD in the direction of fine-grained anomaly detection through experimental results.", "experimental_designs_or_analyses": "I have reviewed the experimental designs and analyses proposed in the paper. The methodology is sound, with appropriate use of the UCF-Crime and XD-Violence datasets, which are relevant and widely recognized within the field. However, I suggest that the authors consider adding additional experiments to further validate the robustness of Ex-VAD under various conditions.", "relation_to_broader_scientific_literature": "The key contributions of this paper are closely related to the broader scientific literature on fine-grained video anomaly detection and anomaly explanation. The paper builds upon previous work by enhancing the capability of fine-grained anomaly detection while also leveraging large language models to generate explanations for anomalies, which is an area of growing research interest. It would be beneficial if the authors could provide a more detailed discussion on how Ex-VAD compares and contrasts with other recent advancements in the field.", "essential_references_not_discussed": "While the paper does an adequate job of citing relevant literature, there may be additional recent works that are essential for a comprehensive understanding of the context of the contributions. For instance, any recent studies on the integration of visual and textual information in AI systems or advancements in explainable AI could provide further context. It would be beneficial if the authors could include these in their discussion to give a more complete picture of the research landscape.", "other_strengths_and_weaknesses": "### Strengths\n1. Ex-VAD not only detects abnormal behaviors but also generates detailed explanations, which is valuable for applications requiring precise decision-making, such as surveillance. This integration of fine-grained classification and explainability is an innovation in the field.\n2. The experimental results on two benchmark datasets demonstrate Ex-VAD's superior performance in both fine-grained and coarse-grained detection tasks, proving its competitiveness in video anomaly detection.\n### Weaknesses\n1. The LAAM module enhances detection performance by expanding label semantics, but this approach may be sensitive to the initial quality of the labels. If the labels are not accurately defined or descriptive enough, performance may degrade. The authors could explore how to maintain robustness when label quality is poor.\n2. While Ex-VAD generates anomaly explanations, the paper lacks quantitative assessment of the quality of these explanations. For example, user studies or comparisons with existing explainable methods could be introduced to more comprehensively demonstrate its explainability advantages.", "other_comments_or_suggestions": "1. It would be helpful if the authors could provide a more detailed discussion on the limitations of Ex-VAD, including scenarios where the model might not perform optimally.\n2. I recommend including a more comprehensive comparison with other recent state-of-the-art methods in video anomaly detection. This would provide a clearer picture of Ex-VAD's advantages and potential areas for improvement.", "questions_for_authors": "1. Could you elaborate on how the quality and usefulness of the generated explanations were quantitatively assessed? \n2. Are there any plans to optimize the computational efficiency of Ex-VAD for real-time applications, and if so, what are the expected improvements in terms of speed and resource usage?\"\n3. What are the next steps in your research after this study? Are there any plans to address the limitations you've identified or to expand the capabilities of Ex-VAD in new directions?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yIHPNCIiYx", "reviewer": "Reviewer_VnC3", "summary": "This paper proposed an explainable approach named Ex-VAD for fine-grained video anomaly detection, which consists of three modules, Anomaly Explanation Generation Module (AEGM), Multimodal Anomaly Detection Module (MADM), and Label Augment and Alignment Module (LAAM). AEGM tries to extract and refine frame-level captions using VLMs, and then generates video-level anomaly explanations with an LLM. MADM encodes the text and extracts temporal and spatial features to detect coarse-grained anomalies. LAAM expands and aligns anomaly category labels with multimodal features to achieve fine-grained anomaly detection. Experimental results verify the effectiveness of the proposed method.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewers for their valuable comments. We will add these valuable comments to the revised manuscript.\n\n**R1: Evaluation of Inference Time.** Thanks for your suggestion. We would like to point out that Table 7 in our main paper provides a comparison of relevant computational metrics, including Trainable Parameters, Inference Time, and Multiply-Add Operations (MACs). Our approach achieves similar inference speeds as previous methods while improving interpretability and fine-grained anomaly detection through VLM and LLM.\n\n|Method|Trainable Params|Inference Time|MACs|UCF-Crime(AUC%)|\n| :- | :-: | :-: | :-: | :-: |\n|RTFM|24\\.72M|**8.28ms**|126\\.59G|85\\.66|\n|DMU|**6.49M**|16\\.60ms|_21\\.00G_|86\\.75|\n|CLIP-TSA|16\\.41M|18\\.33ms|102\\.63G|87\\.58|\n|VADCLIP|35\\.17M|22\\.30ms|29\\.17G|_88\\.02_|\n|ExVAD|_9\\.97M_|_15\\.37ms_|**12.04G**|**88.29**|\n\n**R2: Reasons to choose BLIP2.** The use of VLMs and LLMs is an essential part of the solution to the lack of anomaly classification and anomaly explanations in existing studies. To reduce its required computational cost and resources, we tried various ways to generate captions during the experiment, such as BILP-2, LLAMA3.2, cogVLM, etc. We found that the optimal choice that combines both efficiency and performance is BLIP-2, which generates video captions with lightweight deployment and less time.\n\nRegarding BLIP-2's intermediate features, we tried to integrate them into the model, e.g. by splicing them with video visual features. Experiments show that this method has an accuracy of 88.09% on the UCF-Crime dataset, while the anomaly explanation generated by LLM can achieve 88.29%. Therefore, we finally chose the explanation text generated by LLM which could describe the cause of the anomaly in addition to improving the accuracy by 0.2%.\n\n**R3: Explainable texts are not shown in the main text.** Because of the 8-page limit, necessary experiments (such as accuracy of fine-grained and coarse-grained anomaly detection, ablation experiments, parameter selection, model complexity, and inference time) have taken up a major part of the paper, so we have placed the explainable text in the appendix section. An example showing the logic and results of anomaly text generation is given in Figure 3 in the main text. Additionally, we'll add more explainable text in camera-ready paper.\n\n**R4:** **More methods to compare.** Thanks for your suggestion. We have compared our method with existing explainable video anomaly detection methods such as LAVAD[1], VERA[2], and VADor[3], and we will compare and discuss the suggested method Holmes-vad[4] in the camera-ready paper. \n\nDifferent from Homles-vad which focuses on the coarse-grained anomaly detection task (determining whether a video frame is abnormal), our method can achieve coarse-grained anomaly detection and fine-grained anomaly classification (identify the abnormal category). \n\nAs Homles-vad requires more supervised information, additional introduction of WSVAD [5] to generate pseudo-labels as well as fine-tuning LLM, its coarse-grained anomaly detection results outperform our method. Three main reasons are as follows:\n\n1. Holmes-vad incorporates more supervisory information, performs single-frame temporal annotation in constructing the dataset, and generates interpretable textual descriptions based on it, whereas our method stays consistent with the traditional video anomaly detection methods without any additional annotation.\n1. Holmes-vad is based on the existing WSVAD [5] method which first performs prediction to get the pseudo-labels and then selects the frames with anomaly scores higher than the threshold, which focuses on the keyframes as compared to uniform sampling.\n1. Holmes-vad needs to fine-tune the multimodal large language model and requires more computational resources for training. Experiments demonstrate that our method has only 9.97M training parameters and 15.37ms inference speed. Our method has a better balance between computational resources and effectiveness.\n\n[1] LAVAD: Harnessing large language models for training-free video anomaly detection. CVPR2024\n\n[2] Vera: Explainable video anomaly detection via verbalized learning of vision-language models. ArXiv2024.\n\n[3]VADor: Video anomaly detection and explanation via large language models. ArXiv2024.\n\n[4] Holmes-vad: Towards unbiased and explainable video anomaly detection via multi-modal llm. ArXiv2024.\n\n[5] UR-DMU: Dual memory units with uncertainty regulation for weakly supervised video anomaly detection. AAAI 2023.", "claims_and_evidence": "The claims made in the submission are supported by clear and convincing evidence", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "No theoretical claims included in the paper", "experimental_designs_or_analyses": "The designs of Anomaly Explanation Generation Module (AEGM), Multimodal Anomaly Detection Module (MADM), and Label Augment and Alignment Module (LAAM) are reasonable.", "relation_to_broader_scientific_literature": "results", "essential_references_not_discussed": "[1] Holmes-vad: Towards unbiased and explainable video anomaly detection via multi-modal llm. ArXiv2024.", "other_strengths_and_weaknesses": "Strengths:\n\n1.\tAn intuitive and motivative method is proposed for fine-grained video anomaly detection and can conduct video-level anomaly explanations.\n\n2.\tThis paper is well written and easy to follow.\n\n3.\tExperimental results verify the effectiveness of the proposed method.\n\nWeaknesses:\n\n1.\tThis paper incorporates multiple existing techniques, such as VLM and LLM, to generate explainable text and uses them to improve fine-grained video anomaly detection performance. However, this is an intuitive method, and more model designs or insightful ideas should be included. Additionally, the long pipeline will affect the inference speed. Authors should discuss the inference speed.\n\n2.\tMultiple large and powerful models such as BLIP-2 are used, which makes the comparison with existing methods unfair. Additionally, why can’t use the intermediate features of BLIP-2 to improve the performance? For example, we can concatenate the intermediate features of BLIP-2 and visual features of videos or treat the intermediate features of BLIP-2 as visual features.\n\n3.\tExplainable texts are not showed in the main text. Some cases should be showed to indicate the behaviors of the proposed method.\n\n4.\tExisting explainable video anomaly detection methods such as [1] are not discussed and compared.\n\n[1] Holmes-vad: Towards unbiased and explainable video anomaly detection via multi-modal llm. ArXiv2024.", "other_comments_or_suggestions": "Please refer to the Weaknesses for more details.", "questions_for_authors": "Please refer to the Weaknesses for more details.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_flag": "Flag this paper for an ethics review."}}]

Author rebuttal:

Meta-review: This introduces Ex-VAD, a fine-grained anomaly detection method enhanced by textual explanations from large models. 

Most reviewers appreciated the integration of VLMs and LLMs for richer interpretations and better performance on UCF-Crime and XD-Violence, improved fine-grained results over previous methods. 

However, there were concerns about efficiency and that some preprocessing steps may not included in the reported inference time. Also, folks want wanted broader benchmarking on datasets such as ShanghaiTech (partially addressed by the team) and UCSD-Ped2. 

The authors attempted to address these points by clarifying where processing times are accounted for and adding partial experiments on ShanghaiTech. Although one reviewer remains unconvinced about practical overhead and reliability, the majority view is that the work offers a useful step toward more interpretable video anomaly detection. I am leaning to accept.
Decision: Accept (poster)