Leveraging Multimodal LLM Descriptions of Activity for Explainable Semi-Supervised Video Anomaly Detection
Authors: Furkan Mumcu
Michael J. Jones
Anoop Cherian
Yasin Yilmaz
Venue: Accepted by TMLR ()

Abstract: Existing semi-supervised video anomaly detection (VAD) methods often struggle with detecting complex anomalies involving object interactions and generally lack explainability. To overcome these limitations, we propose a novel VAD framework leveraging Multimodal Large Language Models (MLLMs). Unlike previous MLLM-based approaches that make direct anomaly judgments at the frame level, our method focuses on extracting and interpreting object activity and interactions over time. By querying an MLLM with visual inputs of object pairs at different moments, we generate textual descriptions of the activity and interactions from nominal videos.  These textual descriptions serve as a high-level representation of the activity and interactions of objects in a video.  They are used to detect anomalies during test time by comparing them to textual descriptions found in nominal training videos.  Our approach inherently provides explainability and can be combined with many traditional VAD methods to further enhance their interpretability. Extensive experiments on benchmark datasets demonstrate that our method not only detects complex interaction-based anomalies effectively but also achieves state-of-the-art performance on datasets without interaction anomalies.

Published in Transactions on Machine Learning Research (03/2026)
Leveraging Multimodal LLM Descriptions of Activity for Explainable Semi-Supervised Video Anomaly Detection
Furkan Mumcu"
furkan Brusf. edu
Department of Electrical Engineering University of South Florida
Michael J. Jones
mjones@merl.cm com
Mitsubishi Electric Research Laboratories (MERL)
Anoop Cherian
cherian@merl.com
Milsubishi Electric Research Laboratories (MERL)
Yasin Yilmaz
yasinySusf edu
Department of Electrical Engineering University of South Florida
Reviewed on OpenReview: /ttps://openreeet net/ forum? id= dfc2HpDSLH
Abstract
Existing semi-supervised video anomaly detection (VAD) methods often struggle with de- tecting complex anomalics involving object intoractions and generally lack explainability. To overcome these limitations, we propose novel VAD framework leveraging Multimodal Large Language Models (MLLMs). Unlike previous MLLM-based approaches that make direct anomaly judgments at the frame level, our method focuses on extracting and in- terpreting object activity and interactions over time. By querying an MLLM with visual inputs of object pairs at different moments, we generate textual descriptions of the activ- ity and interactions from nominal videos. These textual descriptions serve as a high-level representation of the activity and interactions of objects in video. They are used to detect anomalies during test time by comparing them to textual descriptions found in nom- inal training videos. Our approach inherently provides explainability and can be combined with many traditional VAD methods to further enhance their interpretability. Extensive experiments on benchmark datasets demonstrate that our method not only detects com- plex interaction-based anomalies effectively but also achieves state-of-the-art performance on datasets without interaction anomalies.
1 Introduction
Video anomaly detection (VAD) has emerged as an active research field due to its significant applications in security and public safety. The rapid growth of surveillance vidco data makes it impractical for humans to monitor thoroughly. Consequently, VAD algorithus play a vital role identifying unusual events in video footage, enabling human operators focus their attention on reviewing these flagged incidents. In this paper, we focus on the formulation in which nominal videos (also called training videos) containing only normal activities in particular scene provided for learning model. The goal is to identify when and where anomalous activities happen within test. videos of the same scene. Along with other papers (Baradaran & Bergevin 2024 Wu et al. 2024a), we refer to this version of the problem as semi-supervised VAD because of the availability of normal, but not anomalous, training video. In Table [ we summarize the properties of some recent papers in this area to highlight the differences between our work and other recent approaches.
"Furkan Mumcu did part of this work intern MERL

---

Published in Transactions on Machine Learning Research (03/2026)
Existing semi-supervised video anomaly detection methods suffer from several weaknesses. Recent studies struggle to provide explainability for detected anomalies. While some existing work (Singh et al. 2023; 2024) explored more explainable approaches by interpreting the input features, they do not offer direct, textual explanations. Furthermore, the ComplexVAD dataset (Muneu et al. 2025b) introduced interaction- based anomalies and demonstrated that detecting these anomalies even more challenging than identifying the simpler anomalies presented in earlier datasets. Hence, we propose new method to address the drawbacks present in existing works, including the lack of focus complex anomalies and explainability. By utiliz- ing a Large Language Model-based detection approach, we introduce an effective video anomaly detection framework that excels at identifying complex anomalies and features built-in explainability.
Large Language Models, especially in their multimodal forms (MLLMs), have demonstrated strong capa- bilities in interpreting and accurately describing complex situations from both text and images. GPT-4 with vision (Achiam al., 2023), for example, can analyze visual inputs and produce detailed, contextually relevant descriptions. Existing work has explored the use of MLLMs for vidco anomaly detection. Most of these methods are designed for mulli-scene, weakly supervised setting. While such approaches are effective for detecting common set of anomalics across multiple scenes, they fall short in identifying scene-specific anomalies, which are central to the semi-supervised, single-scene VAD task. To address this limitation, we propose a novel MLLM-based framework that: (1) considers object interactions in addition to individual object activities, (2) leverages an MLLM to generate high-level descriptions used an exemplar-basod model that can be casily extended without retraining deep networks for each new scene, and (3) enables both spatial and temporal localization of anomalies.
In our method, we adopt an object-level feature extraction strategy similar to (Mumeu et al, 2025b) using object detector and tracker. We then employ an MLLM agent to describe object activities and interactions by querying it with pairs of frames, separated in time by fixed interval, to capture temporal information. A key novelty is using these MILLM-generatr textual descriptions directly as features for modeling normal activity. By removing redundant descriptions (Section 3). we construct a final set of textual exemplars representing nominal videos. This exemplar set is then used detect anomalies: descriptions from anomalous interactions are expected to deviate from nominal ones. In summary, our contributions are:
We introduce the first Massed approach for video anomaly detection specifically designed to identify complex anomalics caused by object interactions.
We propose novel use of MLLMs for VAD: instead of directly deciding the presence of an anomaly, as done in prior work, our method first derives representation (using MLLM descriptions) of what is normal in scene and then identifies anomalies based on deviations from this normal representation.
We demonstrate that our method offers built-in explainability. Furthermore, we show that it can be integrated with many existing VAD approaches to enhance their interpretability.
We evaluate our method on multiple benchmark datasets and show that it achieves state-offthe-art performance.
2 Related Work
Many different formulations of the video anomaly detection (VAD) problem have been studied in recent
years: semi-supervised (Hasan et al., 2016 Ionescu et al.] 2019 Liu et al. 2018 Wang et al.) 2021). weakly
supervised (Sultani et al.] 2018). unsupervised (Zaheer et al. 2022 training-free (Zanella el. al. 2024).
single-scene (Ramachandra et al. 2020b) and multi-scene (Luo et al. 2017 Each of these formulations has applicability in different scenarios. This paper focuses on the singlo-scone, semi-supervised version of vidco anomaly detection because it corresponds to the most common use case that we have observed in the real- world; namely, that of a surveillance camera at a particular location in the world in which video of normal activity abundant but video containing anomalies is rare and expensive to collect. An important attribute of the single-scene VAD problem formulation is that anomalies are scene dependent, i.e., what is anomalous in one scene may be normal in another. This is in contrast to the multi-scene problem formulation in which
2

---

Published in Transactions on Machine Learning Research (03/2026)
Foahures
Task Objootivve
Method
Foeus
MI..M
Interaation
Explain- ability
Seerie Supervised Specific
Usage Ancmalless
VačCLIP
Wu et
21244
Using CLIP for mulii-ocnee wealdy supervised VAD
Q
Vera
Oplinising prouple for MLLMM-beod weskly sugervisdd VAD
Holmes.
VAO
Arang
2025
MLL.M fineraming for anomaly understancingg armes rudiissemme
AnomalyRulerr
Wi ds
Seeme-spocific role generation and frame-level deseriptions vin MLLMMs
VLAVAD
L
& latu
212
Using framg level MLLM descriptions for seuissapervied VAD
EVAL
Staph of
Using object level fecturs for explainable VAD
T-EVAL
Singh es
Objeet-level, tracklet- heard expluinable ble VAD
12
Scene-Graph
Murnc
2025b
Exemplar and graph for interaction-based VAD
Ours
Object-level MLLM descriptions for interaction- VAD
Table 1: Summary of properties and differences of some recent work in VAD. Our work is the only one designed for semi-supervised, scene-specífic VAD that handles interaction anomalies and provides textual explanations of anomalies.
anomaly types are common across different scenes. Finally, the single-scene VAD formulation typically contains anomalies that are pocation-eppedent (e.g. pedestrian may be anomalous in some locations but not others). Such location-dependent anomalies do not occur in multi-scene VAD because they require scene-specific models. These differences imply that the multi-scene version of VAD is not a generalization of
single-scene VAD (Ramachandra et al. 2020b). Solutions for one version generally do not work well on the other version.
Early work on the single-scene, semi-supervised version of VAD includes many papers based on frame re- construction (Hasan et al.] 2016 Tonescu et al.] 2019) or frame prediction (Liu et al.) 2018 Wang et al. 2 One big disadvantage of such approsches is that they require training deep network on video from each new scene. To avoid this disadvantage, others (Doshi & Yilmaz 2020b Ramachandra et al., 2020a Ramachandra & Jones 2020 Singh et al. 2023: 2024 Mumcu et al. 2025b) used an exemplar-base model that extracts a set of representative features from normal video but does not require any deep network training for each scene. We also use this idea in our work.
2.1 MLLMs in Video Anomaly Detection
Recently, researchers have explored ways of taking advantage of large language models vision language models for VAD. These approaches, however, vary significantly depending the VAD problem formulation.
Weakly-Superrvised and Training-Fris Free VAD. Many approaches to VAD that use an MLLM try to use its built-in knowledge of what is normal instead of learning normality from normal training video Zanella et al. 2024 Ye et al. 2025 Lv & Sun 2024).. A disadvantage of such approaches is that they cannot handle scenarios such as a boxing gym in which activity that is typically rare and anomalous (fighting) is actually very common. Another important difference between our work and most other MLLM-based approaches to
VAD is that others have focused on the weakly supervised, multi-scene version of VAD (Li et al. 2025 Wu
et al.
2024b Zanella et al. 2024 Ye et al. 2025 Lv & Sun 2024
setting, the capabilities of MLLMs are under- studied in the semi-supervised setting. Wu et al. (2024c) addresses semi-supervised, single-scene VAD, but with a fundamentally different. methodology. Their approach uses an MLLM to first generale a set of textual rules from normal video and then applies these rules during inference. In contrast, our method directly compares MLLM-generated text descriptions between normal and test vidcos without an intermediate rule- generation step. Furthermore, their model processes full video frames, whereas we employ an object-centric approach that analyzes regions around objects and their pairs. This key design choice allows our method to model object interactions and perform both spatial and temporal localization, while their approach is limited to temporal localization only.
Seni-Supervised VAD. Compared to the weakly
Li & Jiang (2024) also focuses on semi-supervised single scene anomaly detection and proposes an ML.MM- based method. Like our method, they first detect and track objects and then query an MLLM yield high-level descriptions of objects and their activity. However, they do not model object interactions, and do not use an exemplar-based model.
3

---

Published in Transactions on Machine Learning Research (03/2026)
In real-world scenarios, many anomalies are characterized by an unusual interaction between two objects. Therefore, one of the important aspects of our method is its modeling of object interactions in the scene to identify anomalous intoractions. There have been a few other mothods that also modcled interactions (Chen et al. 2018 Doshi & Yilmaz, 2023; Sun et al.; 2020; Szymanowicz el al. 2021 Mumeu el al., 2025b). While there are many differences among these methods, one of the main differences compared to our current work is that none of them take advantage of MLLMs. In our approach, the textual descriptions of object activity and interactions provided by an MLLM yield powerful cues for a high-level understanding of a scene. Language provides a natural abstraction for compositional structure and relational semant (e.g.; interactions, roles, and intent), which are difficult to capture reliably with low-level features alone. In the one-class setting, this abstraction enables a compact and human-aligned representation of normal behavior, where deviations correspond lo semantically meaningful changes rather than pixel-level noise.
3 Our Approach
We propose a novel method to detect anomalics in vidco. The pipcline of our method (which we call MLLM- EVAD, for MLLM-based Explainable Video Anomaly Detection) is depicted in Figure [ The main idea is to use a set of textual descriptions of the activity seen in normal training video of a scene as the model of normality. These textual descriptions are obtained using an MLLM. Anomalies in testing video are found by comparing textual descriptions of the activity in testing frames with the set of normal descriptions. A test description that is dissinilar from all normal descriptions indicates an anomaly. The initial step of our method is detecting and tracking objects followed by identifying pairs of objects that are in close spatial proximity and therefore likely to be interacting. Then, for each frame and a future frame offset by a fixed time interval, we generate crops around pairs of interacting objects as well as single objects that are not close to other objects. The crops are given to an MLLM to obtain natural language descriptions of objects' activities. These descriptions are embedded into vector space using a sentence embedding model. By applying exemplar selection algorithm to all of the sentence embeddings found in nominal training video, we construct representative sets of embeddings for both object pairs and single objects. Anomaly detection is performed by comparing sentence embeddings extracted from testing frames to the exemplar sets, and assigning anomaly scores based on their distance to the most similar exemplar.
3.1 Object Detection and Tracking
In our proposed method, we want to build a model of what objects do and how they interact in video containing normal activity. This requires detecting objects and tracking them across frames in training videos. We also find pairs of objects that are likely to be interacting according to their spatial proximity.
A video V is a collection of M frames {Fi)M,, such that = [Fi, F2,..., FM]. Each frame F; is sent to the object detector O, which returns X number of detected objects. The object detector provides, for each object o, the location = (I.. Uo) which is the x and y coordinates of the center of the object, b = (w.. ho) which is the width and height of the bounding box for the object, and class id The output of the object detector is then OFF)= = [01, 0x.-..,x)] ox], where each object og is represented by o = [b. c, 1]. After detecting objects in a frame, they are then tracked using an object tracker. Each detected object o is sent to the object tracker, which returns æ and y coordinates for that object in the subsequent frames. In our method, we track objects for 30 frames. Therefore, for every object, we acquire the trajectory 0 {I1 91), (z2, (22..2)).... (x.0. y30)).
Next, we pair objects according to their spatial proximity in the frame. To determine which objects should be paired, we need to calculate the 3D distances between each pair of objects. To calculate the 3D distance we need to derive the 3D coordinates of the object locations by estimating a pseudo-depth since we do not have access to actual depth measurements. Given two objects 0n and 02, we have 2D coordinates 4 = (z V1 and l/ = (x2, y2). Then we define a relative depth, 2, between two objects by taking the absolute difference of y values such that 2 1y - yzl- This estimate of pseudo-depth assumes that objects are resting on the ground plane and the ground plane is farther from the camera the closer it is to the top of the image. The 3D distance d can then be calculated by taking the Euclidean distance between 3D coordinates (11.81.2) 2) and (12, y2, 0). Any objects that have 3D distance d smaller than a predetermined threshold h are paired.
4

---

Published in Transactions on Machine Learning Research (03/2026)
Object Desection and Tradiing
Visual Inou:: Crop of two anjects enclssed by bounding boses, from two frames time rand 1+30.
Prompt: Branfly describe what the people in the endosed regiona of these images doing The two
Video Framers
MILLM
Answerr sentence Two people are walking side by side the croeswellk witheut interacting.
Soore calculction: Eq (1) Eq,
Text Encoder
Embadding of
Exemplar sot of embedcings
Training mode:
Inference mode: Use score for anomaly dotaction.
If acorn greater than the threchold add embedding of examplar sot.
The pipeline of our method (MLLM-EVAD) for frame textual description generation with the help of an object detector, object tracker and MLLM. Please the text for explanation.
Figure 1:
Due to applying the threshold, not every single object necessarily connected to another object, which leads to having single objects in addition to object pairs.
We use temporal gap of approximately one second between the paired frames as practical trade-off between capturing meaningful temporal changes and maintaining object. identity consistency in a static- camera, single-scene setting. Shorter temporal gaps often result in nearly identical visual content and textual descriptions with limited motion cues, while significantly larger gaps increase the risk of object drift, missed interactions, changes in object visibility. Importantly, the proposed framework is not tightly coupled to a specific temporal gap; the paired-frame formulation serves lightweight mechanism for incorporating local temporal context, and similar behavior can be captured with other reasonable temporal offsets depending the scene dynamics.
3.2 Generating Textual Descriptions
At the end of the object detection and tracking stage, for each frame F,, we obtain a set of object pairs and a set of single objects. For cach of these, we generate two cropped regions that capture the relevant spatial area for both the current. frame F, and future frame Fi+a (e.g., A = 30), assuming a static camera (as depicted in Figure D.. This ensures that both crops contain the same background and object regions, allowing for consistent comparisons across time. We provide the details of the cropping process in the Appendix. Red reclangles are drawn around each object in the cropped images using the bounding boxes delected by the object detector. The rectangles indicate to the MLLMM which objects are being referred to in the prompt.
We then obtain a sentence r that describes the interaction between objects and 02 by querying an MLLM agent with the corresponding image crops C and Ci+A, along with the user prompt: "Briefly describe what the <object name>s in the enclosed regions of these images are doing. The two images were faken one second apart." " For single-object cases, the prompt is adjusted to: "Briefly describe what the <object name> in the enclosed region of these images is doing. The two images were taken second apart." In both prompts, the <object name> placeholder is replaced by the class label obtained during feature selection.

---

Published in Transactions on Machine Learning Research (03/2026)
An additional "system" prompt is also used with the MLLM: "You will be provided with two frames from video and asked to describe what objects that are indicated by bounding bozes in the video are doing. Your task is to answer the query in a simple sentence. If there is any interaction between the indicated objects, a descriplion of the interaction should given"
The MLLM-generated responses are then used in both the model building and anomaly detection stages.
We note that our use of image-based multimodal language models is deliberate design choice. During development, explored video-language models that process short clips or frame sequences; however, we observed that such models tend to produce coarse, scene-level descriptions that are less sensitive to fine- grained object interactions and localized activity changes. In contrast, qucrying an image-based MLLM with paired object-centric crops from lemporally separated frames provides sufficient temporal context while preserving spatial grounding and object identity. This design aligns naturally with our goal of scene-specific modeling of normal behavior through localized object interactions in semi-supervised setting, rather than general video understanding.
3.3 Model Building
We follow similar exemplar selection strategy as introduced in recent works (Ramachandra & Jones, 2020; Singh et al. 2023 2024 Mumeu et al., 2025b). For all frames in video, we collect all textual descriptions generated from object pairs into one set and all textual descriptions generated from single objects into another set. Then for each of the sets we run an exemplar selection algorithm which selects a subset of the elements of the set such that no two members of the subset are near each other according to a distance function. The intuition behind exemplar selection is to simply remove redundant (or nearly redundant) elements from the set Icaving behind compact, representative subsot of exemplars. Various distance functions between sentences can be used to compare textual descriptions. In most of our experiments we use Sentence-BERT Reimers & Gurevych 2019), but we also test using BLEU (Papineni et al. 2002) and METEOR (Banerjee & Lavie 2005).
Given a set S, the exemplar selection algorithm proceeds as follows: (1) Initialize the exemplar set to NULL. (2) Add the first element of S to the exemplar set. (3) For each subsequent element of S, find its distance to the nearest instance in the exemplar set. this distance above a threshold, th, then add the element to the exemplar set.
Since we obtain textual description for each object pair or single object, we transform these descriptions into sentence embeddings using a Natural Language Processing (NLP) model N, such that 4, = N(r). The distance D between two textual descriptions, and r2, is then computed as:
D(11, = - cos (Ir, Ir.)
(1)
where cos(..-) denotes the cosine similarity between the corresponding embeddings. Using the distance function D and the described exemplar selection algorithm, construct two separate exemplar sets: Eppir and Esingle: containing sentence embeddings of textual descriptions for object pairs and single objects, respectively.
3.4 Anomaly Detection in Test Video
After constructing the exemplar sets from nominal videos, the next step is to detect anomalies in test video from the same scene. Similar to the model-building phase, we extract textual descript ions and encode them into sentence embeddings I,., The anomaly score for a given object pair (or single object) is then computed based on its similarity to the corresponding exemplar sot. Specifically, the score is defined as:
score(r)
max (cos (Tr, e)), e E Epuair or Fringle
(2)
depending whether the imput is an object pair single object. A higher score indicates greater deviation from the nominal descriptions and is thus more likely to be anomalous.

---

Published in Transactions on Machine Learning Research (03/2026)
3.5
Combination with Previous Methods
While the textual descriptions generated by an MLLM provide powerful high-level features for modeling the normal activity in a scene and for detecting anomalies in the scene, some anomalies require more detailed, fino-grained features for detection. For this reason, it can be advantageous to combine our method with previous methods that use more detailed features such as object sizes and trajectories. Combining with another method is especially easy to do if the other method is also object- based and uses an exemplar- based model.
In our experiments, we show results when combining our method separately with two other methods; namely, the scene-graph method of Muucu et al. (025b) and the tracklet method of (ingh et al. (2024).
Our method of modeling interactions was inspired by the scene-graph method of Mumcu et al. (2025b) Their method builds a scene graph for each frame where each object is a node and objects that are nearby and, therefore possibly interacting, are connected by an edge. The attributes stored at cach node are the object's location, size, object class, trajectory and skeletal pose. Then an exemplar-based model is created for every connected pair of nodes across all scene graphs in all nominal frames. A separate exemplar- based model is also created for all singleton nodes across all scene graphs in all nominal frames. To combine our method with theirs, we can simply add the MLLM-generater textual descriptions to each connected pair of nodes and to each singleton node. The distance function used to compare nodes takes the maximum over all attribute distances including the distance between textual descriptions.
In the case of the tracklet method of [Singh et al. (2024), objects in each frame are also detected and tracked. Similar to Mumeu et al. (025bb), for each object, a set of attributes are stored including the location, size, object class and trajectory. An exemplar-based model is then learned from all of the objects found in the nominal training videos using distance function for comparing objects that takes the maximum over each attribute distance. To combine our method with theirs, we again only need to add our textual descriptions as another attribute associated with each object. In this case, object interactions are not modeled. We use this combination to test on datasets that do not contain anomalous object interactions.
4 Experiments
In our experiments, we use the following configurations:
Datasets:
We evaluate MLLM-EVAD on three video anomaly detection benchmark datascts which are all publicly available under a CC-BY-SA-4.0 license. ComplexVAD (Mumcu et al. 2025b) is a large-scale dataset with a total of over 3.6 million frames, with 2,069,941 frames for training and 1,611,49) 1,1,,997 for testing. It features interaction- anomalies and serves as our main focus, since our method leverages descriptions of object activity and interactions for video anomaly detection. The scene features a crosswalk, pedestrian sidewalks, and two-lane street, which are heavily used. Tt demonstrates interaction anomalies such person leaving an object on the ground, person sitting car, a dog walking without person leash, where the individual objects are normal, but their interactions are anomalous.
In addition, we evaluate performance on the Avenue and Street Scene datasets. Avenue (Lu et al. 2013) contains short surveillance vidcos of a campus walkway, with anomalics such as running, throwing objects, or walking in the wrong direction. Street Scene (Ramachandra & Jones] (020) includes longer videos of a city street environment, where anomalies involve vehicles or pedestrians behaving unusually, including cars driving the wrong side of the road, pedestrians crossing at non-designated arcas, or bikers riding on the sidewalk. Avenue contains 30,652 frames, split nearly evenly between 15,328 training frames and 15,324 testing frames. Street Scene is a larger dataset with a total of 203,257 frames, of which 56,847 are used for training and 146,410 for testing.
Implementation Details:
We use Detectron2 (Wu et al. 2019 and ByteTrack /Zhang et al. 2022
as the object detector and object tracker our method. Sentence-BERY (Reimers & Gurevych 2019) (using the pretrained weights of oL-- variant) is used as the text embedding model to encode used as the multi-modal MLLM agent.
activity descriptions. Initially, GPT-4o (Hurst et al. 2024)
However, with the recent introduction of Gemma 3 TTam et al. 2025). we re-conducted the experiments

---

Published in Transactions on Machine Learning Research (03/2026)
Method
RBDC
TBDC
Frame
MemAE
Gong et al.
2019
0.05
0.0
58.0
EVAL Singh et 2023)
10.0
62.0
54.0
AnomalyRuler (Wu et al.
2024c)
N/A
N/A
56.0
Scene-Graph (Mumcu et al.
2025b)
19.0
64.0
60.0
MLLM-EVAD
24.0
68.0
61.0
Scene-Graph + MLLM-EVAD
25.0
70.0
63.0
Table 2: Results on the Complex VAD dataset. Each entry is the area under the curve (AUC) as percentage for the three benchmark methods using the RBDC, TBDC and Frame- evaluation criteria. Bold face indicates the best score and blue indicates second best.
Method
RBDC
TBDC
Frame
Auto-encoder (Hasan et al.
2016
0.3
2.0
61.0
Dictionary method (Lau al.
2013)
1.6
10.0
48.0
Flow baseline
Ramachandra & Jones
2020
11.0
52.0
51.0
FG Baseline RRamachandra & Jones
2020
21.0
53.0
61.0
EVAL (Sighh et 2023
24.3
64.5
N/A
Contextual CMM Yang & Radke 2025
34.0
62.5
67.0
T-EVAL Singh al.
2024
30.9
72.9
66.0
T-EVAL MLLM-EVAD
31.1
73.5
66.8
Table 3: Results on the Street Scene dataset using the area under the curve (AUC) (as percentage) for the RBDC and TBDC evaluation criteria for different hods Bold face indicates the best score and blue indicates second best.
ComplexVAD using Gemma 3. Hence, the results on ComplexVAD VAD are reported with both Gemma 3 and GPTT-4o, while the results on the other datasets are reported with GPT-4o. We choose a threshold th = 0.65 for exemplar selection, which rosults a modest number of total exemplars while maintaining strong detection performance. As observed in prior exemplar-based VAD methods, the choice of th primarily affects model size rather than test accuracy, with performance remaining stable across a reasonable range of threshold values (Singh et al. 2023 2024). In our framework, exemplar selection serves to remove near- duplicate descriptions and construct compact reprosentation of normal behavior, and anomaly detection relies relative distances to this representation rather than precise threshold tuning. We provide additional implementation details in the Appendix.
Evaluation Criteria: We use the Region- Based Detection Criterion (RBDC) and the Track-Basel De- tection Criterion (TBDC) as proposed
(Ramachandra & Jones 2020) as our primary evaluation criteria and report the arca under the curve (AUC) for false positive rates per frame from to 1. We also report frame-level AUC (Mahadevan et al. 2010) scores. As highlighted in previous works (Ramachandra & Jones 2020) frame-level AUC only evaluates temporal accuracy and disregards spatial localization of anomalies. RBDC and TBDC measure a method's capacity to accurately identify anomalous spatio-temporal regions within a given video sequence.
4.1
Quantitative Results
The results of our method on ComplexVAD, as well as those of EVAL (iiggh et al. 2023 MemAE (Gong ct al. 2019), AnomalyRuler (Wu et al., 2024c), and Scene-Graph Mumcu et al. (2025b). using the three evaluation criteria described above, are reported in Table 2 We also provide the result for the combination of our method and Scene-Graph on the Complex VAD dataset. Our MLLM-EVAD method outperforms the next best method (Scene-Graph) by 5, and percentage points in RBDC, TBDC, and frame-level evaluations, respectively. We achieve the best result by combining the Scene- Scene-Graph method with ours, reaching 25%, 70%, and 63% in RBDC, TBDC, and frame-level evaluations, respectively. AnomalyRuler,
8

---

Published in Transactions on Machine Learning Research (03/2026)
Method
RBDC
TBDC
Frame
Siamese AE
RRanachandra et al.
2020a
41.2
78.6
87.2
Hybrid AE
(Liu
2021
41.1
86.2
91.1
BG-Agnostin [Georgescu et al.
2021
65.1
66.9
92.3
Hybrid AE
Liu et al.
2021
PCAB Ristea al. 2021
62.3
89.3
90.9
BG-Agnostic
Georgescu al.
2021 + PCAB
Ristea et al. 2021)
66.0
64.99
92.9
SSMTL++vl
Barbalau et al.
2023
40.9
82.1
93.7
S
Barbalau et al.
2023
47.8
85.2
91.6
AnomalyRuler (Wu et al.
2024c)
N/A
N/A
89.7
EVAL Singh et al.
2023
68.2
87.6
86.0
T-EVAL Singh al.
2024
67.5
89.7
88.0
T-EVAL + MLLM-EVAD
68.9
90.1
88.4
Table 4: Results the CUHK Avenue dataset using the area under the curve (AUC) percentage for
the RBDC and TBDC evaluation criteria for different methods. Bold face indicates the best score and blue indicates second best.
Anomaly
The person holding a phone and appearing take video while being pushed large box by another person.
The two people are walkingg across the crosswalk, with the stumble jump.
Closest Exemplas lar
The person walking along sidowalk
Two people are walking the crosswllk the
Anomaly
person prouching down on the ground.
being pushed forword by another person is riding sconter.
Closest Exemplar
The person is walking grass.
cyclst nding blcycle alongside them.
Figure 2: Examples of anomalies in the Complex VAD dataset that are delected by our method. In each box, the top row shows two frames the anomalous video clip followed by the textual description generated by Gemma The sccond row shows two frames from the closest matching exemplar to the anomaly. The textual description for the closest exemplar is also shown. By contrasting the anomalous description with the closest normal description, it is clear why the testing video clip was found to be anomalous. The nominal description shown corresponds to the closest exemplar in the learned normal set and is presented for contrast; the anomaly is detected because its description is dissimilar from all nominal exemplars, and this semantic distance determines the anomaly score.
recent MLLM-based method Wu al. 2024c), achieves 54% frame-level AUC. However, were not able to obtain results for the RBDC and TBDC criteria, this method does not support spatial localization of anomalies. The MemAE method does very poorly for the two criteria that measure spatial localization. This implies that the regions of an image that MemAE predicts as anomalous are usually normal.
For the experiments on Avenue and Street Scene, we combine our method with the best-performing existing exemplar-based method, namely Tracklet EVAL (Singh et al. (024)). These datasets include many anomalies that need more fine-grained attributes (such as the direction and speed of travel for people and cars), the high-level textual descriptions used by MLLM-EVAD are not sufficient on their own to detect all of

---

Published in Transactions on Machine Learning Research (03/2026)
Method
Explanation Quality (t)
Human-written
4.2 + 0.7
MLLM-EVAD
3.8 + 1.1
Table 5: Results of the human evaluation of explainability. Scores are reported as mean + standard deviation of 5-point Likert ratings, aggregated across evaluators and anomaly clips.
the anomalies. We show that our textual descriptions do add an important new attribute, however, since, when combined with the more fine-grained attributes of Tracklet EVAL, the results with the MLLM textual descriptions improve accuracy for both datasets across all three evaluation criteria.
As shown in Table 4 on the Avenue dataset, the Tracklet-EVAL + MLLM-EVAD method improves the state-of-the-art for both the RBDC and TBDC criteria. On Street Scene (results shown in Table 3), the state-of-the-art is surpassed for the TBDC criterion. Since the frame-level criterion does not measure spatial localization accuracy which is the focus of these datasets, we would argue that the slightly-bloow SOTA frame-level numbers are not representative of the true accuracy of these methods. We should note that we were not able to exactly reproduce the results reported for Tracklet EVAL (Singh et al. 2024) on the Avenue dataset, most likely due to different, parameters, so we report the results we could reproduce for that method and use that version in combination with MLLM-EVAD for the results on the combined method.
All quantitative results reported in Tables 2 4 and 3 are computed using raw, unsmoothed anomaly scores. Although temporal smoothing (e.g., Gaussian filtering) is commonly used for frame-level AUC evaluation, we do not apply smoothing in our main comparisons to ensure consistency across methods. For completeness, we additionally report noussian-smootlle level results for our method on ComplexVAi VAD in the appendix as supplementary post peot-processing analysis.
4.2 Qualitative Results: Explainability
In addition to the quantitative results, we present qualitative results to demonstrate the explainability capabilitics of our method. The combination of using an MLLM agent and an exemplar selection strategy enables intuitive explanations for detected anomalies by providing descriptions of both the anomaly and the most similar event from the nominal training video. In Figure 2 we demonstrate four detected anomalies from CompleXVAD along with their textual descriptions and the textual descriptions of the closest exemplars, as generated by our method. For example, one detected anomaly is described as "The person is crouching down on the ground." The closest matching exemplar from the nominal set is "The person is walking across the grass." " The action of crouching deviates from the typical behavior observed in the training data. This difference highlights how our method leverages semantic understanding to identify uncommon activity, offering an interpretable explanation for the anomaly.
Another example involves an anomaly described as "The person is holding a phone up and appearing to take picture video while being pushed in a large box by another person." The closest matching exemplar from the nominal set is "The person is walking along the sidewalk.", which is, in fact, the most common activity involving a person in this dataset. The anomalous event, a person being pushed in a box, does not occur in the nominal video and represents an unusual and complex behavior. This example further demonstrates our method's ability to detect rare interaction patterns that deviate from typical activities in the scene. In addition, these examples illustrate how our method also provides reasonable explanations for detected anomalies. By leveraging doscriptive comparisons with nominal exemplars, our method enables enhanced and explainable video anomaly delection.
As discussed in Section 5] the absence of ground-truth textual benchmarks makes quantitative evaluation of explainability difficult in the semi-supervised, single-scene VAD setting. To complement our qualitative analysis under this constraint, we perform a small-scale human evaluation of the generated explanations. We randomly select 10 anomalous video clips from ComplexVAD and, for each clip, consider two textual explanations: (i) a human-written annotation and (ii) the description generated by our system. For each clip, evaluators are shown the video and the two anonymized descriptions (labeled as "Description A" and "Description B*) presented in randomized order, without revealing their source, Evaluators rate each
10

---

Published in Transactions on Machine Learning Research (03/2026)
Method
RBDC
TBDC
Frame
Sentence-ERRT
24.0
68.0
61.0
BLEU
17.0
69.0
58.0
METEOR
16.0
69.0
57.0
Table 6: The table reports results on the ComplexVAD datasets using the area under the curve (AUC)
percentage for the RBDC, TBDC and Frame-Tevel evaluation criteria for the our method using 3 different sentence distance functions.
MLLM
RBDC
TBDC
Frame
GPT-4o
19.0
67.0
59.0
Gemma 3
24.0
68.0
61.0
Table 7: The table reports the performance of our proposed method on the ComplexVAD datasets using the area under the curve (AUC) as a pereentage for the RBDC, TBDC and Frame-Level evaluation criteria with different MLLM agents GPT-4o and Gemma 3.
description on a 5-point Likert scale according to the following question: "How well does this description explain why the event in the video is anomalous?" (1: not at all informative, 5: extremely informative). We collect ratings from independent evaluators and report the mean and standard deviation of the scores aggregated across clips and evaluators in table The results show that the explanations generated by MLLM- EVAD are consistently rated as informative and closely aligned with human-written explanations, supporting the interpretability claims of our approach. All human-written and MLLM-generated explanations used in this study are provided in Appendix D for qualitative inspection.
4.3 Ablation Study
We conduct two ablation studics by modifying the default settings of method. In the first ablation study, instead of using sentence embeddings and cosine dissinilarity the distance metric, we use the raw sentences and compute distances using BLEU (Papineni et al. 2002) and METEOR (Banerjee & Lavie 2005). We emphasize that BLEU and METEOR are used here solcly as alternative sentence similarity functions within the anomaly detection pipeline; no ground-truth textual annotations are involved, and evaluation is performed using standard VAD metrics. In Table we compare the results of cosine distance over Sentence- BERT embeddings with those obtained using BLEU and METEOR, based on the descriptions generated by method using GPT-4o as the MLLM agent. The results show that Sentence- BERT embeddings yield slightly higher accuracy in the RBDC and frame-level criteria while BLEU and METEOR perform slightly better on the TBDC criterion. Sentence-BEil BERT is the preferred similarity function considering its greater efficiency due to embeddings which can be saved and only require dot product for comparison.
In the second ablation study, we compare our method's performance on ComplexVAD using different MILM agents, namely Gemma 3 and GPT-4o. Table we show that Gemma achieves better performance, with 24%, 68%, and 61% in the RBDC, TBDC, and frame-level criteria, respectively, while GPT-4o yields 19%, 67%, and 59%. We attribute this difference to the fact that Gemma 3 generates more detailed and descriptive sentences compared to GPT-4o. This level of detail is particularly important for detecting interaction- anomalies, where subtle contextual cues are crucial, and likely contributed to the observed performance gap. We provide a comparison of the descriptions generated by Gemma and GPT-40 in the Appendix.
5 Limitations and Future Directions
While our framework shows promising results, we identify several limitations that pave the way for future research. First, our implementation relies on powerful but computationally expensive multimodal models like Gemma 3. Their high inference latency poses challenge for real-time applications. Recent studies have shown that smaller, task-sp models can perform exceptionally well on specialized tasks (Wang et al.
11

---

Published in Transactions on Machine Learning Research (03/2026)
2020 Schick & Schütze 2020 Marafioti et al. 2025; Mumcu & Yilmaz 2024). We will explore fine-tuning such models for single scenes, specifically for describing object activities and interactions.
Second, a primary contribution of our method is its semantic explainability, but quantitatively evaluating this feature is difficult due to a lack of suitable benchmarks. Current datasets with textual ground truth are designed for multi-scene or weakly-superviser tasks and do not fit the semi-supervised, single-scene VAD problem (Zhu et al. 2024 Du et al. 2024; Zhang et al. 2025). Therefore, a key future direction is the development of new dalasets with ground-truth textual descriptions for scene-specific anomalies.
Our framework relies on relative consistency across nominal textual descriptions rather than absolute textual correctness. Anomaly detection is performed by comparing sentence embeddings to a set of nominal exem- plars, such that deviations from the learned nominal distribution determine anomaly scores. As a result, minor variations in wording or phrasing do not directly affect detection decisions, provided that nominal descriptions remain semantically consistent. In addition, we employ fixed system prompt and con- strained task formulation that focuses describing object activities and interactions, which further reduces sensitivity to prompt wording. Nevertheless, more systematic evaluation of prompt sensitivity remains an important direction for future work, particularly given the computational cost of large-scale MLLM querying.
Finally, our framework currently leverages a robust traditional object detector, which is well-suited for scenarios with known object classes. To further generalize our approach, a promising future direction the integration of open-vocabulary object detoction mothods (Liu al., 2024 Cheng al.) 2024 Mumcu et al., 2025a). By enabling the recognition of an open set of object categories, this advancement. would significantly broaden system's applicability, allowing it to analyze and describe more diverse range of anomalies in unconstrained, real-world environments.
6 Conclusions
In this work, we presented novel video anomaly detection framework that leverages multimodal Large Language Models to detect and explain complex anomalies. By extracting object-level activity and inter- actions and converting them into textual descriptions, our method moves beyond traditional pixel level modeling and introduces a language-based, interpretable layer for video anomaly detection. Unlike many prior MLLM-based methods that rely on direct frame-level judgments, our approach builds set of nominal textual descriptions and identifies anomalies through language- comparison, offering both accuracy and explainability.
We demonstrate that our method outperforms existing approaches on ComplexVAD which is challenging benchnark specifically designed to test interaction-based anomalies. In addition our method also improves of the state-of-the-art results on standard datasets like Avenue and Street Scene when combined with the fine- grained method Tracklet EVAL (Singh et al.) 2024). Through ablation studics, we showed the importance descriptions. Notably, we found that Gemma
of using sentence emboddings and detailed MLLM- provided more informative descriptions than GPT-4o, contributing to higher detection accuracy.
Beyond strong quantitative results, our method offers clear and interpretable explanations for detected anomalies, highlighting its potential for real world deployment in safety critical surveillance applications. We believe this framework opens new directions for integrating multimodal large language models into video understanding tasks and sets the stage for future research in explainable and high level language based video anomaly detection.
7 Broader Impact Statement
This work studies video anomaly detection with a focus on modeling scene-specific object interactions using multimodal large language models. While the proposed method is intended as a research contribution to representation learning and explainable anomaly detection, it is important to consider potential broader impacts and risks associated with its use.
12

---

Published in Transactions on Machine Learning Research (03/2026)
A primary application domain of video anomaly detection is surveillance, which raises ethical concerns related to privacy, misuse, and surveillance overreach. If deployed irresponsibly, anomaly detection systems may contribute to unwarranted monitoring or automated enforcement. We emphasize that the proposed approach is intended as a decision-support lool rather than an automated decision-making system, and should be used in -nohe-- settings where outputs are interpreted by responsible operators rather than acted upon autonomously.
The use of mullimodal large language models introduces additional considerations. MLLMs are compula- tionally expensive, and converting large volumes of video data into textual descriptions increases energy consumption and carbon footprint compared to conventional VAD pipelincs. For this reason, our mothod is not designed for real-time or large-scale deployment, and the MLLM component is used sparingly (offline and at sparse temporal intervals). Future work should consider more efficient models or distillation strategies reduce environmental impact.
MLLM-gennrated textual descriptions may also hallucinate include spurious details. In safety-critical scenarios, such hallucinations could lead to misleading explanations or mischaracterization of events. Our framework mitigates this risk by relying on relative consistency across nominal data and by using language- based representations as an auxiliary explanatory signal rather than as a sole decision criterion. hevertheless, we caution against interpreting generated text as ground truth or using it without human oversight.
Finally, like many vidco analysis methods, our approach inherits biases present in training data and object detection models. Dataset bias may affect which behaviors are considered "normal" or "anomalous," poten- tially leading to unfair incorrect outcomes if deployed without careful consideration. We encourage future work to examine dataset diversity, bias mitigation, and domain-specific validation prior to any real-world deployment.
References
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint Xrx::2330300874, 2023.
Satanjocy Banerjee and Alon Lavic. Metcor: An automatic metric for mt evaluation with improved correla- tion with human judgments. In Proceedings of the ACL Workshop on Intrinsic and Eatrinsic Evaluation Measures for Machine Tronslation and/or Summarization, 2005.
Mohammad Baradaran and Robert Bergevin. eritical study on the recent deep learning based semi- supervised video anomaly detection methods. Multimedia Tools and Applications, 83, 2024.
Antonio Barbalau, Radu Tudor lonescu, Mariana-Luliana Georgescu, Jacob Dueholm, Bharathkumar Ra- machandra, Kamal Nasrollahi, Fahad Shahbaz Khan, Thomas B Moeslund, and Mubarak Shah. Ssmtl++: Revisiting self-supervised multi-task learning for video anomaly detection. Computer Vision and Image Understanding, 229:103656, 2023.
Nicholas F.Y. Chen, Zhiyuan Du, and Khin Hua Ng. Scene graphs for interpretable video anomaly classifi- cation. In NIPS, 2018.
Tianheng Cheng, Lin Song, Yixiao Ge, Wenyu Lin, Xinggang Wang, and Ying Shan. Yolo-world: Real-time open-vocabulary object detection. Proceedings of the IEEE/CVI CCF Conference on Computer Vision and Pallern Recognition, pp. 16901-16911, 2024.
Keval Doshi and Yasin Yilmaz. Any-shot sequential anomaly detection in surveillance videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 934 -935,, 2020a.
Keval Doshi and Yasin Yilmaz. Continual learning for anomaly detection in surveillance videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognation Workshops, pp. 254-255 2020b.
13

---

Published in Transactions on Machine Learning Research (03/2026)
Keval Doshi and Yasin Yilmaz. Towards interpretable video anomaly detection. In JEEE Winter Conference 0n Applications of Computer Vision (WACV), 2023.
Hang Du, Sicheng Zhang, Binzhu Xie, Guoshun Nan, Jiayang Zhang, Junrui Xu, Hangyu Liu, Sicong Leng, Jiangming Liu, Hehe Fan, et al. Uncovering what why and how: A comprehensive benchmark for causation understanding of video anomaly. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 18793-1888 -18803, 2024.
Mariana Iuliana Georgescu, Radu Ionescu, Fahad Shahbaz Khan, Marius Popescu, and Mubarak Shah. A backgrounddaggoosiic framework with adversarial training for abnormal event. detection in video. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.
Dong Gong, Lingqiao Liu, Vuong Le, Budhaditya Saha, Moussa Reda Mansour, Svetha Venkatesh, and Anton van den Hengel. Memorizing normality to detect anomaly: Memory-augmented deep autoencoder for unsupervised anomaly detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019.
Mahmudul Hasan, Jonghyun Choi, Jan Neumann, Amit K Roy-Chowdhury, and Larry S Davis. Learning temporal regularity in video sequences. In Proceedings of the IEEE conference on computer vision and pottern recognition, pp. 733-742, 2016.
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-40 system card. arXiv preprint or XrXiv22410.21276 2024.
Radu Tudor lonescu, Fahad Shahbaz Khan, Mariana-luliant Georgescu, and Ling Shao. Object-centric auto- encoders and dummy anomalies for abnormal event detection in video. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pallern Recognition, pp. 7842 2442-7851, 2019.
Changkang Li and Yalong Jiang, Vlavad: Vision-language models assisted unsupervised video anomaly detection. In Proceedings of the British Machine Vision Conference, 2024.
Fei Li, Wenxuan Liu, Jingjing Chen, Ruixu Zhang, Yuran Wang, Xian Zhong, and Zheng Wnag. Ano- mize: Better open vocabulary video anomaly detection. In Proceedings of the IEFE/CVF Conference on Computer Vision and Pattern Recognition, 2025.
Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European Conference on Computer Vision, pp. 38-55. Springer, 2024.
Wen Liu, Weixin Luo, Dongze Lian, and Shenghua Gao. Future frame prediction for anomaly dctection-a new baseline. In Proceedings of the IEEE conference on compuler vision and pollern recognition, pp. 6536-6545, 2018.
Zhian Liu, Yongwei Nie, Chengjiang Long, Qing Zhang, and Guiqing Li. A hybrid video anomaly detection framework via memory-augmented flow reconstruction and flow-guided frame prediction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 13588-13597, 2021.
Cewu Lu, Jianping Shi, and Jiaya Jia. Abnormal event detection at 150 fps in matlab. In Proceedings of the IEEE International Conference on Compuder Vision, pp. 2720 2727, 2013.
Weixin Luo, Wen Liu, and Shenghua Gao. A revisit of sparse coding based anomaly detection in stacked rnn framework. In Proceedings of the IEEE international conference on computer vision, pp. 341-349, 2017.
Hui Lv and Qianru Sun. Vidco anomaly detection and explanation via large language models. arXiv preprint arXiv:2401.057 050020 2024.
Vijay Mahadevan, Weixin Li, Viral Bhalodia, and Nuno Vasconcelos. Anomaly detection in crowded scenes. In 2010 IEEE computer society conference computer vision and pottern recognition, pp. 1975-1981. IEEE, 2010.
14

---

Published in Transactions on Machine Learning Research (03/2026)
Andrés Marafioti, Orr Zohar, Miquel Farré, Merve Noyan, Elie Bakouch, Pedro Cuenca, Cyril Zakka, Toubna Ben Allal, Anton Lozhkov, Nouamane Tazi, et al. Smolvlm: Redefining small and efficient multi- modal models. arXiv preprint arii:2250 05299, 2025.
Furkan Mumeu and Yasin Yilmaz. Fast and lightweight vision-language model for adversarial traffic sign detection. Electronics, 13(11):2172, 2024.
Furkan Mumcu, Michael Jones, Anoop Cherian, and Yasin Yilmaz. Llm-guided agentic object detection for open-worie world understanding. arXiv preprint arXiv:2507.10844 2025a.
Furkan Mumcu, Michael J. Jones, Yasin Yilmaz, and Anoop Cherian. Complexvad: Detecting interaction anomalies in video. In Proceedings of the IEEE/CVF Winter Conference Applications of Computer Vision Workshops, 2025b.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, 2002.
Bharathkumar Ramachandra and Michael Jones. Street scene: A new dataset and evaluation protocol for video anomaly detection. In Proceedings of the IEEE/CVE Winter Conference on Applications of Computer Vision, pp. 2569 2578, 2020.
Bharathkumar Ramachandra, Michael Jones, and Ranga Vatsavai. Learning distance function with siamese network to localize anomalies in videos. In Proceedings of the IEEE/CV, Winler Conference Applications of Computer Vision, pp.
2020a.
Bharathkumar Ramachandra, Michael Jones, and Ranga Raju Vatsavai. survey of single-scene video anomaly detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020b.
Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2019.
Nicolae-Catalie Ristea, Neelu Madan, Radu Tudor Ionescu, Kamal Nasrollahi, Fahad Shahbaz Khan, Thomas B Moeslund, and Mubarak Shah. Self-supervised predictive convolutional attentive block for anomaly detection. arXiv preprint arXiv:2111.09099, 2021.
Timo Schick and Hinrich Schütze. It's not just size that matters: Small language models are also few-shot learners. Xiv preprint ar XxXx:2009.00 011200999.1118 07118, 2020.
Ashish Singh, Michael J. Jones, and Erik Leared--Mller. Eval: Explainable video anomaly localization. In Proceedings of the IEEF/CVR Conference on Computer Vision and Pattern Recogration, 2023.
Ashish Singh, Michael J. Jones, and Erik Learned-Miller. Tracklet- based explainable video anomaly lo- calization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognátion Workshops, 2024.
Waqas Sultani, Chen Chen, and Mubarak Shah. Real-world anomaly detection in surveillance videos. In Proceedings of the IEFE conference on compuler vision and pattern recognilion, pp. 6479-6488, 2018.
Che Sun, Yunde Jia, Yao Hu, and Yuwei Wu. Scene-aware context reasoning for unsupervised abnormal event detection in videos. In ACMMM, 2020.
Stanislaw Szymanowicz, James Charles, and Roberto Cipolla. X-man: Explaining multiple sources of anoma- lies in video. In CVPR, 2021.
Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et al. Gemma 3 technical report. arXiv preprint ar Xrx::22503. 19786, 2025.
15

---

Published in Transactions on Machine Learning Research (03/2026)
Chenxu Wang, Yanxin Yao, and Han Yao. Video anomaly detection method based on future frame prediction and attention mechanism. In JEEE 11th Annual Computing and Communicaatin Workshop and Conference (CCWC). IEEE, 2021.
Jianfeng Wang, Xiaowei Hu, Pengchuan Zhang, Xiujun Ti, Lijuan Wang, Lei Zhang, Jianfeng Gao, and Zicheng Liu. Minivlm: A smaller and faster uisionllanguae model. arXiv preprint orXiv:2012.06946, 2020.
Peng Wu, Chengyu Pan, Yuting Yan, Guansong Pang, Peng Wang, and Yanning Zhang, Deep learning for video anomaly detection: A review. arXiv preprint arXiv:2409. 05383, 2024a.
Peng Wu, Xuerong Zhou, Guansong Pang, Yujia Sun, Jing Liu, Pong Wang, and Yanning Zhang. Open- vocabulary video anomaly detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024b.
Peng Wu, Xuerong Zhou, Guansong Pang, Yujia Sun, Jing Liu, Peng Wang, and Yanning Zhang. Follow the rules: Reasoning for video anomaly detection with large language models. In Proceedings of the European Conference Computer Vision, 2024c.
Peng Wu, Xuerong Zhou, Guansong Pang, Lingru Zhou, Qingsen Yan, Peng Wang, and Yanning Zhang, Vadclip: adapting vision-language models for weakly supervised video anomaly detection. In AAAT Con- ference on Artificial Intelligence, pp. 6074-6082, 2024d.
Yuxin Wu, Alexander Kirillov, Francisco Massa, Wan-Yen Lo, and Ross Girshick. Detectron2. https:
//github. //github.com/ffeee http:////cccccccooorraaa///tttta 2019.
Zhengye Yang and Richard J. Radke. Detecting contextual anomalies by discovering consistent spatial regions. In Proceedings of the IEEE/CVF Winler Conference Applications of Compuler Vision Work- shops, 2025.
Muchao Ye, Weiyang Liu, and Pan He. Vera: Explainable video anomaly detection via verbalized learning vision-language models. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025.
M. Zaigham Zaheer, Arif Mahmood, M. Haris Khan, Mattia Segu, Fisher Yu, and Seung-Ik Lee. Gener- ative cooperative learning for unsupervised video anomaly detection. In Proceedings of the IEEE/CVF Conference Compuler Vision and Patlern Recognition (CVPR), 2022.
Luca Zanella, Willi Menapace, Massimiliano Mancini, Yiming Wang, and Elisa Ricci. Harnessing large language models for training-free vidco anomaly dotection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
Huaxin Zhang, Xiaohao Xu, Xiang Wang, Jialong Zuo, Xiaonan Huang, Changxin Gao, Shanjun Zhang, Li Yu, and Nong Sang. Holmes-vau: Towards long-term video anomaly understanding any granularity. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 13843 13853, 2025.
Yifu Zhang, Peize Sun, Yi Jiang, Dongdong Yu, Fucheng Weng, Zehuan Yuan, Ping Luo, Wenyu Liu, and Xinggang Wang. Bytetrack: Multi-object tracking by associating every detection box. In Proceedings of the European Conference on Computer Vision (ECCV), 2022.
Liyun Zhu, Lei Wang, Arjun Raj, Tom Gedeon, and Chen Chen. Advancing video anomaly detection: A concise review and new datasct. Advances Neural Information Processing Systems, 37:89943-89977 2024.
16

---

Published in Transactions on Machine Learning Research (03/2026)
A
Compute Resources
All of our experiments were run on a cluster eight 80 GB NVIDIA A100 GPUs. single 80 GB GPU would be sufficient and multiple GPUs were used only to reduce the total compute time by processing multiple video sequences in parallel.
For experiments using the locally deployed MLLM, report inference latency measurements using the Gemma model. On a single NVIDIA A100 GPU, querying Gemma with paired object-centric crops and the associated prompt takes on average 0.58 seconds per query, with a standard deviation of 0.16 seconds (measured over repeated runs). This provides a reproducible lower bound on the inference cost of the MLLM component under local deployment.
Our experiments that used GPT-4o accessed it through an API via the cloud.
B Cropping Frames
For each selected object pair on and 02, we first compute a single bounding box that tightly encloses both objects in given frame. Let the individual bounding boxes for the two objects in frame F3 be denoted as bbox,- = (I Ma., T2a, 1/aa) and bbox,, = (# VIa, F2b, V28), where (x1, represents the top-left corner and (x2, y2) the bottom-right corner of a bounding box. These are merged into a single bounding box bbox, by taking the clement-wise minimum for the top-left corner and maximum for the bottom-right corner:
bbox (min( Zid), min v1b),
max(22a x26,, max(1/2a (26))))))))))))))))))))))))))))
We apply the same procedure obtain bboxq in the future frame FAt using the tracked locations of the same object pair.
Let bbox; = 91,1, and let bbox2 = (f1,2, 31,2, 22,2, 42,2) denote the merged bounding box the same objects tracked in the future frame FI+. A unified bounding box that covers both time steps is computod as:
denole the merged bounding box of the selected object pair in frame Fe:
bbOXmerged (1,2) min(y1.1,
m max 22,2))
Let (I1, and (I2, 32) represent the top-left and bottom-right corners of this merged bounding box. Let Wnin and Amin denote half of the minimum desired crop width and height, respectively. In our case, we set Wmin = 240 and Amin = 135, corresponding to a minimum crop size of 480 x 270 pixels. This size was chosen as a practical minimum that ensures sufficient context around objects while keeping computational overhead low. In our experiments, we found that a resolution of 480 270 served as a reliable lower bound, providing adequate spatial detail for capturing object activities without reducing the quality of the LLM generated descriptions. While larger crops may be used when available, we observed this setting to be consistent and effective baseline across different datasets.
We then compute the crop dimensions as
w max
2
Wimin
= max
y2 2
Amnn
(5)
The crop boundaries determined as:
Imin max( w, 0),
Umin max(v) - h, 0),
Fmax = min(r2 + W),
Fmax min(yz + h, H)
(6)
17

---

Published in Transactions on Machine Learning Research (03/2026)
where W and H denotes the width and height of the full frame. The final cropped region C is extracted from the current frame Fe as:
C = Ft [uuin
Umax: Emin
Tmax]
To ensure both temporal frames share the same spatial context, we apply the exact same crop coordinates to the future frame FetA:
C++a PV+alVmin Mmax Tmii m
This guarantees that both cropped frames contain the same background and object regions, allowing for consistent comparisons across time. In the case of a single object, the same procodure is applied with one exception: instead of merging two object bounding boxes in each frame, we directly use the bounding box of the single object in frame F, and its tracked location in Fita. The rest of the cropping process, including the computation of the unified bounding box across time, minimum size enforcement, and cropping, is performed exactly described above.
C Comparison of Gemma 3 and GPT-4o Activity Descriptions
we share activity descriptions generated by Gemma 3 and GPT-4o for given images and the promt: "Briefly describe what the <object name>> in the enclosed regions of these images are doing. The two images were taken one second apart."
In Figure 3
We have observed that Gemma 3 often gives more concise descriptions with fewer unnecessary details such as the color of a person's clothes. Moreover, Gemma 3 sometimes includes important details that GPT-4o omits, such as the fact that person using his her phone.
18

---

Published in Transactions on Machine Learning Research (03/2026)
Gemma
GPT-4o
person enclosed region is bending and appears same poetlion between the images
The person is crouching down the ground.
In the images, the person standing pushing or interacting with cart box that the seated be inside using. The standing person maintains their position or posture relative to cart between the images.
The person is holding phone up and appearing take picture while being pushod large box another person.
appears
person blue shirt and the person bike interacting, the person blue be nyclist from moving forward.
bicycle being pushed forward by another is riding a scooter.
preventing
person walking acroes crosswall looking their phone, and cyclist passing by them right.
person enclosed region stopping allow cyclist pass by.
In the first image, the the black shorts and dark appears to be jumping falling bald person. In second image, the the black shorts dark standing up straight, while bald remains standing. There is no apparent interaction between
The two people are walking crosswalk, with the person on the right sppearing stumble or jump.
Figure The activity descriptions generated by Gemma 3 and GPT-4o with given images.
19

---

Published in Transactions on Machine Learning Research (03/2026)
Human Evaluation Examples
Clip ID
Human-written Explanation
MLLM-EVAD Explanation
4201
car stops abruptly after crosswalk, causs- ing the car behind to halt on the crosswalk and block pedestrians before both continue.
The car is driving forward as person with backpack walks across the crosswalk in front of it.
4254
Two people cross the crosswalk when third runs up and jumps onto one of them, who ap- pears distressed.
The person in the black jacket appears to be pushing the person in the grey shirt, causing them stumble forward.
1288 The skateboarder falls in the crosswalk as a scooter rider passes by. After a moment, the skateboarder gets up and leaves the crosswalk. dog runs across the crosswalk without a leash, moving away from its owner.
The person is running and then falls to the ground.
4318
The person was walking with the dog, but the dog has broken away and is now running ahead.
1360 A person hits another person the head with a bat and the other person is visibly hurt.
The two people are walking across the cross- walk together.
4371 A scooter and bicycle collide while cross- ing the crosswalk, then both riders continue through.
'hee person on the skateboard is being pulled forward by the person riding the bicycle.
4405
Two people play volleyball near the crosswalk, and their ball eventually falls toward the road.
The people in the enclosed region are playing with ball, passing back and forth to each other.
4414
golf cart parked near a crosswalk and then drives away.
Thee golf cart driving forward as the person walks alongside it, and car is passing in front of the golf cart.
4426 A scooter rider crosses crosswalk, then stops, turns around, and rides back.
The person is riding the scooter forward across the pavement.
4431
A person climbs pole the crosswalk, sits briefly, then gets down and walks away.
The person is climbing onto pole.
Table 8: Examples used in the human explainability evaluation. For each anomalous clip, we show the human-written explanation and the explanation generated by MLLM-EVAD. Anomalous clips are randomly selected from ComplexVAD dataset.
E Effect of Temporal Gaussian Smoothing
Method
Frame AUC (Raw)
Frame AUC (Gaussian Smoothed)
MLLM-EVAD
0.61
0.68
Scene-Graph MLLM-EVAD
0.63
0.71
Table 9: Effect of temporal Gaussian smoothing on framo-level anomaly detection performance on the CompleXVAD dataset. Smoothing is applied only as a post- cost-processing step for this supplementary analysis and is not used in any main quantitative comparisons.
As stated in the main text, all quantilative results reported in Tables 2 and 3 are compuled using raw, unsmoothed anomaly scores to ensure consistency across methods.
As a supplementary analysis, we report the effect of applying Gaussian temporal smoothing to the frame-level anomaly scores of our best berf-peerorming configuration on the ComplexVAD dataset. Temporal smoothing is a commonly used post-processing step in video anomaly detection to reduce temporal noise in frame-level scores. We emphasize that smoothing is not part of the proposed method and is not used in any of the main quantitative comparisons; the results shown here are provided solely for completeness and illustrative purposes.
20


Reviews:
[{"review_id": "vIaVbgRsn8", "reviewer": "Reviewer_oo5N", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes MLLM-EVAD, a single-scene video anomaly detection framework. It uses multimodal large language models (MLLMs) to extract normal object activities and interactions in surveillance videos. Specifically, the MLLM takes in object-centric crops as input and generates natural language descriptions of object activities and interactions in the videos. These descriptions are then embedded, and an exemplar selection algorithm is applied to construct representative sets of embeddings for both object pairs and single objects in the scene. At test time, the sentence embeddings extracted from testing frames and the embeddings of the textual exemplar set are compared to detect and localize anomalies. In the experiments, MLLM-EVAD demonstrates strong empirical performance on the ComplexVAD dataset and the Avenue and Street Scene datasets when combined with prior exemplar-based methods.\n\nThe authors claim that this is the first MLLM-based semi-supervised VAD approach explicitly designed for interaction-based anomalies and addresses an important limitation of prior work. Rather than directly using the MLLM to make anomaly classifications or judgements, it generates textual descriptions as high-level semantic features. The authors argue that this approach provides inherent explainability through the textual descriptions.", "claims_and_evidence_yn": "Yes", "claims_explanation": "- The quantitative results on the ComplexVAD dataset show that MLLM-EVAD outperforms strong baselines on a challenging benchmark. The results on the RBDC and TBDC metrics highlights the proposed framework's spatiotemporal localization capability, which aligns well with the paper’s core claims.\n- The experiments on Scene-Graph and Tracklet-EVAL support the claim that MLLM-EVAD is complementary to fine-grained, non-linguistic features.\n- The ablation studies and the qualitative results further supports the claim that the proposed framework has competitive performance against strong baselines and improved detection of interaction-based anomalies.\n\nSome claims could be strengthened with additional evidence - see \"requested changes\" below. The \"built-in explainability\" claim is also not very well supported, though it can be challenging to quantitatively evaluate, as the authors acknowledged in the limitations section.", "audience_yn": "Yes", "audience_explanation": "Video anomaly detection is a research topic in machine learning, and new video anomaly detection methods and results are something that at least some individuals in TMLR's audience would be interested in knowing. MLLMs is a very popular research topic and how we can use MLLMs to best extend the capabilities of prior methods in video anomaly detection is highly relevant to the topic of TMLR.", "requested_changes": "Critical changes:\n- As the authors acknowledged in the limitations section, the high inference latency of MLLMs can make it challenging for real-time applications. The paper should quantify and compare the latency of different models, provide concrete measurements of inference time and cost for MLLM querying, and discuss the feasibility for longer videos or real-time settings in more detail.\n- The paper should include more analysis on the robustness of the framework, including sensitivity to prompt wording and exemplar-selection threshold. This is necessary since the model purely relies on the exemplars of embeddings of textual descriptions make anomaly detection decisions.\n\nNon-critical changes:\n- The paper could include more qualitative examples on failure cases, e.g. when the model produces ambiguous explanations.\n- Add a broader impact statement (see section below).\n\nMinor changes:\n- The 'year' column could be removed from table 1 since the information is already available in the citation.\n- The citation format might need to be changes in tables 2, 3, and 4 (maybe from \\citet to \\citep)", "broader_impact_concerns": "A broader impact statement section is not present in this manuscript. I would strongly encourage the authors to add one, give that the paper focuses on surveillance video anomaly detection, which raises potential ethical concerns related to privacy, misuse, and surveillance overreach. This new method could amplify existing risks if deployed irresponsibly in mass surveillance systems (e.g. when used as a automated enforcement system instead of a decision-support tool for responsible, human-in-the-loop decision making).", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "THt6tgotVk", "reviewer": "Reviewer_c3eN", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper presents a video anomaly detection (VAD) approach that leverages multimodal large language models. The proposed idea is integrated with state-of-the-art VAD methods and evaluated on three challenging benchmarks: ComplexVAD, CUHK-Avenue, and Street Scene. While the idea of transforming visual information into the textual domain is interesting, the proposed approach is highly resource-intensive and still relies on vision-based models, which limits its practical applicability.", "claims_and_evidence_yn": "No", "claims_explanation": "The paper claims that the proposed VAD approach is explainable; however, no quantitative measures are provided to support this claim. The evaluation relies primarily on qualitative comparisons, but such assessments in the language domain are inherently subjective and can vary significantly across individuals. For example, the paper claims that the following two textual descriptions are close to each other:\n\n“The person is holding a phone up and appearing to take a picture or video while being pushed in a large box by another person.”\nvs.\n“The person is walking along the sidewalk.”\n\nAt first glance, it is difficult to understand how these descriptions are considered similar. It is unclear whether this similarity is derived from the underlying data representation or from the language model itself, and further clarification is needed to justify this claim.", "audience_yn": "No", "audience_explanation": "Overall, the idea of transforming visual information into the textual domain using MLLMs is interesting, and the resulting performance is comparable to existing approaches when this modality transformation is applied. However, this transformation incurs a substantial computational cost, which raises concerns about the practical motivation for adopting such a method. Unless the approach provides clear advantages, such as significantly improved anomaly detection performance or capabilities beyond existing methods, the incentive to switch to this framework remains limited.\n\nFurthermore, a more intuitive explanation would be valuable to help understand why text-based descriptions perform effectively for anomaly detection. Since anomaly detection is fundamentally a one-class learning problem focused on modeling normalcy, it remains unclear whether converting visual scenes into textual representations is inherently better or even sufficient for this task. Clarifying this point would strengthen the conceptual contribution of the paper.\n\nTherefore, so additional clarification and justifaction might be needed to make this research relevant for TMLR's audience", "requested_changes": "1.  The paper claims that the proposed approach is an explainable method for anomaly detection. However, multimodal LLMs are known to be prone to hallucinations, as also discussed in _Holmes-VAU_ (Zhang et al.). Given this, it is unclear how the generated text can serve as a reliable measure of explainability. Furthermore, the paper explicitly states that there is no quantitative evaluation of explainability, which appears to contradict the claim of being an explainable method.\n    \n2.  The datasets used for evaluation—ComplexVAD, CUHK-Avenue, and Street Scene—do not contain video-level or event-level textual descriptions in their original versions (as acknowledged in the paper). It is therefore difficult to understand how the generated textual descriptions are validated or compared to ground truth. This raises questions about how the results reported in Table 5 are obtained:\n    - Where do the ground-truth descriptions come from?\n    - Are the results simply comparisons against outputs from a stronger MLLM?  \n        Additional clarification on this point would be helpful.\n        \n3. The paper demonstrates that converting visual information into textual modality using an MLLM can improve anomaly detection performance. However:\n    \n    - The reported improvements are shown only in combination with a scene-graph–based approach on the ComplexVAD dataset (Mucmu et al.). This suggests that the contribution may be limited to a single baseline and raises concerns about the generality and novelty of the approach.\n    - Integrating the proposed method with other baselines using different architectural styles would help assess whether the idea generalizes beyond scene-graph–based methods and whether the observed improvements truly stem from the MLLM-based evaluation.\n    - For ComplexVAD, results are reported for MLLM-EVAL, the scene-graph baseline, and their combination. However, similar comparisons are not provided for the CUHK-Avenue and Street Scene datasets. Including these results would strengthen the experimental evaluation. \n\n4. Qualitative results are presented only for the ComplexVAD dataset. Since the authors also evaluate performance on CUHK-Avenue and Street Scene, it would be beneficial to include qualitative visualizations for these datasets as well.\n\n5. Regarding explainability, the metrics used to measure textual embedding similarity (e.g., BLEU and METEOR) are inherently subjective and sensitive to phrasing, which limits their quantification for explainability a difficult task. Moreover, based on the examples provided in the supplementary material, it is difficult to interpret how textual distance in the embedding space reflects semantic similarity or dissimilarity. For instance:\n    \n    > “The person is holding a phone up and appearing to take a picture or video while being pushed in a large box by another person.”  \n    > vs.  \n    > “The person is walking along the sidewalk.”  \n    > It is unclear how such differences are quantified meaningfully in the proposed framework. \n\n6. The authors employ Gemma and GPT-4o for video description. However, these models are fundamentally image-based vision–language models and are not designed to explicitly model temporal dynamics in videos. As a result, it is unclear whether the generated descriptions adequately capture motion, event progression, and temporal consistency, which are critical for video anomaly detection. \n\n\t- Moreover, the paper does not compare against existing video-language models that are specifically developed for video understanding and temporal reasoning, such as: Video-LLaMA, VideoChat,  InternVideo, Video-ChatGPT, LLaVA-Video. The absence of such comparisons significantly weakens the empirical evaluation. Without benchmarking against video-centric models, it is difficult to determine whether the reported improvements arise from the proposed methodology or are merely an artifact of using strong image-based MLLMs. Including these baselines is necessary to substantiate the paper’s claims and to demonstrate the generality and novelty of the approach.\n\t- Furthermore, an ablation study using textual descriptions generated separately by Gemma and GPT-4o would be valuable. Such an analysis would help clarify whether state-of-the-art anomaly detection methods consistently benefit from MLLM-based textual representations. A subsequent comparison of different MLLMs to identify which performs best in this setting would be a natural next step.", "broader_impact_concerns": "There could be some broader impact caused by the approach, especially related with the used MLLMs for VAD\n1. The proposed method relies heavily on multimodal large language models, which are computationally expensive to run and scale. Converting large volumes of video data into textual descriptions significantly increases energy consumption and carbon footprint compared to conventional VAD pipelines. This raises sustainability concerns, especially for real-world deployment at scale\n\n2. MLLM-generated textual descriptions are known to hallucinate or introduce spurious details. In safety-critical applications of anomaly detection (e.g., public surveillance, traffic monitoring), such hallucinations could: mischaracterize events, introduce false explanations,\n\n3. Also, textual output as a measure of explainability is flawed, as LLM might generate the text unrelated with the visual scene, which can further leads to the mis-classification of critical events.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "89lOgKueRR", "reviewer": "Reviewer_k2CH", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "- This paper introduces a method that leverages the captioning capability of multimodal large language models (MLLMs) to detect complex video anomalies involving interactions.\n- The paper proposes an exemplar-based approach that uses MLLMs to construct compact and explainable feature representations.\n- The proposed method demonstrates performance improvements on the evaluated benchmarks and exhibits complementary effects when combined with existing methods.", "claims_and_evidence_yn": "Yes", "claims_explanation": "- The paper is clearly written and easy to follow.\n- The effectiveness of the proposed method is supported by evaluations on multiple standard datasets in the video anomaly detection community.\n- Both frame-level metrics (AUC) and region-/track-level metrics (RBDC/TBDC) are used, providing a comprehensive evaluation and showing consistent improvements over baseline methods.\n- Comparisons with relevant baselines and ablation studies clearly identify the contributions of the proposed approach.", "audience_yn": "Yes", "audience_explanation": "The paper proposes a novel way of leveraging multimodal large language models (MLLMs) to construct compact and explainable feature representations for video anomaly detection. This perspective provides useful insights for the TMLR audience, particularly researchers interested in interpretable representation learning and the integration of foundation models into video understanding systems.", "requested_changes": "- Method acronyms in Table 4:\nThe acronyms used for several referenced methods in Table 4 could be refined to better match commonly adopted nomenclature in the literature. It would be helpful to use acronyms consistent with prior work to improve readability and recognizability. For example, Autoregressive Denoising Score Matching is commonly abbreviated in recent literature (e.g., ICCV 2025) and could be aligned accordingly.\n- Inconsistency between Table 5 and Table 1:\nThe performance reported for SentenceBERT in Table 5 does not appear to align with the RBDC/TBDC and AUC values reported in Table 1. This discrepancy should be corrected or explicitly explained, for example by clarifying differences in evaluation protocols or experimental settings.\n- Choice of frame gap:\nThe selection of a 1-second frame gap between input frames appears somewhat arbitrary. Providing additional justification for this choice, or including an ablation study over different frame gaps, would strengthen the empirical motivation.", "broader_impact_concerns": "The proposed method could potentially be applied in surveillance scenarios. While this is a common consideration in video analysis research, the paper does not explicitly discuss dataset bias or downstream deployment risks. A brief Broader Impact Statement acknowledging these aspects could strengthen the paper.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)