An Attribute-based Method for Video Anomaly Detection
Authors: Tal Reiss
Yedid Hoshen
Venue: Accepted by TMLR ()

Abstract: Video anomaly detection (VAD) identifies suspicious events in videos, which is critical for crime prevention and homeland security. In this paper, we propose a simple but highly effective VAD method that relies on attribute-based representations. The base version of our method represents every object by its velocity and pose, and computes anomaly scores by density estimation. Surprisingly, this simple representation is sufficient to achieve state-of-the-art performance in ShanghaiTech, the most commonly used VAD dataset. Combining our attribute-based representations with an off-the-shelf, pretrained deep representation yields state-of-the-art performance with a $99.1\%, 93.7\%$, and $85.9\%$ AUROC on Ped2, Avenue, and ShanghaiTech, respectively.

Published in Transactions on Machine Learning Research (01/2025)
An Attribute-based Method for Video Anomaly Detection
Tal Rciss
tal. lal.reiss@@maillugg...
School of Computer Science and Engineering The Hebrew University of Jerusalem, Israel
Yedid Hoshen
yodiid. @emaslhheemmmmmmmm huji, a.c
School of Computer Science and Engineering The Hebrew University of Jerusalem, Israel
Reviewed on OpenReview: Atips: // openreview. net/ formfo id- XL XL1NNGL
Abstract
Video anomaly detection (VAD) identifies suspicious events in videos, which is critical for crime prevention and homeland security. In this paper, we propose simple but highly effective VAD method that relies on attribute-based representations. The base version of our method represents every object by its velocity and pose, and computes anomaly scores by density estimation. Surprisingly, this simple representation is suffi- cient to achieve state-of-the-art performance in ShanghaïTech, the most commonly used VAD dataset. Combining our attribute-based representations with an off-the-shelf, pre- trained deep representation yields stale-of-the-arl performance with a 99.1%, 93.7%, and 85.9% AUROC on Ped2, Avenue, and ShanghaiTech, respectively. Our code is available at http://hhabbco/aaaaaess///crottt ottp://alleaecccccoooeoeeeaaaaaa
1 Introduction
Video anomaly detection (VAD) aims to discover interesting but rare events video. The task has attracted much interest it is critical for crime prevention and homeland security. One-class classification (OCC) is one of the most popular VAD settings, where the training set consists of normal videos only, while at test time the trained model needs to distinguish between normal and anomalous events. The key challenge for learning-based VAD is that classifying an event as normal or anomalous depends on the human operator's particular definition of normality. Differently from supervised learning, there are no training examples of anomalous events, this essentially requires the learning algorithms to have strong priors.
Many previous VAD methods use a combination of deep networks with sule-super objectives. A popular line of work consists of training neural network to predict the next frame and classifying it anomalous if the predicted and observed frames differ significantly. While such methods can achieve good performance, they do not make their priors explicit i.c., it is unclear why they consider some frames more anomalous than others, making them difficult to debug and improve. More recent methods involve preliminary object extraction stage from video frames, implying that objects are significant for VAD. However, they typically use object-level self-supervised approaches that do not make their priors explicit. Our hypothesis is that making priors more explicit will improve VAD performance.
In this paper, propose new approach that directly represents each video frame by simple attributes that are semantically meaningful to humans. Our method extracts objects from every frame, and represents each object by two attributes: its velocity and body pose (in the case the object is human). These attributes are well known to be important for VAD (Markovitz et al. 2020 Georgescu el al. 2021a). We detect anomalous values of these representations by using density estimation. Concretely, our method classifies a frame as anomalous if it contains one or more objects that have unusual values of velocity and/or pose (sce Fig. 1 Our simple velocity and pose representations achieve state-of-the-a performance (85.9% AUROC) on the most popular VAD dataset, ShanghaiTech.

---

Published in Transactions on Machine Learning Research (01/2025)
Avenue
ShanghaiTech
Normal
Anomalous
Nomal
Anomalous
Velocity
Test Video 21
Test Video 02_ 0161
Pose
Test Video
Deep
Video
Test Video 0100053
Figure 1: The Avenue and ShanghaiTech datasets. We present the most normal and anomalous frames for each feature. For anomalous frames, we visualize the bounding box of the object with the highest anomaly scorc. Best viewed in color.
While the velocity and pose representations are highly effective, they ignore other attributes, most impor- (antly object category. As an example, we never see a lion in normal videos, a lest video containing a lion is anomalous. To represent these residual attributes, we model them with an off-the-shelf, deep representation (here, we use CLIP features). Our final method combines velocity, pose and the deep representations. It achieves state-of-the-art performance on the three most commonly reported datasets.
2 Related Work
Classical video anomaly detection methods were typically composed of two steps: handcrafted feature extrac- tion and anomaly scoring. Some of the manual features that were extracted were: optical flow histograms
(Chaudhry et al. 2009 Colque et al. 2016 Perš et al.] 2010) and SIFT Lowe 2004). Commonly used
scoring methods include: density eslimation (Eskin al., 2002: Glodek el. al.] 2013 Latecki el. al. 2007).
reconstruction (Jollitfe 2011), and one-class classification (Scholkopf al. 20000.
In recent years, deep learning has gained in popularity as alternative to these carly works. The majority of video anomaly detection methods ut utilize at least one of three paradigms: reconstructions bonebas,,, prediction- based, skeletal-based, or auxiliary classification-bast methods.
Reconstruction & prediction based methods. In the reconstruction paradigm, the normal training data typically characterized by an autoencoder, which then used to reconstruct input video clips. The assumption is that model trained solcly on normal training clips will not be able to reconstruct anomalous frames. This assumption does always hold true, neural networks can often generalize some extent
Notable works are (Nguyen & Meunier 2019 Chang et al. 2020 Hasan et al.) 2016b;
Luo et al. 2017b Yu et al. 2020 Park et al. 2020).
Prediction-based methods learn to predict frames or flow maps in video clips, including inpainting inter-
mediate frames, predicting future frames, and predicting human trajectories (Liu et al. 2018a Feng et al.
2021b
Chen et al. 2020 Lee al.) 2019 Lu et al. 2019 Park al. 2020 [Wang al. 2021 Feng et al.
2021a Yu et al. 2020). Additionally, some works take hybrid approach combining the two paradigms [Liu]
et al. 2021b Zhao et al. 2017 Ye et al. 2019 Tang al. 2020 Morais et. al. 2019 As these methods
2

---

Published in Transactions on Machine Learning Research (01/2025)
trained to optimize both objectives, imput frames with large reconstruction or prediction errors considered anomalous.
Self-supervised auxiliary tasks. There has been a great deal of research on learning from unlabeled data, A common approach is to train neural networks on suitably designed auxiliary tasks with automatically generated labels. Tasks include: video frame prediction (Mathieu et al. 2016). image colorization (Zhang
iet al. 2016 Larsson al.) 2016), puzzle solving (Noroozi & Favaro (0166) rotation prediction (Gidaris al, 2018 arrow of time (Wei et al. 2018), predicting playback velocity (Doersch et al.) 2015), and verifying frame order (Misra et al. 2016). Many video anomaly detection methods use self-supervised learning. In fact, self-supervied lcarning is a key component in the majority of reconstruction-basee and prediction-based methods. SSMTL (Georgescu et al.) (021a)) trains a CNN jointly on three auxiliary tasks: arrow of time, motion irregularity, and middle-box prediction, in addition to knowledge distillation. Jigsaw-Puzzle (Wang ct al. 2022 trains neural networks to solve spatio-temporal jigsaw puzzles. The networks are then used for VAD.
Skeletal methods. Such methods rely a pose tracker to extract the skeleton trajectories of each person in the video. Anomalies are then detected using the skeleton trajectory data. Our attribute-based method outperforms previous skeletal methods (e.g.. (Markovitz et al. 2020; Rodrigues et al.) 2020 Yu et al. 2021
Sun & Gong 2023) by a large margin. In concurrent work, (Hirschorn & Avidan| (023)) proposed a skeletal- based method that achieved comparable results to ours on the ShanghafTech dataset but was outperformed
by large margins on the rest of the evaluated benchmarks. Different from skeletal approaches, our method does not require pose tracking, which is extremely challenging in crowded scenes. Our pose features only use single frame, while velocity features only require pair of frames. In contrast, skeletal approaches require pose tracking across many frames, which is expensive and error-prone. It is also important to note that skeletal features by themselves are ineffective in detecting non- non-human anomalies, therefore, being insufficient for providing a complete VAD solution.
Object-level video anomaly detection. Early methods, both classical and deep learning, operated on entire video frames. This proved difficult for VAD as frames contain many variations, as well as a large number of objects. More recent methods (Georgescu et al., 2021a]] Liu et al.] 2021b Wang et al.) 2022) operate at the object level by first extracting object bounding boxes using off-tho-shelf object detectors. Then, they detect if each object is anomalous. This is easier lask, as objects contain much less variation than whole frames. Object- based methods yield significantly better results than frame-level methods.
It is often believed that due to the complexity of realistic scenes and the variety of behaviors, it is difficult to craft leatures that will discriminate between them. As object detection was inacourate prior to deep learning, classical methods were previously applied at the frame level rather than at the object level, and therefore underperformed on standard benchmarks. We break this misconception and demonstrates that it is possible to craft semantic features that are accurate.
3 Method
3.1
Preliminaries
Our method assumes a training set consisting of N₂ vidco clips {c,, (2.,c...CN,, E Xtrain that are all normal (i.e., do not contain any anomalies). Each clip & is comprised of N, frames, 6; = [f,, Ji,,y-..i,,,]. The goal is classify each frame f c in an inference clip e as normal or anomalous. Our method represents each frame f ((f) E Rd. where d E N is the feature dimension. We compute the anomaly score of frame f using an anomaly scoring function s(o(f)), and classify it anomalous if s(o(f)) exceeds threshold.
3.2 Overview
We propose a method that represents each video frame as a set of objects, with each object characterized by its attributes. This contrasts with most previous methods that do not explicitly represent attributes. Specifically, we focus on two key attributes: velocity and body pose. Object velocity is probably the most important attribute as it can detect. if an object is moving unusually fast. e.g., running away, or in a strange
3

---

Published in Transactions on Machine Learning Research (01/2025)
frame
frame
Velocity Extractor
FlowNet
Pose Extractor
Object Detector
CLIP
Figure 2: An overview of our method. We first extract optical flow maps and bounding boxes for all of the objects in the frame. We then crop each object from the original image and its corresponding flow map. Our representation consists of velocity, pose, and deep (CLIP) features.
direction e.g., against the direction of traffic. Given the challenges of capturing true 3D velocity without depth information, we use optical flow as an effective proxy to represent apparent motion between frames. Similarly, human body poses can reveal anomalous activities, such as throwing an item. Instead of using the full 3D human pose; we represent it with 2D landmarks, which are easier detect. Both attributes have been used in previous VAD studies, such Georgescu al. (2021a) and Markovitz al. (2020), as part of more complex approaches.
We compute the anomaly score based on density estimation of object-level feature descriptors. This is done in three stages: pre-processing, feature extraction, and density estimation. In the pre-processing stage, our method (i) uses an off off-the-shelf motion estimator to estimate the optical flow for each frame; (ii) localizes and classifies the bounding boxes of all objects within a frame using an off-the-shelf object detector. The outputs of these models are used to extract object-level velocity, posc, and deep representations (sce Sec. 3.4
Finally, our method uses density estimation calculate the anomaly score of each test frame. See Fig. for illustration.
3.3
Pre- processing
Our method extracts velocity and pose from each object each video frame. To do so, we compute optical flow and body landmarks for each object in the video.
Optical flow. Our method uses optical flow proxy for its velocity. We extract the optical flow map for each frame f E in every video clip e using an off-tho-shelf optical flow model.
Object detection. Our method models frames by representing every object individually. This follows many recent papers, e.g., (Georgescu et al. 2021a Liu et al.) 2021b Wang et al. 2022) that found that object-basod representations are more effective than global, frame-level representations. We first detoct all objects in each frame using an off-the-shelf object detector. Formally, our object detection generates a set of m bounding boxes bbj bbm for each frame, with corresponding category labels y1, 1/, Unn-

---

Published in Transactions on Machine Learning Research (01/2025)
7
Vector Representation
Average magnitude
2
5
8
7
Bin
Figure 3: An illustration of our velocity feature vector. Left: We quantize the orientations into B = 8 equi-spaced bins, and assign each optical flow vector in the object's bounding box is single bin. Right: The value of each bin is the average magnitude of the optical flow vectors assigned to this bin. Best viewed in color.
3.4 Feature extraction
Our method represents each object by two semantic attributes, velocity and pose, and by an implicit, deep representation.
Velocity features. Our working hypothesis is that an unusual speed or direction of motion can often identify anomalics in video. As objects can move both x and y axes and both the magnitude (spood) and orientation of the velocity may be anomalous, we compute velocity features for each object in each frame. We begin by cropping the frame-level optical flow map o by the bounding box bb of each detected object. Following this step, we obtain a set of cropped object flow maps (see Fig. 2. We rescale these flow maps to a fixed size of ftow X Wiowww Next, we represent each flow map with the average motion for cach orientation, where orientations are quantized into B E N equi-spaced bins, following classical optical flow representations e.g., (Chaudhry et al. 2009). The final representation is a B-dimensional vector consisting of the average flow magnitude of the flow vectors in each bin (sce Fig. 3). This representation is capable of describing motion in both radial and langential directions. We denote our velocity feature extractor as: Pvelacity flow x Wloow - B
Pose features. Most anomalous events in video involve humans, so we include activity features in our representation. While full understanding of activity requires temporal features, we find that human body pose from a single frame can detect many unusual activities. Even though human pose is essentially a 3D feature, 2D body landmark positions already provide much of the signal. We compute pose features for each human object bb using an off-tho-shelf 2D keypoint extractor that outputs the pixel coordinates of each landmark position, denoted by Spose (bb) 2 where d E is the number of keypoints. To ensure invariance the human's position and size, normalize the keypoints. First, subtract the coordinates of the top-left corner of the object bounding box from each landmark. We then scale the x and y axes so that the object bounding box has a fixed size of Hpose X W pose, where Hpose and W Wosse are constants. Formally, let 1 E R be the top-left corner of the human bounding box. The pose descriptor becomes:
H posc height(bb)
(Gpasee - -)
Opose (bb)
(1)
W Wppen (o6dth(66)
Here, height(bb) and width(bb) are the height and width of the object bounding box bb, respectively and Le ? be the top-left corner of 6b. Finally, we flatten Opose to obtain the final pose feature vector.
Deep features. While velocity is the most discriminative attribute, other attributes beyond pose may also matter. Deep features implicitly bundle together many different altributes. Hence, we use them to model the residual attributes which are not described by velocity and pose. We follow previous anomaly detection works [Reiss al. 2021 Reiss & Hoshen| 2023), that used generic, pretrained encoders to implicitly represent image attributes. Concretely, we use pretrained CLIP encoder (Radford et al. 2021). Φdeep(-), to represent the bounding box of each object in each frame. Note that CL.P representations do not achieve competitiveness
5

---

Published in Transactions on Machine Learning Research (01/2025)
their own; in fact, they perform much worse than the velocity representations (see Tab, 3 However, together, velocity, pose, and CLIP features represent video sufficiently well to outperform the state-of-the-ari.
3.5 Density Estimation
We density estimation for scoring samples normal anomalous, where low estimated density indicates anomaly. To estimate density, we fit separate cstimator for cach feature. As velocity features are low-dimensional,
use Gaussian mixture (GMM) estimator. As our pose and deep features are high-dimensional, we estimate their density using ANN. Specifically, we compute the L2 distance between the feature x of a target object and the nearest & exemplars in the corresponding training feature set. We compare different exemplar selection methods is in Sec. 4.5 We denote our density estimators by (-), (.), $.cep(.).
Score calibration. Combining the three density estimators requires calibration. To do so, estimate the distribution of anomaly scores the normal training set. We then scale the scores using min-max normalization. The KNN used for scoring pose and deep features present a subtle point. When computing kNN on the training set, the exemplars must not be taken from the same clip as the target object. The reason is that the same object appears in nearby frames with virtually no variation, distorting &NN estimates. Instead, we compute the &NN between each training set object and all objects in the other video clips provided the training set. We can now define Vf {velocity, pose, deep): P = (s)(())(0 and Vy min. (=))())())(()))
3.6 Inference
We extract a representation for each object bb in each frame in each inference clip, as describe above. We then compute an anomaly score for each attribute feature of each object bb. The score for every frame is simply the maximum score across all objects. The final anomaly score is the sum of the individual feature scores normalized by our calibration parameters:
Svel (Qvelocity (bbx)) - Pvelocity } max
Spose (Opose (bbx)) - Ppose
sf)) = max {
}+
Mvclocity Veclocity
Mpose
Ppase
Sdeep Ceoeep(oaaee (ddeep (bbu)) )) Videey
max
Pdeep Vdeep
As anomalous events span multiple frames, smooth the frame scores using a temporal smoothing filter.
4 Experiments
4.1 Datasets
We evaluated our method on three publicly available VAD datasets, using their training and test splits. Only test videos included anomalous events. We report the statistios of the datasets in Tab. [
UCSD Ped2. This dataset [Mahadevan et al. 2010 contains 16 normal training videos and 12 test videos at a 240 x 360 pixel resolution. Videos show a fixed scene with a camera above the scene and pointed downward. The training video clips contain only normal behavior of pedestrians walking, while examples of abnormal events are bikers, skateboarding, and cars.
CUHK Avenue. This dataset (Lu et al.) 2013) contains 16 normal training videos and 21 test videos 360 x 640 pixel resolution. Videos show fixed scene using ground-level camera. Training video clips contain only normal behavior. Examples of abnormal events are strange activities (e.g. throwing objects, loitering, and running), movement. in the wrong direction, and abnormal objects.
ShanghaiTech Campus. This dataset (Liu al., 2018a is the largest publicly available datasct for VAD. There are 330 training videos and 107 test videos from 13 different scenes at 480 856 pixel resolution. ShanghaiTech contains video clips with complex light conditions and camera angles, making this dataset

---

Published in Transactions on Machine Learning Research (01/2025)
Table 1: Statistics of the evaluation datasets.
Number of Frames
Dataset
Scenes
Anomaly Types
Total
Train set
Test set
Normal
Anomalous
UCSD Ped2
4,560
2,550
2,010
2,924
1,636
5
CUHK Avenue
30,652
15,328
15,324
26,832
3,820
5
ShanghafTech
317,398
274,515
42,883
300,308
17,090
13
11
more challenging than the other two. Anomalics include robberies, jumping, fights, car invasions, and bike riding in pedestrian areas.
4.2
Implementation Details
We use ResNet50 Mask-RCNN (He et al. 2017] pretrained on MS-COCO (Lin et al. 2014) to extract object bounding boxes. To filter out low confidence objects, we follow the same configurations as in (Georgescu ct al. 2021a). Specifically for Ped2, Avenue, and ShanghaiToch, we set confidence thresholds of 0.5, 0.8, and 0.8. In order to generate optical flow maps, we use FlowNet2 (llg et al., 2017]. For our landmark detection, we use AlphaPose (Fang et al.] 2017) pretrained VIT B-16 CLIP (Dosovitskiy et al. 2020; Radford et al. 2021) image encoder as our deep feature extractor. Our method is built around the extracted objects and flow maps. We use Hvelocity x Weelocity = 224 x 224 to rescale flow maps. As for Hpose x W Woose rescaling, we calculate the average height and width from the bounding boxes of the train set and use those values. The lower resolution of Ped2 prevents objects from filling a histogram, and to extract pose representations, therefore we use = orientations and rely solely on velocity and deep representations, We use for anomaly scoring we ANN for the pose and deep representations with k 1 nearest neighbors. For velocity, we use GMM with = 5 Gaussians. Finally, the anomaly score of a frame represents the maximum score among all the objects within that frame.
MS-COCO with d = 17 keypoints. We
pretrained
orientations for Avenue and Shanghai
When testing,
4.3 Evaluation Metrics
Our study uses standard VAD evaluation metrics. We vary the threshold over the anomaly scores to measure the frame-level Area Under the Receiver Operation Characteristic (AUROC) with respect to the ground-truu -tuth annotations. We report two types of AUROC: (i) micro-averaged AUROC, which computes the score by on all frames from all videos; (ii) macro-averaged AUROC, which compules the AUROC score individually for each video and then averages the scores of all videos. Most existing studies report micro-averaged AUROC, while only a few report macro-averaged AUROC.
4.4 Quantitative Results
We compare our method and the state-of-the-art from recent years in Tab. numbers of the baseline methods directly from the original papers.
2
We took the performance
Ped2 results. Most methods obtained over 94% on Pod2, indicating that of the three public datasets, it is the simplest. While our method is comparable to the current state-of-the-art method (HF2 (Liu et al. 2021b)) in terms of performance, the near-perfect results Ped2 indicate it is practically solved.
Avenue results. Our method obtained new state-of-the-art micro-averaged AUROC of 93.7%. Our method also outperformed the current state-of-the-art in terms of macro-averaged AUROC by a considerable margin of 2.8%, reaching 96.3%.
ShanghaïTech results. Our method outperforms all previous methods on the largest dataset, Shang- haiTech, by a considerable margin. Accordingly, our method achieves 85.9% AUROC, higher than the best performance previous methods achieved, 8% (MS-VAD (Zhang et al. 2024 We note that in concurrent
7

---

Published in Transactions on Machine Learning Research (01/2025)
Table 2: Frame-level AUROC (%) comparison.
The best and second-best results are bolded and
underlined, respectively.
Ped2
Avenue
Shang SlanghaiTer Tech
Year
Method
Micro
Macro
Micro
Macro
Micro
Macro
HOOF (Chaudhry et al. 2009
61.1
HOFM Colque et al. 2016
89.9
SCL LL et al.) 2013
80.9
CE Hasan al. 2016a
90.0
70.2
StackRNN Luo et al. 20178
92.2
81.7
68.0
STAN Lee et al. 2018
96.5
87.2
<
2019
MC2ST Liu et al.) 2018b)
87.5
84.4
Frame- Pred. (Liu et al. 2018a)
95.4
85.1
72.8
Mem-AN AE. Gong et al.)
2019
94.1
83.3
71.2
CAE-SVM Jooneccu
2019
94.3
97.8
87.4
90.4
78.7
84.9
BMAN Lee et al.) 2019)
96.6
90.0
76.2
AM-CorT Nguyen & Meunier
2019
96.2
86.9
MNAD-Re Recon. (Park et al.
2020)
97.0
88.5
70.5
CAC Waag et al. 2020
87.0
79.3
2020
Scene- Aware (Sun el al.
2020
89.6
74.7
VFC Yu et al. 2020)
97.3
90.2
74.8
ClusterAE (Chang et al.
2020
96.5
86.0
73.3
AMMCN (Cai et al. 2021
96.6
86.6
73.7
SSMTL Georgescu
al. 2021a
97.5
99.8
91.5
91.9
82.4
89.3
MPN (Lv al.
2021
96.9
89.5
73.8
2021
HF2 (Liu al.
2021a
99.3
91.1
93.5
76.2
CT-D2GAN
Feng et al. 2021a
97.2
85.9
77.7
BA-AED Georgescu et al.
2021b
98.7
99.7
92.3
90.4
82.7
89.3
SSPCAB Ristea et al. 2022
92.9
91.9
83.6
89.5
2022
DLAN-AC Yang et al. 2022
97.6
89.9
74.7
Higaaw-Puzzle (Wang et al. 2022)
99.0
99.9
92.2
93.0
84.3
89.8
USTN-DSC (Yang al. 2023)
98.1
89.9
73.8
EVAL Singh
2023
86.0
76.6
FB-SAE Cao al.
2023
97.1
99.2
86.8
89.1
79.2
80.2
2023
FPDM Yan et al. 2023
90.1
78.6
LMPT Shi et al. 2023
97.6
90.9
78.8
STF-NF Hirschorn & Avidan
2023
93.1
91.2
60.1
63.5
85.9
87.8
SD-MAE Ristea et al.
2024
95.4
98.4
91.3
90.9
79.1
84.7
2024
MS-VAD Zhang et al.
2024
92.4
92.9
85.1
89.8
Ours
99.1
99.9
93.7
96.3
85.9
89.6
work, STF-NF (Hirschorn & Avidan 2023) achieved comparable results to ours on the ShanghaïTech dataset.
Our method outperforms it by large margins on Ped2 (by 6.0% AUROC) and Avenue (by 33.6% AUROC).
To summarize, our method achieves the highest performance on the three most popular public benchmarks. It simply consists of three simple representations and does not require training.
4.5 Analysis
Ablation study. We report Tab, 3 the anomaly detection performance on the Ped2, Avenue and Shang- haiTech datasets of all attribute combinations. Our findings reveal that the velocity fcatures provide the high- est frame-level AUROC on Ped2, Avenue and ShanghaiTech, with 98. 8%, 86.0% and 84.4% micro-averager velocity features their own already state-of-the-ar com-
AUROC, respectively. In Shanghai

---

Published in Transactions on Machine Learning Research (01/2025)
Table 3: Ablation study. Result are in frame-level AUROC (%). bold and underline, respectively.
The best and second-best results are
Ped2
Avenue
Shanghai'Tedh
Pose Features
Deep Features
Velocity Features
Micro
Macro
Micro
Macro
Micro
Macro
73.8
76.2
74.5
81.0
96.4
95.3
85.4
87.7
72.5
82.5
98.8
99.6
86.0
89.6
84.4
84.8
89.3
88.8
76.7
84.9
99.1
99.9
93.0
95.5
84.5
88.7
86.8
93.0
85.9
88.8
V
93.7
96.3
85.1
89.6
Table 4: Comparison of different numbers of velocity features bins (B). Frame-level AU-
Table 5: Our final results when &NN is replaced by k-means. Frame-level AUROC (%). Time is expressed in average ms per frame. Best in bold.
ROC (%) results. Best in bold.
Avenue
Shanghai Tech
Avenue Mac.
ShanghaiTech
Bins (B)
k
Micro
Macro
Micro
Macro
Mic.
Time
Mic.
Mac.
Time
B
83.5
83.5
81.2
80.9
91.8
94.0
0.51
84.2
87.2
0.45
B
84.1
83.8
82.1
82.7
92.0
94.2
0.52
84.3
88.1
0.45
B 4
85.5
89.2
84.0
84.6
10
92.1
94.5
0.52
84.6
88.1
0.45
B =3b 8
86.0
89.6
84.4
84.8
100
92.9
95.2
0.53
84.8
88.6
0.46
B = 16
84.1
88.4
83.1
84.2
All
93.7
96.3
4.93
85.1
89.6
36.0
pared with all previous VAD methods. We expect this to be due to the large number of anomalies associated with speed and motion, such as rumning people and fast-moving objects, e.g. cars and bikes. Adding either pose or CLIP improved performance, mostly macro-AUROC, presumably as it provided information about human activity which accounts for some of the anomalies in this dataset. Velocity features were still the most performant on Ped2 and Avenue. However, combining them with deep features improved performance significantly. Overall, observe that using all three features performed the best Avenue. Due to the extremely low resolution of the Ped2 dataset, pose feature extraction is not feasible, so rely solely on velocity and deep features for this dataset.
Number of velocity bins. We ablated the impact of diffcrent numbers of bins (B) in our velocity features in Tab. 4 We compared AUROC scores on the Avenue and ShanghaiTech datasets. The results indicate that the choice of influences detection accuracy. Specifically, we observed that increasing the number of bins from B to B 8 led to consistent improvements in both micro and macro AUROC scores on both datasets. This suggests that finer quantization of velocity orientations represents motion better and improves anomaly detection. Performance gains diminish beyond B 8.
k-Means as a faster alternative. Computing KNN has linear complexity in the number of objects in the datasets, which may be slow for large datascts. We can speed it up by reducing the number samples via k-means. In Tab. 5 we compare the performance of our method with &NN and k-means. Note that k-means still uses &NN to calculate anomaly scores the sum of distances to ncarest neighbor mcans. This much faster than the original ANN as there are fewer means than the number of objects in the training set. We observe that it improves inference time with a small accuracy loss.
Pose fcatures for non-human objects. We extract pose representations exclusively for human objects and not for non-human objects. We calculate the pose anomaly score for each frame by taking the score of the object with the most anomalous pose. Non-human objects are given a pose anomaly score -o0 and therefore do not contribute to the frame-wise pose anomaly score. While we acknowledge that non-human objects can also exhibit anomalies, our method leverages velocity and deep representations to capture these types of events.
9

---

Published in Transactions on Machine Learning Research (01/2025)
Table 6: Comparison of FlowNet2 vs. RAFT for flow map extraction. Frame-level AUROC (%) based on velocity features.
Table 7:
Comparison of Mask R-CNN vs.
YOLO-v8 for object detection. Frame-level AU-
ROC (%) based on velocity features.
Avenue
ShanghaiTech
Avenue
Shanghai Tech
Backbone
Backbone
Micro
Macro
Micro
Macro
Micro
Macro
Micro
Macro
RAFT
85.7
89.7
84.3
84.2
YOLO-v8
84.8
87.4
83.1
82.7
FlowNet2
86.0
89.6
84.4
84.8
Mask-RONN
86.0
89.6
84.4
84.8
Table 8: Comparison of video encoders and CLIP. Frame-level AUROC (%) results. Best in bold.
Avenue
Shanghai
Encoder
Level
Micro
Macro
Micro
Macro
TimeSformer
Bertasius al.
2021
Frame
61.2
64.1
58.2
60.1
TimeSformer
Bertasius et al.
2021
Object
63.1
64.0
59.1
59.2
VideoMAE V2
(Wang et al.
2023
Frame
68.3
67.9
60.3
60.5
VideoMAE V2
(Wang el al.
2023
Object
67.0
68.1
60.0
59.9
DINO
Object
77.6
81.2
71.2
80.3
CLIP (ours)
Object
85.4
87.7
72.5
82.5
Backbone analysis. We performed additional ablation studies to evaluate the impact of different backbone networks on the overall performance of our method. Specifically, we tested alternative backbones for optical flow (e12 vs. RAFT (Teed & Deng 2020)): as shown in Tab. t and object detection (Mask R-CNN vs. YOLO-v8) in Tab. 7 The results indicate that the effectiveness of our approach primarily driven by the feature design rather than any specific choice backbone.
Why do we use an image encoder instead of a video encoder? Recent self-supervised learning methods such TimeSformer (Bertasius al. 2021 VideoMAE (Tong et al. 2022: Wang et al. 2023 X-
CLIP (Ni et al. 2022), and CoCa (Yu et al. 2022 have significantly improved the performance of pretrained video encoders on downstream tasks like Kinetics-400 (Kay et al. (017). It is therefore reasonable to expect that video encoders, which capture both temporal and spatial information, would outperform image encodors in vidco anomaly detection (VAD). However, in our experiments, we found that fcatures extracted by paetrained video encoders did not perform as well as those extracted from pretrained image encoders on VAD benchmark datasets. We hypothesize that this weaker performance is due the video encoders' focus capturing frame-level temporal dynamics, whereas tested video encoders 10-frame windows of fixed object bounding baxes (centered around time t), we observed no performance gain, likely due to resolution constraints and the need for high-quality contextual information. Tab. S summarizes our findings on the limited effectiveness of video encoders in this setting. Additionally, we evaluated DINO (Caron et al. 2021) as a comparison to CLIP and found that while DINO performed slightly worse than CL.PP, still outperformed video encoders. This result, with DINO showing only slight performance drop compared to CLIP, demonstrates that our deep features are not dependent a specific image encoder.
method is object-centric. Additionally, when we
We carried out all our experiments stage, which includes object detection and optical flow extraction, takes approximately 80 milliseconds (ms) per frame. It takes our method approximately 5 ms to compute the velocity extraction, pose extraction, and deep features extraction stages, combined with anomaly scoring. Our method runs at 12FPS with an average of objects per frame. For comparison, we evaluated two other methods on the same hardware: BA-AED (Georgescu al. (2021b) runs at 24 FPS, while HF2 Liu al. (2021a). 2021) runs at 12 FPS. Our method's running speed comparable HF2 but slightly slower than BA-AED.
Running times.
NVIDIA RTX 2080 GPU. Our preprocessing
10

---

Published in Transactions on Machine Learning Research (01/202)
Anomaly score
Grundituhh
Goundsrruh
3
Anomaly 0.6
2-
230
aon
600
A00
100
200
300
400
Frame number
Frame number
Figure 4: Frame-level scores and anomaly localiza- tions for Avenue's 's test video 04. Best viewed in color.
Figure 5: Frame-level scores and anomaly localiza- tions for ShanghaiTech'ss test video 03 03_0059. Best viewed in color.
4.6 Qualitative Results
We visualize the anomaly detection process for Avenue and ShanghaiTech Fig. and Fig. 5 where we plot the anomaly scores across all frames of video. Our anomaly scores are clearly highly correlated with anomalous events, demonstrating the effectiveness of our method.
5 Discussion
Exploring other semantic attributes. There are other important attributes for VAD beyond velocity and pose. Identifying other relevant attributes that correlate with anomalous events can further improve anomaly detection systems. For example, attributes related to object interactions, spatial arrangements, or temporal patterns may be very discriminative for some types of anomalies. Finding ways to systematically discover such altributes may significantly speed up research.
Guidance. Finding relevant attributes for anomaly detection may require user guidance. In real-world scenarios, operators have domain knowledge about factors that lead to anomalous behavior. Efficiently incorporating this guidance, such selecting velocity or pose features in our work, is essential for leveraging this knowledge effectively.
Other academic benchmarks. While our method, using simple attributes, was effective on the three most popular VAD datasets, extending datasots such as UCF-Crime (Sultani al. 2018) and XD-Violence (Wu et al., 2020), which feature wider variety of anomalies and larger scales, present additional challenges. These datasets are essentially different from the ones tested here they contain distinct scenes in training and testing data, and include moving cameras, which also change the scene. So far, only weakly-supervised VAD has been successful on these datasets as they labeled anomalous data in training. The field needs new, more complex datasets within the fixed camera setting to further stress-test. one-class classification VAD methods such as ours.
more complex datasets may require more work. Publicly available
6 Ethical Considerations
While VAD offers significant potential for enhancing public safety and security, is crucial to acknowledge and address the ethical implications of such technology. VAD systems, including our proposed method, can be used in surveillance applications, which raises important privacy concerns. The continuous monitoring of public spaces may lead to sense of constant observation, potentially infringing individuals' right to privacy and freedom of movement. Morcover, there is risk that VAD systems could be misused for unauthorized tracking or profiling of individuals.
To mitigate these ethical risks, several strategies should be considered in VAD systems development and deployment. First, strict data protection protocols should be implemented to ensure that collected video data is securely stored, accessed only by authorized individuals, and deleted after a defined period. Second, VAD should be transparent, with clear warnings informing individuals when they entering areas
11

---

Published in Transactions on Machine Learning Research (01/2025)
under surveillance. Third, VAD systems should be designed with crivac-prreervvngg techniques, such immediate data anonymization or the use of low-resolution data that can detect anomalies without identifying individuals. By implementing these measures, we can work towards harnessing VAD technology benefits while respecting individual privacy and civil rights.
In addition to technical safeguards, it is also necessary to consider regulatory and oversight mechanisms ensure responsible deployment. We recommend that VAD systems be subject to civilian oversight, where independent authorities evaluate their use, especially in sensitive contexts like law enforcement public monitoring. Such oversight would help prevent potential misuse, ensuring that VAD systems are applied in ways that benefit socicty without compromising human rights. Furthermore, restrictions should be placod on VAD deployment for purposes other than public safely, with guidelines that limit its use to specific cases where the benefits clearly outweigh the risks. These guidelines could include requiring legal authorization for certain VAD uses, particularly private spaces or applications that extend beyond standard anomaly detection use-cases.
7 Conclusion
We propose simple yet highly effective attribute-based method for video anomaly detection (VAD). Our method represents each object in each frame using velocity and pose representations and uses density esti- mation to compute anomaly scores. These simple representations are sufficient to achieve state-of-tho-art performance on the ShanghaiTech and Ped2 datasets. By combining attribute-based representations with implicit deep representations, we achieve top VAD performance with AUROC scores of 99.1%, 93.7%, and 85.9% on Ped2, Avenue, and ShanghaiTech. respectively. Our extensive ablation study highlights the relative merits of the three representations. Overall, our method is both accurate and easy to implement.
Acknowweedgent
This research was partially supported by funding from the Israeli Science Foundation and the KLA Corpo- ration. Tal Reiss supported by the Google Fellowship and the Isracli Council for Higher Education.
References
Gedas Bertasius, Heng Wang, and Lorenzo Torresani. Is space-time attention all you need for video under- standing? In 1CML, volume 2, pp. 4, 2021.
Ruichu Cai, Hao Zhang, Wen Liu, Shenghua Gao, and Zhifeng Hao. Appearance-motion memory consistency network for video anomaly detection. In AAAI, 2021.
Congqi Cao, Yue Lu, Peng Wang, and Yanning Zhang. A new comprehensive benchmark for semi-supervised video anomaly detection and anticipation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20392 20401, 2023.
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 9650-9660, 2021.
Yumpeng Chang, Zhigang Tu, Wei Xie, and Junsong Yuan. Clustering driven deep autoencoder for vidco anomaly detection. In European Conference on Computer Vision, pp. 329 345. Springer, 2020.
Rizwan Chaudhry, Avinash Ravichandran, Gregory Hager, and René Vidal. Histograms of oriented optical flow and binet-cauchy kernels on nonlinear dynamical systems for the recognition of human actions. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1932-1939. IEEE, 2009.
Dongyue Chen, Pengtao Wang, Lingyi Yue, Yuxin Zhang, and Tong Jia. Anomaly detection in surveillance video based on bidirectional prediction. Image and Vision Computing, 98:103915, 2020.
12

---

Published in Transactions on Machine Learning Research (01/2025)
Rensso Victor Hugo Mora Colque, Carlos Caetano, Matheus Toledo Lustosa de Andrade, and William Robson Schwartz. Histograms of optical flow orientation and magnitude and entropy to detect anomalous events in videos. LEEE Transactions on Circuits and Systems for Video Technology, 27(3))673 682, 2016.
Carl Doersch, Abhinav Gupta, and Alexei Efros. Unsupervised visual representation learning by context prediction. In Proceedings of the IEEE international conference on computer vision, pp. 1422 1430, 2015.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Un- terthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, al. An image is worth 16x16 words:
Transformers for image recognition at scale. arXiv preprint mxXiv:2010.19920 2020.
Eleazar Eskin, Andrew Arnold, Michael Prerau, Leonid Portnoy, and Sal Stolfo. A geometrie framework for unsupervised anomaly detection. In Applications of data mining in computer security, pp. 77-101. Springer, 2002.
Hao-Shi Fang, Shuqin Xie, Yu- lu-ning Tai, and Cewu Lu. RMPE: Regional multi-person pose estimation. In ICCV, 2017.
Xinyang Feng, Dongjin Song, Yuncong Chen, Zhengzhang Chen, Jingchao Ni, and Haifeng Chen. Convolu- tional transformer based dual discriminator generalive adversarial networks for video anomaly detection. Proceedings of the 29th ACM International Conference on Multimedia, 2021a.
Xinyang Feng, Dongjin Song, Yuncong Chen, Zhengzhang Chen, Jingchao Ni, and Haifeng Chen. Convolu- tional transformer based dual discriminator generative adversarial networks for video anomaly detection. In Proceedings of the 29th ACM International Conference Multimedia, pp. 5546-5554, 2021b.
Mariana-Tuliana Georgescu, Antonio Bärbäläu, Radu Tudor lonescu, Fahad Shahbaz Khan, Marius Popescu, and Mubarak Shah. Anomaly detection in video via self-supervised and multi-task learning. 2021 IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12737-12747, 2021a.
Mariana Iuliana Georgescu, Radu Tudor lonescu, Fahad Shahbaz Khan, Marius Popescu, and Mubarak Shah. aggossti framework with adversarial training for abnormal event detection in video. TEEE transactions pattern analysis and machine intelligence, 44(9):4505-4522 2021b.
Spyros Gidaris, Praveer Singh, and Nikos Komodakis. Unsupervised representation learning by predicting image rotations. arXiv preprint atXiv:1803.07723 2018.
Michael Glodek, Martin Schels, and Friedhelm Schwenker. Ensemble gaussian mixture models for probability density estirnation. Computational Statistics, 28(1):127-133 2013.
Dong Gong, Lingqiao Liu, Vuong Le, Budhaditya Saha, Moussa Reda Mansour, Svetha Venkatesh, and Anton den Hengel. Memorizing normality to detect anomaly: Memory-augmented deep autoencoder for unsupervised anomaly detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1705-1714, 2019.
Mahmudul Hasan, Jonghyun Choi, Jan Neumann, Amit Roy-Chowdhury, and Larry S Davis. Learning temporal regularity in video sequences. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 733-742, 2016a.
Mahmudul Hasan, Jonghyun Choi, Jan Neumann, Amit K Roy-Chowdhury, and Larry S Davis. Learning temporal regularity in video sequences. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 733-742, 2016b.
Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In Proceedings of the TEEE international conference on computer vision, pp. 2961 2969, 2017.
Or Hirschorn and Shai Avidan. Normalizing flows for human pose anomaly detection. In Proccedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 13545-1355 October 2023.
13

---

Published in Transactions on Machine Learning Research (01/2025)
Eddy Ilg, Nikolaus Mayer, Tonmoy Saikia, Margret Keuper, Alexey Dosovitskiy, and Thomas Brox. Flownet 2.0: Evolution of optical flow estimation with deep networks. In Proceedings of the IEFE conference on computer vision and pattern recognition, pp. 2462 2470, 2017.
Radu Tudor Tonescu, Fahad Shahbaz Khan, Mariana-Iuliane Georgescu, and Ling Shao. Object-centric auto- encoders and dummy anomalies for abnormal event detection in video. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2019.
lan Jolliffe. Principal component analysis. Springer, 2011.
Will Kay, Joao Carreira, Karen Simonyan, Brian Zhang, Chloe Hillier, Sudheendra Vijayanarasimhan, Fabio Viola, Tim Green, Trevor Back, Paul Natsev, et al. The kinetics human action video dataset. arXiv preprint MxXiv:1705006500 2017.
Gustav Larsson, Michael Maire, and Gregory Shakhnarovich. Learning representations for automatic col- orization. In ECCV, 2016.
Longin Jan Latecki, Aleksandar Lazarevic, and Dragoljub Pokrajac. Outlier detection with kernel density functions. In International Workshop on Machine Learning and Data Mining in Pattern Recognition, pp. 61-75. Springer, 2007.
Sangmin Lee, Hak Gu Kim, and Yong Man Ro. Stan: Spatio-temporal adversarial networks for abnor- mal event detection. In 2018 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 1323 1323-1327. IEEE, 2018.
Sangmin Lee, Hak Gu Kim, and Yong Man Ro. Bman: bidirectional multi-scale aggregation networks for abnormal event detection. TEEE Transactions on Image Processing, 29:2395-2408, 2019.
Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference computer vision, pp. 740 755. Springer, 2014.
Wen Liu, Weixin Luo, Dongze Lian, and Shenghua Gao. Future frame prediction for anomaly detection-a new baseline. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 6536 6545, 2018a.
Yusha Liu, Chun-Liang Li, and Barnabás Póczos. Classifier two-sample test for video anomaly detections. In BMVC, 2018b.
Zhian Liu, Yongwei Nic, Chongjiang Long, Qing Zhang, and Guiqing Li. A hybrid vidco anomaly detection framework via menory-augmentedd flow reconstruction and flow-guided frame prediction. In Proceedings of the IEEE/CVF International Conference Computer Vision (ICCV), pp. 13588-1181 11599, October 2021a.
Zhian Liu, Yongwei Nie, Chengjiang Long, Qing Zhang, and Guiqing Li. A hybrid video anomaly detection framework via memory-augmentee flow reconstruction and flow-guided frame prediction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 13588 13597, 2021b.
David G Lowe. Distinctive image features from scale-invariant keypoints. International journal of computer vision, 60(2):91-110, 2004.
Cewu Lu, Jianping Shi, and Jiaya Jia. Abnormal event detection at 150 fps in matlab. In Proceedings of the IEEE international conference on computer vision, pp. 2720 2727, 2013.
Yiwei Lu, K Mahesh Kumar, Seyed shahabeddin Nabavi, and Yang Wang. Future frame prediction using convolutional vrnn for anomaly detection. In 2019 16Th IEEE international conference on advanced vidco and signal based surveillance (AVSS), pp. 1 8. IEEE, 2019.
Weixin Luo, Wen Liu, and Shenghua Gao. A revisit of sparse coding based anomaly detection in stacked rnn framework. In Proceedings of the IEEE international conference on computer vision, pp. 341-30 -499, 2017a.
14

---

Published in Transactions on Machine Learning Research (01/2025)
Weixin Luo, Wen Liu, and Shenghua Gao. A revisit of sparse coding based anomaly detection in stacked rnn framework. In Proceedings of the IEFE international conference on computer vision, pp. 341-349, 2017b.
Hui Lv; Chen Chen, Zhen Cui, Chunyan Xu, Yong Li, and Jian Yang. Learning normal dynamics in videos with meta prototype network. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 15425 15434, June 2021.
Vijay Mahadevan, Weixin Li, Viral Bhalodia, and Nuno Vasconcelos. Anomaly detection in crowded scenes. In 2010 IEEE computer society conference computer vision and pattern recognition, pp. 1975-1981. IEEE, 2010.
Amir Markovitz, Gilad Sharir, Itamar Friedman, Lihi Jelnik-Manor, and Shai Avidan. Graph embedded pose clustering for anomaly detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10539-10544 2020.
Michael Mathieu, Camille Couprie, and Yann LeCun. Deep multi-scale video prediction beyond mean square error. ICLR, 2016.
Ishan Misra, C Lawrence Zitnick, and Martial Hebert. Shuffic and learn: unsupervised learning using temporal order verification. In European conference on computer vision, pp. 527 544. Springer, 2016.
Romero Morais, Vuong Le, Truyen Tran, Budhaditya Saha, Moussa Mansour, and Svetha Venkatesh. Learn- ing regularity skeleton trajectories for anomaly detection in videos. In Procoedings of the TEEE/CVF conference on computer vision and pattern recognition, pp. 11996 12004, 2019.
Trong-Nguyen Nguyen and Jean Meunier. Anomaly detection in video sequence with appearanco-motion correspondence. In Proceedings of the IEEE/CVF international conference computer vision, pp. 1273 1283, 2019.
Bolin Ni, Houwen Peng, Minghao Chen, Songyang Zhang, Gaofeng Meng, Jianlong Fu, Shiming Xiang, and Haibin Ling. Expanding language-image pretrained models for general video recognition. arXiv preprint, 2022.
Mehdi Noroozi and Paolo Favaro. Unsupervised learning of visual representations by solving jigsaw puzzles. In ECCV, 2016.
Hyunjong Park, Jongyoun Noh, and Bumsub Ham. Learning memory-guided normality for anomaly detec- tion. 2020 IEEE/CV /CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 14360-14369, 2020.
Janez Pers, Vildana Sulié, Matej Kristan, Matej Perše, Klemen Polanec, and Stanislav Kovačič. Histograms of optical flow for efficient representation of body motion. Pattern Recognition Letters, 31(11):369--1376 2010.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, pp. 8748 8763. PMLR, 2021.
Tal Rciss and Yedid Hoshen. Mcan-shifted contrastive loss for anomaly detection. In Proceedings of the AAAI Conference Artificial Intelligence, volume 37, pp. 2155-2162 2023.
Tal Reiss, Niv Cohen, Liron Bergman, and Yedid Hoshen. Panda: Adapting pretrained features for anomaly detection and sogmentation. In Procoedings the TEEE/CVF Conference on Computer Vision and Pattern Recoguition, pp. 2806 2814, 2021.
Tal Reiss, Niv Cohen, Elihhu Horwitz, Ron Abutbul, and Yedid Hoshen. Anomaly detection requires better representations. In European Conference Computer Vision, pp. 56 68. Springer, 2022.
15

---

Published in Transactions on Machine Learning Research (01/2025)
Tal Reiss, George Kour, Naama Zwerdling, Ateret Anaby-Tavor, and Yedid Hoshen. From zero to hero: Cold-start anomaly detection. arXiv preprint arXiv:2405.22 20341, 2024.
Nicolae-C Ristea, Florinel-Alin Croitoru, Radu Tudor Tonescu, Marius Popescu, Fahad Shahbaz Khan, Mubarak Shah, al. Self-distilled masked auto-encoders are efficient video anomaly detectors. In Proccodings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 15984- 15995, 2024.
Sicolae-Cätälle Ristea, Neclu Madan, Radu Tudor Ionescu, Kamal Nasrollahi, Fahad Shahbaz Khan, Thomas B Moeslund, and Mubarak Shah. Self-supervised predictive convolutional attentive block for anomaly detection. In Proceedings of the LEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13576-13586, 2022.
Royston Rodrigues, Neha Bhargava, Rajbabu Velmurugan, and Subhasis Chaudhuri. Mulli-timescale tra- jectory prediction for abnormal human activity detection. In Proceedings the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 2626-2666 2224, 2020.
Bernhard Scholkopf, Robert C Williamson, Alex J Smola, John Shawe- Shawe-Taylor, and John C Platt. Support vector method for novelty detection. In NIPS, 2000.
Chenrui Shi, Che Sun, Yuwei Wu, and Yunde Jia. Video anomaly detection via sequentially learning multiple pretext tasks. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 10300- October 2023.
Ashish Singh, Michael Jones, and Erik G Learned-Miller. Eval: Explainable video anomaly localization. Proceedings of the IEFE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18717-18726, 2023.
Waqas Sultani, Chen Chen, and Mubarak Shah. Real-world anomaly detection in surveillance vidcos. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 6479 6488, 2018.
Che Sun, Y. Jia, Yao Hu, and Y. Wu. Scene-aware context reasoning for unsupervised abnormal event detection in videos. Proceedings of the 28th ACM International Conference Multimedia, 2020.
Shengyang Sun and Xiaojin Gong. Hierarchical semantic contrast for scene-aware video anomaly detection. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22846 22856, 2023.
Yao Tang, Lin Zhao, Shanshan Zhang, Chen Gong, Guangyu Li, and Jian Yang. Intograting prediction and reconstruction for anomaly detection. Pattern Recognition Letters, 129:123 130, 2020.
Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23 28, 2020, Proceedings, Part II 16, pp. 402 419. Springer, 2020.
Zhan Tong, Yibing Song, Jue Wang, and Limin Wang. Videomae: Masked autoencoders are data-efficient learners for self-supervised vidco pre-training. arXiv preprint arXiv:2203. 2022.
Guodong Wang, Yunhong Wang, Jie Qin, Dongming Zhang, Xiuguo Bao, and Di Huang. Video anomaly detection by solving decoupled spatio-temporal jigsaw puzzles. In European Conference on Computer Vision (ECCV), 2022.
Limin Wang, Bingkun Huang, Zhiyu Zhao, Zhan Tong, Yinan He, Yi Wang, Yali Wang, and Yu Qiao. Videomae v2: Scaling video masked autoencoders with dual masking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14549-14560, 2023.
Xuanzhao Wang, Zhengping Che, Bo Jiang, Ning Xiao, Ke Yang, Jian Tang, Jieping Ye, Jingyu Wang, and Qi Qi. Robust unsupervised video anomaly detection by multipath frame prediction. IEEE transactions neural networks and learning systems, 2021.
16

---

Published in Transactions on Machine Learning Research (01/2025)
Ziming Wang, Yuexian Zou, and Zeming Zhang. Cluster attention contrast for video anomaly detection. Proceedings of the 28th ACM International Conference Multimedia, 2020.
Donglai Wei, Joseph J Lim, Andrew Zisserman, and William Freeman. Learning and using the arrow of time. In Proceedings of the TEEE Conference Computer Vision and Pattern Recognition, pp. 8052 8052-8060, 2018.
Peng Wu, jing Liu, Yujia Shi, Yujia Sun, Fangtao Shao, Zhaoyang Wu, and Zhiwei Yang. Not only look, but also listen: Learning multimodal violence detection under weak supervision. In European Conference on Computer Vision (BCCV), 2020.
Cheng Yan, Shiyu Zhang, Yang Liu, Guansong Pang, and Wenjun Wang. Feature prediction diffusion model for video anomaly detection. In Proceedings of the JEEE/CVF International Conference on Computer Vision (ICCV), pp. 5527-5537, October 2023.
Zhiwei Yang, Peng Wu, Jing Liu, and Xiaotao Liu. Dynamic local aggregation network with adaptive clusterer for anomaly detection. In European Conference on Computer Vision, pp. 404 421. Springer, 2022.
Zhiwei Yang, Jing Liu, Zhaoyang Wu, Peng Wu, and Xiaotao Liu. Video event restoration based on key frames for video anomaly detection. In Proceedings of the IEEE/ Conference on Computer Vision and Pattern Recognition, pp. 14592 14601, 2023.
Muchao Ye, Xiaojiang Peng, Weihao Gan, Wei Wu, and Yu Qiao. Anopen: Video anomaly detection via deep predictive coding network. In ACM MM, 2019.
Guang Yu, Siqi Wang, Zhiping Cai, En Zhu, Chuanfu Xu, Jianping Yin, and Marius Kloft. Cloze test helps: Effective video anomaly detection via learning to complete video events. In Proceodings of the 28th ACM International Conference Multimedia, pp. 583-591, 2020.
Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung, Mojtaba Seyedhosseini, and Yonghui Wu. Coca: Contrastive captioners are image-text foundation models. arXiv preprint arXiv:2205.0191 2022.
Shoubin Yu, Zhongyin Zhao, Haoshu Fang, Andong Deng, Haisheng Su, Dongliang Wang, Weihao Gan, Cewu Lu, and Wei Wu. Regularity learning via explicit distribution modeling for skeletal video anomaly detection. arXiv preprint arXiv:2112.03649, 2021.
Menghao Zhang, Jingyu Wang, Qi Qi, Haifeng Sun, Zirui Zhuang, Pengfei Ren, Ruilong Ma, and Jianxin Liao. Multi-scale video anomaly detection by multi-grained spatio-temporal representation learning. In Proceedings of the IEEE/ IEEE/CVF /CVF Conference on Computer Vision and Pattern Recognition, pp. -1394, 2024.
Richard Zhang, Phillip Tsola, and Alexei A Efros. Colorful image colorization. In ECCV, 2016.
Yiru Zhao, Bing Deng, Chen Shen, Yao Liu, Hongtao Lu, and Xian-Sheng Hua. Spatio-temporal autoencoder for vidco anomaly detection. In ACM MM, 2017.
17

---

Published in Transactions on Machine Learning Research (01/2025)
A More Qualitative Results
We provide more qualitative results of our methods the evaluation datascts.
In Ped2, Fig. 6 and Fig. 7 demonstraae the effectiveness of our method, which can easily detect fast-moving objects such as trucks and bicycles. Accordingly, we can conclude that Ped2 has been practically solved based on the near-perfect results obtained by our method (as well as many others). Fig. shows that our mothod is capable of detecting anomalies within a short timeframe. Fig. and Fig. 10 provide more qualitative information regarding our method's ability detect anomalies of various types. In this way, our method achieves a new state-of-the-art in Avenue and ShanghaiTech, surpassing other approaches by a wide margin.
12.5
Anomaly score
7.5
5.0
2.5
25
50
75
100
125
150
175
Frame number
Figure 6: Frame-level scores and anomaly localization examples for test video 04 from Ped2.
Anomall score
Ground truth
Ours: AURCC-10
60
100
140
Frame number
Figure 7: Frame-level scores and anomaly localization examples for test video 05 from Ped2.
Anomall score
3
Ground-truth
Ours: 5UR0C-90.57
2
200
400
600
800
Frame number
Figure 8: Frame-level scores and anomaly localization examples for test video 03 from Avenue.

---

Published in Transactions on Machine Learning Research (01/2025)
Anomaly score
Ground-truth
1.5
AUROC
0.5
100
200
300
400
500
600
Frame number
Figure 9: Frame-level scores and anomaly localization examples for test video 0025 from Shanghai Tech.
Anomaly score
Ground truth
1.5
Ours: AUROC-URCC1E 100
1.0
0.5
Frame number
Figure 10: Frame-level scores and anomaly localization examples for test video 07_0048 from ShanghaiToch.
B More Analysis & Discussion
Per-scene breakdown. In Tab. g we present the per-scene performance of our method on the Shang- haiTech dataset, which is the only dataset among the three benchmarks that includes multiple scenes. The results demonstrate that our method performs consistently well across most scenes, with a few exceptions. Specifically, scenes 8, 9, and 10 demonstrate lower performance compared to others. These scenes (i.e., vidcos with the prefix 08 **, ** 10 10_**) feature anomalies involving complex actívitics such erratic jumping, throwing objects, and pushing people, as well frequent occlusions. Such activities involve high
Table 9: ShanghaiTech per-scene frame-level micro AUROC (%) results.
Scene Number
Total Test Frames
Total Test Anomalies
Anomaly Ratio
AUROC
01
11, 894
4, 884
0.41
88.2
02
1, 155
662
0.57
87.8
03
4, 090
1, 212
0.29
90.8
04
4, 761
1, 874
0.39
87.0
05
4,1
1, 016
0.24
94.6
06
1,470
702
0.47
94.5
07
3,368
886
0.26
92.7
08
3, 708
1, .92
0.53
67.8
09
361
84
0.23
72.6
10
2,213
1, 539
0.69
64.1
11
337
141
0.41
99.3
12
3.74
234
0.71
81.1
2

---

Published in Transactions on Machine Learning Research (01/2025)
levels of motion and interaction between multiple subjects, which likely challenges the velocity- based feature representations, leading to reduced performance.
What are the benefits of pretrained features? Previous anomaly detection works (Reiss et al. 2021
Reiss Hoshen 2023 Reiss et al. 2022; 2024) demonstrated that using feature extractors pretrained on external, generic datasets (e.g. ResNet on ImageNet classification) achieves high anomaly detection per- formance. This was demonstrated on a large variety of datasets across sizes, domains, resolutions, and symmetries. These representations achieved state-of-the-an performance distant domains, such as acrial, microscopy, and industrial images. As the anomalies in these datasets typically had nothing to do with velocity or human posc, it is clear the pretrained features model many attributes beyond velocity and pose. Consequently, by combining our attribule-based representations with CLIP's image encoder, we are able to emphasize both explicit attributes (velocity and pose) derived from real-world priors and attributes that cannot be described by them, allowing us achieve the best of both worlds.
3


Reviews:
[{"review_id": "0w9AqGB3db", "reviewer": "Reviewer_GfMb", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n- The paper is overall easy to follow.\n- The simplicity of the proposed method is appealing to me, particularly considering its high performance compared to the previous approaches. It not only offers ease of use but also provides efficiency and interpretability, e.g., the velocity and pose features are quite straightforward to analyze.\n- The experimental results successfully support the effectiveness of the proposed approach, and provides a detailed component-wise analysis, e.g., for their uses of the three different features.\n\n\n**Weaknesses**\n- The proposed method relies on the pose features, which is only available for humans. Ablation study says that the use of pose features have a small-but-consistent effectiveness (compared to the other two) to the final performance. This raises a question about any possible alternative when such a pose features are not available. \n- The proposed method highly depends on the performance of the (pre-trained) backbone feature extractors, viz., for obtaining optical flows, bounding boxes, and CLIP features. A missing ablation study here is the individual effects of the choice of these backbones. For example, could the method further advance the state-of-the-art if we just take larger backbones? Which backbone network is the most influential for the final performance?\n- On the flip side of its simplicity, I found few technical significance in the proposed method; it can be essentially regarded as a straightforward combination of basic features. Additional justification for the method design, particularly addressing why this approach is unique and effective in the context of VAD, would be be helpful.\n- Although the paper provides an extensive comparison with previous methods up to 2023, it lacks of comparison with more recent methods from 2024 - for example, [1].\n- The ablation study was not performed on Ped2 - wondering if there is any particular reason for this.\n- The paper reports that its method runs at 12FPS overall. But there is no comparison of this value with other baselines, making it hard to evaluate in terms of its efficiency.\n\n[1] Micorek et al., MULDE: Multiscale Log-Density Estimation via Denoising Score Matching for\nVideo Anomaly Detection, CVPR 2024.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper claims a new state-of-the-art in video anomaly detection (VAD) tasks, where the goal is to identify anomalous frames within a given video clip. The method is quite simple. Specifically, it is based on the following three features - (a) the velocity feature: an 8-dimensional aggregation of its optical flow map, (b) the pose feature: the normalized coordinates of $d$ keypoints, and (c) the CLIP image embedding. A per-object anomaly score is obtained via density estimation of these features, either using Gaussian mixture (for the velocity feature) and kNN (for the others). Experimental results show that the proposed method outperforms previous methods on three public benchmarks - UCSD Ped2 (99.1% AUROC), CUHK Avenue (93.7%), and ShanghaiTech Campus (85.9%) - despite its simplicity.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Any discussion about scenarios when the pose features are not available, e.g., for anomaly detection of general objects.\n- A further ablation study about the choice of backbone networks, e.g., flow net, object detector, and deep feature extractor.\n- Comparison with more recent state-of-the-art methods.\n- (minor) The overall notations could be improved; for example, the current notation states that the anomaly score $s(\\phi(f))$ is solely based on the feature $\\phi(f)$, i.e., per-frame. In my understanding this is not true, as the the features are actually defined using the optical flow information that sees other frames as well. The notation in Eq. 2 is also confusing to me; here $s(\\cdot)$ is used inside the definition of $s(\\cdot)$ itself.\n- (minor) Starting sentences with I.e. or E.g. looks a bit informal to me.", "broader_impact_concerns": "As discussed in Section 6 of the paper, the technique developed in this work - VAD systems with a human focus - naturally carries potential risks of misuse: particularly in applications related to privacy concerns, e.g., for unauthorized human surveillance. The paper does suggest several technical strategies to mitigate such risks; it may be beneficial to further include a discussion about any recommendations for regulatory oversight beyond the technical aspects, for ensuring that VADs are deployed responsibly - e.g., limiting use cases to public safety, ensuring civilian oversight over the VAD system, etc.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "vdksupjnyJ", "reviewer": "Reviewer_z24s", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**(Strength) The method is simple yet effective.**\nThe proposed method leverages features readily available from the off-the-shelf models, such as optical flows (velocity) and object detectors (pose), and CLIP (a deep feature). Although the method has a simple design to combine the anomaly scores from each feature, it still achieves state-of-the-art performance in the experiment.\n\n**(Weakness) The motion attribute design solely depending on optical flows.**\nThe proposed method relies on velocity features based on optical flows, which might be effective for (single) object-centric anomalies, but might struggle in more complex scenarios, such as those involving interactions between multiple objects causing occlusions. In such a case, the optical flow method may introduce significant errors in the motion attribute.\n\n**(Weakness) The ablation study lacks the comparison of the feature extraction methods.**\nThe proposed method essentially combines the velocity, pose, and deep features for a better video anomaly detection, and only one existing model for each feature is considered throughout the paper. However, there are different approaches for these features, such as point tracking (instead of optical flow, which cannot model the occlusions) [1, 2] and video backbones (instead of CLIP, which only models the per-frame feature) [3, 4]. Since video anomaly detection fundamentally requires an understanding of object interactions (including occlusions) and video representation learning, the reviewer believes it is important to consider these backbones in the proposed method.\n\n**(Weakness) Important details are missing in the experiment and discussions.**\n- While the authors argue that replacing kNN with k-means provides a faster alternative, its runtime speed is not presented.\n- Most baselines are missing the macro AUROC results in Table 1. However, these results could be filled in for those baselines that provide the open source implementation and checkpoints. For example, the official repository for the latest baselines considered in Table 1 can be readily found by searching in Google, e.g., [5, 6].\n- What are the ground truth statistics for the benchmark? For example, it would be useful to present the duration, the proportion of anomalous frames in each video. Since the benchmarks are run with only 12 / 21 / 107 test scenes, it would also be useful to present discussions per scene (possibly in the appendix).\n\n**(Weakness) Some claims need more support and description.**\n- On page 3, it is claimed that \"Since determining true velocity requires 3D understanding, we represent it using optical flow\". However, this reviewer does not fully understand how optical flow can provide 3D understanding, since it only provides frame-by-frame 2D translation of pixels.\n\n[1] Doersch et al., \"TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinement\", ICCV 2023\n\n[2] Karaev et al., \"CoTracker: It is Better to Track Together\", ECCV 2024\n\n[3] Wang et al., \"VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking\", CVPR 2023\n\n[4] Li et al., \"VideoMamba: State Space Model for Efficient Video Understanding\", ECCV 2024\n\n[5] Hirschorn et al., \"STG-NF\"; https://github.com/orhir/STG-NF\n\n[6] Cao et al., \"FB-SAE\"; https://campusvad.github.io/", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a method for Video Anomaly Detection (VAD; a binary classification task to detect whether each frame in a given video is an anomaly or not) based on a frame representation that combines several different features focusing on object attributes such as velocity and pose, along with deep representations from a pre-trained CLIP. Given this representation, a K-nearest neighbor method (for the pose and deep features) and a Gaussian Mixture Model (GMM; for the velocity feature) are used to compute the anomaly score as given in Equation 2, and the final decision is made if the score exceeds a threshold.\n\nThe experiment is designed to reveal the binary classification performance in terms of AUROC, and the authors claim that the proposed method leads to state-of-the-art performance on three major VAD benchmarks: Ped2, Avenue, and ShanghaiTech. As for the ablation study, different combinations of the three features (i.e., the velocity feature, the pose feature, and the CLIP feature) are tested, the influence of the number of bins hyperparameter in the velocity feature is studied, and finally, an alternative model that replaces the kNN method with a lightweight k-means is evaluated, which trade-offs some classification performance for a practical runtime speed of the method.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please refer to the weaknesses, and provide further explanations or possibly the update in the manuscript.", "broader_impact_concerns": "I have no concerns about broader impacts.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "UFGIhI9YRg", "reviewer": "Reviewer_zhpF", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. The paper is well-written and easy to follow.\n\n2. The experimental part is solid, including sufficient baselines, ablation studies, and hyperparameter analysis.\n\n\nWeaknesses:\n\n1. My main concern is the generalizability of the proposed method. In my opinion, the anomaly scenarios in a video can be associated with non-human objects. In addition, for both human and non-human objects, attributing their anomaly behaviors purely to their poses, velocities, and depths may be over-simplified. As a result, although the proposed method achieves encouraging performance in the existing datasets, the tasks associated with these datasets are too simple to reflect real-world video anomaly detection problems. Actually, many baselines can also achieve high performance (>90% accuracy), which verifies my concern to some extent.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "In this submission, the authors proposed a simple but effective video anomaly detection method based on the attributes of visual objects in video. In particular, the authors found that in existing datasets, most anomaly scenarios are associated with anomaly pose, velocity, and depth of human beings. Accordingly, the authors combined a flow-based model with a visual detector, extracting representations for the three attributes. Based on these representations, the proposed video anomaly detector achieves higher accuracy in existing datasets.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "See the weaknesses above.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a simple-yet-effective method for video anomaly detection (VAD). It claims that combining simple features, such as velocity, pose, and CLIP-based deep representations, is sufficient to achieve near state-of-the-art performance on VAD. The claim is validated through experiments conducted primarily on three public benchmarks. 

All reviewers have acknowledged the simplicity and effectiveness of the proposed method, affirming that the paper's main claim is well-supported. Additionally, the authors successfully addressed reviewers’ comments in their revision, leading all reviewers to agree on the paper's acceptance. Therefore, AE recommends acceptance.
Decision: Accept (poster)