Abnormal Event Detection at 150 FPS in MATLAB
Cewu Lu
Jianping Shi
Jiaya Jia
The Chinese University of Hong Kong{cwlu, jpshi, leojia}@cse.cuhk.edu.hk

Abstract

15, 6, 2], which utilize co-occurrence patterns. Amongthese methods, normal patterns were fitted in a spacetime Markov random field in [21, 11, 10, 3]. Kwon and
Lee [9] used a graph editing framework for abnormal event
detection. Topic models, such as latent Dirichlet allocation,
were employed [19, 15]. Recently, sparse representation[12, 17] has attracted much attention and sparsity-basedabnormality detection models [22, 5] achieved state-of-theart performance reported in many datasets.
Although realtime processing is a key criterion to anpractically employable system given continuously captured
videos, most sparsity-based methods cannot be performed
fast enough. The major obstruction to high efficiency is
the inherently intensive computation to build the sparse
representation. Note a slow process could delay alarm andpostpone response to special events.
We provide brief analysis below about this issue withrespect to the general sparsity strategies and present our
new framework with an effective representation. It fits
the structure of surveillance video data and leads to an
extremely cheap testing cost.

Speedy abnormal event detection meets the growing
demand to process an enormous number of surveillance
videos. Based on inherent redundancy of video structures,
we propose an efficient sparse combination learning framework. It achieves decent performance in the detection phasewithout compromising result quality. The short running
time is guaranteed because the new method effectively turns
the original complicated problem to one in which only
a few costless small-scale least square optimization steps
are involved. Our method reaches high detection rates
on benchmark datasets at a speed of 140∼150 frames per
second on average when computing on an ordinary desktop
PC using MATLAB.

1. Introduction
With the increasing demand of security, surveillancecameras are commonly deployed. Detecting abnormal
events is one critical task based on what cameras capture,
which is traditionally labor-intensive and requires non-stop
human attention. What makes this interminable and boringprocess worse is that abnormal events generally happen
with a frustratingly small chance, making over 99.9% of
the effort for one to watch videos wasted.
This predicament catalyzes important research in computer vision, aiming to find abnormal events automatically [8, 1, 13, 23, 16, 21, 11, 22, 5]. It is not a typicalclassification problem due to the difficulty to list all possible
negative samples. Research in this area commonly follows
the line that normal patterns are first learned from training
videos, and are then used to detect events deviated from thisrepresentation.
Specifically, in [8, 20], extracted trajectories were utilized by tracking object-of-interest to represent normal
patterns. Outliers are regarded as abnormal. Another line isto learn normal low-level video feature distributions, suchas exponential [1], multivariate Gaussian mixture [13] orclustering as a special case [23, 16]. Graph model normal
event representations were proposed in [21, 11, 10, 3, 19,

1.1. Sparsity Based Abnormality Detection
Sparsity is a general constraint [22, 5] to model normal
event patterns as a linear combination of a set of basis
atoms. We analyze abnormality detection in one local region to show that this process is computationally expensive
by nature.
Given training features [x1 , . . . , xn ] extracted from thehistory video sequence in a region, a normal pattern dictionary D ∈ Rp×q is learned with a sparsity prior. In
the testing phase for a new feature x, we reconstruct it bysparsely combining elements in D, expressed asmin x − Dβ22
βs.t. β0 ≤ s(1)where β ∈ Rq×1 contains sparse coefficients. x − Dβ22
is the data fitting term; β0 is the sparsity regularization
term; and s ( q) is a parameter to control sparsity. With
this representation, an abnormal pattern can be naturally
defined as one with large error resulted from x − Dβ22 .
1

Figure 1. Our testing architecture. X denotes testing data.
{S1 , . . . , SK } are learned combinations, with each Si ∈Rp×s (s  q). Ei is the corresponding least square reconstruction
error. The final error is the minimum among all combinations.

Previous work verified that this form can lead to highdetection accuracy.
Efficiency Problem A high testing cost is inevitable
when adopting Eq. (1), which aims to find the suitablebasis vectors (with scale s) from the dictionary (with scaleq) to represent testing data x. The search space is verylarge, as (qs ) different combinations exist. Although much
effort has been put to reducing the dictionary size [5] andadopting fast sparse coding solvers [22], in general, seconds
are needed to process a frame as reported in prior papers.
The efficiency problem is thus critical to address before
this type of methods can be deployed practically. A realtime
process needs to be 100 times faster than the current
fastest sparsity-based methods, which is difficult without
tremendous hardware advancement. We tackle this problem
from an algorithm perspective. Our method yields decent
performance and naturally accelerates sparse coding by
400+ times even using MATLAB implementation.

1.2. Our Contribution
We propose sparse combination learning for detection.
With high structure redundancy in surveillance videos,
instead of coding sparsity by finding an s basis combinationfrom D in Eq. (1), we code it directly as a set of possiblecombinations of basis vectors. Each combination here
corresponds to a set with s dictionary bases in Eq. (1).
With this change, other than searching s bases from p ofthem for each testing feature, we only need to find the mostsuitable combination by evaluating the least square error.
The testing framework is shown in Fig. 1.
This framework is efficient since only small-scale leastsquare optimization is required in detection with simple
matrix projection. In our experiments, testing is on a small
number of combinations, each takes 10−6 ∼ 10−7 secondin MATLAB.
The effectiveness of our approach is well guaranteed by
the inherent sparsity constraint on the combination size.

Figure 2. Pyramid region architecture. A frame is resized into 3
different scales. In each scale the frame is partitioned into several
regions.

Compared to original sparse coding, our model is morefaithful to the input data. When freely selecting s basis vectors from a total of q vectors by Eq. (1), the reconstructedstructure could much deviate from input due to the large
freedom. But in our trained combinations, it is unlikely
to happen, since each combination finds its corresponding
input data, better constraining reconstruction quality. Our
method therefore is robust to distinguish between normal
and abnormal patterns.
We have verified our model on a large set of surveillance
videos in Sec. 3.2. We also benchmark it on existingdatasets for abnormal event detection. It reaches 140∼150
FPS using a desktop with 3.4GHz CPU and 8G memory in
MATLAB 2012.

2. Method
We describe our framework that learns sparse basis combinations. To extract usable data, we resize each frame into
different scales as [5] and uniformly partition each layer
to a set of non-overlapping patches. All patches have thesame size. Corresponding regions in 5 continuous frames
are stacked together to form a spatial-temporal cube. An
example is illustrate in Fig. 2. This pyramid involves local
information in fine-scale layers and more global structuresin small-resolution ones.
With the spatial-temporal cubes, we compute 3D gradient features on each of them following [11]. These featuresin a video sequence are processed separately according totheir spatial coordinates. Only features at the same spatial
location in the video frames are used together for training
and testing.

2.1. Learning Combinations on Training Data
For each cube location, 3D gradient features in all frames
are denoted as X = {x1 , . . . , xn } ∈ Rp×n , gathered
temporally for training. Our goal is to find a sparse basis
combination set S = {S1 , . . . , SK } with each Si ∈ Rp×s
containing s dictionary basis vectors, forming a unique
combination, where s  q. Each Si belongs to a closed,
convex and bounded set, which ensures column-wise unit

norm to prevent over-fitting.
Our sparse combination learning has two goals. The
first goal – effective representation – is to find K basis
combinations, which enjoy a small reconstruction error t.
It is coarsely expressed as
t 

n 
K

min
γji xj − Si β ij 22S,γ , β j=1 i=1

s.t.

K


(2)

γji = 1, γji = {0, 1}

i=1

where γ = {γ 1 , . . . , γ n } and γ j = {γj1 , . . . , γjK }. Eachγji indicates whether or not the ith combination Si is chosen
for data xj . βij is the corresponding coefficient set for repre
senting xj with combination Si . The constraints γji = 1
and γji = {0, 1} require that only one combination is
selected. The objective function makes each training cubex constructible by at least one basis combination in S.
The second goal is to make the total number K of combinations small enough based on redundant surveillance video
information. It is natural and inevitable because a very large
K could possibly make the reconstruction error t in Eq. (2)always close to zero, even for abnormal events.

2.2. Optimization for Training
Our two objectives contradict each other in a sense.
Reducing K could increase reconstruction errors. It is not
optimal to fix K as well, as content may vary among videos.
This problem is addressed in our system with a maximum
representation strategy. It automatically finds K while not
wildly increasing the reconstruction error t. In fact, error t
for each training feature is upper bounded in our method.
We obtain a set of combinations with a small K by
setting a reconstruction error upper bound λ uniformly for
all elements in S. If the reconstruction error for each featureis smaller than λ, the coding result is with good quality. So
we update function (2) to∀j ∈ {1, · · · , n}, tj =Kγji {xj − Si βij 22 − λ} ≤ 0,

maximum commonness. This process ends until all training
data are computed and bounded. The size of combination
K reflects how informative the training data are.
Specifically, in the ith pass, given the leftover training
data Xc ⊆ X that cannot be represented by previous
combinations {S1 , . . . , Si−1 }, we compute Si to boundmost data in Xc . Our objective function becomesγji (xj − Si β ij 22 − λ)minSi ,γ , β j∈Ω
c
(4)
K

i
i
γj = 1, γj = {0, 1}
s.t.
i=1

where Ωc is the index set for Xc . It is easy to prove that thiscost satisfies condition (3) and the resulting Si can represent
most data. Specifically, if xj − Si β ij 22 − λ ≥ 0, setting
γji = 0 yields a smaller value compared to setting γji =1. Contrarily, γji should be 1 if xj − Si β ij 22 − λ < 0,complying with condition (3).
In each pass i, we solve the function in Eq. (4) bydividing it into two steps to iteratively update {Sis , β} andγ using the following procedure.
Update {Sis , β} With fixed γ, Eq. (4) becomes aquadratic functionγji xj − Si β ij 22 .
(5)L(β, Si ) =j∈ΩcFollowing the traditional procedure, we optimize β while
fixing Si for all γji = 0 and then optimize Si using blockcoordinate descent [14]. These two steps alternate. The
closed-form solution for β is
βij = (STi Si )−1 STi xj .

(6)

Sis finds its solution asSi =[Si − δt ∇Si L(β, Si )],(7)where δt is set to 1E − 4 and denotes projecting the basis
to a unit column. Block-coordinate descent can converge
to a global optimum due to its convexity [4]. Therefore,
the total energy for L(β, Si ) decreases in each iteration,
guaranteeing convergence.

i=1

s.t.

K

i=1

Update γ With the {Si , β} output, for each xj , theobjective function becomes

γji = 1, γji = {0, 1}(3)

Algorithm Our method performs in an iterative manner.
In each pass, we update only one combination, making it
represent as many training data as possible. This process
can quickly find the dominating combinations encoding
important and most common features. Remaining training cube features that cannot be well represented by thiscombination are sent to the next round to gather residualγji xj − Si βij 22 − λγji
min
i
γj

s.t. γji = 0 or 1 .
γji has a closed-form solution
1
if xj − Si βij 22 < λiγj =0otherwise
The update step guarantees condition (3).

(8)

(9)

Algorithm 1 Training for Sparse Combination Learning
Input: X , current training features Xc = X
initialize S = ∅ and i = 1
repeatrepeatOptimize {Si , β} with Eqs. (6) and (7)Optimize {γ} using Eq. (9)until Eq. (5) converges
Add Si to set S
Remove computed features xj with γji = 0 from Xc
i=i+1
until Xc = ∅Output: S
Algorithm Summary and Analysis In each pass, welearn one Si . We repeat this process to obtain a few
combinations until the training data set Xc is empty. This
scheme reduces information overlap between combinations.
We summarize our training algorithm in Algorithm 1. The
initial dictionary Si in each pass is calculated by clustering
training data Xc via K-means with s centers.
Our algorithm is controlled by λ, the upper bound ofreconstruction errors. Reducing it could lead to a larger
K. Our approach is expressive because all training normal
event patterns are represented with controllable reconstruction errors under condition (3). We train 20K-sample data,
whose scales are in R100 , in 20 minutes on a PC with 8GB
RAM and an Intel 3.4GHz CPU.

2.3. TestingWith the learned sparse combinations S = {S1 . . . SK },in the testing phase with new data x, we check if there exists
a combination in S fitting the reconstruction error upperbound. It can be quickly achieved by checking the leastsquare error for each Si :
x − Si β i 22minβi∀ i = 1, . . . , K(10)

It is a standard quadratic function with the optimal solution
i = (ST S )−1 ST x.
β
i i
i

(11)

The reconstruction error in Si isi 2 = (S (ST S )−1 ST − I )x2 ,x − Si β
i
p
2
i i
i
2

It is noted that the first a few dominating combinations
represent the largest number of normal event features,
which enable us to determine positive data quickly. In
our experiments, the average combination checking ratio
is 0.325, which is the number of combinations checked
divided by the total number K. Also, our method can beeasily accelerated via parallel processing to achieve O(1)complexity although it is generally not necessary.

2.4. Relation to Subspace Clustering
Our approach can be regarded as enhancement of subspace clustering [7] with the major difference on the working scheme. The relationship between subspace clusteringand our method is similar to that between K-means andhierarchical clustering [18]. Specifically, subspace clustering method [7] takes the number of clusters k as known
or fixed beforehand, like K-means. In video abnormality
detection applications, it is however difficult to know the
optimal number of bases in prior. Our approach utilizes theallowed representation error to build combinations, where
the error upper bound is explicitly implemented with clear
statistical meaning. There is no need to define the cluster
size in this method. Our extensive experiments manifest
that our strategy is both reliable and efficient.

3. Experiments
We empirically demonstrate that our model is suitableto represent general surveillance videos. We apply our
method to different datasets. Quantitative comparisons areprovided.

3.1. System Setting(12)

where Ip is a p × p identity matrix. To further simplify
computation, we define an auxiliary matrix Ri for each Si :Ri = Si (STi Si )−1 STi − Ip .

Algorithm 2 Testing with Sparse Combinations
Input: x, auxiliary matrices {R1 , . . . , RK } and threshold T
for j = 1 → K do
if Rk x22 < T thenreturn normal event;end ifend forreturn abnormal event;(13)

Reconstruction error for Si is accordingly Ri x22 . If it issmall, x is regarded as a normal event pattern. The final
testing scheme is summarized in Algorithm 2.

In our method, size of Si ∈ Rp×s controls the sparsity
level. We experimentally set s = 0.1 × p where p is the data
dimension. λ in Eq. (4) is the error upper bound, set to 0.04
in experiments.
Given the input video, we resize each frame to 3 scaleswith 20×20, 30×40, and 120×160 pixels respectively anduniformly partition each layer to a set of non-overlapping
10 × 10 patches, leading to 208 sub-regions for each frame
in total shown in Fig. 2. Corresponding sub-regions in 5

Figure 3. A few frames from the surveillance videos used forverification.

Figure 5. Spatial distribution of combination numbers to represent
normal structures in the Avenue data.

6000

Ground Truth
Ours

5000
4000

Run
4
4

Loiter
5
4

Throw
5
4

False Alarm
N/A
1

Table 1. Detection results in the Avenue dataset.

3000
2000
1000
0

20

40

60

80

100

120

Figure 4. Different numbers of basis combinations to represent
normal events in 31,200 groups (x-axis: K; y-axis: number ofgroups that use K combinations).

continuous frames are stacked together to form a spatialtemporal cube, each with resolution 10 × 10 × 5. We
compute 3D gradient features on each of them following[11]. Those gradients are concatenated to a 1500-dimension
feature vector for each cube and are then reduced to 100
dimensions via PCA. Normalization is performed to makethem mean 0 and variance 1.
For each frame, we compute an abnormal indicator V bysumming the numberof cubes in each scale with weights.
n
It is defined as V = i=1 2n−i vi , where vi is the numberof abnormal cubes in scale i. The top scale is with index
1 while the bottom one is with n. All experiments are
conducted using MATLAB.

3.2. Verification of Sparse Combinations
Surveillance videos consist of many redundant patterns.
For example, in subway exit, people generally move insimilar directions. These patterns share information codedin our sparse combinations. To verify it, we collect 150
normal event surveillance videos with a total length of 107.8
hours. The videos are obtained from sources including
dataset UCSD Ped1 [15], Subway datasets [1] (excluding
abnormal event frames), 68 videos from YouTube, and 79
videos we captured. The scene includes subway, mall,traffic, indoor, elevator, square, etc. We show a few example

frames in Fig. 3.
Each video contains 208 regions as illustrated in Fig. 2.
With the 150 different videos, we gather a total of 31,200(208 × 150) groups of cube features with each groupcorresponding to a set of patches (cubes). They are used
separately to verify the combination model. Each groupcontains 6,000-120,000 features. The number of combinations for each group is denoted as K. We show in Fig. 4 the
distribution of K in the 31,200 groups. Its mean is 9.75 andvariance is 10.62, indicating 10 combinations are generally
enough in our model. The largest K is 108. About 99% of
the Ks are smaller than 45.
We illustrate the K distributions spatially in the Avenue
data (described below) in Fig. 5. Many regions only need
1 combination because they are static. Largely varying
patches may need dozens of combinations to summarize
normal events. The statistical regression error is as small as
0.0132±1.38E−4, which indicates our dictionaries containalmost all normal patterns.

3.3. Avenue Data Benchmark
We build an avenue dataset, which contains 15 sequences. Each sequence is about 2 minutes long. The total
number of frames is 35,240. There are 14 unusual events
including running, throwing objects, and loitering. 4 videos
are used as training data with 8,478 frames in total.
A video sequence and its abnormal event detection result
are demonstrated in Fig. 6. Fig. 7 contains two importantframes and their abnormal event regions in two image
scales. We list the detection statistics in Table 1. The
performance of our method is satisfactory with the average
detection rate of 141.34 frames per second.

100
50
0
0

100

200

300

400

500

600

Figure 6. Detection results in a video sequence. The bottom plot is the response. A peak appears when an abnormal event – paper throwing– happens. The x value indexes frames and y-index denotes response strength.

(a) events(b) maps(c) maps
Figure 7. Two abnormal events and their corresponding abnormal
patches under two different scales in the Avenue dataset.

Ground Truth
[22]
[10]
[5]
subspace
Ours

WD
9
9
9
9
6
9

LT
3
3
3
3
3

MISC
7
7
7
5
7

Total
19
19
19
14
19

FA
0
2
3
2
4
2

Table 2. Comparison with other sparsity-based methods [22, 5] on
the Exit-Gate Subway dataset. WD: wrong direction; LT: loitering;
FA: false alarm. “-” means the results are not provided. Subspace:results by replacing our combination learning by subspace clustering [7].

(a) events(b) maps(c) maps
Figure 8. Subway dataset (Exit-Gate): Three abnormal events andtheir corresponding detection maps in two different scales in the
Subway-Exit video.

GT
[22]
[10]
[5]
subspace
Ours

WD
26
25
24
21
21
25

NP
13
9
8
6
6
7

LT
14
14
13
9
13

II
4
4
4
3
4

MISC
9
8
8
7
8

Total
66
60
57
46
57

FA
0
5
6
4
7
4

Table 3. Comparison using the Subway-Entrance video with several previous methods. GT: ground truth; WD: wrong direction;
NP: no payment; LT: loitering; II: irregular interactions; MISC:
misc; FA: false alarm. “-” means results are not provided.
Subspace: replacing our combination learning by subspace clustering [7].

3.4. Subway Dataset

Second/Frame2
4.6
0.00641

Platform
MATLAB 7.0
MATLAB 2012

CPU
2.6 GHz
2.6 GHz
3.4 GHz

Memory
2.0GB
2.0GB
8.0GB

We conduct quantitative comparison with previous methods on the Subway dataset [1]. The videos are 2 hours longin total, containing 209,150 frames with size 512 × 384.
There are two types of videos, i.e., “exit gate” and “entrancegate” videos.

[22]
[5]
Ours

Exit Gate The subway exit surveillance video contains
19 different types of unusual events, such as walking in
the wrong direction and loitering near the exit. The video
sequence in the first 15 minutes is used for training. Thisconfiguration is the same as those in [10, 22].
The abnormal event detection results for a few frames
are shown in Fig. 8. Table 2 lists the comparison with othermethods. Our false alarm rate is low mainly because eachcombination can construct many normal event features, thus

reducing the chance of constructing an abnormal structurewith a small error. This representation tightens feature modeling and makes it not that easy to misclassify abnormality
as normal events. In this dataset, our combination number
K varies from 1 to 56 for different cube features.

Table 4. Running time comparison on the Subway dataset.

Entrance Gate In this video, again, the first 15 minutes
are used for training. Detection statistics are listed in Table
3. Our results are comparable to those of [22, 10, 5]. The

1

[13]
[5]
[2]
Ours

TPR

0.8
SparseDMTSFMPPCAMPPCA+SF
Adam
Antic
Saligrama
Subspace
Ours

0.6
0.4
0.2
0
0

0.2

0.4

0.6

0.8

TPR

0.8

1

SparseDMTSFMPPCAMPPCA+SF
Adam
Subspace
Ours

0.6
0.4
0.2
0
0

0.2

0.4

0.6

0.8

CPU
3.0 GHz
2.6 GHz
3.4 GHz

Memory
2.0GB
2.0GB
8.0GB

show the results via ROC curves, Equal Error Rate (EER),
and Equal Detected Rate (EDR).

Figure 9. Frame-level comparison of the ROC curves in UCSD
Ped1 dataset. Method abbreviation: MPPCA+SF [13], SF [13],
MDT [13], Sparse [5], Saligrama [16], Antic [2], Subspace:
replacing our combination learning by subspace clustering [7].

1Platform
MATLAB
MATLAB 2012

Table 5. Running time comparison on the UCSD Ped1 dataset.

FPR

1.2

Second/Frame25
3.8
5 ∼ 100.00697

1

FPR

Figure 10. Pixel-level comparison of the ROC curves in UCSD
Ped1 dataset. Method abbreviation: MPPCA+SF [13], SF [13],
MDT [13], Sparse [5], Saligrama [16], Antic [2], Subspace:
replacing our combinations learning by subspace clustering [7].

proposed method yields high detection rates together withlow false alarm.
Running Time Comparison We compare our systemwith other sparse dictionary learning based methods [22, 5]
in terms of running time on the Subway dataset in Table 4.
The speed of methods [22, 5] is reported in their respective
papers. The difference on detection speed is much larger
than that of working environment.

3.5. UCSD Ped1 Dataset
The UCSD Ped1 dataset [13] provides 34 short clips
for training, and another 36 clips for testing. All testing
clips have frame-level ground truth labels, and 10 clips havepixel-level ground truth labels. There are 200 frames ineach clip.
Our configuration is similar to that of [13]. That is, theperformance is evaluated on frame- and pixel-levels. We

ROC Curve Comparison According to [13] in framelevel detection, if a frame contains at least one abnormal
pixel, it is considered as successful detection. In our
experiment, if a frame contains one or more abnormal
patches, we label it as an abnormal event. For frame-levelevaluation, we alter frame abnormality threshold to produce
a ROC curve shown in Fig. 9. Our method has a reasonably
high detection rate when the false positive value is low. It isvital for practical detection system development.
In pixel level evaluation, a pixel is labeled as abnormal,
if and only if the regions it belongs to in all scales areabnormal. We alter threshold for all pixels. Following [13],if more than 40% of truly anomalous pixels are detected,
the corresponding frame is considered as being correctly
detected. We show the ROC curve in Fig. 10. Besides
all methods that are compared in [13], we also include
the performance of subspace clustering [7]. Our method
achieves satisfactory performance.
EER and EDR Different parameters could affect detection and error rates. Following [13], we obtain these rates
when false positive number equals to the missing value.
They are called equal error rate (EER) and equal detected
rate (EDR). We compute the area under the ROC curve(AUC). We report EER, ERD and AUC in the pixel-levelcomparison (Table 6) and calculate EER and AUC in theframe-level (Table 7). These results indicate that our results
are with high quality in both measures.
We compare the running time in Table 5. The detection
time per frame and working platforms of [13, 5, 2] areobtained from the original papers.

3.6. Separate Cost Analysis
Our testing includes two main steps: feature extraction(3D cube gradient computing and PCA) and combination
testing using Algorithm 2. Other minor procedures areframe resizing, matrix reshape, etc. We list the averagerunning time spent for each step to process one frame in
the three datasets in Table 8.

4. Conclusion
We have presented an abnormal event detection method
via sparse combination learning. This approach directly

EDR
AUC

SF [13]21 %19.7 %MPPCA [13]18 %20.5 %

SF-MPPCA [13]18 %21.3 %MDT [13]45 %44.1 %

Sparse[5]
46 %
13.3%

Adam[1]
24 %46.1 %

Antic [2]
68 %
76 %

Subspace [7]39.3%
43.2 %

Ours
59.1 %
63.8 %

Table 6. Comparison of pixel-level EDR and AUC curves on the UCSD Ped1 dataset.

EER
AUC

SF-MPPCA [13]
40 %
59 %

SF [13]31 %67.5 %MDT [13]
25 %
81.8%

Sparse[5]
19 %
86 %

Saligrama [16]16 %92.7 %

Antic [2]
18 %
91 %

Subspace [7]29.6%
68.4 %

Ours
15 %91.8 %

Table 7. Comparison of frame-level EER and AUC curves on the UCSD Ped1 dataset.

AvenueUCSD Ped1
SubwayFeature extraction (ms)
4.513
4.496
4.634

Combinations testing (ms)
1.792
1.724
1.409

Others (ms)
0.770
0.743
0.625

All (ms)
7.075
6.965
6.412

FPS
141.34143.57155.97

Table 8. Average running time of processing one frame in each step on the three datasets. “ms” is short for millisecond.

learns sparse combinations, which increase the testing
speed hundreds of times without compromising effectiveness. Our method achieves state-of-the-art results in several
datasets. It is related to but differ largely from traditional
subspace clustering. Our future work will be to extend
the sparse combination learning framework to other video
applications.

Acknowledgments
This research has been supported by General Research
Fund (No. 412911) from the Research Grants Council ofHong Kong.

References[1] A. Adam, E. Rivlin, I. Shimshoni, and D. Reinitz. Robust real-time unusual event detection using multiple fixedlocation monitors. IEEE TPAMI, 30(3):555–560, 2008.
[2] B. Antic and B. Ommer. Video parsing for abnormality
detection. In ICCV, pages 2415–2422, 2011.
[3] Y. Benezeth, P.-M. Jodoin, V. Saligrama, and C. Rosenberger. Abnormal events detection based on spatio-temporalco-occurences. In CVPR, 2009.
[4] D. Bertsekas. Nonlinear programming. Athena ScientificBelmont, MA, 1999.
[5] Y. Cong, J. Yuan, and J. Liu. Sparse reconstruction costs
for abnormal event detection. In CVPR, pages 3449–3456,
2011.
[6] X. Cui, Q. Liu, M. Gao, and D. Metaxas. Abnormal detection
using interaction energy potentials. In CVPR, pages 3161–
3167, 2011.
[7] E. Ehsan and R. Vidal. Sparse subspace clustering. In CVPR,
2009.
[8] F. Jianga, J. Yuan, S. A. Tsaftarisa, and A. K. Katsaggelosa. Anomalous video event detection using spatiotemporal context. Computer Vision and Image Understanding, 115(3):323–333, 2011.
[9] K. Jouseok and L. Kyoungmu. A unified framework for event
summarization and rare event detection. In CVPR, 2012.

[10] J. Kim and K. Grauman. Observe locally, infer globally:a space-time mrf for detecting abnormal activities withincremental updates. In CVPR, pages 2921–2928, 2009.
[11] L. Kratz and K. Nishino. Anomaly detection in extremely
crowded scenes using spatio-temporal motion pattern models. In CVPR, pages 1446–1453, 2009.
[12] C. Lu, J. Shi, and J. Jia. Online robust dictionary learning.
In CVPR, 2013.
[13] V. Mahadevan, W. Li, V. Bhalodia, and N. Vasconcelos.
Anomaly detection in crowded scenes. In CVPR, 2010.
[14] J. Mairal, F. Bach, J. Ponce, and G. Sapiro. Online learning
for matrix factorization and sparse coding. The Journal of
Machine Learning Research, 11:19–60, 2010.
[15] R. Mehran, A. Oyama, and M. Shah. Abnormal crowd
behavior detection using social force model. In CVPR, 2009.
[16] V. Saligrama and Z. Chen. Video anomaly detection basedon local statistical aggregates. In CVPR, pages 2112–2119,
2012.
[17] J. Shi, X. Ren, G. Dai, J. Wang, and Z. Zhang. A non-convex
relaxation approach to sparse dictionary learning. In CVPR,
pages 1809–1816, 2011.
[18] H. Trevor, T. Robert, and J. H. Friedman. The elements ofstatistical learning. Springer New York, 2001.
[19] X. Wang, X. Ma, and E. Grimson. Unsupervised activity
perception by hierarchical bayesian models. In CVPR, pages
1–8, 2007.
[20] S. Wu, B. E. Moore, and M. Shah. Chaotic invariantsof lagrangian particle trajectories for anomaly detection incrowded scenes. In CVPR, 2010.
[21] D. Zhang, D. Gatica-Perez, S. Bengio, and I. McCowan.
Semi-supervised adapted hmms for unusual event detection.
In CVPR, 2005.
[22] B. Zhao, L. Fei-Fei, and E. Xing. Online detection of unusual
events in videos via dynamic sparse coding. In CVPR, 2011.
[23] H. Zhong, J. Shi, and M. Visontai. Detecting unusual activity
in video. In CVPR, 2004.