Journal of Robotics, Networking and Artificial Life
Vol. 8(1); June (2021), pp. 41–46
DOI: https://doi.org/10.2991/jrnal.k.210521.010; ISSN 2405-9021; eISSN 2352-6386
https://www.atlantis-press.com/journals/jrnal

Research Article

Anomaly Detection Using Support Vector Machines
for Time Series Data
Umaporn Yokkampon1,*, Sakmongkon Chumkamon1, Abbe Mowshowitz2, Ryusuke Fujisawa1, Eiji Hayashi1
Graduate School of Computer Science and Systems Engineering, Kyushu Institute of Technology, 680-4 Kawazu, Iizuka, Fukuoka 820-8502, Japan
Department of Computer Science, The City College of New York, 160 Convent Avenue, New York, NY 10031, USA

1
2

ARTICLE INFOABSTRACT

Article History

Analysis of large data sets is increasingly important in business and scientific research. One of the challenges in such analysis
stems from uncertainty in data, which can produce anomalous results. This paper proposes a method for detecting an anomaly
in time series data using a Support Vector Machine (SVM). Three different kernels of the SVM are analyzed to predict anomalies
in the UCR time series benchmark data sets. Comparison of the three kernels shows that the defined parameter values of the
Radial Basis Function (RBF) kernel are critical for improving the validity and accuracy in anomaly detection. Our results showthat the RBF kernel of the SVM can be used to advantage in detecting anomalies.

Received 26 November 2020
Accepted 12 April 2021
Keywords
Anomaly detection
support vector machinedata mining
factory automation© 2021 The Authors. Published by Atlantis Press B.V.
This is an open access article distributed under the CC BY-NC 4.0 license (http://creativecommons.org/licenses/by-nc/4.0/).

1. INTRODUCTION
Research on anomaly detection is of great interest in machinelearning and data mining. Detecting anomalies or finding outliers
involves identifying abnormal or inconsistent patterns in a dataset. Abnormal data often results from unauthorized activity. Credit
card fraud offers a well-known example. Transactions with a stolen
or fake credit card can produce suspicious data. A fake card can bemade by copying information from an authorized card and using it
to create a new unauthorized one. Data such as personal identifying
information may be obtained through phishing or from employees
who work in credit card companies [1]. Another source of abnormal data may derive from unauthorized intrusions in networks.
Abnormal traffic or user actions are common signs of intrusions,which may occasion breaches of sensitive or confidential data.
Intrusions may also cause sensor networks to generate erroneous
data. When a sensor malfunctions, it is unable to capture data correctly and thus may produce anomalies. Abnormal changes in data
sources may also result in anomalies [2].
Anomaly detection typically uses data mining and machine learning methods for detecting abnormal activities in systems. Many
anomaly detection techniques have been developed, including
Support Vector Machines (SVM), which can solve classification
and regression problems. The performance of SVM depends on theselection of kernel function and kernel parameters. The selection
quality of SVM parameters and kernel functions has an effect onlearning and generation performance. Appropriate kernel function
and associated parameters should be selected to obtain optimalclassification performance. When an appropriate kernel function
Corresponding author. Email: may@mmcs.mse.kyutech.ac.jp*

and parameters are selected, the prediction error of SVM can be
minimized.
This paper reports on application of the support vector machine
method to eight real world time series data sets to detect anomalies
using three different kernels for analysis and prediction. In addition, SVM kernels are compared for effectiveness based on Area
under the Curve (AUC), Precision, Recall, F1-Score, Specificity,and Jaccard index criteria.

2. SUPPORT VECTOR MACHINEALGORITHM
The SVM algorithm’s goal is to create the best line or decisionboundary that can decompose an n-dimensional space into sets
supporting categorization of new data points. Hyperplanes define
the boundaries in this space.
In our proposed method, we used SVM provided by Scikit-learn todetect the anomaly in time series data.
For a given dataset x with a number i of training data, SVM finds
the maximum margin hyperplane separating different classes ofdata [3]:   x = ( xi , yi ), xi ∈  , yi ∈{−1,1} , ∀ i = 1, 2, ..., N (1)pwhere xi is the p-dimensional input vector, yi is the output value(1 or −1) and “·” is the dot product which has a formula form byx  y   xi yi . A decision vector separating two classes is givenby:Tw ⋅ x + b = 0 (2)		  

U. Yokkampon et al. / Journal of Robotics, Networking and Artificial Life 8(1) 41–46

42

where wT is the optimal weighting vector and b is the bias. The margins of linearly separable training data can be defined as:	  wT  x  b  1 and wT  x  b   1 (3)The distance between the margins can be defined by 2/‖wT‖.
Therefore, minimizing ‖wT‖ is an objective function. In practice, it
is not easy to linearly decompose the training dataset. Let C be theregularization parameter that defines the separation of two classes
and the error when using a training dataset. The hyperplane isdetermined by minimizing:NC ∑ei +
		 
i =1

1 2
w (4)
2

with constraints ti y(xi) ≥ 1‒ei, i = 1, …, N where ti is the target value
and ei is the set of slack variables.
Instead of using a minimization model (4), the problem may beformulated using Lagrangian dual multipliers a as:N	   max  a i 
i 1

1 N N
  a a y y (x , x ) (5)2 i 1 j 1 i j i j i jsubject to:n

0  a i  C  i  1, 2, ..., n and  a i yi  0 (6)i 1Kernel trick can be applied to reduce the complexity of the optimization problem.
Support vector machine with nonlinear kernel has an objectivefunction form as follows:Nmax  a i 
	 
i 1

1 N N
  a a y y k(x , x ) (7)2 i 1 j 1 i j i j i jsamples, small or large with both high and low dimensions [5]. The
SVM performance for each kernel will be evaluated in this study todetermine the optimal kernel.

4. EXPERIMENTS
This section introduces the data sets and the evaluation metricused. We have compared the three kernels and evaluated theireffectiveness for anomaly detection in SVMs.

4.1. Data Sets
Time series data obtained from UCR public data set [6] were used
to evaluate effectiveness. Table 1 shows the details of the datasets.
All datasets are presented in time series form, and every data point
is manually labeled. For all datasets, we designated the minority
class as an anomaly class. Twenty percent of the data was used fortesting.

4.2. Evaluation Metrics
The accuracy of an anomaly detection method is evaluated using
the AUC of the Receiver Operating Characteristic (ROC), Precision(Pre), Recall (Rec), F1-Score, Specificity, and Jaccard index, defined
as follows (Figure 1):
		 Precision =Table 1 | Summary of the datasets
Datasets

3. KERNELS
In machine learning, kernel methods are a popular class for a variety of tasks. Kernel methods can generate the model complex data
through the kernel trick, which is an important feature [4].
In this paper, two types of kernel functions are chosen and evaluated, namely the linear and Radial Basis Function (RBF). The
mathematical formula for the said functions are as follows:

TP
(10)
TP + FP

ItalyPowerDemand
Wafer
SonyAIBORobotSurface2
ECGFiveDays
TwoLeadECG
MoteStrain
Herring
Strawberry

LengthNumber of
instances

Anomaly
ratio

24
152
65
136
82
84
512
235

1096
7164
980
884
1162
1272
128
983

0.49
0.11
0.38
0.50
0.50
0.46
0.40
0.36

3.1. Linear Kernel		 k(xi , x j ) = (xi , x j ) (8)3.2. Radial Basis Function Kernel −  xi − x j  2  (9)
(2s 2 ) 	   k( xi , x j ) = exp K(xi, xj) = xi · xj is the kernel function, which is generalized dot
products. “·” denotes the dot product. The two kernels have theirown advantages and limitations. The linear kernel offers ease of
performance as well as an ability to deal with small and linearly
separable samples. The RBF kernel, on the other hand, is known
as a good mapping function since it can be used for all kinds ofFigure 1 | Structure of anomaly detection in time series data used SVM.
We used eight time series data sets processed by SVM, and three different
kernels based on AUC, Precision, Recall, F1-Score, Specificity, and Jaccard
index criteria.

U. Yokkampon et al. / Journal of Robotics, Networking and Artificial Life 8(1) 41–46

	   Recall = Sensitivity =	    F1-Score  2 		 Specificity 

43

TP
(11)
TP + FN

Pre  Rec
Pre +Rec

(12)

TN
(13)
TN  FPTP

		 Jaccard  TP  FP  FN (14)where TP is the accurately detected abnormal, FP is the false
detected abnormal, TN is the accurately assigned normal, and FN
is the false assigned normal.

Figure 4 | The kernel performance comparison of Linear, RBF and RBF1
for testing SonyAIBORobotSurface2 dataset using ROC.

5. RESULTS AND DISCUSSION
The efficiency of the following three SVM kernels are compared:
1. Linear Kernel
2. RBF Kernel (Default parameters value)
3. RBF1 Kernel (We define the parameters C = 20, γ = 0.02)
We performed experiments on accuracy of analysis and predictionof anomalies for eight time series data sets using the three different
SVM kernels. Accuracy of analysis and prediction can be measured
by the AUC as shown in the ROC in Figures 2–9. The blue line is
the Linear Kernel of SVM, the orange line is the RBF kernel, andthe green line is the RBF1 kernel.

Figure 2 | The kernel performance comparison of Linear, RBF and RBF1
for testing ItalyPowerDemand dataset using ROC.

Figure 3 | The kernel performance comparison of Linear, RBF and RBF1
for testing Wafer dataset using ROC.

Figure 5 | The kernel performance comparison of Linear, RBF and RBF1
for testing ECGFiveDays dataset using ROC.

Figure 6 | The kernel performance comparison of Linear, RBF and RBF1
for testing TwoLeadECG dataset using ROC.

Figure 7 | The kernel performance comparison of Linear, RBF and RBF1
for testing MoteStrain dataset using ROC.

U. Yokkampon et al. / Journal of Robotics, Networking and Artificial Life 8(1) 41–46

44

Figure 2 shows that the RBF1 kernel is slightly more efficient thanthe Linear and RBF kernels for the ItalyPowerDemand data set.
However, all three kernels yield almost 100% accuracy.

accurate. In particular, the RBF1 kernel gives a ROC value perfectly
for the Wafer data set.
Figure 4 reveals that RBF1 is slightly more efficient than thelinear and RBF kernels for the SonyAIBORobotSurface2 data set.
In particular, the RBF and RBF1 kernels provide almost 100%
accuracy.

Figure 3 shows that the RBF and RBF1 kernels are more efficientthan the Linear kernel, and that the RBF kernel is almost 100%

Figure 5 shows that all three kernels give perfect ROC values for the
ECGFiveDays data set.
Figure 6 shows that the RBF kernel is almost 100%. Linear andRBF1 kernels give nearly perfect ROC values for the TwoLeadECG
data set.
Figure 7 shows that the RBF1 kernel to be slightly more efficientthan the linear and RBF kernels for the MoteStrain data set.
Figure 8 reveals that the Linear kernel gives slightly more accurate
ROC values than does the RBF kernel, but the RBF1 kernel is themost accurate for the Herring data set.
Finally, Figure 9 shows that the Linear kernel is more accurate thanthe RBF kernel. In particular, the RBF1 kernel is almost 100% for
the Strawberry data set.

Figure 8 | The kernel performance comparison of Linear, RBF and RBF1
for testing Herring dataset using ROC.

Table 2 shows the summary of anomaly detection results andcomparisons. The results show that SVM with RBF1 kernel gives
the highest accuracy, specificity, Jaccard index, and F1-Scoreon all aspects and data sets, except for the Herring data set, forwhich the RBF kernel gives the highest of F1-Score and Jaccard.
All three kernels gave perfect results for AUC, Precision, Recall,
F1-Score, Specification, and Jaccard index on the ECGFiveDays
data set.
We compared our results to the latest research in “Time Series
Anomaly Detection with Variational Autoencoders [7]”, which useda different method. There are six data sets utilized in our method,
namely, ItalyPowerDemand, Wafer, ECGFiveDays, TwoLeadECG,
MoteStrain, and Herring. Table 3 shows the AUC results of thelatest research. These results demonstrate that our proposed

Figure 9 | The kernel performance comparison of Linear, RBF and RBF1
for testing Strawberry dataset using ROC.

Table 2 | Summary of the kernel performance comparison of Linear, RBF, and RBF1
Datasets
ItalyPowerDemand
Wafer
SonyAIBORobotSurface2
ECGFiveDays
TwoLeadECG
MoteStrain
Herring
Strawberry
Datasets
ItalyPowerDemand
Wafer
SonyAIBORobotSurface2
ECGFiveDays
TwoLeadECG
MoteStrain
Herring
Strawberry

Kernel: Linear

Kernel: RBF (C = 20, γ = 0.02)

Kernel: RBF (Default)

AUC

Precision

Recall

F1-Score

AUC

Precision Recall F1-Score

AUC

Precision Recall F1-Score

0.99940.75100.98191.00001.00000.95400.66670.98840.98180.96870.96461.00000.99150.95520.76470.99080.98180.9946
0.94781.00001.00000.87670.76470.87100.98180.98150.95611.00000.99570.91430.76470.92700.99900.99990.99741.00000.99940.97890.62750.9366

0.99101.00000.99101.00000.99150.97830.65380.63270.99550.99850.97351.00000.99570.95070.79070.77500.99991.00000.99971.00001.00000.98400.72550.9904

0.99101.00001.00001.00001.00000.9716
0.76920.9911

Kernel: Linear

1.00000.99690.95651.00001.00000.92471.00001.0000Kernel: RBF (Default)

1.00000.99850.98261.00001.00000.9384
0.58820.89520.99550.99920.99121.00001.00000.9547
0.66670.9407

Kernel: RBF (C = 20, γ = 0.02)

Specificity

JaccardSpecificity

JaccardSpecificity

Jaccard0.98180.67190.95061.00000.99140.94500.5556
0.98630.96430.96360.91601.00000.99150.84210.61900.86400.99091.00000.98771.00000.9914
0.97250.00000.0137

0.99100.99690.94831.00000.99150.90600.65380.63260.99091.00001.00001.00001.00000.96330.66670.98630.99100.99850.98261.00001.00000.91330.50000.8880U. Yokkampon et al. / Journal of Robotics, Networking and Artificial Life 8(1) 41–46

Table 3 | Comparing AUC of VAE results from latest research
Datasets

OUR

ANOGAN

ItalyPowerDemand
Wafer
ECGFiveDays
TwoLeadECG
MoteStrain
Arrhythmia
KDD99
GunPointAgeSpan
ToeSegmentation2
Herring

0.761
0.965
0.970
0.891
0.840
0.758
0.958
0.881
0.846
0.659

0.516
0.558
0.970
0.554
0.746
0.576
0.887
0.515
0.547
0.488

ALAD MLP-VAE

IF

0.538
0.587
0.694
0.515
0.504
0.515
0.950
0.547
0.544
0.569

0.763
0.847
0.678
0.760
0.762
0.530
0.929
0.612
0.787
0.698

0.768
0.790
0.910
0.731
0.750
0.747
0.622
0.821
0.816
0.627

method achieves superior results compared to the approach taken
in Zhang and Chen [7] for all six datasets.
This shows that the RBF kernel with parameter values C = 20,
γ = 0.02 exhibits good performance in anomaly detection for timeseries data.

45

In the future, we intend to implement the variational autoencoder
method for detecting and predicting anomalies in time series andspectrum data to compare it with the autoencoder method.

CONFLICTS OF INTEREST
The authors declare they have no conflicts of interest.

ACKNOWLEDGMENT
This work was supported by the Religion revitalization project by
the Japanese Government.

REFERENCES[1]

6. CONCLUSION[2]

In this paper, we presented an analysis of anomaly detection intime series data using SVM with three different kernels, namely,
Linear, RBF and RBF1. We evaluated the accuracy of anomaly
detection methods based on AUC, Precision, Recall, F1-Score,
Specificity, and Jaccard index criteria. The evaluation results showthat the kernel with defined parameters can improve accuracy on
all aspects and data sets. This application of the SVM method,with the RBF kernel, can be efficient for detecting anomalies
in time series data. The results for data set ECGFiveDays show
100% accuracy with all three kernels, and the results for the
TwoLeadECG show almost 100% with all three kernels. Moreover,
the results indicate a high degree of accuracy for the three kernels on all the data sets, perhaps because our data was trained insupervised conditions. Since we train the machine using data thatis well labeled, and the algorithms learn to predict output from the
input data. It means some data is already tagged with the correct
answer.

[3]

[4]

[5]

[6]
[7]

E. Hormozi, M. Kazem Akbari, H. Hormozi, M.S. Javan, Accuracy
evaluation of a credit card fraud detection system on Hadoop
MapReduce, The 5th Conference on Information and Knowledge
Technology, IEEE, Shiraz, Iran, 2013, pp. 35–39.
V. Chandola, A. Banerjee, V. Kumar, Anomaly detection: a survey,
ACM Comput. Surv. 41 (2009), 15.
P. Batta, M. Singh, Z. Li, Q. Ding, L. Trajković, Evaluation of support vector machine kernels for detecting network anomalies,
2018 IEEE International Symposium on Circuits and Systems(ISCAS), IEEE, Florence, Italy, 2018, pp. 1–4.
B. Schölkopf, A.J. Smola, Support vector machines and kernel
algorithms, in: P. Armitage, T. Colton (Eds.), Encyclopediaof Biostatistics, John Wiley & Sons, New York, USA, 2005,
pp. 5328–5335.
W.F. Cao, L. Li, X.L. Lv, Kernel function characteristic analysis based on support vector machine in face recognition, 2007
International Conference on Machine Learning and Cybernetics,
IEEE, Hong Kong, China, 2007, pp. 2869–2873.
H.A. Dau, E. Keogh, K. Kamgar, C.C.M. Yeh, Y. Zhu, S. Gharghabi,
et al., The UCR time series classification archive, 2019. Available
from: https://www.cs.ucr.edu/~eamonn/time_series_data_2018/.
C. Zhang, Y. Chen, Time series anomaly detection with variational autoencoders, CoRR, abs/1907.01702, 2019.

AUTHORS INTRODUCTION
Ms. Umaporn Yokkampon
She received Master of Science degree inApplied Statistics from King Mongkut’s
University of Technology North Bangkok
in 2019. Presently, she is a doctoral student
in the department of Computer Scienceand Systems Engineering at Kyushu
Institute of Technology. Her researchinterests include data mining, anomaly
detection, and time series analysis.

Dr. Sakmongkon Chumkamon

and social robots.

He received Doctor of Engineering degreefrom Kyushu Institute of Technology in2017. He was a postdoctoral researcher
at Guangdong University of Technology
in 2017–2019. Presently, he is a postdoctoral researcher in Kyushu Institute ofTechnology since 2019. His research interests include factory automation robots

46

U. Yokkampon et al. / Journal of Robotics, Networking and Artificial Life 8(1) 41–46

Prof. Abbe Mowshowitz

Prof. Eiji Hayashi

He received the PhD degree from
University of Michigan in 1967. He has
been a professor of computer science at theCity College of New York and member ofthe doctoral faculty at the Graduate Centerof the City University of New York since1984. His current research interests lie in
two areas are organizational and managerial issues in computing, and network science. In addition to teaching and research,he has acted as consultant on the uses and impacts of information technology (especially computer networks) to a wide
range of public and private organizations in North America andEurope.

He is a professor in the department ofIntelligent and Control Systems at Kyushu
Institute of Technology. He received the
PhD (Dr. Eng.) degree from Waseda
University in 1996. His research interests
include intelligent mechanics, mechanicalsystems and perceptual information processing. He is a member of The Instituteof Electrical and Electronics Engineers(IEEE), and The Japan Society of Mechanical Engineers (JSME).

Assoc. Prof. Ryusuke Fujisawa
He is an associate professor in thedepartment of Intelligent and Control
Systems at Kyushu Institute of Technology.
He received the PhD (Dr. Eng.) degreefrom The University of ElectroCommunications in 2009. His researchinterests include Swarm intelligence,
Swarm robotics and Distributed autonomous system. He is amember of Japan Society of Civil Engineers (JSCE), Information
Processing Society of Japan (IPSJ), The Society of Instrument
and Control Engineers (SICE) and The Robotics Society ofJapan (RSJ).