Support Vector Machine (SVM) for Anomaly Detection | by Mahbubul Alam | Towards Data Science

An expert or a novice in machine learning, you probably have heard about Support Vector Machine (SVM) — a supervised machine learning algorithm frequently cited and used in classification problems. SVMs use hyperplanes in multi-dimensional space to separate one class of observations from another. Naturally, SVM is used in solving multi-class classification problems.
However, SVM is also increasingly being used in one class problem, where all data belong to a single class. In this case, the algorithm is trained to learn what is “normal”, so that when a new data is shown the algorithm can identify whether it should belong to the group or not. If not, the new data is labeled as out of ordinary or anomaly. To learn more about one-class SVM check out this lengthy article by Roemer Vlasveld.
One last thing to mention, if you are familiar with sklearn library you will notice that there’s an algorithm specifically designed for what is known as “novelty detection”. It works in a similar fashion as the one I just described in anomaly detection using one-class SVM. In my mind, it’s just the context that determines whether to call it novelty detection or outlier detection or anything like that.
Today’s article is a continuation of my series on anomaly, outlier and fraud detection algorithms with hands-on example codes. My previous 7articles touched on different tools and techniques available in the field of anomaly detection, if you are interested to learn about them the following are the links:
DBSCAN, an unsupervised algorithm
Elliptic Envelope
Local Outlier Factor (LOF)
Z-score
Boxplot
Statistical techniques
Time series anomaly detection
Below is a simple demonstration of One-Class SVM in Python programming language. Note that I am using outlier and anomaly interchangeably.
For this demo we need three core libraries — for data wrangling python and numpy, for model building sklearn and for visualization matlotlib.
# import librariesimport pandas as pdfrom sklearn.svm import OneClassSVMimport matplotlib.pyplot as pltfrom numpy import where
I am using the famous Iris dataset from an online source, so you can practice along without worrying about where to get the data from how to clean that up.
# import datadata = pd.read_csv("https://raw.githubusercontent.com/uiuc-cse/data-fa14/gh-pages/data/iris.csv")# input datadf = data[["sepal_length", "sepal_width"]]
Instead of hyperparameter tuning as in other classification algorithms, one-class SVM uses nu as a hyperparameter which is used to define what portion of data should be classified as outliers. nu = 0.03 means that the algorithm will designate 3% data as outliers.
# model specificationmodel = OneClassSVM(kernel = 'rbf', gamma = 0.001, nu = 0.03).fit(df)
The predicted dataset will have either 1 or -1 values, where -1 values are outliers detected by the algorithm.
# predictiony_pred = model.predict(df)y_pred
# filter outlier indexoutlier_index = where(y_pred == -1) # filter outlier valuesoutlier_values = df.iloc[outlier_index]outlier_values
# visualize outputsplt.scatter(data["sepal_length"], df["sepal_width"])plt.scatter(outlier_values["sepal_length"], outlier_values["sepal_width"], c = "r")
In this article, I wanted to give a gentle introduction to One-Class SVM — a machine learning algorithm used for fraud/outlier/anomaly detection. I showed some simple steps to build the intuition, but of course, a real-world implementation would require much more experimentation to find out what works and what doesn’t for a particular context and industry.
Thanks for following along, to learn more about my work you can follow me on Twitter or LinkedIn