Anomaly detection with Local Outlier Factor (LOF) | by Mahbubul Alam | Towards Data Science

Today’s article is my 5th in a series of “bite-size” article I am writing on different techniques used for anomaly detection. If you are interested, the following are the previous four articles:
Z-score for anomaly detection
Boxplot for anomaly detection
Statistical techniques for anomaly detection
Time series anomaly detection with “anomalize” library
Today I am going beyond statistical techniques and stepping into machine learning algorithms for anomaly detection.
LOF is an unsupervised (well, semi-supervised) machine learning algorithm that uses the density of data points in the distribution as a key factor to detect outliers.
LOF compares the density of any given data point to the density of its neighbors. Since outliers come from low-density areas, the ratio will be higher for anomalous data points. As a rule of thumb, a normal data point has a LOF between 1 and 1.5 whereas anomalous observations will have much higher LOF. The higher the LOF the more likely it is an outlier. If the LOF of point X is 5, it means the average density of X’s neighbors is 5 times higher than its local density.
In mathematical terms,
LOF(X)=[(LRD(1st neighbor) + LRD(2nd neighbor ) + .................+ LRD(kth neighbor))/LRD(X)]/k
where LRD is Local Reachability Distance and is computed as follows.
LRD(X) = 1/(sum of Reachability Distance (X, n))/k)where n is neighbors upto k
The algorithm has four different components:
Hyperparameter k: determines the number of neighbors
Reachability distance: distances measured using 3 methods — Euclid, Minkowski, Manhattan
Local reachability distance: (LRD) (X) = 1/(sum of Reachability Distance (X, n))/k), where n is neighbors upto k
Local Outlier Factor (LOF)
Enough of theory and mathematics. If you didn’t understand much of it, no hard feelings. As I use to say, to drive a car we don’t need to know about its mechanics, but we do need to know how to drive! So jump right into the next section on the implementation of LOF in Python.
We are going to implement LOF for anomaly detection in Python environment using Scikit-Learn library. Let’s first import the required libraries:
# data preparationimport pandas as pdimport numpy as np# data visualzationimport matplotlib.pyplot as pltimport seaborn as sns# outlier/anomaly detectionfrom sklearn.neighbors import LocalOutlierFactor
Now let’s create a hypothetical dataset containing 5 data points.
# datadf = pd.DataFrame(np.array([[0,1], [1,1], [1,2], [2,2], [5,6]]), columns = ["x", "y"], index = [0,1,2,3,4])
If you plot the data points, finding out the outlier with visual inspection is not so difficult.
# plot data pointsplt.scatter(df["x"], df["y"], color = "b", s = 65)plt.grid()
So indeed, we don’t need a machine learning algorithm to find out that the 5th data point is an outlier. But let’s see if the algorithm can detect it.
# model specificationmodel1 = LocalOutlierFactor(n_neighbors = 2, metric = "manhattan", contamination = 0.02)# model fittingy_pred = model1.fit_predict(df)# filter outlier indexoutlier_index = where(y_pred == -1) # negative values are outliers and positives inliers# filter outlier valuesoutlier_values = df.iloc[outlier_index]# plot dataplt.scatter(df["x"], df["y"], color = "b", s = 65)# plot outlier valuesplt.scatter(outlier_values["x"], outlier_values["y"], color = "r")
There you go! The algorithm correctly detected the outlier.
The purpose of this article was to introduce a density-based anomaly detection technique — Local Outlier Factor. LOF compares the density of a given data point to its neighbors and determines whether that data is normal or anomalous. The implementation of this algorithm is not too difficult thanks to the sklearn library. The interpretation of the results is also pretty straight forward.
To focus on just one thing I left out another important use case of LocalOutlierFactor() algorithm — Novelty Detection. This is the subject of another article but briefly, LOF is a semi-supervised ML algorithm where the algorithm is trained only on normal data. After training the algorithm, new data is shown to identify whether it is novel or not.
Hope you liked this article, feel free to follow me on Medium or Twitter.