Isolation Forest: A Tree-based Algorithm for Anomaly Detection | by Mahbubul Alam | Towards Data Science

This is the 10th in a series of small, bite-sized articles I am writing about algorithms that are commonly used in anomaly detection (I’ll put links to all other articles towards the end). In today’s article, I’ll focus on a tree-based machine learning algorithm — Isolation Forest — that can efficiently isolate outliers from a multi-dimensional dataset.
My objective here is to give an intuition of how the algorithm works and how to implement it in a few lines of codes as a demonstration. So I am not going deep into the theory, but just enough to help readers understand the basics. You can always search and lookup for details if there’s a specific part of the algorithm that you are interested. So let’s dive right in!
Isolation Forest or iForest is one of the more recent algorithms which was first proposed in 2008 [1] and later published in a paper in 2012 [2]. Around 2016 it was incorporated within the Python Scikit-Learn library.
It is a tree-based algorithm, built around the theory of decision trees and random forests. When presented with a dataset, the algorithm splits the data into two parts based on a random threshold value. This process continues recursively until each data point is isolated. Once the algorithm runs through the whole data, it filters the data points which took fewer steps than others to be isolated. Isolation Forest in sklearn is part of the Ensemble model class, it returns the anomaly score of each instance to measure abnormality.
In most unsupervised methods, “normal” data points are first profiled and anomalies are reported if they do not resemble that profile. Isolation forest, on the other hand, takes a different approach; it isolates anomalous data points explicitly.
It is important to mention that Isolation Forest is an unsupervised machine learning algorithm. Meaning, there is no actual “training” or “learning” involved in the process and there is no pre-determined labeling of “outlier” or “not-outlier” in the dataset. So there is no accuracy test in the conventional machine learning sense.
Now let’s get into the implementation of the algorithm in just 5 steps.
You will need pandas and numpy for data wrangling, matplotlib for data visualization and, of course, the algorithm itself from sklearn library.
# librariesimport pandas as pdimport numpy as npimport matplotlib.pyplot as pltfrom sklearn.ensemble import IsolationForest
So that you can follow along, I’m not importing any external dataset, rather creating a simple two dimensional numpy array with just one outlier as presented in the two-dimensional plot.
# datasetX = np.array([[1, 2], [2, 1], [3, 2], [0, 1], [25, 30], [3, 5]])# using just numpy array for visualizationplt.scatter(X[:,0], X[:,1], color = "r", s = 50)plt.grid()
Like most algorithms in the Scikit Learn library, instantiating and fitting the model takes only a couple of lines of code.
Two important parameters in building the model are n_estimators and contamination, the latter specifying the percentage of data to be identified as outliers.
# instantiate modelmodel = IsolationForest(n_estimators = 10)# fit modelmodel.fit(X)
When it comes to the prediction, the process involves showing the model a two-dimensional array and the model will spit out either 1 for normal data and -1 for an outlier.
# predict on new data new_data = np.array([[10, 15]])model.predict(new_data)>> array([-1])
Like I said in the beginning, the algorithm uses anomaly scores as a metric, which represents an average anomaly score of the input sample.
# average anomaly scoremodel.decision_function(np.array(new_data))>> array([-0.11147288])
Thanks for reading, hope this article was useful to get an intuition of what Isolation Forest is and how to implement it in Python sklearn library. I wrote a series of articles in this publication focusing on several other algorithms. If you are interested to check out, below are the links. And as always, feel free to reach out via Twitter or LinkedIn and follow me on Medium to get notification of latest articles.
1. K-Nearest Neighbors (kNN)
2. Support Vector Machines (SVM)
3. DBSCAN, an unsupervised algorithm
4. Elliptic Envelope
5. Local Outlier Factor (LOF)
6. Z-score
7. Boxplot
8. Statistical techniques
9. Time series anomaly detection
References
[1] Liu, Fei Tony, Ting, Kai Ming and Zhou, Zhi-Hua. “Isolation forest.” Data Mining, 2008. ICDM’08. Eighth IEEE International Conference on.
[2] Liu, Fei Tony, Ting, Kai Ming and Zhou, Zhi-Hua. “Isolation-based anomaly detection.” ACM Transactions on Knowledge Discovery from Data (TKDD) 6.1 (2012): 3.