Detecting the Onset of Machine Failure using Anomaly Detection Techniques
By: Animesh Goyal
Tags: Data Science, Anomaly Detection, Time Series Analysis

Introduction

Numerous factors can contribute to the quality of a product, and not every one of these factors is under manufacturers control. One of the most common sources of quality problems is faulty equipment which has not been properly maintained. Hence, monitoring the condition of machines and components such as cooling fans, bearings, turbines, gears, belts and maintaining a desirable working state becomes very crucial.

When a machine or a component fails, corrective maintenance is performed to identify the cause of failure and decide on repair procedures required to maintain and re-initiate the machine to its normal working conditions. However, because the machine has already failed without any prior warnings, time is required for procuring and repairing of the failed component. Therefore, a maintenance strategy needs to be considered to minimize the downtime of service. But machines and their components degrade through time, and the time of failure is not known in advance. Hence, time-based maintenance strategies are predominantly used for maintaining the conditions of machines and equipment.

In this article, I will discuss various techniques that can be used to detect the onset of failure occurring in machines.

Anomaly Detection

There are always some students in a classroom who either outperform the other students or failed to even pass with a bare minimum when it comes to securing marks in subjects. Most of the times, the marks of the students are generally normally distributed apart from the ones just mentioned. These marks can be termed as extreme highs and extreme lows respectively. In Statistics and other related areas like Machine Learning, these values are referred to as Anomalies or Outliers.

Anomaly detection (or outlier detection) is the identification of rare items, events or observations which raise suspicions by differing significantly from the majority of the data. Typically, anomalous data can be connected to some kind of problem or rare event such as e.g. bank fraud, medical problems, structural defects, malfunctioning equipment etc. This connection makes it very interesting to be able to pick out which data points can be considered anomalies, as identifying these events are typically very interesting from a business perspective.

Anomaly Detection

This triggers a question in mind: How can we identify if the data is normal or anomalous? There are various techniques which can help us identify this.

Anomaly Detection Techniques

There exist numerous anomaly detection methods. A rough grouping of some of these methods includes statistical algorithms, clustering-based, nearest-neighbor- based, classification-based, spectral-based, space subsampling-based and deep learning methods.

Statistical-based methods

Statistical-based methods assume that the data follow a specific distribution, so the model is created from a specified probability distribution. The simplest approach for detecting anomalies in data would be flagging data points that deviate from common statistical properties of a distribution. For example, one could define an anomaly based on a certain standard deviation away from the mean. The advantage of these models is that they output a probability as a measure of outlierness.

Statistical based methods

Clustering-based methods

In clustering-based anomaly detection, the assumption is that data points that are similar belong to a similar group. This is determined by the distance to the cluster centroid. The anomaly score is then calculated by setting a threshold for the size of a cluster or the distance to the cluster centroid; if cluster has data points less than the value of threshold they are marked as anomalies, or if the data points distance to the center of the cluster exceeds the set threshold it is flagged as anomalous. K means clustering is an example of such methods.

Clustering-based method (K Means Clustering)

Nearest Neighbor based methods

The nearest-neighbor-based anomaly detection generally assumes that normal data samples appear in neighborhoods that seem to be dense, while anomalies are far from their closest neighbors. Nearest neighbor methods can be generally grouped into distance-based and density-based methods. Both approaches require a similarity or a distance measure in order to make the decision on the degree of abnormality of a data instance. Some examples include k-NN, Mahalanobis distance, and Local Outlier Factor (LOF).

Nearest Neighbor method (KNN)

Classification based methods

Classification-based anomaly detection can be divided into one-class (normal labels only) and multi-class (multiple classes) classification depending on the availability of labels. Anomaly detection based on a classifier comprises of two steps:

In the training phase, a classifier is learned using available labeled training data. Then the test instances are classified as normal or abnormal using the classifier trained in the initial step.

The One-Class Support Vector Machine (OCSVM) and neural network methods are examples of such detection methods.

Classification based method (One class SVM)

Spectral or subspace-based methods

Spectral or subspace-based methods try to extract features that best describe the variability of the training data. These methods assume that the normal data can be presented in a lower dimension subspace where normal data is distinguished from abnormal data. Principle component analysis (PCA) is considered as a subspace-based approach to anomaly detection.

Subspace based method (PCA)

Subsampling based methods

Many outlier detection methods suffer from the curse of dimensionality; as the dimensionality of a given dataset increases, distance-based approaches fail because the relative distance between any pair of points become relatively the same. To overcome this problem sub-sampling-based methods the high dimensional space is divided into smaller sub-spaces and the outlier ranking in different sub-spaces is monitored. Outliers are points that consistently rank high in the smaller sub-spaces. Isolation forest is an algorithm that divides the high dimensional space to smaller dimensions and tries to find anomalies in the lower dimensional space.

Subsampling based methods (Isolation forest)

Gaussian distribution-based methods

Gaussian distribution is also called a normal distribution. We will be using the Gaussian distribution to develop an anomaly detection algorithm, that is, we’ll assume that our data are normally distributed. This’s an assumption that cannot hold true for all data sets, yet when it does, it proves an effective method for spotting outliers.

Elliptic Envelope is a method that tries to figure out the key parameters of our data’s general distribution by assuming that our entire data is an expression of an underlying multivariate Gaussian distribution.

Gaussian based methods (Elliptic Envelope)

Implementation

I have downloaded the dataset from Numenta Anomaly Benchmark (NAB). NAB is a novel benchmark for evaluating algorithms for anomaly detection in streaming, real-time applications. Specifically, I will be working with Ambient Temperature Dataset which has real-time observations from an office setting.

The dataset contains 22695 observations taken at a 5-second interval from 2nd December 2013 to 19th February 2014. The dataset contains no null values. The temperature of the building is in Fahrenheit which was later converted to degree Celcius.

Following is the graphical representation of the temperature reading over the time period.

Temperature Distribution

Feature Engineering

I performed feature engineering on the dataset which would help build a better model.

The day was divided into daylight or night time which can be really useful to find out if the failures occur during day time or night time.

Daytime — 7 am to 10 pm

Nighttime — 10.01 pm to 6.59 am

2. The week was divided as weekday or weekend which will be useful information if failures occurred during the time when most of the employees are in the office (weekday) or otherwise.

3. Four distinct categories were made to divide the observations into “Weekday night”, “Weekday light”, “Weekend light”, “Weekend night”

As we can see, the temperature is more stable during daylight of the business day.

Models

K Means Clustering

k-means clustering aims to partition n observations into k clusters in which each observation belongs to the cluster with the nearest mean, serving as a prototype of the cluster. Here k is a hyperparameter which is selected using the elbow method.

Using the elbow method, k was found to be 15. The data was then separated into 15 different clusters as seen in the figure.

K means clustering with k=15

The next step is to get the distance between each point and its nearest centroid. The biggest distances are considered as an anomaly. Below is the visual representation of anomaly with cluster view.

Anomalies (Red color) and normal readings (Blue color)

As we can clearly see that K means clustering was able to successfully separate the anomalies (red color) from the normal ones (blue color). Anomalies can also be visualized throughout time.

Anomalies visualized throughout time (K Means)

Anomalies can also be visualized with temperature repartition.

Anomaly detection using temperature repartitioning (K Means)

One disadvantage of using K Means Clustering is that it detects the low temperature around the end of the record as an anomaly but it doesn’t detect anomalies near the higher temperature.

Isolation Forest

Following anomalies were detected using Isolation forest method.

Anomalies visualized throughout time (Isolation Forest)

Anomaly detection using temperature repartitioning (Isolation Forest)

One Class SVM

Anomalies visualized throughout time (One Class SVM)

Anomaly detection using temperature repartitioning (One Class SVM)

Elliptic Envelope

Anomalies visualized throughout time (Elliptic Envelope)

Anomaly detection using temperature repartitioning (Elliptic Envelope)

Result

One Class SVM performed the best among all the algorithms and was able to find anomalies in high as well as low-temperature ranges.

The code for the blog can be found here.

References