Unsupervised Learning and Convolutional Autoencoder for Image Anomaly Detection
Tags: Deep Learning, Anomaly Detection, Mnist, Unsupervised Learning, Autoencoder

The rest of this post is organized as follows. First, we will explain the whole datasets and the organization of the training/validation/testing dataset. Next, we will brief the concept of autoencoder and the idea about applying it to anomaly detection. Finally, we will walk through the complete process of our solution then evaluate the results.

1. MNIST and Fashion-MNIST

We consider that images from the MNIST handwritten digit dataset (the left part in Figure 3) will be the normal data and images from the Fashion-MNIST fashion product dataset (the right part in Figure 3) will be the anomaly data. Both datasets have been included in the deep learning library Keras.

Figure 3. MNIST and Fashion-MNIST.

Table 1 shows the data used for training, validation, and testing.

Table 1. Training/Validation/Testing Dataset.

Training Dataset : 54000 28x28 MNIST images are used to train the convolutional autoencoder;

: 54000 28x28 MNIST images are used to train the convolutional autoencoder; Validation Dataset : 6000 28x28 MNIST images are used for two purposes: 1. verifying if the average loss of the validation dataset is in the neighborhood of the average loss of the training dataset so as to see if the convolutional autoencoder is overfitting or not; 2. developing the threshold of the reconstruction error for separating the normal data and the anomaly data;

: 6000 28x28 MNIST images are used for two purposes: 1. verifying if the average loss of the validation dataset is in the neighborhood of the average loss of the training dataset so as to see if the convolutional autoencoder is overfitting or not; 2. developing the threshold of the reconstruction error for separating the normal data and the anomaly data; Testing Dataset: 500 28x28 MNIST images and 500 28x28 Fashion-MNIST images are used for the final check.

2. What is an Autoencoder and How to Train an Autoencoder to Learn MNIST

An autoencoder is basically an encoder-decoder system that reconstructs the input as the output. This is achieved by two subsystems: the encoder takes the input (e.g., an image, a piece of audio/sound, a text, etc.) and maps it into an internal representation (e.g., a code, an embedding, or a feature vector), while the decoder takes the internal representation and maps it back to the original input. Figure 4 shows the architecture of an autoencoder. Typically, the encoder and the decoder are implemented as deep neural networks.

Figure 4. the architecture of an autoencoder [ref].

It is interesting to note that from the outset the goal of an autoencoder is to learn the representation of a given dataset under unsupervised learning. However, it can be used in many different scenarios, for example, feature extraction, dimensionality reduction, noise cancellation, or data generation.

In our case, we take a convolutional autoencoder to learn the representation of MINST and hope that it can reconstruct images from MNIST better than reconstruct images from Fashion-MNIST during testing because the former case is from known representations while the latter case is from unknown representations. Following the idea from the blog of Keras, the code of our autoencoder to learn MNIST is shown in Figure 5.

Figure 5. the autoencoder implemented by Keras.

The history of loss in binary cross-entropy for the training dataset and the validation dataset are shown in Figure 6.

Figure 6. training loss and validation loss.

3. How to define the metrics and threshold for the reconstruction error

After the autoencoder completes the learning process, there are two major steps for building the anomaly detection mechanism: (1) define the metrics of the reconstruction error between the original image and the reconstructed image, and (2) determine the threshold of the reconstruction error so as to separate the normal data and the anomaly data.

Figure 7. the concept of the reconstruction error.

Here we consider cross-entropy as the reconstruction error between the original image and the reconstructed image, and 3 standard deviations of the mean of the reconstruction error as the threshold. Note that the implementation of cross-entropy seems to be tricky. Please refer to [here] and [here] for details. The idea of relating normal distribution and anomaly detection can be found in [here].

Note: Mean Squared Error (MSE) or Structural Similarity Index (SSIM) could be also used as the reconstruction error. Check [here] and [here] for further details. But, if the autoencoder is optimized for cross-entropy, it is better to use cross-entropy as the reconstruction error.

4. Evaluating the testing dataset

500 MNIST images and 500 Fashion-MNIST images are used for evaluating our anomaly detection process.

Step 1: Pass all the images into the autoencoder and get its corresponding reconstructed images; (Figure 8 shows some image pairs as examples.)

Step 2: Calculate the cross-entropy reconstruction error for each image pairs; (Figure 9 shows the reconstruction error for MNIST images in green and the reconstruction error for Fashion-MNIST images in blue.)

Step 3: Use the 3 standard deviations of the mean of the reconstruction error as the threshold to identify the normal data and the anomaly data; (Figure 9 shows the threshold line in red.)

Figure 8. some original images (row 1/3) and reconstructed images (row 2/4) from the testing dataset.

Figure 9. cross-entropy of MNIST images and Fashion-MNIST images.

Figure 9 shows that the accuracy of identifying the normal data and the anomaly data is 99.80%.

5. Conclusions

This post has described the process of image anomaly detection using a convolutional autoencoder under the paradigm of unsupervised learning. However, from my experience, a lot of real-life image applications such as examining medical images or product defects are approached by supervised learning, e.g., image classification, object detection, or image segmentation, because it can provide more information on abnormal conditions such as the type and the location (potentially size and number) of abnormal conditions. On the other hand, supervised learning requires labels on all the images which is not only labor-intensive but also potentially noisy. As a result, unsupervised learning could be a reasonable approach or companion in some anomaly detection problems.

Thanks for reading. If you have any questions or comments, please feel free to drop a note.