Anomaly Detection in Images. Classify and Individuate anomalies with... | by Marco Cerliani | Towards Data Science

In Machine Learning is normal to deal with Anomaly Detection tasks. Data Scientists frequently are engaged in problems where they have to show, explain and predict anomalies.
I also made a post about Anomaly Detection with Time Series, where I studied internal system behaviors and I provided anomaly forecasts in the future.
In this post, I try to solve a different challenge. I change the domain of interest: swapping from Time Series to Images. Given an image, we want to achieve a dual purpose: predict the presence of anomalies and individuate them, giving a colorful representation of the results.
I got the data from the internet: The crack dataset contains images of wall cracks. Half of the images show new and uncorrupted pieces of the wall; the remaining part shows cracks of various dimensions and types.
As you can see from the samples below, our data are present different types of wall cracks, some of them aren’t so easy to identify also for me.
We want to build a machine learning model which is able to classify wall images and detect at the same time where anomalies are located. To achieve this dual purpose, the most efficient method consists in building a strong classifier. It will be able to read and classify our input images as ‘damaged’ or ‘not damaged’. At the last step, we’ll make use of knowledge learned by our classifier to extract useful information which will help us to detect also where are anomalies.
But let’s proceed with order and start to assemble our Neural Net...
For this kind of task, I’ve chosen a silver bullet of computer vision, the loyalty VGG16. We load and remake the train of VGG16. This is very easy to do in Keras with only a few lines of code.
vgg_conv = vgg16.VGG16(weights='imagenet', include_top=False, input_shape = (224, 224, 3))for layer in vgg_conv.layers[:-8]:    layer.trainable = False
In detail, we imported the VGG architecture allowing training of the last two convolutional blocks. This will permit our model to specialize in our classification task. For this purpose, we’ve also excluded the top layers of the original model replacing them with another structure.
x = vgg_conv.outputx = GlobalAveragePooling2D()(x)x = Dense(2, activation="softmax")(x)model = Model(vgg_conv.input, x)model.compile(loss = "categorical_crossentropy", optimizer = optimizers.SGD(lr=0.0001, momentum=0.9), metrics=["accuracy"])
At the classification stage, the GlobalAveragePooling layer reduces the size of the preceding layer by taking the average of each feature map. This choice, plus the omitted usage of intermediate dense layer, permits avoiding overfitting.
The training is simple and easy if you have at disposal a GPU. COLAB or Kaggle gave us the weapons we needed to speed up this process. We also used a simple data generator provided by Keras for image augmentation.
In the end, we were able to achieve an overall accuracy of 0.90, not bad!
Now, with our model trained, we play with it in order to extract all the useful information which permits us to show cracks in our wall images. We try to make this process easy and nice to see at the end with heat map representation.
The useful information we need is located at the top. Particularly we access to:
Convolutional layers: upper we go in our VGG structure and more important features the network has created. We’ve selected the last convolutional layer (‘block5_conv3’) and cut here our classification model. We’ve recreated an intermediate model which, given the original image as input, output the related activation map. Thinking about dimensionality, our intermediate model augments the channels (new features) and reduces dimensions (height and width) of our initial image.
Final Dense layer: for each class of interest we need these weights, which are responsible to provide the final results of the classification.
With these compressed objects in our hands, we have all the knowledge to localize cracks. We want to ‘paint’ them on an original image in order to make the results easy to understand and nice to see. ‘Unzip’ this information is easy in python: we only have to make bilinear upsampling to resize each activation map and compute a dot product.
This magic is accessible executing a simple function:
def plot_activation(img):    pred = model.predict(img[np.newaxis,:,:,:])    pred_class = np.argmax(pred)    weights = model.layers[-1].get_weights()[0]    class_weights = weights[:, pred_class]    intermediate = Model(model.input,                         model.get_layer("block5_conv3").output)    conv_output = intermediate.predict(img[np.newaxis,:,:,:])    conv_output = np.squeeze(conv_output)    h = int(img.shape[0]/conv_output.shape[0])    w = int(img.shape[1]/conv_output.shape[1])    act_maps = sp.ndimage.zoom(conv_output, (h, w, 1), order=1)    out = np.dot(act_maps.reshape((img.shape[0]*img.shape[1],512)),                  class_weights).reshape(img.shape[0],img.shape[1])    plt.imshow(img.astype('float32').reshape(img.shape[0],               img.shape[1],3))    plt.imshow(out, cmap='jet', alpha=0.35)    plt.title('Crack' if pred_class == 1 else 'No Crack')
I display the results in the image below, where I’ve plotted the crack heat maps on test images classified as crack. We can see that the heat map is able to generalize well and point pieces of wall containing cracks.
In this post, we produce a machine learning solution for anomaly identifications and localizations. All these functionalities are accessible by implementing a single classification model. During training our Neural Network acquires all the relevant information which permits it to operate classification. After this phase, we’ve assembled the final pieces which have told us where the crack is in the image, without additional work!
CHECK MY GITHUB REPO
Keep in touch: Linkedin