The power of transfer learning with FASTAI: Crack Detection in Concrete Structure
By: Muhammad Farooq
Tags: Machine Learning, Image Classification, Transfer Learning, Fastai, Deep Learning

The power of transfer learning with FASTAI: Crack Detection in Concrete Structure Muhammad Farooq Follow May 7 · 7 min read

This blog post presents a practical application of computer vision classification. We will learn how to use the fascinating fastai library to do transfer learning for identifying cracks (damages) in concrete structures. This is critical for the field of Structure Health Monitoring. We will start with small patches to train and test the model and then will present an application on how to do prediction on larger real-life road images which are different from those used to train the model. We will compare our results with an academic paper published on the topic.

Pain Point

Automatic detection of damages in aging infrastructure is critical for smart structure health monitoring. This can help alleviate the need of manual monitoring of the structures such as building and bridges; which is labor intensive and time consuming. This blog presents an approach to use transfer learning for automatic detection of damages (cracks) in concrete structures. This blog was inspired by the paper and a related blog post. Related codes are available at Github.

Data set:

The data set is available here. According to the description:

“The dataset contains concrete images having cracks. The data is collected from various METU Campus Buildings. The dataset is divided into two as negative and positive crack images for image classification. Each class has 20000images with a total of 40000 images with 227 x 227 pixels with RGB channels. The dataset is generated from 458 high-resolution images (4032x3024 pixel) with the method proposed by Zhang et al (2016). High-resolution images have variance in terms of surface finish and illumination conditions.”

Let’s get the data..

Download the data and extract it to a folder called train. There are two folders “Positive” and “Negative”. According to the original paper, the authors used 15% of the data for testing. We will move randomly pick 15% from both classes and move it to a test directory. Check out the related Github repo to follow along. Here is a function to achieve this.

Distribution of training and test images:

Train: Number of cracked images: 17000

Train: Number of non-cracked images: 17000

Test: Number of cracked images: 3000

Test: Number of non-cracked images: 3000

Few examples of the Positive (cracks) and Negative classes. This is a binary classification problem.

Tools of trade

We will be using fastai v2 for fine tuning a pre-trained ResNet34 model. Fastai is my go to library when it comes to transfer learning for computer vision tasks. Fastai has implementation of the state of the art techniques with a user-friendly API enables rapid prototyping possible. Check out fastai website for more details.

Data Pre-processing and data augmentation

In the original paper, authors didn’t include any data augmentation. We will utilize fastai to include data augmentation to increase our dataset size.

Reading data with fastai is very easy. All you need to do it to provide the root folder that contains images in different folders separated by classes. Fastai uses DataBlocks and DataLoaders for reading and processing data. We will use the function below to read and process the data. Don’t worry, we are going to discuss each line.

Let’s go over the code. The function expect path of the training directory which in this case is arrange as follows:

train

|---Positive

|---Negative

Next,

blocks=(ImageBlock, CategoryBlock)

This tells fastai about the nature of the data (images in the case) and the type of the dependent variable (categorical in this case). fastai uses this information to set the appropriate loss function and proper encoding for the dependent variable e.g. one-hot encoding. Fastai will also set the default optimizer (Adam).

get_items=get_image_files, get_y=parent_label,

In this case the DataBlock is arranged such that it expects image data where the function “get_image_file” will be used for reading image file and “parent_label” will be used for extracting the corresponding labels (classes) from the folder names.

splitter=RandomSplitter(valid_pct=0.2, seed=42)

fastai will use a RandomSplitter to randomly select 20% of the training data for validation.

item_tfms=Resize(resize_image),

Sometimes the available images are of different sizes, here we resize all images to the same size before doing any other transformations.

batch_tfms=[*aug_transforms(do_flip=True, flip_vert=True, max_rotate=10, max_zoom=1.1, max_lighting=0.2, min_scale=0.75, size=image_size), Normalize.from_stats(*imagenet_stats)]

List of all the transformation that are done on each batch. This includes data augmentations (listed above) as well as normalization using ImageNet weights since we are using a pre-trained network.

image_dataloader = get_dataloader(path="data/train", 128, 227, 224)

we define our image_dataloader by providing the path, batch_size and the image sizes that we want.

The following line will randomly show some images from the dataloader.

image_dataloader.show_batch(max_n=9)

Now the fun part…

we are going to create a classifier based on the ResNet -34 architecture:

learn=cnn_learner(image_dataloader, resnet34, metrics=[error_rate, accuracy])

Here we are tell fastai where to get the data from (image_dataloader), the architecture to use (resnet34) and then the metrics to use.

What about the hyper-parameters?

Fastai takes care of setting a whole bunch of hyper-parameter for you with appropriate defaults. One of the most important one is the learning rate that requires experimentation. Fastai makes easier to choose that as well. You can use the lr_find tool as follow:

learn.lr_find()

You want to choose the point with maximum slope. We choose 10^(-3). We are all set to train the model. We train it for 5 epochs as follows:

learn.fit_one_cycle(5, 1e-3)

Wow.. that was fast and the result look great on the validation set. You can get the confusion matrix on the validation set using the following code.

We can also check the miss-classified images as follows:

This is a great tool for understanding why wrong predictions are happening. Some of the original labels seem to be wrong in this case. Some are hard to judge. This can give you good insight of your model performance.

But, how about the test data?

We can check the performance of the model on the independent test set using the code segment below:

As you can see that after 5 epochs, we have 99.87% on the validation set and 99.95% on the independent test set. This shows how easy it is to train powerful models with minimum effort. The model was trained on concrete images taking from the buildings. We have trained the model on relatively small images of 224x224 pixels. Can we extend this work to be used on real-life images? Well the answer is yes.

Testing on real-life images that were not seen before.

The model is trained on relatively small images, however, we want to test them on much larger images e.g. images taken by a self-driving car. In this case, we will divide the images into smaller patches and run the prediction on each patch and then stitch all the images back together to get prediction on different parts of the original image. There will be some patches whose dimension is smaller than the size of the images used for training, we will discard those. Here is the code used for dividing the larger images into smaller patches and getting predictions. This code utilizes code from the blog here.

Some results:

Predictions on a road image. Patches with cracks are highlighted with red color.

Another image. The results look great.

This image is tricky. The model was able to correctly classify cracks on the road. The curbside is something that is not present in the training data and the model thinks its a crack or not the normal class.

Take aways…

Its easy to train powerful models with transfer learning and libraries like fastai

The model was trained on images of cracks and no-cracks in concrete structures. However, the model is still able to generalize to road size images. This shows the power of transfer learning.

Anyone can use this code as a template for their classification problems.

Check out my github repo to follow along and play with the code. Would appreciate feedback from the community.

References