Face Mask Detection using Mask R-CNN(Part-2)
By: Rishikesh Pathak
Tags: Object Detection, Face Mask

Photo by Kate Trifo on Unsplash

In the previous article (Part-1), We were learning how to prepare the dataset and make the dataset object to feed it to the model to train. We have also tested the dataset model and it works fine.

Now, In this part we have to move on to the next steps for training, evaluating, and testing the model. So, let’s get into it.

Train Mask R-CNN Model for Face Mask Detection

Although we can train a Mask R-CNN model from scratch like other computer vision tasks time and computational power can be saved by using transfer learning.

So, the first step will be to download the pre-trained weights. Download the model weights to a file with the name ‘mask_rcnn_coco.h5‘ in your current working directory.

Next, a configuration object for the model must be defined.

We will have to make a config class that extends the mrcnn.config.Config class and defines properties of both the prediction problem (such as name and the number of classes) and the algorithm for training the model (such as the learning rate).

# define a configuration for the model class FacemaskConfig(Config): # Give the configuration a recognizable name NAME = "Facemask_cfg" # Number of classes (background + without_mask and with_mask) NUM_CLASSES = 1 + 2 # Number of training steps per epoch STEPS_PER_EPOCH = 131 # prepare config config = FacemaskConfig()

Next, we can define our model.

This is achieved by creating an instance of the mrcnn.model.MaskRCNN class and specifying the model will be used for training via setting the ‘mode‘ argument to ‘training‘.

# define the model

model = MaskRCNN(mode='training', model_dir='./', config=config)

Load weights using the load_weights() function.

# load weights (mscoco)

model.load_weights('mask_rcnn_coco.h5', by_name=True, exclude=["mrcnn_class_logits", "mrcnn_bbox_fc", "mrcnn_bbox", "mrcnn_mask"])

To start training use this-

# train weights (output layers or 'heads')

model.train(train_set, test_set, learning_rate=config.LEARNING_RATE, epochs=5, layers='heads')

Note: Your results may vary given the stochastic nature of the algorithm or evaluation procedure or differences in numerical precision. Consider running the example a few times and compare the average outcome.

By now you should have been done with training the model.

Epoch 1/5

131/131 [==============================] - 106s 811ms/step - loss: 0.8491 - rpn_class_loss: 0.0044 - rpn_bbox_loss: 0.1452 - mrcnn_class_loss: 0.0420 - mrcnn_bbox_loss: 0.2874 - mrcnn_mask_loss: 0.3701 - val_loss: 1.3402 - val_rpn_class_loss: 0.0160 - val_rpn_bbox_loss: 0.7913 - val_mrcnn_class_loss: 0.0092 - val_mrcnn_bbox_loss: 0.2263 - val_mrcnn_mask_loss: 0.2975

Epoch 2/5

131/131 [==============================] - 69s 526ms/step - loss: 0.4774 - rpn_class_loss: 0.0025 - rpn_bbox_loss: 0.1159 - mrcnn_class_loss: 0.0170 - mrcnn_bbox_loss: 0.1134 - mrcnn_mask_loss: 0.2285 - val_loss: 0.6261 - val_rpn_class_loss: 8.9502e-04 - val_rpn_bbox_loss: 0.1624 - val_mrcnn_class_loss: 0.0197 - val_mrcnn_bbox_loss: 0.2148 - val_mrcnn_mask_loss: 0.2282

Epoch 3/5

131/131 [==============================] - 67s 515ms/step - loss: 0.4471 - rpn_class_loss: 0.0029 - rpn_bbox_loss: 0.1153 - mrcnn_class_loss: 0.0234 - mrcnn_bbox_loss: 0.0958 - mrcnn_mask_loss: 0.2097 - val_loss: 1.2998 - val_rpn_class_loss: 0.0144 - val_rpn_bbox_loss: 0.6712 - val_mrcnn_class_loss: 0.0372 - val_mrcnn_bbox_loss: 0.2645 - val_mrcnn_mask_loss: 0.3125

Epoch 4/5

131/131 [==============================] - 66s 502ms/step - loss: 0.3934 - rpn_class_loss: 0.0026 - rpn_bbox_loss: 0.1003 - mrcnn_class_loss: 0.0171 - mrcnn_bbox_loss: 0.0806 - mrcnn_mask_loss: 0.1928 - val_loss: 0.6709 - val_rpn_class_loss: 0.0016 - val_rpn_bbox_loss: 0.2012 - val_mrcnn_class_loss: 0.0244 - val_mrcnn_bbox_loss: 0.1942 - val_mrcnn_mask_loss: 0.2495

Epoch 5/5

131/131 [==============================] - 65s 493ms/step - loss: 0.3357 - rpn_class_loss: 0.0024 - rpn_bbox_loss: 0.0804 - mrcnn_class_loss: 0.0193 - mrcnn_bbox_loss: 0.0616 - mrcnn_mask_loss: 0.1721 - val_loss: 0.8878 - val_rpn_class_loss: 0.0030 - val_rpn_bbox_loss: 0.4409 - val_mrcnn_class_loss: 0.0174 - val_mrcnn_bbox_loss: 0.1752 - val_mrcnn_mask_loss: 0.2513



A model file is created and saved at the end of each epoch in a subdirectory that starts with ‘facemask_cfg‘ followed by random characters.

A model must be selected for use; in this case, the loss continues to decrease for the bounding boxes on each epoch, so we will use the final model at the end of the run.

So, we have trained our model!

Next, let’s look at evaluating the performance of this model.

Evaluate a Mask R-CNN Model

The performance of a model for an object recognition task is often evaluated using the mean absolute precision, or mAP.

First, we will be needing a new config object to use for making predictions, we can just extend our previously defined config class to reuse the parameters.

Then, we can define the model with the config class and set ‘mode’ argument to ‘inference’ instead of training.

# create config

cfg = PredictionConfig()

# define the model

model = MaskRCNN(mode='inference', model_dir='./', config=cfg)

Next, we can load the weights from our saved model.

We can do that by specifying the path to the model file. In this case, the model file is ‘mask_rcnn_facemask_cfg_0005.h5‘ in the current working directory.

# load model weights

model.load_weights('mask_rcnn_facemask_cfg_0005.h5', by_name=True)

Next, we can evaluate the model. This involves enumerating the images in a dataset, making a prediction, and calculating the AP for the prediction before predicting a mean AP across all images.

we will be needing a evaluate_model() function to calculate AP and append them to a list to calculate the mAP(mean Absolute Precision).

We can now calculate the mAP for the model on the train and test datasets.

# evaluate model on training dataset

train_mAP = evaluate_model(train_set, model, cfg)

print("Train mAP: %.3f" % train_mAP)

# evaluate model on test dataset

test_mAP = evaluate_model(test_set, model, cfg)

print("Test mAP: %.3f" % test_mAP)

Running the example will make a prediction for each image in the train and test datasets and calculate the mAP for each.

Train mAP: 0.928

Test mAP: 0.902

Now that we have some confidence that the model is sensible, we can use it to make some predictions.

To make predictions on new images. we can use this function.

# plot a number of photos with ground truth and predictions

def plot_actual_vs_predicted(dataset, model, cfg, n_images=5):

# load image and mask

for i in range(n_images):

# load the image and mask

image = dataset.load_image(i)

mask, _ = dataset.load_mask(i)

# convert pixel values (e.g. center)

scaled_image = mold_image(image, cfg)

# convert image into one sample

sample = expand_dims(scaled_image, 0)

# make prediction

yhat = model.detect(sample, verbose=0)[0]

# define subplot

pyplot.subplot(n_images, 2, i*2+1)

# plot raw pixel data

pyplot.imshow(image)

pyplot.title('Actual')

# plot masks

for j in range(mask.shape[2]):

pyplot.imshow(mask[:, :, j], cmap='gray', alpha=0.3)

# get the context for drawing boxes

pyplot.subplot(n_images, 2, i*2+2)

# plot raw pixel data

pyplot.imshow(image)

pyplot.title('Predicted')

ax = pyplot.gca()

# plot each box

for box in yhat['rois']:

# get coordinates

y1, x1, y2, x2 = box

# calculate width and height of the box

width, height = x2 - x1, y2 - y1

# create the shape

rect = Rectangle((x1, y1), width, height, fill=False, color='red')

# draw the box

ax.add_patch(rect)

# show the figure

pyplot.show()

Also to visualize images use this —

class_names = ['BG', 'without_mask', 'with_mask']

visualize.display_instances(image, r['rois'], r['masks'], r['class_ids'], class_names, r['scores'])

And with this, we complete our tutorial.

Summary

In this tutorial, we discovered how to develop a Mask R-CNN model for Facemask detection in photographs.

Thank you for reading!