MaskOn — A Simple Mask Detector WebApp
By: Vishwanatha H M
Tags: Detection, Image Processing, Covid 19, Masks, Deep Learning

With Covid-19 shutting down the world around us, I got enough time to work on a fun project that involved both software development and data science. After reading an article about someone building their version of mask detectors, I got inspired to build a web app and a custom data science pipeline to detect face masks.

After two weeks of 3hr/day work, Finally, I had a stable working version. Below are the technical details of the implementation. If you want to check out the application, you can check it here.

The basic flow of how the application works is given below.

Application flow

WebApp

MaskOn is a simple web application built using React (Frontend) and NodeJS (backend). Some of the technical details are provided below

No redux is used as it will be a very simple application and default state management within the components should suffice the requirement.

Face-api.js — A Javascript API is used for face detection and drawing labels over the image

CSS is made to be mobile friendly using Material UI and few media queries

Website is optimized to have a faster initial load and load models in the background

Version control is managed on Github and the application is hosted on Heroku

In short, it is a simple web app that provides features of selecting an image or use a webcam to detect a mask and display it.

Face detection

For face detection, a pre-trained model called tinyFaceDetector ` was used. It was chosen as it is small in size and gives a very good performance. It was not re-trained for the current use case.

Mask detection

For mask detection, a custom CNN network was trained. The training was done on a local computer using Tensorflow. Other tools and libraries used are — Jupyter Notebook, OpenCV, Numpy, Pandas.

Training details are as below,

Image-preprocessing

A simple pre-processing was done by

Size reduced to 150x150 pxs

Values reduced to the range 0–1

Data augmentation is done on rotation & horizontal flip to account for different angles of the face

Model selection

I wanted to use the simplest(or somewhere near to it) model possible, this was because of the reasons,

It is being used in a web app where the download time of the model matters for the user experience.

Mask/No mask seems like a fairly straight forward visual problem that can be tackled with smaller networks.

The architecture I choose was,

12 layers of Conv(6) + MaxPool(6) (with increasing filter size and filter depth)

One Dense layer with 128 neurons

One Sigmoid output neuron

Model training

The training was done with,

80/20 split was done for train/test

30 epochs were run

batch size of 32 was used

Training and Validation accuracy reached very close to 1 by the end of 30 epochs.

Model usage

As we will be using the model with tensorflow.js, the model which is saved in a .h5 file needs to be saved into a format required by tensorflow.js. This was done using a tensorflowjs converter.

import tensorflowjs as tfjs

tfjs.converters.save_keras_model(model, './modelData')

Learnings

A Complete Data science-based application implementation — The project gave an Idea about how to set up everything required for a data science-based web app.

Some of the things I learned specifically while doing this project are,

The trade-off between the size of the model and the accuracy and how to reach a middle ground.

Improving frontend performance by auditing the app and improving things like initial load, image formats, etc

Building clean and mobile-friendly layout

Conclusion

The current implementation serves as an MVP for the idea. It is a very simple and lightweight web app that provides fast and fairly accurate mask detection. Further improvements can be done from here on.

Future improvements

Improve some of the frontend edge scenarios — Currently, the application is not completely tested and it might have some of the user scenarios where it might malfunction.

Improve face detector model — Current face detection model is a pre-trained model that does not specifically have faces with masks in the training set. This decreases the face detection in cases of masked-images. This can be reduced by using masked faces during face detection training.

Improve mask detection model— As I have not looked at other architectures that might suit the use case, it would be a good exercise the brainstorm on different network architectures and try out a few of them.