How to Detect Face Masks in Images with TensorFlow.js
By: Krissanawat Kaewsanmuang
Tags: TensorFlow, JavaScript, Image Classification, Heartbeat, Machine Learning

Project Setup

The structure of this project is simple. We are not going to download a pre-trained model—instead, we’re going train our own model. This tutorial aims to educate beginners on TensorFlow and image recognition. Hence, we’ll start the project from scratch.

Here, we have the datasets for two types of images—people wearing and not wearing a mask—using which, we’re going to train our model. You can download the training face mask dataset images from Kaggle.

We also have two images for testing purposes, which are not included in the training image data set. You can get these from free-to-download image websites like Unsplash.

With the dataset, testing images, and programming files, our final project structure should look as shown in the screenshot below:

index.html is our main landing file. Hence, we need to include every script library in it, as shown in the code snippet below:

<!DOCTYPE html>

<html lang="en">

<head>

<meta charset="UTF-8">

<title>TensorFlow.js Mask detection</title>

<link href="./style.css" rel="stylesheet">

</head>

<body> <div class="train-images"></div> <img id="test-img" src="./images/no_mask.jpg">

<script src="

<!-- Load MobileNet -->

<script src="

<!-- Load KNN Classifier -->

<script src="

<script src="app.js"></script>

</body>

</html> https://cdn.jsdelivr.net/npm/@tensorflow/tfjs</a> "> https://cdn.jsdelivr.net/npm/@tensorflow-models/mobilenet</a> "> https://cdn.jsdelivr.net/npm/@tensorflow-models/knn-classifier</a> ">

Here, we have also included a style.css file. For HTML elements, we have an image element with an image source. We have also included our local script as an app.js file, where we are going to code all our logic.

Now, for CSS design codes, we can refer to the codes in the following snippet:

body {

margin: 0;

padding: 0;

background-color: antiquewhite;

} .train-images {

display: none;

} #test-img {

max-height: 300px;

position: absolute;

top: 50%;

left: 50%;

transform: translate(-50%, -50%);

border: 10px solid red;

} #test-img.mask {

border: 10px solid green;

} #test-img.no-mask {

border: 10px solid red;

}

We need to place the above CSS design codes in our style.css file. Here, we simply have CSS styles for images with subjects wearing a mask and those with subjects not wearing a mask. We will distinguish the pictures using the border colors we’ve coded in this CSS file.

Now onto the main logic to be included in the app.js file. First, we need to fetch all data and add it to this file in order to prepare to train the model. The code to do that is provided below:

window.onload = async () => {

const maskImageCount = 5;

const noMaskImageCount = 6; const trainImagesContainer = document.querySelector('.train-images');

// Add mask images to the DOM and give them a class of `mask-img`

for (let i = 1; i <= maskImageCount; i++) {

const newImage = document.createElement('IMG');

newImage.setAttribute('src', `images/mask/${i}.jpg`);

newImage.classList.add('mask-img');

trainImagesContainer.appendChild(newImage);

}

// Add no mask images to the DOM and give them a class of `no-mask-img`

for (let i = 1; i <= noMaskImageCount; i++) {

const newImage = document.createElement('IMG');

newImage.setAttribute('src', `images/no_mask/${i}.jpg`);

newImage.classList.add('no-mask-img');

trainImagesContainer.appendChild(newImage);

}

Next, we need to create a function to classify images using the KNN classifier. This function is used to train the classifier and receives a MobileNet object as a parameter. Using the mobilenet object, we will return the already trained objects. The code for the asynchronous function trainClassifier is provided in the code snippet below:

async function trainClassifier(mobilenetModule) {

// Create a new KNN Classifier

const classifier = knnClassifier.create(); // Train using mask images

const maskImages = document.querySelectorAll('.mask-img');

maskImages.forEach(img => {

const tfImg = tf.browser.fromPixels(img);

const logits = mobilenetModule.infer(tfImg, 'conv_preds');

classifier.addExample(logits, 0); // has mask

});

// Train using no mask images

const noMaskImages = document.querySelectorAll('.no-mask-img');

noMaskImages.forEach(img => {

const tfImg = tf.browser.fromPixels(img);

const logits = mobilenetModule.infer(tfImg, 'conv_preds');

classifier.addExample(logits, 1); // no mask

}); return classifier;

}

Next, we need to load the MobileNet module and pass the mobilenet object to the function we implemented previously. Then, after we receive the classified data from the classifier, we run inference in order to get the prediction result.

In order to classify the images as containing a mask or no mask, we’re going to add the above-defined border colors to the image using CSS styles.

The overall code for this is provided in the code snippet below:

// Load mobilenet module

const mobilenetModule = await mobilenet.load({version: 2, alpha: 1});

// Add examples to the KNN Classifier

const classifier = await trainClassifier(mobilenetModule); // Predict class for the test image

const testImage = document.getElementById('test-img');

const tfTestImage = tf.browser.fromPixels(testImage);

const logits = mobilenetModule.infer(tfTestImage, 'conv_preds');

const prediction = await classifier.predictClass(logits); // Add a border to the test image to display the prediction result

if (prediction.label == 1) { // no mask - red border

testImage.classList.add('no-mask');

} else { // has mask - green border

testImage.classList.add('mask');

}

Hence, we get the results as displayed in the demo screenshots below:

As we can see, we get a red border for an image in which a person is not wearing a mask.

And, for an image where people are wearing masks, we get a green border, as shown in the demo screenshot below:

Hence, we have successfully implemented the first version of a program that can determine if people in a given image are wearing a mask or not using a simple HTML project with the TensorFlow library and MobileNet.

Conclusion

It’s awesome that we now have the ability to develop solutions to existing real-world problems using programming technologies. In this tutorial, we learned how to detect the existence of face masks in an image by classifying the image with MobileNet and a KNN classifier.

Though this is a simple solution, this can be a starting place for more complex projects and is just a starting point to learn about image classification using TensorFlow.js. Now it’s up to you to take it to the next level and innovate new solutions.

Happy Coding folks!