[Huawei Developers] shikkerimath:
View attachment 5214045

Article Introduction

In this article, we will show how to integrate Huawei ML Kit (Face Detection) and powerful AI engine MindSpore Lite in an android application to detect in realtime either the users are wearing masks or not. Due to Covid-19, the face mask is mandatory in many parts of the world. Considering this fact, the use case has been created with an option to remind the users with audio commands.

Huawei ML Kit (Face Detection)

Huawei Face Detection service (offered by ML Kit) detects 2D and 3D face contours. The 2D face detection capability can detect features of your user's face, including their facial expression, age, gender, and wearing. The 3D face detection capability can obtain information such as the face keypoint coordinates, 3D projection matrix, and face angle. The face detection service supports static image detection, camera stream detection, and cross-frame face tracking. Multiple faces can be detected at a time.

View attachment 5214047

View attachment 5214049

Following are the important features supported by Face Detection service:

View attachment 5214051

MindSpore Lite

MindSpore Lite is an ultra-fast, intelligent, and simplified AI engine that enables intelligent applications in all scenarios, provides E2E solutions for users, and helps users enable AI capabilities. Following are some of common scenarios to use MindSpore:

View attachment 5214053

For this article, we implemented Image classification. The camera stream yield frames. We then process it to detect faces using ML Kit (Face Detection). Once, we have the faces, we process our trained MindSpore lite engine to detect either the face is With or Without Mask.

Pre-Requisites

Before getting started, we need to train our model and generate .ms file. For that, I used HMS Toolkit plugin of Android Studio. If you are migrating from Tensorflow, you can convert your model from .tflite to .ms using the same plugin.

The dataset used for this article is from Kaggle (link is provided in the references). It provided 5000 images for both cases. It also provided some testing and validation images to test our model after being trained.

Step 1: Importing the images

To start the training, please select HMS > Coding Assistance > AI > AI Create > Image Classification. Import both folders (WithMask and WithoutMask) in the Train Data description. Select the output folder and train parameters based on your requirements. You can read more about this in the official documentation (link is provided in the references).

View attachment 5214057

Step 2: Creating the Model

When you are ready, click on Create Model button. It will take some time depending upon your machine. You can check the progress of the training and validation throughout the process.

View attachment 5214059

Once the process is completed, you will see the summary of the training and validation.

View attachment 5214061

Step 3: Testing the Model

It is always recommended to test your model before using it practically. We used the provided test images in the dataset to complete the testing manually. Following were the test results for our dataset:

After testing, add the generated .ms file along with labels.txt in the assets folder of your project. You can also generate Demo Project from the HMS Toolkit plugin.

Development

Since it is on device capability, we don't need to integrate HMS Core or import agconnect-services.json in our project. Following are the major steps of development for this article:

Read full article.

7.2: Final Results

View attachment 5214067

Conclusion

Building smart solutions with AI capabilities is much easy with HUAWEI mobile services (HMS) ML Kit and AI engine MindSpore Lite. Considering different situations, the use cases can be developed for all industries including but not limited to transportation, manufacturing, agriculture and construction.

Having said that, we used Face Detection ML Kit and AI engine MindSpore to develop Face Mask detection feature. The on-device open capabiltiies of HMS provided us highly efficient and optimized results. Individual or Multiple users without Mask can be detected from far in realtime. This is applicable to be used in public places, offices, malls or at any entrance.

Tips & Tricks

Make sure to add all the permissions like WRITE_EXTERNAL_STORAGE, READ_EXTERNAL_STORAGE, CAMERA, ACCESS_NETWORK_STATE, ACCESS_WIFI_STATE.

Make sure to add aaptOptions in the app-level build.gradle file aftering adding .ms and labels.txt files in the assets folder. If you miss this, you might get Load model failed.

Always use animation libraries like Lottie to enhance UI/UX in your application. We also used OwlBottomSheet for the help bottom sheet.

The performance of model is directly propotional to the number of training inputs. Higher the number of inputs, higher will be accuracy to yield better results. In our article, we used 5000 images for each case. You can add as many as possible to improve the accuracy.

MindSpore Lite provides output as callback. Make sure to design your use case while considering this fact.

If you have Tensorflow Lite Model file (.tflite), you can convert it to .ms using the HMS Toolkit plugin.

HMS Toolkit plugin is very powerful. It supports converting MindSpore Lite and HiAI models. MindSpore Lite supports TensorFlow Lite and Caffe and HiAI supports TensorFlow, Caffe, CoreML, PaddlePaddle, ONNX, MxNet and Keras.

If you want to use Tensorflow with HMS ML Kit, you can also implement that. I have created another demo where I put the processing engine as dynamic. You can check the link in the references section.

References

HUAWEI ML Kit (Face Detection) Official Documentation:

https://developer.huawei.com/consum...-Guides-V5/face-detection-0000001050038170-V5

HUAWEI HMS Toolkit AI Create Official Documentation:

https://developer.huawei.com/consumer/en/doc/development/Tools-Guides/ai-create-0000001055252424

HUAWEI Model Integration Official Documentation:

https://developer.huawei.com/consum...ols-Guides/model-integration-0000001054933838

MindSpore Lite Documentation:

Using MindSpore on Mobile and IoT — MindSpore Lite r1.1 documentation

MindSpore Lite Code Repo:

MindSpore/mindspore

MindSpore is a new open source deep learning training/inference framework that could be used for mobile, edge and cloud scenarios.

gitee.com

Kaggle Dataset Link:

Face Mask Detection ~12K Images Dataset

12K Images divided in training testing and validation directories.

www.kaggle.com

Lottie Android Documentation:

Lottie Docs

Lottie is a library for Android, iOS, Web, and Windows that parses Adobe After Effects animations exported as json with Bodymovin and renders them natively on mobile and on the web

airbnb.io

Tensorflow as a processor with HMS ML Kit:

https://github.com/yasirtahir/Huawe...icodelabs/fragments/mlkit/facemask/tensorflow

Github Code Link:

https://github.com/yasirtahir/DetectFaceMask

Read full article.