Face detection with MediaPipe  | Towards Data Science

MediaPipe is one of the most widely shared and re-usable libraries for media processing within Google.” — Kahye Song
Google open-source MediaPipe was first introduced in June, 2019. It aims to make our life easy by providing some integrated computer vision and machine learning features. Media Pipe is a framework for building multimodal(e.g video,audio or any time series data),cross-platform (i.eAndroid,IOS,web,edge devices) applied ML pipelines. Mediapipe also facilitates the deployment of machine learning technology into demos and applications on a wide variety of different hardware platforms.
Notable Applications
Face Detection
Multi-hand Tracking
Hair Segmentation
Object Detection and Tracking
Objection: 3D Object Detection and Tracking
AutoFlip: Automatic video cropping pipeline
and so on..
Why do you need MediaPipe
Effectively manage resources (CPU and GPU) to achieve low latency performance, handles synchronization of time series data such as audio and video frames and more valid reasons why MediaPipe is necessary. MediaPipe abstracts each perception model into a module and connects them with maintain graph. In addition to the above features, MediaPipe also supports TensorFlow and TF Lite inference engines. Any TensorFlow and TF Lite model can be used on MediaPipe. At the same time on mobile and embedded platforms, MediaPipe also supports GPU acceleration of the device itself.
It’s time to step forward to an amazing application of MediaPipe, Face Detection.
Think about a scenario, “a retailer ask you to count the number of visitors and track the movement of visitors through their stores.”
Seems difficult!! How can we solve the problem? hmmm...
Oh yes! we will use Face Tracking to solve the problem. Now, the question is how Face can be detected the face of the customers. The answer to this question is Face Tracking technology use Face Detection as the first step to detect a human face.
Face detection is a problem in computer vision of locating and localizing one or more faces in a photograph.
A general statement of the problem can be defined as follows: Given a still or video image, detect and localize an unknown number (if any) of faces — Face Detection: A Survey, 2001.
Model To Perform Face Detection with MediaPipe:
To perform face detection three models have been used:
Short-range model (best for faces within 2 meters from the camera)Full-range model (dense, best for faces within 5 meters from the camera)Full-range model (sparse, best for faces within 5 meters from the camera)
Short-range model (best for faces within 2 meters from the camera)
Full-range model (dense, best for faces within 5 meters from the camera)
Full-range model (sparse, best for faces within 5 meters from the camera)
Full-range dense and sparse models have the same quality in terms of F-score however differ in underlying metrics. The dense model is slightly better in Recall whereas the sparse model outperforms the dense one in Precision.
It’s time to make our hands dirty with a hands-on face detection model using MediaPipe.
Step by Step Guideline
Installation of Necessary Libraries
To perform face detection you have to install MediaPipe at first in your machine. If you are a windows user then you can run the below code in your computer’s command prompt.
pip install mediapipe
For details instructions, you can follow the link:
https://google.github.io/mediapipe/getting_started/python.html
You also need to install OpenCV for webcam or image input. If you are a windows user, you can run the below code in your command prompt.
pip install opencv-python
For details instructions, you can follow the link:
https://pypi.org/project/opencv-python/
Write Code To Understand The Use Of API:
We use Google Colab to run the code. You can use it at your choice. Import necessary libraries. We need the cv2module, to be able to read and display the image, and the MediaPipe module, which exposes the functionality we need to perform the face detection
[N.B.- Full code for both google colab version and jupyter notebook version are available of the project. The link is given at the end of the conclusion section of this article.]
import cv2import mediapipe as mp
Then we will access two submodules face_detection and drawing_utils. The face_detection is used to load all functionality to perform face detection and the drawing_utils is used to draw the detected face over the image.
mp_face_detection = mp.solutions.face_detectionmp_drawing = mp.solutions.drawing_utils
It’s time to dig deep into the code. At first, we take an image as an input. Here we use two types of image
(i)image that contains face within two meters
(ii)image that contains face within five meters.
We use files from colab to load images directly from our local directory. You can also use cv2.imread() to load images when you work in your local computer.
(a)Taken first image
from google.colab import filesuploaded_short_range = files.upload()
(b)Taken second image
from google.colab import filesuploaded_full_range = files.upload()
You may use while working on a local PC
cv2.imread() #to take input.
Click here to know more about cv2.imread()
Now we are going to resize the images and show the images. To show the image we have to use cv2_imshow module from colab or cv2.imshow(frame name, iamge) in case of working in a local machine. We may use the below code to resize and showing the image in google colab.
Now, we will draw landmarks on the face.
We may change the value of the thickness and circle_radius as follows.
drawing_spec = mp_drawing.DrawingSpec(thickness=1, circle_radius=1)
To know detail information about mp.solutions.face_detection run
below code:
help(mp_face_detection.FaceDetection)
After this, we are going to create an object of class FaceDetection. This object will allow us to process the image and perform the face landmark estimation. The constructor of this class supports the following parameters:
(i)MODEL_SELECTION:An integer index 0 or 1. Use 0 to select a short-range model that works best for faces within 2 meters from the camera, and 1 for a full-range model best for faces within 5 meters. For the full-range option, a sparse model is used for its improved inference speed. We refer to the model cards for details. Default to 0 if not specified.
(ii)MIN_DETECTION_CONFIDENCE: Minimum confidence value ([0.0, 1.0]) from the face detection model for the detection to be considered successful. Default to 0.5.
with mp_face_detection.FaceDetection(min_detection_confidence=0.5, model_selection=0) as face_detection:
Above code model_selection = 0 which means we select short range model for face detection. Using the below code we perform final face detection using a short image model and also draw the landmark.
Now for model_selection = 1 that means we select face detection full-range model. Using the below code we perform final face detection using the full image model and also draw the landmark.
We can also perform this process on group photos with the code for full length face detection model.
We can go beyond face detection using MediaPipe. The following article on algoscale will show you a direction for estimating pose using OpenCV and MediaPipe.
Workout Pose Estimation using OpenCV and MediaPipe
Face detection is one of the most common problems in computer vision. There are many techniques for face detection and face landmarks drawing. The most efficient techniques are created with the help of deep learning models. But if we try to make the models from scratch, it requires huge computational power, sophisticated knowledge as well as a dataset. In most cases, it’s a problem for the common people. Mediapipe library is amazing in case of making the difficult task easy for us. The library facilities a customized built-in model. In this article, we have just shown the simple and easy process of face detection and face landmarks drawing using MediaPipe. In the next articles, many more easy and useful techniques will be shown with the help of MediaPipe.
Source code in github
Colab source code........
github.com
Jupyter notebook source code.....
github.com
towardsdatascience.com
towardsdatascience.com
If you enjoy the article follow me on medium for more.
Connect me on LinkedIn for collaboration.