Write a Selfie and Face Detection App
Tags: Opencv, Selfie, Machine Learning Tools, Pretrained Model, Face Detection App

Face recognition is one of the earliest practical applications of artificial intelligence (AI) and machine learning (ML). As fb has started to use face recognition technology a few years ago, it can automatically recognize people (and their names) in photos through machine learning, facial recognition technology has become well known. In recent years, it has also been applied to surveillance systems which draws a lot of people’s attention.

I think many of us may want to learn more about the technology by developing our own app rather than just being recognised by other people’s facial recognition apps. For a starter, one may try using Google’s Teachable Machine which provides some hands-on experience on how to train a machine to recognise different people. Here is my youtube (Yiu, 2021) showing you how to do that (Figure 1).

Figure 1 How to Use Google’s Teachable Machine to Train a Machine to Recognise People. Source: Yiu (2021)

It’s easy to use, but you can’t change the app. This article introduces very simple coding examples for you to selfie (take photos) by a computer and detect faces in the photos.

Years ago, coding for a Machine Learning App was relatively difficult. Take face detection app as an example, it requires writing a training app to train the machine to differentiate between human faces from all other entities in photos. That is similar to the source codes generated by the Teachable Machine, you can realise how complicated it can be.

However, today we have many pre-trained ML libraries to do all these tasks, including face detection, smile detection, etc. and many experts share their program codes and make videos to teach us how to use the libraries, it becomes much easier for us to do ML by ourselves.

Before introducing the codes, let me show you the outcomes first. In the first part, the app program the computer to take a photo of the scene facing the camera (selfie). Figure 2 shows my example. I intentionally held my mobile with my photo in it to test the face detection app whether it can detect two faces or not.

Figure 2 a photo taken by a Selfie App.

I use the following face detection library from OpenCV based on Haar Cascade Classifier:

!wget — no-check-certificate \ https://raw.githubusercontent.com/computationalcore/introduction-to-opencv/master/assets/haarcascade_frontalface_default.xml \-O haarcascade_frontalface_default.xml

This pre-trained machine learning program can recognize the two faces in the photo, and it displays them by blue square frames. Figure 3 shows the result. There are also libraries for eye, mouth and smile detections that you may try.

Figure 3 the two faces in the photo are detected by the App.

How about if a face is not completely shown or a side-face? Here is my second test. Figure 4 shows the second selfie that I took, which the mobile phone obstructs part of my face and the photo shown in the mobile phone is a side-face. The background is also dimmer.

Figure 4 another photo taken by the Selfie App. with the face being obstructed

Uh-oh! This time the detector can only detect the side-face in the mobile phone, but cannot recognize the face behind it (Figure 5). Is it because of lighting or being blocked by the phone? You can test and find the answer by your own.

Figure 5 only one face in the photo is detected by the App.

Of course, what I introduced here is only the most basic application of face detection technology. It only detect faces, but cannot tell who the person is. It is far away from facial recognition technology.

Regarding the coding content, you may following the following two references. The first one is the complete codes in colab, which you can directly run at your computer.(https://colab.research.google.com/github/dortmans/ml_notebooks/blob/master/face_detection.ipynb#scrollTo =N77mENRp_cDV).

It says the codes are based on Rosebrock’s (2018) “Open CV and Deep Learning program” Blog.

Figure 6 Face Detection by OpenCV and Deep Learning, by Adrian Rosebrock (2018) on February 26, 2018 https://www.pyimagesearch.com/2018/02/26/face-detection-with-opencv-and-deep-learning/

I also used the CascadeClassifier program, which simplifies the coding of face detection a lot. Please refer to https://colab.research.google.com/github/computationalcore/introduction-to-opencv/blob/master/notebooks/ 4-Cascade_classification.ipynb#scrollTo=rOG12Rey7g9C, which is also a program on the Colab platform. They use Haar feature-based cascade classifiers, a pre-training program that recognizes human faces, smiles and eyes.

But the performance is not quite satisfactory. There are some mistakes and omissions. Their webpage also shows the mistakes of detecting smiles as shown in Figure 7. The T-Shirt folds are regarded as smiles!

Figure 7 ‘it detected the smiles correctly (the right guy, who is actually me, is not giving a full smile) but there are a lot of false positives, as usual in cascade models.’ https://colab.research.google .com/github/computationalcore/introduction-to-opencv/blob/master/notebooks/4-Cascade_classification.ipynb#scrollTo=_IEdRb4-gU3a

There was a Youtube video that introduced how to make an automatic selfie machine capturing all smiles automatically. It doesn’t require any button pressing, as long as you smile that the machine detects, a selfie will be taken. It demonstrates how ML can help keep our memories of good times in life. If you are interested, you can browse Programming Hero (2021).

References:

Programming Hero (2021) Take Automatic Selfie Using Python, Youtube, June 23. https://youtu.be/xhG2AeyJXcU

Rosebrock, A. (2018) Open CV and Deep Learning program Blog, Feb 26. https://www.pyimagesearch.com/2018/02/26/face-detection-with-opencv-and-deep-learning/.

Yiu, C.Y. (2021) Learning Machine Learning — Train People to Train a Machine, Medium, Feb 2. https://ecyy.medium.com/learning-machine-learning-7e273c1a4728