Two Step Facial Recognition With Colab | by Dev Dash MD MPH | Towards Data Science

It’s Monday morning and I’m channeling my inner Jeremy Howard (again). Over the last few months, I have seen plenty of facial recognition guides and some of them are really quite good.
The problem is, each student of deep learning or machine vision becomes interested through different venues. Maybe you’re a musician interested in sounds, or a statistics student interested in tabular data.
I know I personally became interested when started reading about TensorFlow and image classification — this idea that if I have a well curated dataset, I might be able to train an architecture to classify images with high accuracy.
Google Colab didn’t exist then and I recently ported something similar over to Colab and I’m in the process of a walkthrough. Check out Part 1 here:
towardsdatascience.com
My initial foray was done on my iMac using Anaconda, Spyder, no GPU usage, just letting my Core i5 trudge through thousands of images while I left to do other things (read: study for emergency medicine boards). Of course, before I could train, I had to read through two dozen articles of people using different styles to write code. What resulted was a chimeric hodge-podge of code that was difficult to understand when I looked at it weeks later.
Then Colab entered the scene and while I still use Jupyter and Spyder on my machine, I use Colab to speak to you and efficiently share my code. It’s not perfect but they’ve done a great job in making high end computing available to the masses and all you need is a Google account.
Enough rambling. Let’s walk through the notebook:
I’m being a bit facetious here — the two steps are “Run All” and uploading your picture of faces.
The notebook is designed to hide most of the code so you can cleanly look at the results. Look at the output first and convince yourself that we can not only pick out (hopefully most) the faces in the picture and later on, assign an emotion using an emotion detector function.
I’ve also left in some code to try to time the calculation process.
This one uses the ‘face_recognition’ library, found here. We will be timing how long this takes, too.
This line in particular does all the heavy lifting:
face_locations = face_recognition.face_locations(image)
The rest of the code uses a for loop to iterate through the number of faces, and draw a box around the faces using these two lines:
rect = patches.Rectangle((left,top), (right - left),     (bottom-top),linewidth=3,edgecolor='r',facecolor='none')ax.add_patch(rect)
Note: Not all faces are detected; hats and sunglasses throw the detection off.
Number of faces detected: 24
Time taken: 2.43 seconds
The code is almost identical to above except for one line (if block at line 11):
There’s a reason the ‘if’ statement is present as we’ll see in the image below. The ‘detect_emotions’ line doesn’t work with all the faces.
Here’s the result:
Note: Emotion detection mostly works. Not all faces register an emotion and it thinks the baby is angry. I will let you take a closer look to see if you feel the labels match up perfectly — I don’t think they do.
Number of faces detected: 24
Time taken: 2.83 seconds (slightly longer to add emotion detection)
Haar cascade based recognition is another way of doing facial recognition. Discussion of how these work is beyond this post. There are plenty of technical resources available online that discuss this method.
Here’s the result:
Again, using Haar cascades is another method and there are some missteps here as well. In particular, it misses some faces and thinks the combination of a pant leg, ankle, top of a shoe is a face.
However, in every iteration I have run, using Haar cascades and OpenCV is much faster than using the face_recognition library — almost 20–30% faster at times. This kind of speed benefit is significant, especially if going through hundreds of images (e.g. video). It also speaks to a process that might be computationally less expensive and likely runs on less expensive hardware that requires less power.
Of course, these are not the only two facial detection methods and proprietary ones are likely much better. If you had to pick between the two, I don’t think there’s a clear answer — one is better at detection (face_recognition) but the other is much faster (Haar cascades).
Number of faces detected: 24
Time taken: 1.95 seconds (faster)
[1] Wikipedia-Jeremy Howard, Accessed October 2020
[2] Facial Recognition Library, PyPi, Accessed October 2020
[3] Open CV Haar Cascades, Accessed October 2020