I Implemented a Face Detection Model. Here’s How I Did It. | by Chi-Feng Wang | Towards Data Science

Last week, I started an internship at Augentix Inc., aiming to learn more about neural networks. There, I came across a model for facial detection which achieved high accuracy while keeping real time performance (link here). This model uses Multi-task Cascaded Convolutional Networks (MTCNN), which is essentially several convolutional networks strung together that give out several pieces of information.
After downloading the model from github (link here), I opened up and ran example.py, which produced this image:
As seen in the image above, the neural network detects individual faces, locates facial landmarks (i.e. two eyes, nose, and endpoints of the mouth), and draws a bounding box around the face. The code from example.py supports this.
First, they import OpenCV (to open, read, write, and show images) and MTCNN. Checking ./mtcnn/mtcnn.py showed the MTCNN class, which performed the facial detection.
#!/usr/bin/env python3# -*- coding: utf-8 -*-import cv2from mtcnn.mtcnn import MTCNN
Then, a detector of the MTCNN class was created, and the image read in with cv2.imread. The detect_faces function within the MTCNN class is called, to “detect faces” within the image we passed in and output the faces in “result”.
detector = MTCNN()image = cv2.imread("ivan.jpg")result = detector.detect_faces(image)
To check what was in “result”, I printed it out to get:
[{'box': [277, 90, 48, 63], 'confidence': 0.9985162615776062, 'keypoints': {'left_eye': (291, 117), 'right_eye': (314, 114), 'nose': (303, 131), 'mouth_left': (296, 143), 'mouth_right': (313, 141)}}]
Result seems to be a dictionary that included the coordinates of the bounding box and facial landmarks, as well as the network’s confidence in classifying that area as a face.
We can now separate the coordinates, passing in the bounding box coordinates to bounding_box and the facial landmark coordinates to keypoints.
bounding_box = result[0]['box']keypoints = result[0]['keypoints']
Now we draw the rectangle of the bounding box by passing in the coordinates, the color (RGB), and the thickness of the box outline. Here, bounding_box[0] and bounding_box[1] represent the x and y coordinates of the top left corner, and bounding_box[2] and bounding_box[3] represent the width and the height of the box, respectively.
Similarly, we can draw the points of the facial landmarks by passing in their coordinates, the radius of the circle, and the thickness of the line.
cv2.rectangle(image,              (bounding_box[0], bounding_box[1]),              (bounding_box[0]+bounding_box[2], bounding_box[1] + bounding_box[3]),              (0,155,255),              2)cv2.circle(image,(keypoints['left_eye']), 2, (0,155,255), 2)cv2.circle(image,(keypoints['right_eye']), 2, (0,155,255), 2)cv2.circle(image,(keypoints['nose']), 2, (0,155,255), 2)cv2.circle(image,(keypoints['mouth_left']), 2, (0,155,255), 2)cv2.circle(image,(keypoints['mouth_right']), 2, (0,155,255), 2)
Finally, we create another file named “ivan_drawn.jpg” and show the image.
cv2.imwrite("ivan_drawn.jpg", image)cv2.namedWindow("image")cv2.imshow("image",image)cv2.waitKey(0)
This was good, but facial detection is a big useless if we could only pass in images. I wanted to modify it so I could use it with my webcam. In doing so, I can also test the paper’s claim that it would be able to locate faces in real time.
Once again, I imported OpenCV and MTCNN, then created a detector:
import cv2from mtcnn.mtcnn import MTCNNdetector = MTCNN()
To use my webcam, I created a VideoCapture object. Since I only have 1 camera, I passed in 0.
cap.read() returns a boolean (True/False) that states whether or not a frame is read in correctly. If an error occurs and a frame is not read it, it will return False and the while loop will be broken.
Then, similar to example.py, I called detect_faces on each frame. Since sometimes a face may not be in the frame (and result will be empty), I added “if result != []” for the program to continue running even when there are no faces in the frame.
In addition, there may be more than one face in the frame. In that case, result will return back multiple sets of coordinates, one for each face. Hence, I ran a for loop in result to iterate through every individual face. For each face, I drew out the bounding box frame and dotted the 5 facial landmarks.
Finally, I displayed each individual frame. Each frame will wait for 1 millisecond for the key ‘q’, then move on to the next frame. To close the window, all I have to do is press ‘q’. Once I do that, I release the video capture and close the window.
cap = cv2.VideoCapture(0)while True:     #Capture frame-by-frame    __, frame = cap.read()        #Use MTCNN to detect faces    result = detector.detect_faces(frame)    if result != []:        for person in result:            bounding_box = person['box']            keypoints = person['keypoints']                cv2.rectangle(frame,                          (bounding_box[0], bounding_box[1]),                          (bounding_box[0]+bounding_box[2], bounding_box[1] + bounding_box[3]),                          (0,155,255),                          2)                cv2.circle(frame,(keypoints['left_eye']), 2, (0,155,255), 2)            cv2.circle(frame,(keypoints['right_eye']), 2, (0,155,255), 2)            cv2.circle(frame,(keypoints['nose']), 2, (0,155,255), 2)            cv2.circle(frame,(keypoints['mouth_left']), 2, (0,155,255), 2)            cv2.circle(frame,(keypoints['mouth_right']), 2, (0,155,255), 2)    #display resulting frame    cv2.imshow('frame',frame)    if cv2.waitKey(1) &0xFF == ord('q'):        break#When everything's done, release capturecap.release()cv2.destroyAllWindows()
Running this new file, I see that the MTCNN network can indeed run in real time. boxing my face and marking out my features. If I move my face out of the frame, the webcam continues running as well.
Click here to read about the structure of the MTCNN model!
Click here to read about how MTCNN does face detection!
Download the MTCNN paper and resources here:
Github download: https://github.com/ipazc/mtcnn
Research article: http://arxiv.org/abs/1604.02878