# Facial Expression Recognition Using Computer Vision: A Systematic Review
## 4 Pre-Processing
### Face Detection  
Generally, face detection is the very beginning of an FER system. This technique is responsible for selecting the ROI of an input image that will be fed to the next steps of the FER system. Most reviewed papers used the classic Viola-Jones face detector [68] from 2004.  
The Viola-Jones face detector is a Machine Learning based approach where a cascade function is trained from a lot of positive images (images with faces) and negative images (images without faces). Haar features are used in this algorithm and they are applied in all training images to find the best threshold which will classify the faces as positive or negative detections. Figure 5 shows how these features are selected by AdaBoost (learning algorithm that selects a small number of critical visual features from a very large set of potential features).  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Database** & **Capacity** & **Emotion** & **Environment** & **Facial Expressions** & **Website** \\ \hline CK+ & 593 videos & Posed & Controlled & 8 & [59] \\ \hline JAFFE & 213 images & Posed & Controlled & 7 & [60] \\ \hline BU-3 DFE & 606 videos & Posed and Spontaneous & Controlled & 7 & [61] \\ \hline FER-2013 & 35,887 images & Posed and Spontaneous & Uncontrolled & 7 & [62] \\ \hline EmotiW & 1268 videos and 700 images & Spontaneous & Uncontrolled & 7 & [63] \\ \hline MMI & 2900 videos & Posed & Controlled & 6 & [64] \\ \hline ENTERFACE’05 & 1166 videos & Spontaneous & Controlled & 6 & [65] \\ \hline KDEF & 4900 images & Posed & Controlled & 7 & [66] \\ \hline RaFD & 8040 images & Posed & Controlled & 8 & [67] \\ \hline \hline \end{tabular}
\end{table}
Table 1: Summarized FER databases.  
Figure 4: Examples of the introduced FER databases [50; 51; 52; 53; 54; 55; 56; 57; 58].  
Two examples of features are shown in the top row and then overlaid on a training face in the bottom row. Basically, these features measure the difference in intensity between the white region and the black region. Typically, one region is darker than the other, and that is how the features are selected to detect faces.  
Very few works used other face detectors, such as the Dlib library [69] and Multi-task Cascade Convolution Neural Network (MTCNN) [70]. The Dlib face detector uses an ensemble of regression trees to regress the location of 68 facial landmarks from a sparse subset of intensity values extracted from an input image and, consequently, detect where the faces are. Figure 6 shows a detected face from the CK+ database and its 68 facial landmarks using the Dlib library.  
As for MTCNN, it goes through three stages to output a proper face detection, and, in each stage, the face that is being analyzed goes through a CNN. The first stage obtains the candidate windows and their bounding box regression vectors, merging highly overlapped candidates [71]. The second stage feeds those candidates to another CNN, which rejects a large number of false candidates. The third stage is similar to the second one, but it also outputs five facial landmarks' positions. Figure 7 shows a detected face from the CK+ database and its five facial landmarks using MTCNN.  
Despite Viola-Jones being the most used face detector choice in the reviewed papers, MTCNN outperforms it across several challenging benchmarks for face detection and face alignment, while keeping real-time performance. The main reason Viola-Jones was the most used face detector is  
Figure 5: Features selected by the AdaBoost learning algorithm in a face from the CK+ database.  
Figure 6: Detected face from the CK+ database using the Dlib library.  
Figure 7: Detected face from the CK+ database using MTCNN.  
that most reviewed works tested their FER systems in controlled environment databases, where Viola-Jones can smoothly detect the faces. Works that tested their FER systems in uncontrolled environment databases generally used MTCNN.