# Hybrid Approach for Facial Expression Recognition Using Convolutional Neural Networks and SVM
## 3 Experiments
### Facial Expression Recognition Using CNN-Based Facial Motion Flow for CK+ Database  
The CK+ database was released for the purpose of promoting research into automatically detecting individual facial expressions. Since then, the CK+ database has become one of the most widely used datasets for algorithm development and evaluation as it provides AU coding. Based on the facial action unit criteria, it provides seven facial expression categories: Anger (An), Contempt (Co), Disgust (Di), Fear (Fe), Happiness (He), Sadness (Sa), and Surprise (Su). We used these seven facial expression categories as the class labels of the facial expression recognition system. In all of the following experiments, the CK+ database with seven class labels is used. The number of class samples in the database is different in different expressions. We evaluated the facial expression recognition performance using four-fold cross-validation for the CK+ database. From the training database, 20% of the data are used for the validation test for the learning of the CNN-based facial expression recognition. For the training of the proposed CNN-based facial expression recognition from facial motion flow, we trained the proposed model with batch size 256 and learning rate starting from 0.0003 and reducing gradually. Figure 6 shows the feature map visualization of the second convolution layer of the model, when we applied training data in different expressions. We selected representative feature maps of each facial expression, which shows feature maps that capture distinguishing characteristics of each facial expression. The visualized feature map of each expression shows different activation areas of each expression, which may be corresponding to the action unit of each expression type. The result implies that the network properly extracts the features of different expression types. Table 2 shows the CK+ facial expression recognition results, with an average recognition rate of 83.3% using CNN-based classification with the network architecture presented in Figure 4a. By the modified network architecture presented in Figure 4b, we achieve higher facial expression recognition performance than the basic network architecture with an average 90.5% recognition rate, as shown in Table 3. We evaluated the facial expression recognition performance in different input data types. Table 4 shows the performance of facial expression recognition for the CK+ database. Three channels are used for the input in the case of \(\Delta x\), \(\Delta y\), angle(\(\theta\)), and \(\Delta x\), \(\Delta y\), Intensity(\(I\)). The table shows that the proposed two-channel motion flow (\(\Delta x\), \(\Delta y\)) outperforms other input data types. The performance of the modified CNN-based facial expression recognition using two-channel motion flow (\(\Delta x\), \(\Delta y\)) is 7.2% better than the basic CNN-based model's performance. \begin{table}
\begin{tabular}{c c} \hline
**Input Data Type** & **Recognition Performance** \\ \hline Texture image & 64.8 \\ \hline \(\Delta x\) & 79.51 \\ \hline \(\Delta y\) & 83.79 \\ \hline abs(\(Delta\), \(\Delta y\)) & 77.98 \\ \hline Intensity of motion flow(\(I\)) & 75.54 \\ \hline Angle of motion flow(\(\theta\)) & 78.29 \\ \hline Intensity(\(I\)) and angle(\(\theta\)) of motion flow & 78.29 \\ \hline \(\Delta x\), \(\Delta y\), \(\theta\) & 76.76 \\ \hline \(\Delta x\), \(\Delta y\), \(I\) & 88.69 \\ \hline Motion flow(\(\Delta x\),\(\Delta y\)) & 90.52 \\ \hline \end{tabular}
\end{table} 
Table 4: Modified CNN-based facial expression recognition performance according to different input data types.