# Hybrid Approach for Facial Expression Recognition Using Convolutional Neural Networks and SVM
## 4 Conclusions and Future Works  
This paper presents the state-of-the-art performance of a facial expression classification system obtained by using a hybrid model: a mixture of CNN-based facial motion flow classification and SVM-based facial geometry displacement classification. The hybrid model exhibits higher facial expression classification performance than each individual facial expression classifier. The facial motion flow, which is the extracted facial motion displacement in every pixel, provides data that can be used to extract motion characteristics without a specific network architecture for motion extraction. A CNN architecture that can model facial motion flow is also presented. In addition, we have compared the proposed model with the state-of-the-art models and shows recognition performance of 99.69% and 94.69% for the CK+ and BU4D database, respectively. For the CK+ database, if we compare our recognition performance with [48], which uses a combination of CNN and SIFT, there is an improvement of 0.59%, and if we compare the proposed model with [38], which uses DTAN and DTGN, there is an improvement of 3%. For the BU4D database, if we compare the recognition performance of the proposed model with [54], which uses HOG3D for onset frames, there is a slight degradation of 1.95%. However, if we compare performance with all frames, our proposed model performs better by 11.84%. In addition, if we compare our proposed model with [46], there is an improvement of.51%, which shows the effectiveness of the proposed approach. In conclusion, the proposed method outperformed the others in the case of the CK+ database and is still comparable for the BU4D database with the previous schemes.  
The limitation of proposed system is that we have used only western databases for the recognition of facial expressions, but in the future, we will add an eastern database for facial expression recognition. Even though our system is suitable to detect facial expressions, the detection should be interpreted carefully, because of the limited congruency between the facial expression and the emotion actually experienced by a person. The proposed hybrid model could be improved by improving the CNN-based facial expression classification performance. The weighting factor can be further optimized for better classification by determining the optimal weighting for each expression category or by determining another  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Approach** & **Details** & **Recognition Performance** \\ \hline [PERSON] et al. [44] & Using BoMW & 63.8 \\ \hline [PERSON] et al. [51] & Using 2D-HMM & 68.3 \\ \hline [PERSON] et al. [53] & Using LBP-TOP & 71.6 \\ \hline [PERSON] et al. [49] & Using FFD & 73.4 \\ \hline [PERSON] et al. [45] & Using DGIN & 92.22 \\ \hline [PERSON] et al. [54] & Using HOG3D (all frames) & 82.80 \\ \hline [PERSON] et al. [46] & Using DSF (whole video) & 94.18 \\ \hline [PERSON] et al. [54] & Using HOG3D (onset frames) & 96.64 \\ \hline [PERSON] et al. [54] & Using HOG3D (all frames) & 82.80 \\ \hline Proposed method & Using modified CNN + SVM & 94.69 \\ \hline \hline \end{tabular}
\end{table}
Table 12: Facial expression recognition performance in different approaches for BU4D database.  
nonlinear classifier based on each classifier output. In the future, the proposed model can be used for entertainment applications, such as changing music tracks by identifying the mood of a person by using his/her facial expressions.  
Conceptualization, C.-S.L.; Data curation, H.-E.S.; Methodology, J.-C.K.; Software, J.-C.K., M.-H.K. and H.-E.S.; Supervision, C.-S.L.; Writing--original draft, C.-S.L.; Writing--review & editing, M.T.N. All authors have read and agreed to the published version of the manuscript.  
This research was funded by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (2021R1A1A03040177).  
The authors declare no conflict of interest.