# Facial Micro-Expression Recognition Based on Deep Local-Holistic Network
## 4 Experiments
### DLHN Performance Analysis  
#### 4.4.1 Comparison with SOTA Methods  
In the choice of comparison methods, among the handcrafted-feature-based methods, we choose the classical FDM features and LBP features [50], as well as the variant of LBP features (LBP-SIP) [51]. Among the deep learning methods, we choose the first place method for Micro-Expression Grand Challenge 2019 and two deep learning-based methods with codes released in the last two years, which are STSTNet [52], RCN(_a_,\(w\),\(s\), and _f_) [33], and Feature Refinement (FR) [35], respectively. Moreover, we all reproduced these methods with the same data configuration. Table 6 shows the overall accuracy of all algorithms. The best algorithm based on traditional methods for micro-expression recognition is LBP-TOP\((4\times 4)\), which achieves 58.38% mean accuracy. Among the deep learning approaches, the Feature Refinement (FR) approach achieved the highest accuracy rate of 56%. Our proposed DLHN method, with global sparse and local detailed spatio-temporal feature extraction, achieves a better performance than all methods, i.e., an accuracy rate of 60.31%. Two reasons mainly cause the lack of high performance in micro-expression recognition:  
* First, micro-expressions are involuntary and rapidly flowing facial expressions of individuals, which are subtle, brief, and localized. The recognition rate of micro \begin{table}
\begin{tabular}{l c c c c c c c c c c} \hline \hline
**Method** & **Fold1** & **Fold2** & **Fold3** & **Fold4** & **Fold5** & **Fold6** & **Fold7** & **Fold8** & **Fold9** & **Fold10** & **Mean** \\ \hline \hline FDM+SVM & 36.92 & 41.54 & 52.31 & 43.08 & 33.85 & 43.08 & 43.08 & 52.31 & 33.85 & 50.77 & 43.08 \\ LBP\({}_{1}\)+SVM & 55.38 & 52.31 & 50.77 & 56.92 & 58.46 & 47.69 & 53.85 & 55.38 & 56.92 & 53.85 & 53.85 \\ LBP\({}_{2}\)+SVM & **66.15** & 58.46 & 64.62 & 58.46 & **63.08** & 58.46 & 49.23 & 52.31 & **61.54** & **61.54** & 58.38 \\ LBP\({}_{3}\)+SVM & 55.38 & 58.46 & 58.46 & 53.85 & 46.15 & 50.77 & 43.08 & 58.46 & 58.46 & 53.85 & 43.08 \\ LBP\({}_{4}\)+SVM & 60 & 55.38 & 41.54 & 49.23 & 60 & 47.69 & 46.15 & 55.38 & 55.38 & 49.23 & 46.15 \\ STSTNet & 46.15 & 60.00 & 58.46 & 55.38 & 50.77 & 53.85 & 50.77 & 49.23 & 52.31 & 55.38 & 53.23 \\ RCN\_w & 47.69 & 61.54 & 53.85 & 52.31 & 49.23 & 56.92 & 46.15 & 58.46 & 55.38 & 53.85 & 53.54 \\ RCN\_s & 38.46 & 63.08 & 49.23 & 56.92 & 46.15 & 53.85 & 46.15 & 55.38 & 60.00 & 36.92 & 50.61 \\ RCN\_a & 35.38 & 61.54 & 47.69 & 61.54 & 46.15 & 46.15 & 49.23 & 64.62 & 47.69 & 36.92 & 49.69 \\ RCN\_f & 46.15 & 72.31 & 56.92 & 53.85 & 46.15 & 50.77 & 53.85 & 50.77 & 58.46 & 47.69 & 53.69 \\ FR & 46.15 & 61.54 & 58.46 & **66.15** & 61.54 & 56.92 & 50.77 & 44.62 & 56.92 & 56.92 & 56.00 \\ DLHN & 53.85 & **70.77** & **61.54** & **66.15** & 60 & **64.62** & 53.85 & 58.46 & 53.85 & 60 & **60.31** \\ \hline \hline \end{tabular}
\end{table} 
Table 6: The overall accuracy (%) of DLHN and other SOTA methods. LBP\({}_{1}\), LBP\({}_{2}\), LBP\({}_{3}\), and LBP\({}_{4}\) represent LBP-TOP\((2\times 2)\), LBP-TOP\((4\times 4)\), LBP-SIP\((2\times 2)\), and LBP-SIP\((4\times 4)\), respectively. The maximum value in each fold and the maximum mean value are bolded. \begin{table}
\begin{tabular}{l c c c c c c c c c c c} \hline \hline
**Method** & **Fold1** & **Fold2** & **Fold3** & **Fold4** & **Fold5** & **Fold6** & **Fold7** & **Fold8** & **Fold9** & **Fold10** & **Mean** \\ \hline HCRNN & 53.85 & 63.08 & 58.46 & 63.08 & 56.92 & 56.92 & 40 & 52.31 & 55.38 & 50.77 & 55.08 \\ RPRNN & 56.82 & 64.62 & 60 & 61.54 & 56.92 & 60 & **56.92** & **60** & 56.92 & **61.54** & 59.53 \\ DLHN & 53.85 & **70.77** & **61.54** & **66.15** & 60 & **64.62** & 53.85 & 58.46 & 53.85 & 60 & **60.31** \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Ablation study on the HCRNN and RPRNN modules. The maximum value in each fold and the maximum mean value are bolded. expressions in videos by the naked eye is less than 50%, even for professionally trained experts [8].