# An Approach for Selecting the Most Explanatory Features for Facial Expression Recognition
## 6 Conclusions  
In this work, we studied the different contributions of various combination methods to a facial recognition system. To achieve this, we utilized the concept of sparse representations. The results demonstrated that the combination of classifiers can lead to improved performance when compared to individual classifiers. In particular, the MB, MP, and ML combination methods were significantly superior to other combination rules in facial expression recognition problems. In all cases, the combination of different features improved the results, compared to when using only one feature. Individually, the best features were **Ho**G (in 82% of combinations) and LPQ (in 83% of combinations); however, when combining different features, the best selection was **R**AW/**GW/**L**BP/**LPQ/**GEO**. Curiously, the best schemes (R/W/H/P/G/F+MP and R/W/B/P/G+MP; accuracy of 0.992 and 0.828, respectively) combined the best and the worst individual schemes. This increases the diversity of the system, providing the opportunity to select the best representation subspace for each class. Different features were found to specialize in different expressions; for example, the feature space RAW specializes in the neutral class, LPQ specializes in the angry and happy classes, LBP specializes in the disgust class, and HOG specializes in the surprise and sad classes, while the fear class did not have a predominant feature space. We observed that the LPQ, HOG, and RAW methods were present in more than 80% of the selected combinations. This demonstrates that they play a significant role in the classification of some expressions. We presented the most explanatory features for facial expression recognition. Feature relevance techniques appear to be some of the most-used schemes for post hoc explanations [4; 5], as they are capable of providing an explicit description of the inner behavior of the model. There was no combination of features in a model that appeared to be clearly the best, leading to the conclusion that the recognition of a facial expression is the result of a combination of many features. To the best of our knowledge, there has been no other study focused on the explainability of features. In this work, a system based on SRC was presented to provide an explanation, thus determining the most important features for each expression. This information can be useful in designing neural networks or performing fine-tuning in existing ones. CNN-based methods can be stripped from the top classification layer and that vector is, in fact, a feature representation, but we cannot have any knowledge of how these features are calculated and every time the CNN is trained, those features can change. Our method is based on well-known feature extraction methods; therefore, we can establish what features are more revealing. We can take advantage of this knowledge to improve the performance of all the models, including the CNN-based methods. In our future work, we will explore the most focused-upon face regions for each feature vector, in order to determine the most interesting regions used for features and for each expression. \begin{table}
\begin{tabular}{c c c c c} \hline
**Feature** & **CK** & **BU-3 DFE** & **JAFFE** & **Total** \\ \hline R & 0.86 & 0.76 & 0.82 & 0.82 \\ W & 0.72 & 0.86 & 0.59 & 0.73 \\ B & 0.66 & 0.62 & 0.59 & 0.40 \\ H & 0.69 & 0.81 & 1.00 & 0.82 \\ P & 0.83 & 1.00 & 0.68 & 0.83 \\ G & 0.42 & 0.71 & 0.54 & 0.55 \\ V & 0.83 & 0.62 & 0.28 & 0.59 \\ F & 0.79 & 0.48 & 0.68 & 0.75 \\ \hline \end{tabular}
\end{table} 
Table 10: Frequency of features in selected combinations for the CK+, BU-3 DFE, and JAFFE data sets. RAW: (R), GW: (W), LBP: (B), HOG: (H), LPQ: (P), GEO: (G), VGG: (V), VGGF: (F). **Author Contributions:** Conceptualization, J.M.B.-R. and A.J.-i.-C.; methodology, P.D.M.-F., A.J.-i.-C. and T.I.R. ; software, P.D.M.-F.; formal analysis, P.D.M.-F. and J.M.B.-R.; data curation, P.D.M.-F. and J.M.B.-R.; writing--original draft preparation, P.D.M.-F.; writing--review and editing, T.I.R.