## 2 Related Work
### Emotion Feature Extraction and Emotion Recognition Based on Facial Expressions  
Emotion feature extraction and the emotion state recognition method are other main research topics and key technologies of FER. Feature extraction involves extraction and dimension-reduction techniques. They aim to provide more valuable and friendly inputs for classifiers. CNN-based [27] and RNN-based [28] methods, as deep learning (DL)-based methods, can effectively extract features from static and dynamic data, respectively. However, due to issues such as feature dimension explosion and overfitting caused by a lack of sufficient training data (since DL-based methods require more training samples) when using DL-based methods [29], as well as the stronger sensitivity of hand-crafted facial features to target emotions in machine learning (ML)-based methods, ML-based methods perform better than DL-based methods in many scenarios [13]. Most of the popular ML-based facial emotion feature extraction methods extract the peak features of facial expressions from static images; these methods include Gabor wavelet [30], Local Binary Patterns (LBPs) [31; 32], Principal Components Analysis (PCA) [33], Scale-Invariant Feature Transform (SIFT) [34], etc. However, people's emotional expression is a process that occurs over time, and the temporal dynamics of emotional expression also contain richer information [35]. The object used for emotion feature extraction, consequently, should be video data, i.e., a time series of images. Currently, the local binary pattern from three orthogonal planes (LBP-TOP) [36] and its variants have shown good performance in dynamic FER. For example, [PERSON] et al. [37] designed a Facial Micro-Expression Recognition (FMR) framework based on two types of feature extractors (optical strain flow (OSF) and block-based LBP-TOP). Further, [PERSON] et al. [33] introduced  
Figure 1: PAD model. an FER method based on Dynamic Local Ternary Pattern (DLTP) features and a multi-class Kernel Extreme Learning Machine (K-ELM) classifier, which has high classification accuracy. However, due to the complexity of the process or the high dimensionality of the features, the above methods may not be able to meet the real-time detection requirements for tasks. An apparent problem, therefore, is the lack of simple and effective feature extraction methods based on facial dynamic information in ML-based methods because video data have an additional \"time\" dimension and increase the difficulty of feature description and extraction. The most popular models for the classifiers of ML-based FER are Support Vector Machine (SVM) [38; 39], Random Forest (RF) [40], probabilistic neural networks (PNNs) [41], etc. [PERSON] et al. [42] proposed a method using SVM as a classifier, combining PCA with LBPs. [PERSON] et al. [43] designed a new facial expression analysis framework that recognizes Action Units (AUs) from image sequences using a twofold random forest classifier. [PERSON] et al. [44; 45] discussed the application of an improved Active Appearance Model (AAM) based on evolutionary feature extraction in combination with a PNN for the recognition of six facial expressions from static data. Moreover, [PERSON] et al. [46] used 2D Gabor filters to extract features from facial segmentation regions and performed down-sampling to eliminate redundant features, and then K-nearest neighbor (KNN) was used to classify expressions. They all achieved superior performance in several recognized facial expression datasets, such as CK+ and JAFFE. We found that the FER methods based on SVM are the most common, but the methods based on RF often achieve the highest recognition accuracy.