# Facial Micro-Expression Recognition Based on Deep Local-Holistic Network
## 2 Related Works and Background
### Micro-Expression Recognition  
In the early stages of the study, most methods adopt handcrafted features to identify micro-expressions. [PERSON] et al. [18] divided the face into specific regions and recognized the motion in each region using a 3D-Gradients orientation histogram descriptor. [PERSON] et al. [19] designed the first spontaneous micro-expression database (SMIC) and used LBP-TOP [20] to extract dynamic and appearance features of micro-expressions. [PERSON] et al. [21] adopted robust Principal Component Analysis (RPCA) [22] to extract sparse micro-expression information and Local Spatiotemporal Directional Features. [PERSON] et al. introduced a discriminant tensor subspace analysis (DTSA) [23] to preserve the spatial structure information of micro-expression images. Furthermore, they treated a micro-expression video clip as a fourth-order tensor and transformed the color information from RGB into TICS to improve the performance [24]. [PERSON] et al. [25] show a spatiotemporal facial representation to characterize facial movements and used LBP to extract appearance and motion features. [PERSON] et al. [26] proposed a simple, effective Main Directional Mean Optical-flow features (MDMO) and adopted an SVM classifier to recognize micro-expression. [PERSON] et al. [27] analyzed micro-expression by proposing SpatioTemporal Completed Local Quantization Patterns (STCLQP), which exploits magnitude and orientation as complementary features. The above recognition methods are not capable enough to capture subtle facial displacements. This is due to the constant movement of the observed individual, which is common in typical micro-expression applications. Addressing this problem, [PERSON] et al. [28] proposed a Facial Dynamics Map method by depicting micro-expression characteristics in different granularities. [PERSON] et al. [29] proposed a Main Directional Maximal Difference micro-expression recognition method (MDMD), extracting optical flow features from the region of interest (ROIs) based on action units. Furthermore, addressing the SSS problem, [PERSON] et al. [30] performed data augmentation based on their proposed local temporal pattern for micro-expression analysis.  
Recently, the outstanding performance of deep learning has attracted the attention of many researchers to develop micro-expression recognition algorithms. [PERSON] et al. [31] used the pre-trained ImageNet-VGG-f CNN to extract features of each frame in micro-expression  
Figure 2: Global (green clipping head) and local area of interest (yellow arrow) tracking of micro-expression action. (Sample from SAMM dataset [16]).  
video clips. [PERSON] et al. [32] proposed a Transferring Long-term Convolutional Neural Network (TLCNN) method, which uses Deep CNN to extract spatial features per frame and Long Short Term Memory (LSTM) to learn micro-expression temporal information. [PERSON] et al. [33] investigated a low-complexity recurrent convolutional neural network (RCN) based on cross-database micro-expression recognition. [PERSON] et al. [34] performed a joint local and global information learning on apex frame for micro-expression recognition. [PERSON] et al. [35] proposed an expression-specific feature learning and fusion method for micro-expression recognition However, the small sample size of micro-expression samples and the subtle and brief nature of micro-expression limit the combination of deep learning with micro-expression recognition methods. Thus, how to learn the micro-expression features effectively is necessary research for further performance improvement.