# Facial Expression Recognition Based on Vision Transformer with Hybrid Local Attention
## 4 Experimental Results  
### Implementation Details  
The input batch image data are cropped to a fixed size, 224 \(\times\) 224, and randomly flipped horizontally to avoid overfitting. At the same time, the data enhancement method RandomErasing is used to further improve the performance of the model. In this paper, the basic network IR50 pre-trained on the MS1 MV3 [26] face recognition dataset is used as the feature extraction network. Face features are extracted from the third convolution stage of IR50. The depth of the Transformer encoders is set to 8 in this experiment, and the MLP ratio and drop path are set to 2 and 0.01, respectively. The training batch size is set to 128 and the learning rate is initialized to 4 \(\times\) 10\({}^{-5}\) using the ExponentialLR learning rate decay. The entire network is optimized using the Adam optimizer. Using the Adam optimizer allows the network to converge faster and makes tuning the parameters more convenient. Use the standard label smoothing cross-entropy loss to supervise the model, which has better generalization for expression recognition. The method is implemented using the Pytorch 1.11.0 toolbox, with the initial momentum coefficient of the network set to 0.9 and trained on a single NVIDIA RTX 3090 GPU (NVIDIA, Santa Clara, CA, USA).  
The experiments are evaluated on three different commonly used expression datasets in a natural scene. The RAF-DB dataset contains samples of neutral expressions and six basic expression categories labeled with 12,271 images for training and 3068 images for testing. The FERPlus database contains 28,709 training images and 3555 test images, and, in addition to the neutral expressions and six basic expressions, the labels also contain the contempt category. The AffectNet dataset includes six basic expressions and neutral expressions. The training and test sets contain 283,901 and 3500 images, respectively.  
The experimental analysis on the RAF-DB dataset of the depth of the Transformer encoder used in HLA-ViT is shown in Figure 3. The experiment shows that the best results are achieved when using ViT with a depth of 8.  
In order to explore the effect of the two hyperparameters \(\lambda_{1}\) and \(\lambda_{2}\) in the loss function on the HLA-ViT performance, the different recognition results on the RAF-DB dataset with different values from 0.1 to 0.9 are experimentally investigated. As shown in Table 1, when the two branches are fused at the decision level with a local loss weight of 0.4 and a global loss weight of 0.6, the facial expression recognition accuracy can reach 90.45%. This indicates that the importance of the ViT stream is slightly higher than that of the local attention stream. Subsequent ablation experiments will confirm that using only ViT or HLA for facial expression recognition does not achieve the same level of accuracy as when the two are fused. This indicates that the proposed method of fusing hybrid local features and global features can further enhance information complementarity and improve the recognition performance of the network.  
The results in Table 1 show that the importance of the ViT stream is slightly higher than that of the local attention stream for \(\lambda_{1}=0.4\) and \(\lambda_{2}=0.6\), when a good recognition performance is obtained.