# Facial Expression Recognition Based on Vision Transformer with Hybrid Local Attention
## 3 Methodology
### Hybrid Local Attention Module  
The hybrid local attention module proposed in this paper consists of a combination of spatial attention and channel attention. When the dimensionality of the input features is H \(\times\) W \(\times\) C, the new feature dimensions obtained remain the same after passing the hybrid attention module. The previous method us used to segment faces into blocks through facial landmarks or random cropping to obtain effective local features, thereby eliminating interference from facial occlusion and pose variations. However, these methods may lead to misalignment or uncertainty in facial expression recognition. In this paper, the feature mapping based on spatial domain and channel domain information is combined, and the feature mapping is passed through the hybrid attention module, so as to obtain the feature mapping with enhanced important local features. The network first learns what features are key features through the channel attention module, and then uses the spatial attention module to learn where the key features are, enhancing the acquisition of discriminative features in images. The hybrid local attention module is shown in Figure 2.  
The spatial attention module provides varying degrees of attention to different regions of the input image or feature image, increasing the influence of the target region and reducing the influence of irrelevant regions. The spatial attention in CBAM proposed by [PERSON] [19] is to merge two H \(\times\) W \(\times\) 1 feature maps obtained by maximum pooling and average pooling to obtain an H \(\times\) W \(\times\) 2 feature map. Then, a spatial attention weight matrix of H \(\times\) W \(\times\) 1 is output by convolution and sigmoid function. In this paper, the proposed spatial attention module replaces the maximum pooling and average pooling operations with a two-layer 1 \(\times\) 1 convolutional layer with ReLU and sigmoid activation functions. Let \(X\in R^{H\times W\times C}\) denote the input feature maps, where \(H\), W, and \(C\) represent the height, width, and number of feature maps, respectively. After the first 1 \(\times\) 1 convolutional layer, output \(C/r\) feature maps, where r represents the reduction rate of the feature maps. The ReLU layer is then used to enhance the nonlinear relationship between the layers of the network. After the second 1 \(\times\) 1 convolution layer, the number of feature maps is reduced to 1, and the attention maps are generated by the Sigmoid function. Thus, through two successive 1 \(\times\) 1 convolution layers, the model automatically locates the important facial parts. Meanwhile, in order to further enhance spatial attention module, a local attention network, which is a two-branch spatial attention module, is proposed. It is similar to the operation in CBAM, generating two spatial attention maps \(X_{1}\) and \(X_{2}\), but, instead of combining them for the convolution operation, the final attention map \(X_{out}\) is generated by taking the maximum value of each attention map element in the two spatial attention maps. This can guide the model to focus on feature regions that are more useful for facial expression recognition. The specific formula is shown as follows:  
\[X_{out}(w,h)=MAX\{X_{1}(w,h),X_{2}(w,h)\}\ 1\leq w\leq W,\ 1\leq h\leq H \tag{1}\]  
In Equation (1), \(H\) and \(W\) represent the height and width of the feature map, respectively. Finally, \(X_{out}\) is multiplied with the original feature map \(X\) element by element, so that the unimportant regions in the original feature map are suppressed, and the role of important regions in expression recognition is highlighted. The specific structure of the local attention network is an improvement module of spatial attention.