# Occlusion-Robust Facial Expression Recognition Based on Multi-Angle Feature Extraction
## 3 Method
### Multi-Feature Extraction Module
#### 3.2.2 Swin-E  
Researchers typically refer to image recognition as coarse-grained image recognition, such as the recognition of categories, such as distinguishing between cats and dogs. However, within image recognition, there are more specific tasks, called fine-grained image recognition. The goal of fine-grained image recognition is to distinguish between sub-classes within the same category, such as different kitten breeds. This idea can be applied to FER. For example, Figure 4 shows the JAFFE dataset, wherein expressions like happiness, sadness, and fear are labeled as happiness-person, sadness-person, and fear-person. FER is different from other recognition tasks because expression features are more subtly embedded in facial features. To extract finer details, we refer to methods commonly used in fine-grained image recognition.  
Researchers have used the Swin Transformer to develop network models for fine-grained image recognition [33; 34; 35]. This approach has demonstrated that the Swin Transformer, with its shifted window attention mechanism, can effectively extract fine-grained features from images. The shifted window mechanism allows the model to capture detailed local features while maintaining the spatial relationship between different regions, making it particularly well suited for tasks like FER. Consequently, the fine-grained feature extraction branch in our model utilizes the Swin Transformer's Encoder module.  
Figure 3: Cropping process. The input image is processed through Dlib, wherein five feature points are identified and marked with red dots, as shown in Figure 3. Thereafter, the image is cropped.  
The input image is designated as \(X\), and the cropped image is designated as \(X_{\text{Lb}}\), \(X_{\text{Lt}}\), \(X_{\text{Rb}}\), \(X_{\text{Rt}}\). PTIR-50 is designated as \(P(\cdot;\theta_{p})\), and SWIN-E is designated as \(S(\cdot;\theta_{s})\). The features \(F\). and self-attention \(Attn\)., which were acquired after PTIR-50 and SWIN-E for \(X\) and the region blocks \(X_{\text{Lb}}\), \(X_{\text{Lt}}\), \(X_{\text{Rb}}\), \(X_{\text{Rt}}\), are designated as follows:  
\[\begin{array}{c}[F_{X},Attn_{X}]=P(X;\theta_{p}),F_{Swin}=S(X;\theta_{S})\\ [F_{\text{Lb}},Attn_{Lb}]=P(X_{\text{Lb}};\theta_{p}),[F_{\text{Lt}},Attn_{Lt}] =P(X_{\text{Lt}};\theta_{p})\\ [F_{\text{Rb}},Attn_{Rb}]=P(X_{\text{Rb}};\theta_{p}),[F_{\text{Rt}},Attn_{Rt}] =P(X_{\text{Rt}};\theta_{p})\end{array} \tag{1}\]  
where \(\theta_{p}\) is a parameter of PTIR-50 and \(\theta_{S}\) is a parameter of SWIN-E.  
The feature set \(F\) and the self-attention set \(A\) are obtained after sorting:  
\[F=[F_{X},F_{\text{Lb}},F_{\text{Lt}},F_{\text{Rb}},F_{\text{Rt}}] \tag{2}\]  
\[A=[Attn_{X},Attn_{Lb},Attn_{Lt},Attn_{Rb},Attn_{Rt}] \tag{3}\]  
As illustrated in Figure 5, a variety of features were extracted from the distinct region blocks.  
Figure 4: Example of fine-grained image recognition—JAFFE expression recognition dataset. The JAFFE dataset was released in 1998 and comprises images of human facial expressions obtained from 10 Japanese women who were instructed to make a range of expressions in an experimental setting and were then photographed with a camera. The entire database comprises 213 images captured from 10 individuals, all of whom were female. Each subject displayed seven expressions: sad, happy, angry, disgusted, surprised, afraid, and neutral. Approximately 20 images were selected from each expression group.  
Figure 5: Visualization of the regional features. As shown in the figure, different regions focus on distinct aspects, leading to variations in the extracted features.