# Swin-FER: Swin Transformer for Facial Expression Recognition
## 4 Experiment Results and Analysis
### Experimental Results  
Applying a transformer to the FER2013 dataset for facial expression recognition is a challenging task due to the small original image size of 48 \(\times\) 48, while most transformer models are suitable for sizes of 224 and 356. The Swin transformer requires a large image size for operations such as patch embedding, with a step size of 4. Therefore, to obtain the transformer's benchmark, the image size of the facial expression pictures had to be enlarged to 224 \(\times\) 224. We tested four lightweight transformer structures, with specific experimental results presented in Table 4. With almost no change in the parameter count, the recognition accuracy of our method reached 71.11% on FER2013--an increase of 0.41% compared to the original Swin transformer--and 100% on CK+--a 3.75% improvement with respect to the original network. To expand the practicality of transformers, there are usually different types of the same transformer architecture, such as tiny, small, base, and large. Increasing the model in transformer methods typically improves the accuracy, but there is also a corresponding exponential increase in parameters between different models. The transformer models selected in Table 4 are all lightweight structures, such as small and tiny-level structures, emphasizing the use of methods like mean, adaptive pooling, and group convolution to control the number of parameters [19]. As shown in Table 5, while applying a fusion strategy to better extract features, the total parameters of the proposed Swin-Fer network increased by only 70 B compared to the original Swin transformer. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Method** & **Fer2013** & **CK+** \\ \hline EdgeViT & 0.5935 & 0.725 \\ EfficientFormer & 0.6053 & 0.7125 \\ Vision transformer & 0.6848 & 0.85 \\ Swin transformer & 0.7070 & 0.9625 \\
**Swin-Fer** & **0.7111** & **1** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Accuracy of transformer methods on the FER2013 and CK+ datasets. \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameter Name** & **Parameters** \\ \hline Batch size & 80 \\ Learning rate & 0.0001 \\ Optimizer & Adam \\ Seed & 1024 \\ Image size & 224 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Experimental parameters for Swin-Fer. \begin{table}
\begin{tabular}{c c} \hline \hline
**Network** & **Number of Parameters** \\ \hline EdgeViT & 11.30 M \\ EfficientFormer & 19.8 M \\ Vision transformer & 21.42 M \\ Swin transformer & 27.53 M (27530120 B) \\ Swin-Fer & 27.53 M (27530190 B) \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Comparison of network parameter counts for transformer methods. The experimental results for Swin-Fer and the original Swin transformer on the FER2013 dataset are shown in Figure 7. From Figure 7(a), it is evident that due to the use of pre-training methods, both models quickly showed a trend of convergence, stabilizing around 40 iterations. For the facial expression recognition problem, which is indeed a multi-class classification problem, we adopted the One-vs.-Rest strategy to generate the ROC curves. As shown in Figure 7(b), the area under the ROC curve for both models is 0.84, with the curves deviating obviously from the 45-degree diagonal line, indicating good sensitivity and specificity. However, as can be seen in Figure 7(c), it is apparent that the Swin-Fer method has relatively smaller fluctuations in accuracy, exhibiting more stability. Finally, from the confusion matrices presented in Figure 7(d), the accuracy changes in various classifications between Swin-Fer and the original Swin transformer are as follows: anger (+0.2%), disgust (+3.6%), fear (+0.7%), happiness (\(-1.7\%\)), neutral (+1.4%), sadness (+1%), and surprise (+1.9%), with an increase in accuracy for six out of seven classifications, demonstrating the effectiveness of the proposed improvement algorithm. Figure 7: Experimental results on the FER2013 dataset. Figure 8 presents the experimental results for Swin-Fer and the original Swin transformer network on the CK+ dataset.