# Attention-Based Spatiotemporal-Aware Network for Fine-Grained Visual Recognition
## 4 Experiment and Analysis
### Hyperparameter Optimization  
First, we compare and experiment with different learning rate settings. Due to the weak intensity of micro-expression features, the initial learning rate of the network should be relatively low. In this experiment, we set the learning rates to \(1\times 10^{-3}\), \(1\times 10^{-4}\) and \(1\times 10^{-5}\) and tested them on the CASME II dataset. When the learning rate was \(1\times 10^{-3}\), the network training experienced oscillations, and the accuracy could not improve steadily. This was because the excessively high learning rate prevented the network from reaching local minima. When the learning rate was \(1\times 10^{-5}\), the network's fitting speed was too  
\begin{table}
\begin{tabular}{l c c c c} \hline \multirow{2}{*}{**Methods**} & \multicolumn{2}{c}{**SMIC**} & \multicolumn{2}{c}{**CASME II**} \\ \cline{2-5}  & **ACC (\%)** & **F1-Score** & **ACC (\%)** & **F1-Score** \\ \hline LBP-TOP [35] & 43.38 & 0.34 & 46.46 & 0.42 \\ Bi-WOOF+Phase [36] & 68.29 & 0.67 & 62.55 & 0.65 \\ FDM [11] & 54.88 & 0.54 & 45.93 & 0.40 \\ Sparse MDMO [37] & 70.51 & 0.70 & 66.95 & 0.69 \\ DSSN [38] & 63.41 & 0.65 & 70.78 & 0.73 \\ GEME [39] & 65.24 & 0.61 & 75.02 & 0.73 \\ OFF-Apex Net [40] & 67.68 & 0.67 & 68.94 & 0.70 \\
**ASTNet** & **73.57** & **0.72** & **75.61** & **0.74** \\ \hline \end{tabular} Bold text indicates the best results.  
\end{table}
Table 5: Comparison of experimental results.  
slow, resulting in a waste of time. Therefore, considering these factors, the learning rate was set to \(1\times 10^{-4}\) for the experiment.  
When training the network, the initial weights need to be initialized according to a certain distribution to ensure faster convergence of the loss function during training, thereby achieving better optimization results. However, when initializing network weights randomly according to a certain distribution, inappropriate initial weights might cause the loss function to get stuck in local minima during training, preventing it from reaching a global optimum. Momentum can partially solve this problem. The larger the momentum, the more energy is converted into potential energy, making it more likely to escape the confines of local concave regions and enter global concave regions. The most common setting for momentum is 0.9.  
Other parameter settings are as follows: the optimizer used is Stochastic Gradient Descent (SGD), and lr_scheduler is used to automatically update the learning rate. We compared batch sizes of 16, 32, 64 and 128, and dropout rates of 0.3, 0.4, 0.5, 0.6 and 0.7.  
Figures 11 and 12 show the impact of Batch Size and Dropout on network recognition, respectively. When the batch is selected as 32 and the dropout parameter is set to 0.5, the recognition of our network is the best. Dropout prevents overfitting by discarding some features and  
The confusion matrices on SMIC and CASME II are shown in Figure 13. It can be seen from Figure 13 that among the three emotions identified in the SMIC, 'negative' has the highest accuracy, followed by'surprise', and 'positive' has the lowest accuracy. The possible reason is that the number of 'negative' samples is the largest and it has high discrimination from other emotions. Among the five emotions identified in the CASME II, the accuracy of 'others' is the highest, while the accuracy of'repression' is the lowest. The reason is that the number of 'others' is the largest,'repression' has a small number of samples and the class differentiation between other emotions is small. While the number of samples is also small, the accuracies of 'disgust' and'surprise' are better than that of'repression', which is because the former two are highly distinguished from other emotions. It can be seen that the number of samples and the intensity of features can have a great impact on the recognition.