# Attention-Based Spatiotemporal-Aware Network for Fine-Grained Visual Recognition
## 4 Experiment and Analysis
### Ablation and Analysis  
To validate the effectiveness of each module, we conducted ablation experiments, testing the effectiveness of the action amplification module, the spatial attention mechanism and the channel attention mechanism. We generated four variants: the ResNet-ConvLSTM network without the action amplification module and both attention mechanisms. ResNet-ConvLSTM with the action amplification module but without both attention mechanisms. The ResNet-SpatialAttention-ConvLSTM network without the channel attention module. The ResNet-ChannelAttention-ConvLSTM network without the spatial attention module. We compared the results of these variants with our ASTNet. Table 6 illustrates the effectiveness of micro-expression amplification and compares the improvement of different attention modules on the network. It can be seen from Table 6 that although action amplification increases the noise of the micro-expression image and causes a certain degree of image distortion, it has a significant amplification effect on micro-expression features. On the whole, its recognition effect on the micro-expression has been improved. ResNet+ConvLSTM has more degradation without the attention mechanism, which indicates that the importance of pixels in different regions and different channels of the micro-expression image is different and verifies the effectiveness of our attention mechanism. Compared with spatial attention and channel attention, CBAM improves the network recognition effect more significantly. At the same time, it can be seen that the spatial attention mechanism improves the network recognition effect less than the channel attention mechanism. This is because the convolution of ConvLSTM is used for conversion between input and state and state and state, which can better capture the spatial information of images than LSTM. Therefore, it can be considered that there is an overlap between the spatial attention mechanism and the convolution of ConvLSTM. We also try to add CBAM to different locations within ResNet to explore the impact of location and structure on the CBAM module computing attention map and improve the effect of the network model. As shown in Table 7, in our proposed network model, ResNet is composed of five middle layers. We integrate CBAM modules between each two middle  
Figure 13: Confusion matrix. layers to form four different networks. In Table 8, we show the effect of these networks on recognizing micro-expressions and analyze the possible reasons. The results show that they improve the accuracy of recognition less than adding CBAM between ResNet and ConvLSTM. The results show that although integrating the attention mechanism into different locations within the ResNet will improve the recognition rate of our network, their improvement is less than that of integrating CBAM between ResNet and ConvLSTM. It is difficult to find the law of the effect of the attention mechanism integrated into the network with the change in location, but they will affect the effect of transfer learning to a certain extent, so the overall improvement effect is not as good as the effect of integrating attention outside the network. \begin{table}
\begin{tabular}{c c c} \hline \hline \multirow{2}{*}{**Methods**} & **SMIC** & **CASME II** \\ \cline{2-3}  & **ACC (\%)** & **ACC (\%)** \\ \hline \({}^{1}\) ResNet-ConvLSTM & 64.21 & 66.83 \\ \({}^{2}\) ResNet-ConvLSTM & 69.59 & 71.54 \\ \({}^{2}\) ResNet-Spatial Attention-ConvLSTM & 71.83 & 73.96 \\ \({}^{2}\) ResNet-Channel Attention-ConvLSTM & 72.27 & 74.35 \\ \({}^{2}\) ASTNet & **73.57** & **75.61** \\ \hline \({}^{1}\) indicates no action amplification. \({}^{2}\) indicates action amplification. Bold text indicates the best experimental results. & \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Effectiveness analysis of amplification and attention models.