# Attention-Based Spatiotemporal-Aware Network for Fine-Grained Visual Recognition
## 3 Our Method
### Network  
ASTNet consists of three parts: ResNet for extracting spatial features of the micro-expression image sequence, an attention mechanism module for extracting attention maps and ConvLSTM for processing temporal features and recognizing emotions. ResNet can train very deep neural networks and extract deeper, fine-grained spatial features of micro-expression image sequences. The attention mechanism calculates weighted features through attention scores, allowing for a better focus on key features during the micro-expression change process. Since we use micro-expression video sequences that vary over time, temporal modeling is necessary. ConvLSTM excels in capturing spatiotemporal relationships and can simultaneously model spatial relationships to recognize facial emotions. In the following sections, we will detail each component of the model. 1. ResNet The first part of our network uses the transfer of learning; we initialize the parameters of ResNet by using ImageNet, and further train on the macro-expression dataset: CK+. Finally, we extract the spatial features of the optical flow image by using the processed ResNet. ResNet was proposed by [PERSON], [PERSON] et al. (2017). They added a residual structure in their network to solve the degradation problem of the poor stacking effect when they deepened the depth of the network to a certain extent. ResNet also solved the problem of gradient disappearance or explosion of the deep network through the identity mapping method. Figure 6 shows the residual block. We remove the full connection layer of ResNet and directly output the convoluted and pooled graph of features as the input of the next part of our network. 1. Attention Mechanism Each region or visual feature on micro-expression images has different importance for the recognition of the network. Therefore, in order to establish the correlation of the pixels  
Figure 5: Optical flow visualization. Figure 6: Residual block structure of ResNet. of a micro-expression image in the spatial and channel dimensions, we use CBAM [28] to construct the attention mechanism. CBAM is a feedforward convolutional neural network attention module that can be integrated into CNN. CBAM has two sequential submodules: channel and space. The feature map of the network convolution block can be adaptively refined through CBAM. Figure 7 shows CBAM's structure. Given a feature graph \(\mathrm{F}\in\mathrm{C}\times\mathrm{H}\times\mathrm{W}\) as input, CBAM is successively transformed into a one-dimensional channel attention map \(\mathrm{M}_{\mathrm{c}}\in\mathrm{R}^{\mathrm{C}\times 1\times 1}\) and a two-dimensional spatial attention map \(\mathrm{M}_{\mathrm{s}}\in\mathrm{R}^{1\times H\times W}\). The channel attention module uses the channel relationship between features to generate the channel attention map. Firstly, the channel attention module uses the average pooling and maximum pooling to aggregate the spatial information of the feature graph, and generate average pooling \(\mathrm{F}^{\mathrm{c}}_{\mathrm{avg}}\) and maximum pooling \(\mathrm{F}^{\mathrm{c}}_{\mathrm{max}}\). Then, it transmits \(\mathrm{F}^{\mathrm{c}}_{\mathrm{avg}}\) and \(\mathrm{F}^{\mathrm{c}}_{\mathrm{max}}\) to a shared network layer composed of multi-layer perceptrons to generate the channel attention graph \(\mathrm{M}_{\mathrm{c}}\in\mathrm{R}^{\mathrm{C}\times 1\times 1}\). Finally, the feature vectors are merged. The channel attention model structure is shown in Figure 8.  
The spatial attention module uses the spatial relationship between features to generate a spatial attention map. Firstly, the average pooling and maximum pooling operations are used to aggregate the channel information of the characteristic graph along the channel direction to generate two characteristic graphs: \(\mathrm{F}^{\mathrm{s}}_{\mathrm{avg}}\in\mathrm{R}^{1\times\mathrm{H}\times \mathrm{W}}\) and \(\mathrm{F}^{\mathrm{s}}_{\mathrm{max}}\in\mathrm{R}^{1\times\mathrm{H}\times \mathrm{W}}\).