# Attention-Based Spatiotemporal-Aware Network for Fine-Grained Visual Recognition
## 1 Introduction
We employ a deep learning-based video motion magnification network to amplify micro-expression features and use a convolutional neural network (CNN) to estimate optical flow, extracting the optical flow information between adjacent frames in the image sequence. Existing networks may overlook the correlation of image information in channel and spatial dimensions. To address this issue, we combine ResNet and ConvLSTM to extract spatiotemporal information, adding attention mechanisms between them. Our model effectively focuses on the micro-expression information across multiple dimensions, achieving superior recognition performance. Micro-expression datasets are typically small and imbalanced, which can hinder the training of deep networks and lead to overfitting. To address these challenges, we employ data augmentation techniques, such as random rotation, to expand the effective dataset. We exclude classes with very few samples to improve training efficiency. The ResNet parameters are initialized with ImageNet pre-trained weights, and the model is further trained on the macro-expression dataset CK+. Additionally, we incorporate dropout layers into our network to selectively ignore some neurons and reduce their interactions in each training batch, helping to prevent overfitting. Figure 1 shows our research framework. The input to the model consists of video sequences containing micro-expressions. First, facial detection and alignment are performed on each frame to ensure consistent facial feature points across all frames. Next, an action  
magnification network is used to amplify micro-expression features, making them easier to capture. Subsequently, a convolutional neural network (CNN) processes the optical flow information between adjacent frames to capture subtle facial muscle movements. Specifically, the optical flow describes the motion patterns between two consecutive frames. Following this, ResNet is utilized to extract spatial features, and ConvLSTM is employed to capture temporal variations. By combining these two methods, richer feature representations are obtained. Additionally, both channel attention mechanisms and spatial attention mechanisms are adopted to achieve multidimensional focus on micro-expressions, enhancing the model's ability to perceive important features. Finally, a fully connected layer maps the extracted features to the classification space, and a classifier is used for categorization. Overall, our contributions can be mainly summarized as follows:  
* Instead of using traditional RGB images, we leverage video motion to recognize micro-expressions, which allows for better capture of expression dynamics. * We combine spatiotemporal features with static features and introduce two types of attention mechanisms--channel attention and spatial attention--to model micro-expressions in a fine-grained manner from multiple dimensions. * To address the challenge of limited micro-expression datasets, we employ effective data augmentation techniques such as rotation and flipping, improving the training process. * We conduct extensive experimental validation, demonstrating that ASTNet significantly outperforms nine state-of-the-art open set recognition methods. The structure of other parts of this paper is as follows: The second chapter briefly describes the existing feature extraction methods based on traditional or deep learning. The third chapter describes our processing method and proposes our network. The fourth chapter describes the experimental process, use parameters, results of experiments and the analysis of module effectiveness in detail. The last chapter summarizes our main work and contributions and looks forward to the possible direction of follow-up work.