# Attention-Based Spatiotemporal-Aware Network for Fine-Grained Visual Recognition
## 3 Our Method
### Processing  
(1) Micro-expression motion amplification  
Micro-expressions change subtly, which leads to low characteristic intensity and low category discrimination. Therefore, we use the motion amplification technology to properly amplify these subtle dynamic changes and improve the discrimination of emotions. Some methods of video action amplification can be used to complete this part of the work. Their working principles are as follows: First they use Euler Video Amplification [22] to decompose the input continuous segments into different spatial frequency bands. Then, they use the same time filtering for processing and give an amplification factor to amplify the signal of interest. Finally, they superimpose it on the original signal as output; the mathematical description is as follows: Let \(I(x,t)\) be the image intensity of the image at time t and position x, and use the displacement function to express the observed intensity \(\delta(t)\), that is, \(I(x,t)=f(x+\delta(t)))\) and \(I(x,0)=f(x)\). After amplifying the image signal \(\alpha\) times, it is superimposed back to the original image to obtain the synthesized image signal \(I^{\prime}(x,t)\), as shown in Equation (1):  
\[I^{\prime}(x,t)=f(x+(I+\alpha)\delta(t)) \tag{1}\]  
Figure 2: Framework of our network. This paper focuses on the amplification of facial micro-expression movements, setting the amplification factor \(z\) to 2, 3 and 4 for comparison. Figure 3 shows the amplification effects of three types of emotions in the CASME II dataset, with each set of experiments using the same frame comparison. In the emotion category labeled as disgust, as the amplification factor increases, the muscle movements in the eyebrow area of the subjects become more noticeable, and the activation level of the frowning action increases. In the happiness category, the muscle movements in the mouth area of the subjects become more noticeable, and the activation level of the mouth corner lifting action increases. In the surprise category, the muscle movements in the eyebrow and mouth areas are generally amplified to varying degrees compared to other muscle regions.Additionally, it can be observed that in the happiness category, some subjects exhibit eye closure, while in the surprise category, the subjects' eyeballs are amplified. This is because the network enhances eye movements such as pupil changes and blinking through temporal information from consecutive frames. However, these eye movements do not conform to the definition of micro-expressions and do not contribute to their recognition.From the amplification results of the three emotions, although image noise increases when the amplification factor is set to 3, the amplification effect on motion is generally more noticeable than when the factor is set to 2. When the amplification factor is set to 4, facial muscles start to distort excessively, and image noise increases significantly. Considering these factors, this paper adopts an amplification factor of 3 for the network to amplify the actions of micro-expression samples. We magnify images of micro-expression in SMIC and CASME II by 3 times through the Motion Amplification Network [23]. Figure 4 shows the comparison between the images before and after processing. Figure 3: Comparison of before and after micro-expression magnification. 2. Optical flow feature extraction  
In micro-expression recognition, feature extraction is a critical issue. Micro-expressions are brief, involuntary facial expressions, and directly using RGB images of micro-expressions as network input to extract deep features cannot effectively recognize emotions [24]. Therefore, extracting effective features is essential [25]. In this paper, we employ an optical flow-based feature extraction method due to the superior performance of optical flow in handling dynamic information and subtle changes, which is crucial for micro-expression recognition.