## 1 Introduction
Thus far, various attention-based models have been proposed to learn emotion-related information about music by letting models focus on the important parts of features to achieve better performance [24, 25, 26, 27, 28]. Another challenge for traditional neural networks is the potential loss of key information and performance degradation as the network depth increases; therefore, residual learning is introduced to prevent the gradient from disappearing through skip connections [29] and has also been employed in emotion recognition tasks [30,31]. Motivated by existing works, we propose an attention-based spatial-temporal feature extraction approach for music emotion classification and have called it the FFA-BiGRU model. In the proposed method, we first input the log Mel-spectrogram of music clips into the FFA (feature fusion attention) [32] module to extract high-level spatial features of music audios. The FFA module consists of three group architecture blocks, each of which is a stack of channel-spatial attention-based convolutional residual blocks. The three group architecture blocks can fully extract the multi-scale spatial features of music clips, and the channel-spatial attention mechanism can pay more attention to critical information for sentiment classification from both the channel and spatial aspects. Finally, the output features of three group architecture blocks are fused together through channel-spatial attention. In order to further capture the sequence characteristics of music audio, the bidirectional gated recurrent units (BiGRU) module is employed to learn the temporal features after the FFA module. Then, the output feature maps of FFA and those of the BiGRU are concatenated in the channel direction. Finally, the concatenated features are passed through fully connected layers to predict the emotion classification results. In summary, the main contributions of this paper are as follows:  
1. We propose an end-to-end spatial-temporal feature extraction method for MER called FFA-BiGRU. The proposed model fully considers the spatial and temporal propertiesof the log Mel-spectrogram features of music audios and extracts rich emotion-related spatial-temporal features through the combination of the FFA and BiGRU modules. The experimental results show that the integration of FFA and BiGRU is effective and can achieve a better classification performance than the existing baselines. 2. In the proposed FFA-BiGRU model, we extract multi-scale spatial features through three group architecture blocks, and each of them is a stack of multiple channel-spatial attention-based residual blocks. The channel-spatial attention mechanism can effectively highlight features that are critical to music emotion classification at both the channel and spatial levels. Moreover, the concatenation of spatial features from FFA and the temporal features from the BiGRU can fully retain the spatial and temporal features of music, which can discriminate emotions well in the final emotion space. 3. Finally, we conduct sufficient comparison experiments and an ablation study on the EMOPIA dataset [33], and it shows the effectiveness of the network architecture as well as each component of the approach, including the effectiveness of the channel-spatial attention mechanism, the optimal number of group architecture blocks, and the optimal number of network layers in these blocks. The remainder of this paper is organized as follows. Related works of this paper are given in Section 2. Section 3 illustrates the architectural details of the proposed model. Extensive experiments are conducted to demonstrate the effectiveness and efficiency of our method in Section 4. Finally, Section 5 concludes this paper and discusses several future research directions.