# A Multi-Scale Feature Fusion Hybrid Convolution Attention Model for Birdsong Recognition
## 2 Research Method
### Multi-Scale Feature Fusion Hybrid Convolutional Attention Model(MUSCA)
#### 2.2.1 Hybrid Residual Convolutional Attention Block  
Traditional convolution blocks involve convolution operations across channels and spatial dimensions, which limits the capture ability of single-channel features and hinders the extraction of spatial dimension details. To address this issue, this paper proposes a hybrid convolutional attention block. This structure combines depthwise separable convolutions with traditional convolutional blocks to form a hybrid convolutional block. It incorporates channel attention and spatial attention mechanisms to create a hybrid convolutional attention block aimed at extracting finer-grained features. The hybrid convolutional attention block is illustrated in Figure 3, and its specific structure is described in Equation (4). This hybrid convolutional attention block is primarily used for feature extraction and down-sampling operations. If the stride is set to 1, down-sampling is not performed, and the input and output are directly added pointwise. On the contrary, the down-sampling is performed, and the convolution block scales the feature map before adding the feature map to realize the residual connection. The residual operation allows the model to retain more original feature information in the subsequent layers, preventing the gradient vanishing  
Figure 3: Hybrid convolutional attention block structure diagram. Conv2d 1*1 Stride 2 denotes a convolution kernel of length 1 and width 1, Stride is 2.  
problem caused by excessive model depth and enhancing the model's learning efficiency and convergence speed. \[\mathbf{Z}=Conv2d_{3\times 3}(Conv2d_{3\times 3}(\mathbf{X}))+\text{DepthwiseSeparableConv}(\mathbf{X}) \tag{4}\] \[\mathbf{O} =\text{SpatialAttention}(\text{Channel Attention}(\mathbf{Z}))\] \[ResidualOutput = \begin{cases}\mathbf{X}+\mathbf{O},&\text{if $X$ is not down-sampled}\\ \text{conv}_{1\times 1,\text{stride}=2}(\mathbf{X})+\mathbf{O},&\text{if $X$ is down-sampled}\end{cases} \]  
Among them, \(\mathbf{X}\) is the input feature, \(\mathbf{O}\) is the output of spatial attention and self-attention, and \(\mathbf{Z}\) is the mixed convolution result. **Hybrid Convolution:** The mixed convolution block, as depicted in Figure 4, amalgamates depthwise separable convolution with traditional convolution blocks. Depthwise separable convolution autonomously executes spatial convolution operations on each input channel, enabling it to capture feature information from each channel individually. Meanwhile, the convolutional block comprising two traditional convolutions conducts inter-channel and spatial convolution operations on the input to capture feature information across channels. The features derived from depthwise separable convolutions and ordinary convolutional blocks are fused through element-wise addition. **Channel Attention Mechanism:** Figure 5 introduces the structure of channel attention mechanism, which enables the model to learn to assign different weights according to the importance of each channel. The channel attention mechanism performs global average pooling on input features, fully connects and activates the global average pooling, and uses Sigmoid to obtain weighted weights for each channel. The input features are then weighted according to the input channel dimension to achieve channel attention. Figure 4: Hybrid residual convolution block structure diagram. The front of all \({}^{\star}\) in the figure represents the length of the convolution kernel, and the back of \({}^{\star}\) represents the width of the convolution kernel.