## 2 The Proposed Method
### Residual Time-Frequency Mixed Attention Module Network
#### 2.2.1 Residual Time-Frequency Mixed Attention Module  
The attention mechanism was first proposed by [PERSON] et al. [44] based on the observation rules of the visual system. In essence, it is a mechanism for allocating resources to the object of attention, that is, allocating resources according to the importance of the object. The critical parts need to be allocated more than the other parts. In deep learning, the resources allocated by the attention mechanism are reflected in weight. The information related to the recognition task is weighted more heavily, while the irrelevant information is weighted less [45]. Introducing the attention mechanism into the convolutional neural network can make the network model pay more attention to the region of interest in the input information. It makes the model ignore irrelevant features and focus only on the essential features to be extracted. The residual time-frequency mixed attention module proposed in this paper includes the channel, time and frequency. As can be seen from Figure 1, RCA, RTA and RFA are the three components of the residual time-frequency mixed attention module, and this module is stacked with the three components in sequence. When an input feature map is given, RCA pays attention to the time-frequency images of different channels. Then, RTA and RFA pay attention to SCRVF and SCSVF, respectively, and ignore the unimportant interference information. The residual time-frequency mixed attention module does not increase the model's depth but expands the model's width, which further improves the performance of the network model. The details of the RCA, RTA, and RFA components are shown in Figure 3.  
To improve the recognition performance of a convolutional neural network for TFCF dual-channel time-frequency image feature mapping, RCA, RTA and RFA are proposed to focus on the valuable information in feature mapping. Firstly, three dimensions of input feature mapping \(M\) are defined as C, T and F, corresponding to time-frequency image channel, time and frequency, respectively. As you can see from Figure 3, RCA, RFA, and RTA differ only in input and output. RCA does not change the dimension of the input feature map, while RFA and RTA perform generalized transpose of the input feature map to realize the attention mechanism of C, T and F. Take RCA as an example, and the input characteristic map is \(M\in R^{T\times F\times C}\). Then, global average pooling and maximum global pooling are performed on \(M\), and the sum of the two results by element fuses the entire time-frequency plane information into a channel identifier \(M_{c}\in R^{1\times 1\times C}\). Then, to further extract the effective information of \(M_{c}\), this paper uses two convolution operations to process it and adds the ReLU function after each convolution operation to improve the nonlinear expression ability of the attention module. The first convolution layer is mainly used for dimensionality reduction, setting the dimensionality reduction ratio \(r=2\). The second convolution layer is used to restore the dimensions. Finally, the element value of channel identifier \(M_{c}\) is controlled between 0 and 1 by sigmoid function, and the channel weight information \(A_{c}\in R^{1\times 1\times C}\) is obtained. The channel attention feature matrix \(U_{c}\in R^{T\times F\times C}\) is obtained by weighting the input feature map \(M\) with \(A_{c}\). The above process can be expressed as:  
\[U_{c}=[\sigma(W_{2}R(W_{1}M_{c}))]\times M \tag{14}\]  
where \(R(\cdot)\) represents the ReLU function, \(W_{1}\in R^{\hat{\tau}\times C}\) and \(W_{2}\in R^{C\times\hat{\tau}}\) represent the weights of the two convolution operations respectively, and \(\sigma(\cdot)\) represents the Sigmoid activation function.