# A Deep Attention Model for Environmental Sound Classification from Multi-Feature Data
## 3 Time-Frequency Attention Mechanism Model Based on Multi-Feature Parameters
### Residual Model  
The residual network bypasses some processing layers by means of additional Skip Connections, merges their input and output, and solves the gradient exploding problem and the gradient vanishing problem of deep neural networks.  
The proposed network structure (as shown in Figure 3) mainly consists of a time-frequency attention module and four residual blocks. The input data is composed of Log-Mel, Phase, and a time-frequency spectrogram connected in parallel in a new dimension. The size is \((h,\ t,\ c)\), where \(h\) is the frequency dimension, \(t\) is the time dimension of the feature map, and \(c\) is the category dimension of the feature degree. The input data passes first through the attention module, where a preliminary processing of the feature map takes place to mitigate the influence of the noise and irrelevant frequency bands in the audio. After that, it is processed by the first Conv_layer (convolution kernel 3 \(\times\) 3 \(\times\) 24, stride [1,2]), and then features are extracted through four residual blocks, each of which consists of two Conv_layers (convolution kernel 3 \(\times\) 3 \(\times\) (24 \(\times\) 2n), where n is the Residual block number, stride [1,2]), two Conv_layers (convolution kernel 3 \(\times\) 3 \(\times\) (24 \(\times\) 2n), where n is the Residual block number, stride = 1), and one Average Pooling. To ensure that the dimension of the feature map passing through the Average Pooling branch is consistent with the main route, the channel axis is zeroized after Average Pooling. Finally, the category information is obtained after going through two Conv_layers and one Global Average Pooling layer.  
It is noteworthy that although the time-frequency attention structure is simple, requiring no complex computations, it does improve the recognition ability of this model. Its expression is:  
\[\begin{array}{ll}M_{c}(F)&=\sigma(f^{1\times 1}[f^{1\times 1}(AvgPool(F))],f^{1 \times 1}(AvgPool(F))])\\ &=\sigma(f^{1\times 1}[f^{1\times 1}(F^{c}_{avg}),f^{1\times 1}(F^{c}_{avg})]) \end{array} \tag{1}\]  
\(F\), a feature vector of dimension \((h,t,c)\), is input into the time-frequency attention mechanism module. First, average pooling and maximum pooling are performed on this vector \(F\) in the feature channel dimension, and then it goes through two-dimensional convolution of a convolution kernel size of \(1\times 1\times c\), and this produces two feature maps of size \((h/2,\ t,\ c)\). The two feature maps are connected in parallel in the channel dimension, and go through two-dimensional convolution of a convolution kernel of size \(1\times 1\times c\).  
Figure 2: Three characteristic spectra of saw wood sound. (**a**) Log-Mel spectrogram. (**b**) time spectrogram. (**c**) phase spectrogram.  
This gives attention weight coefficients of size \((h,\ t,\ c)\), which are multiplied by the input feature to produce the feature vector, which has been extracted by attention.  
Where, \(c\) is the number of neurons, and \(f\) is the convolution operation, whose formula is shown (2).  
\[f_{i}=\sigma(W_{i}\odot X_{i}+b) \tag{2}\]  
where \(f_{i}\) is the output of the convolution operation, \(\delta\) is the activation functions, \(W\) is the convolution kernel, \(\odot\) is the convolution operation, \(X\) is the current input feature, and \(b\) is the bias.