# Multi-Scale Features for Transformer Model to Improve the Performance of Sound Event Detection
## 3 Proposed Network Architecture  
### Proposed Transformer Network Model  
The architecture of the proposed Transformer model is shown in Figure 2. The architecture was motivated from the feature-pyramid network based on CRNN [22]. It consists of three parts: the baseline Transformer, a multi-scale feature extraction part, and a classification layer. The baseline Transformer consists of seven convolution blocks (ConvBlocks) and one Transformer encoder, which is depicted in Figure 1. The parameters of the CNN block and Transformer encoder in the baseline Transformer were mainly determined by referencing the work in [25]. The input features of the baseline Transformer are 64-dimensional log-Mel spectrograms extracted with a window size of 1024 sample points and a hop length of 323 points. The length of the input frames is 496 frames, corresponding to 10 s of each audio clip with a sampling rate of 16 kHz. The spectrogram is constructed as a two-dimensional (496 \(\times\) 64) feature map.  
In the ConvBlocks, a 3 \(\times\) 3 convolutional filter was applied to the context window of the input feature map, and batch normalization is used to normalize the filter output to zero mean and unit variance. A gated linear unit (GLU) activation function was applied after batch normalization. Non-overlapping [2 \(\times\) 2, 1 \(\times\) 2, 1 \(\times\) 2] average pooling was selectively applied in the frequency and time-domain to reduce the dimension of the input data to 128 \(\times\) 62 \(\times\) 1 (number of feature maps \(\times\) number of time frames \(\times\) number of frequency bands) in the last ConvBlock before the Transformer encoder. We reduced the dimension in the frequency domain to 1, but much of the time-domain information was preserved to use the time-correlation information of the sound signal in the Transformer encoder. Dropout is used before average pooling to reduce overfitting during training. The output of the last ConvBlock is used as input into the Transformer encoder, which has 128 attention units (\(d_{model}=128\)) and 16 heads (\(h=16\)) in MHA. The internal position-wise feed-forward network of the Transformer encoder has 512 units (\(d_{ff}=512\)).  
The classification layer has 10 output units corresponding to the sound classes. These units have a sigmoid activation function, the output of which denotes the posterior probability of the classes for each frame of the sound signal. In addition to the sigmoid activation output used to predict frame-level labels in the classification layer, an attention layer with the softmax activation function is used to predict the clip-level label by the weighted average for the 10 s clip. The weighted average score \(O_{c}\) for class \(C\) was computed by multiplying the softmax activation function output \(O_{c}^{att}(t)\) and the sigmoid output \(O_{c}^{cla}(t)\) as follows:  
\[\small\begin{split} o_{c}=\frac{\sum\limits_{t=1}^{T}o_{c}^{att}( t)\times o_{c}^{cla}(t)}{\sum\limits_{t=1}^{T}o_{c}^{att}(t)}\end{split} \tag{5}\]  
where \(T\) is the total number of time frames in the 10 s audio clip. The clip-level output \(O_{c}\) is used to calculate the loss function at the clip level when using the weakly labeled data for training in the proposed Transformer model.  
In the multi-scale feature extraction part, two more Transformer encoders were employed, each coupled with a ConvBlock. The output of the last Transformer encoder is up-sampled and concatenated with that of the second last Transformer encoder. Then, a 1 \(\times\) 1 convolution is applied to the concatenated feature map to smooth the data and reduce  
Figure 2: Proposed Transformer Network Model.  
their dimensions. Once again, the resulting data are up-sampled and combined with the output of the Transformer encoder in the baseline Transformer, and a \(1\times 1\) convolution is applied to feed the multi-scale feature to the classification layer.  
In Figure 3, we show the simplified procedure of the training and testing in our work using the proposed Transformer Network model.