# Multi-Scale Features for Transformer Model to Improve the Performance of Sound Event Detection
## 1 Introduction  
Sounds in daily life usually contain important information about the surrounding conditions. Many research efforts have been made to extract information automatically from sound signals in machine learning paradigms. With a growing interest in this area, the detection and classification of acoustic scenes and events (DCASE) challenges have been held from 2013 to 2020 with several competitive tasks. Sound event detection (SED), one of the main research topics in the competition, is defined as identifying sound signals as well as the onset and offset times of sound events [1]. The application of SED is varied and includes audio surveillance [2, 3], urban sound analysis [4], information retrieval from multimedia content [5], healthcare monitoring [6], and bird call detection [7]. As deep neural networks (DNNs) have shown state-of-the-art performance in computer vision [8], speech recognition [9], and machine translation [10], current SED studies primarily focus on DNN-based approaches. It has been demonstrated that the feedforward neural networks (FNNs) outperformed the traditional Gaussian mixture model and support vector machines in SED [11, 12, 13]. Convolutional neural networks (CNNs) have exhibited superior performance to that of FNNs in various sound classification tasks, including SED [14, 15, 16]. By using two-dimensional filters, the parameters of which are shared along the time and frequency shift, CNNs can efficiently handle audio signal distortions in the time-frequency domain. However, CNNs are not suitable for modeling the time correlations of audio signals. Recurrent neural networks (RNNs), such as gated recurrent neural networks (GRUs) and long short-term memory, have been successfully used in speechrecognition [9]. They can efficiently model the long-term time correlations in time series signals, and thus, they are expected to facilitate SED. Among them, the convolutional recurrent neural network (CRNN), which combines convolutional neural networks (CNN) and recurrent neural networks (RNN), has exhibited a satisfactory classification performance in SED [11]. It is now considered as a representative DNN architecture for SED and has been widely used in many applications. One of the most important problems in SED is how to model the short and long temporal correlations of sound signals. The large difference in sound durations between competing sound classes makes it crucial to efficiently model the time correlation in sound signals. The short correlation is considered in milliseconds and represents the short-term spectral variations of the sound signal in general. To model the spectral variations, the window size and hop size in the calculation of the spectrum are usually set to tens of milliseconds. Compared with short correlation, long temporal correlation is usually more than hundreds of milliseconds and depends on the nature of the sound signals [17]. Compared with short correlation, which can be modeled from the perspective of signal analysis, long correlation is rather difficult to model. RNNs such as long short-term memory (LSTM) and gated recurrent unit (GRU)-based neural networks have been widely used to model long correlation information in sequential data, but their performance remains unsatisfactory due to the limited length of correlation that they can model. To overcome this problem, multi-scale features have been used as input into RNNs in some studies on sound signal classification [18; 19; 20]. In [18], two bi-directional RNNs were employed to model fine-scale and coarse-scale features. The outputs of the RNNs were then concatenated to feed the following fully connected neural network to classify sound events. The multi-scale (fine and coarse) features were advantageous in modeling long correlations of the sound signal compared with conventional single-scale RNNs. Feature-pyramid networks have been used for object detection problems in computer vision [21].