# Weakly Supervised U-Net with Limited Upsampling for Sound Event Detection
## 1 Introduction  
The purpose of sound event detection (SED) is figuring out the types of sound sources contained in the input audio recordings. SED can be used in a surveillance system that detects the occurrence of a specific sound [1; 2], voice activity detection (VAD) [3], or keyword spotting with the human voice [4]. Currently, SED systems are usually implemented by convolutional neural networks (CNNs) [5; 6], recurrent neural networks (RNNs) [2], or combinations of them [7; 8; 9]. SED solutions early on usually relied on a supervised learning framework that required a completely labeled dataset [10] in which the onset and offset timings of the sound events were transcribed. However, complete labeling of all the timings would require too much effort from human listeners, and often, they are not available in practical situations. In more realistic cases, only the types of sound events of the audio recordings are known, as shown in Figure 1. Strong labels include onset and offset times in the given audio recordings, but weak labels give only the types of audio events. When incomplete information is given, the task is called weakly supervised classification [11; 12; 13; 14; 15; 16; 17]. Using weakly supervised learning with an incompletely labeled dataset might reduce thecosts for dataset construction. One of the weakly supervised classifications is generating a strongly labeled dataset from the given weakly supervised training data [16]. This framework generates strongly labeled sound event timings by clipping sound event audio, normalizing it, and then synthesizing it with normalized background audio or white noise. This is similar to data augmentation, but with this framework, it is possible to increase the quantity of the dataset. In addition, natural environments are polyphonic, meaning that multiple sounds may be active at the same time [1; 2; 17]. Therefore, the sound event occurrences overlap. There are no predefined rules on how sounds can co-occur, and the way to model co-occurrence is through random sampling with the degree of overlap of different classes given by or obtained from the data statistics. To detect polyphonic sound events, multi-label classifiers are required [1]. The SED model is usually based on weakly supervised CNNs [16; 17; 18]. When training CNNs, only the event labels are used as targets. In the detection stage, video object description models (VODMs) such as cascade R-CNN (regions with CNN features) [19], faster R-CNN [20], and the class activation map (CAM) approach [21] are used. The 2D object regions or class activation maps obtained by a trained CNN are used to predict the onset and offset times of the classified audio events. Another type of SED approach is using temporal models such as convolutional recurrent neural networks (CRNNs) [1; 22] and Transformers [23]. These temporal models are able to find the onset and offset timings more accurately because they are inferred simultaneously with the class labels in a single learning stage. The aforementioned weakly supervised learning methods require generating onset and offset timings that are obtained in an unsupervised manner, so the accuracy of the detection varies greatly according to the types of audio sources and the characteristics of the recording environments. Self-supervised learning [24] and pseudo-labeling [25] methods for weakly supervised SED have been proposed, but their performances rely on specific audio events only. Recently, U-Net showed successful performance in image segmentation and also was applied to SED [16; 26]. U-Net was originally designed for segmentation tasks, so in a weakly supervised manner, the segmentation targets are converted to sound event category labels by global pooling techniques such as global average pooling (GAP) [27; 28], global max pooling (GMP) [16], and global weighted rank pooling (GWRP) [18]. However, these methods require averaging or sorting operations on the total feature map, so their is an extensive computational overhead.