# Weakly Supervised U-Net with Limited Upsampling for Sound Event Detection
## 3 Proposed Method
### U-Net with Limited Upsampling  
Figure 5 describes the proposed U-Net architecture with limited upsampling. Even though the segmentation map is not learned by direct loss minimization, its prediction can be obtained while minimizing the classification loss of each time frame. The U-Net-based architecture learns to generate an activation map of each event in the time-frequency domain. However, some events that activate only a few parts of the frequency bins could be ignored after postprocessing because of the average pooling. To handle this problem, reducing the frequency axis to 1 inside the model is not a good way since it removes too much representation in the middle of the model. Therefore, we further applied limited upscaling along the frequency axis in the decoder part of U-Net to keep high activation in the encoded feature. More specifically, the baseline U-Net in Figure 3 uses deconvolution with stride \((2,2)\) for upscaling in the decoder. However, the proposed U-Net uses deconvolution with stride \((2,1)\). Therefore, compared with the existing U-Net, the size of the decoder's feature map is smaller, which reduces the computational cost. In addition, the size of data transmitted in the skip connection between the encoder and decoder is different from the feature of the decoder. To adjust this, we downscaled the data size by using average pooling with \(kernel=stride=(1,2^{n})\).  
The encoder part is unchanged, but the decoder upsamples only the time axis of the encoded features. In the example shown in Figure 5, the frequency range shrinks from 64 to 8, and the shrink size 8 is maintained in the final segmentation map. The time range changes from 311 to 39, and it goes back to 311 to restore the original time range. Upsampling in the deconvolutional layers is often unreliable because the information of the input is lost by the U-Net encoder and should be regenerated, resulting in a great amount of generation errors at the output. In a weakly supervised manner, the target information is incomplete, so more uncertainty is likely to propagate through the network, especially in the training steps. As a result, this structure reduces the size of the feature handled by the decoder, further increasing the computational efficiency. If the upscaling of the frequency axis actually interferes with learning in a weakly supervised manner, removing unnecessary upscaling may yield higher detection performance.  
By comparing the U-Net in Figure 3 and the proposed LUU-Net in Figure 5, the size of the bottleneck layer between the encoder and decoder networks is \(39\times 8\times 128\), and it is the same for both models. The information transferred from the input to the output through the network is the same. In U-Net, the deconvolutional layers in the decoder part restore the original frequency and time axes to obtain the segmentation masks in the frequency and time domain. However, SED does not require the segmentation mask in the frequency domain. The proposed LUU-Net expands the time axis only, and the segmentation mask can be obtained more reliably by removing unnecessary expansion from the bottleneck layers with the same amount of information. Therefore, LUU-Net is more efficient than U-Net in SED tasks without loss of information.