# Weakly Supervised U-Net with Limited Upsampling for Sound Event Detection
## 3 Proposed Method
### Global Threshold Average Pooling  
The GWRP in Section 2.4 requires a sorted list of whole segmentation map elements. The sorting is required both in training and testing, so there is a large computational overhead. Moreover, the hyperparameter \(d\) is adjusted by the amount of events occurring in the map. In Equation (3), \(d^{i-1}\) decreases as \(i\) increases because \(0\leq d\leq 1\), and the value of \(d\) should be relatively large if the sound event occurs densely or the length of the event is long and small if it occurs sparsely or its length is short. When long and short events are mixed in the input recording, which is usual in real situations, it is very hard to determine a single value of \(d\) to guarantee the detection performance. Global max pooling is not influenced by the event lengths; however, it is sensitive to the outliers, and most of the segmentation map values, except the maximum, are discarded, so learning through error backpropagation may not work well.  
To overcome the problems of GWRP, computational overhead, and hyperparameter adjustment, we propose a novel global pooling using a learnable threshold value. To begin with, we define the following threshold function, which is similar to the standard step function:  
\[g(m,\theta) = \begin{cases}1,&\text{if }m\geq\theta\\ 0,&\text{if }m<\theta\end{cases}, \tag{4}\]  
where \(m\) is the input and \(\theta\) is a given threshold value. The key idea of the proposed method is averaging only the values larger than the threshold. We define global threshold average pooling (GTAP) as follows:  
Figure 5: U-Net architecture with limited upsampling in the deconvolutional layers. The left encoder part is the same as U-Net, but in the right, the decoder part, it only upsamples along the time axis to match the input time range.  
\[h_{tf} \triangleq \mathbf{H}_{c}(t,f)\] \[\mathrm{GTAP}(\mathbf{H}_{c}) = E_{\geq\theta}[\mathbf{H}_{c}]\;=\;\frac{\sum_{t}\sum_{f}h_{tf}g(h _{tf},\theta)}{\sum_{t}\sum_{f}g(h_{tf},\theta)}\quad, \tag{5}\]  
where \(h_{tf}\) is the \((t,f)\)-element of the 2D segmentation map for class \(c\), \(\mathbf{H}_{c}\), defined in Equation (1). \(E_{\geq\theta}[\cdot]\) is a conditional expectation, so the GTAP function is the average of the values larger than threshold \(\theta\). In the actual implementation, we used the following equivalent equation for more efficient calculation.  
\[\mathrm{ReLU}(x) = \max(x,0)\] \[\mathrm{GTAP}(\mathbf{H}_{c}) = \frac{\sum_{t}\sum_{f}\mathrm{ReLU}(h_{tf}-\theta)}{\sum_{t}\sum _{f}g(\mathrm{ReLU}(h_{tf}-\theta),0)}+\theta\quad, \tag{6}\]  
where \"ReLU\" is the rectified linear unit commonly used in deep neural networks.  
The threshold \(\theta\) is also relevant to the frequency and the length of the events. With the assumption that the segmentation map is distributed by a Gaussian, the appropriate threshold value is found by  
\[\theta=mean(\mathbf{H})+astd(\mathbf{H})\;, \tag{7}\]  
where \"\(mean\)\" and \"\(std\)\" are the mean and the standard deviation of the segmentation map and \(\alpha\) is a hyperparameter to define how tightly to cut off the segmentation map. Because it is hard to use a class-specific threshold, we used all the training data to compute the mean and the standard variation. If \(\alpha<0\), the threshold value becomes smaller, which results in overestimation over \(\alpha=0\). If \(\alpha>0\), the threshold increases and the trained model may underestimate. A grid search is used to determine the appropriate value of \(\alpha\) using a validation set. The detailed procedure is explained in the Experiments Section.