# Deep Learning for Audio Event Detection and Tagging on Low-Resource Datasets
## 4 Evaluation
### Results  
The same parameters are used for training both WHEN and WHO networks for all three different approaches. Our batchsize is equal to eight recordings. We use the Adam optimiser [34] with a learning rate scheduler that reduces the initial rate of \(1\times 10^{-5}\) by half every 20 epochs until it reaches a minimum rate of \(1\times 10^{-8}\). The loss function used for the predictions of the WHEN network is the proposed MMM loss, while we use a binary cross-entropy loss for the multi-class predictions of the WHO network.  
First, we trained WHEN and WHO independently. WHEN was trained with a HnH input, since not using HnH can cause the network to either ignore negative recordings or mix the negative and positive frames in a recording. On the other hand, WHO was trained with the conventional nonHnH input since using HnH for WHO made its performance worse. This is due to the fact that the active classes are already very sparse (0 to 6 active classes out of 87 per recording) and, for the NIPS4B dataset, the HnH input duplicates negative recordings, hence decreasing the activation rate for each class, making it even harder to predict.  
Next, we trained two versions of the joint network, one of them uses a HnH input while the other a nonHnH input. When training the joint network with HnH, the WHO predictions tend to not have a satisfactory performance due to the increase in negative recordings. When training the joint network with the nonHnH input, the WHEN task performance is degraded. The loss value of the WHO task tends to be an order smaller than the one for WHEN, hence we trained with two different combination of weights for the task. For one of them, both task losses have the same weight of 0.5, while for theother one the weight for the WHO task loss is an order larger than the WHEN; more specifically, we used weight 0.5 for WHEN loss and 5.0 for WHO loss.  
Finally, we performed a tied weights training. This solved the issue of using only one type of input since it can train with both HnH and nonHnH input separately for each task as if the tasks are trained independently, while still sharing the weights of the shared layers like the joint training.  
During the NIPS4B 2013 challenge, systems that performed audio tagging, similar to the task for our WHO network, were submitted. The winning solution [35] was trained on the whole training set (687 recordings) and tested on the NIPS4B testing set and reached 0.92 area under the receiver operating characteristic (ROC) curve (AUC) score. These results can be used as a performance baseline for this dataset. However, the winning NIPS4B method and our method cannot be explicitly compared since our network is trained in a subset of the NIPS4B training set and then evaluated in the rest of the recordings.  
Table 3 shows the area under the ROC curve (AUC) results for each training approach. We can see that even though the tied weights training has a better overall performance compared to the joint training, separate training still has the best overall results. The best overall results for joint training were produced when using weights 0.5 and 5.0 for WHEN and WHO loss, respectively and also using nonHnH input. Hence, we can conclude that the WHO network is sharing important information with the WHEN network that can boost its performance when enough weight is given to its loss. As mentioned before, any type of joint training has so far been proven to outperform independent training, which is not the case in our experiments, when comparing results for both WHEN and WHO. We consider the two tasks to be closely related and use hard parameter sharing approaches. However, the tasks might be more loosely related than we originally considered and a soft parameter sharing approach [36; 37; 38; 39] may increase performance.