# Confidence Learning for Semi-Supervised Acoustic Event Detection
## 4 Results and Analysis
### Comparison with Other Methods  
We compared the proposed model with the following approaches:  
_MT18_: the official baseline for DCASE2019 task4, with the mean teacher structure [17]. _Baseline_: modified MT18 method with attention pooling. _MT-SAED_: the stage one model of C-SAED with power pooling. _Prob0.9_: only predictions with \(prob\geq 0.9\) added to pseudo-labels, samples retrained with equal weight [20]. _Prob_: all samples retrained with probabilities as weights. _Prob0.5_: only predictions with \(prob\geq 0.5\) added to pseudo-labels, samples retrained with confidence. Table 1 lists the performance of different models on the three test sets. The results illustrate that C-SAED models are significantly improved in contrast to the other models. The ER improvement is mainly due to the significant reduction of INS error. That is, the improvement comes from correcting false positives. When the model retrained with parameter \(\alpha=0\), ER also decreased. Although the mean teacher method already makes use of weakly labeled and unlabeled data, applying an appropriate self-training structure can effectively reduce false alarms. However, _Prob0.9_ introduced massive false positive predictions as pseudo-labels and applied them equally, resulting in many insertion errors. _Prob_ caused performance degradation due to the fact that the majority of true negatives owned small weights approximately 0.7 after applying parameter \(\alpha=0.3\). Meanwhile, many false positives (Figure 3) were introduced with high weights. In contrast, C-SAED with confidence further improved ER by successfully strengthening the attention to true negatives and true positives. The poor results of _Prob0.5_ reveal the importance of true negatives. In addition, for most models, the error rate on polyset is 0.13 to 0.2 higher than results on validation 2019. This phenomenon confirms that polyphonic audio event detection is more challenging than monophonic audio event detection. Figure 3: Accuracy of frame-level predictions changes with confidence estimates and probabilities. The accuracy of positive predictions (\(prob\geq 0.5\)) changed with learned confidence estimations and probabilities. Their polynomial fitting curves are displayed. Figure 4 shows the class-wise error rates. We omit \"Prob0.9\" which yielded high ERs, and \"MT18\" whose original paper does not report its detailed error rates. Some conclusions can be drawn. First, the parameter \(\lambda\) and \(\alpha\) effect on the INS and the DEL errors--for example, when \(\lambda=0.01\), large \(\alpha\) (\(\alpha=1.0\)) gives lower INS but higher DEL compared with small \(\alpha\) (\(\alpha=0.3\)). The reason may be that large \(\alpha\) gives low weights on the samples with low confidences, which makes the predicted events accurate but may fail with difficult samples. Second, the proposed 2-stage scheme outperforms the baseline, MT-SAED, and self-training on most events. Third, different events have different difficulties in classification. For example, event alarm/bell/ringing, electric shaver/tooth brush, and vacuum cleaner have relatively lower ERs, while event frying gives the highest ER.