## 4 Evaluation
### Performance Comparison with Other Ensemble-Guided and Vanilla Models

Table 2: Performance comparison between the proposed model, the conventional mean-teacher model, the temporal ensembling models, and the vanilla model without any ensemble guidance. Additionally, there are interesting results regarding the performance-enhancing technique, which is the subject of this paper. As shown in Table 3, several performance improvement techniques specific to their models have been tried, e.g., feature reuse in [21], domain adaptation in [14], and knowledge distillation [33]. The techniques have F\({}_{1}\)-score improvements of 0.2%, 0.1%, and 2%, respectively, and log loss improvements of about 0.01. Therefore, the 1.3% F\({}_{1}\)-score improvement and the 0.04 log loss improvement of the proposed mean-teacher model may be meaningful. Figure 6 shows the classwise F\({}_{1}\)-score performances of the vanilla and proposed models. Comparing the F\({}_{1}\)-scores of the vanilla model and the proposed model, the performance improvements of two classes are notable: the public square (5.74% improvement) and the pedestrian street (5.81% improvement), indicated by green arrows in Figure 6. These two classes showed the lowest performance in the vanilla model, so the results show that the proposed method can improve the low-performance classes. In the training step, additional computation time is required to calculate the prediction result of the teacher model in the proposed mean-teacher structure. Of course, there is no additional time required for updating the coefficient, i.e., backpropagation, because the coefficients of the teacher model are fixed. In our experiment using a GPU (NVIDIA Geforce RTX 3090), the elapsed time to train the vanilla model and the proposed mean-teacher \begin{table}
\begin{tabular}{l c c c} \hline \hline
**Method** & **Log Loss** & **Macro-Averaged F\({}_{1}\)-Score** & **Model Size** \\ \hline DCASE 2021 Task 1A Baseline (in [20]) & 1.47 & 47.7\% & 90.3 KB \\ \hline Vanilla model (ours) & 1.05 & 65.8\% & 125.8 KB \\ Mean-Teacher model (ours) & 1.01 & 67.1\% & 125.8 KB \\ \hline [PERSON]’s model (in [21]) & 0.92 & 68.0\% & 42.5 KB \\ [PERSON]’s model + Feature Reuse (in [21]) & 0.91 & 68.2\% & 106.7 KB \\ \hline [PERSON]’s model (in [14]) & 0.89 & 69.4\% & 127.7 KB \\ [PERSON]’s model + Domain Adaptation (in [14]) & 0.88 & 69.5\% & 127.5 KB \\ \hline [PERSON]’s model (in [33]) & - & 68.5\% & 124.1 KB \\ [PERSON]’s model + Knowledge Distillation (in [33]) & - & 70.5\% & 124.1 KB \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Performance comparison with the state-of-the-art performances. All performances are presented using the values reported in referenced papers. Figure 6: Classwise F\({}_{1}\)-score performances. Green arrows indicate classes with significantly improved F\({}_{1}\)-score performance. model are 105 and 146 s per epoch, respectively. With using a CPU, the elapsed time to train the vanilla and the proposed models are 55 and 76 minutes per epoch, respectively. Each elapsed time was measured as an average of ten epochs for the GPU and three epochs for the CPU. In the inference step, the proposed and the vanilla models have the same elapsed times because the proposed model uses only the student model.