## 4 Evaluation
### Application to State-of-the-Art Model  
In order to evaluate our ensemble-guided model, we applied it to [PERSON]'s wide ResNet (WRN) with 1-bit quantization [21], which achieved fifth in team ranking with an extremely small model size in DCASE 2021 Task 1A. The model that we implemented with reference to the study in [21] is shown in Figure 7. All convolutional layers consisted of 1-bit quantized weights as follows except for the gray-colored layer in Figure 7[21,34]:  
\[\mathbf{W}_{obj,i}=\sqrt{\frac{2}{K_{i}C_{in,i}}}\text{sgn}(\mathbf{W}_{i}) \tag{6}\]  
where \(\mathbf{W}_{i}\) and \(\mathbf{W}_{obj,i}\) are the weight tensors before and after quantization, respectively; \(\text{sgn}(\cdot)\) is the element-wise sign function; \(K_{i}\) is the kernel size; and \(C_{in,i}\) is the number of input channels of \(i\)th convolutional layer. If the convolutional layer has a two dimensional kernel, e.g., (\(a\times b\)), the kernel size is calculated by multiplication of each dimension, e.g., \(ab\). The training parameters and mean-teacher model parameters were set to the same values as Section 4.1 including learning rate scheduling.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**SNR** & **Log Loss** & **Macro-Averaged F\({}_{1}\)-Score** \\ \hline Without noise & 1.041 & 65.97\% \\
70 dB & 1.041 & 66.03\% \\
60 dB & **1.009** & **67.12\%** \\
50 dB & 1.028 & 66.39\% \\ \hline \hline \end{tabular}
\end{table}
Table 6: Performance comparison with various signal-to-noise ratios (SNRs) of additive noise.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Method** & **Log Loss** & **Macro-Averaged F\({}_{1}\)-Score** \\ \hline Without noise & 1.041 & 65.97\% \\ Both models, throughout training & 1.045 & 66.30\% \\ Student model only, throughout training & 1.025 & 66.92\% \\ Student model only, \(n\geq n_{act\_ems}\) & **1.009** & **67.12\%** \\ \hline \hline \end{tabular}
\end{table}
Table 7: Performance comparison with various noise addition strategies.  
Table 8 shows the performances of the ensemble-guided model with 1-bit-quantized WRN. Both the log loss and F\({}_{1}\)-score performances are improved for the cases of the proposed mean-teacher model with \(\beta=0.5,1,2.5\) compared with the vanilla model, and the case with \(\beta=2.5\) shows the best performance. The proposed model with \(\beta=5\) has the same log loss performance as the vanilla model, but it shows improved F\({}_{1}\)-score performance. Therefore, the proposed mean-teacher model is also effective for the 1-bit quantized WRN model structure in our experiment.  
We expect that the proposed technique will be well applied to the models of the top four teams of DCASE 2021 Task 1A, because the models also consist of CNN variants, and various learning strategies or data augmentation techniques do not conflict with the proposed algorithm. It is expected that the proposed method can be applied even when pruning is used to reduce the model size, but additional verification is required in this case. However, some models of [PERSON]'s [35] and [PERSON]'s [33] systems utilize the knowledge distillation technique, which uses teacher-student structure, so it may be necessary to design a different strategy to apply the proposed technique in this case.