## 4 Performance Evaluation
### Objective Quality Evaluation for Voice Activity Detection  
This subsection evaluates the performance of each VAD method using both the matched and mismatched evaluation datasets by measuring the area under the receiver operating characteristic curve (AUC) and the EER scores [38]. First, the noisy input signal was directly applied to the DNN-, bDNN-, and LSTM-based VAD methods. Second, MTU-Net(IRM) was used as a front-end for the DNN-, bDNN-, and LSTM-based VAD methods. Table 4 compares the average AUC and EER scores under the matched noise conditions. As shown in the table, each of the supervised learning-based VAD methods that employed MTU-Net as a front-end achieved higher AUCs and lower EERs than those directly using the noisy input signal. Moreover, all the supervised learning-based VAD methods achieved higher AUCs and lower EERs than the proposed mask-based VAD method. This was because the NN models for VAD were trained to accommodate the characteristics of the noises that commonly appeared in both model training and testing. However, the proposed VAD method operated without any training; thus, its performance was lower than that of the NN-based VAD methods. Nevertheless, the performance degradation of the proposed VAD method was not severe.  
Next, the performance evaluations of the VAD methods were repeated under the mismatched noise conditions; these results are shown in Table 5. As shown in the table, the performance of the NN-based VAD methods was significantly degraded compared to that under the matched noise conditions shown in Table 4. The NN-based VAD methods directly using noisy input speech achieved especially higher AUC and lower EER values than the VAD methods using the estimated clean speech processed by MTU-Net. This was caused by the performance degradation of the SE methods under mismatched noise conditions. However, the performance of the proposed VAD method had similar average AUC and EER values when not considering the noise condition because the proposed VAD methods only depend on the performance of MTU-Net(IRM) operating fairly well with noisy utterances under matched and mismatched conditions. Moreover, the proposed mask-based VAD method provided higher average AUCs and EERs than the three NN-based VAD methods.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**AUC (\%)** & **\(\mathbf{-5}\) dB** & **0 dB** & **5 dB** & **10 dB** & **Average** \\ \hline Noisy + DNN & 86.72 & 90.18 & 91.03 & 92.51 & 90.11 \\ Noisy + bDNN & 86.06 & 90.75 & 89.88 & 91.47 & 89.54 \\ Noisy + LSTM & 86.51 & 91.21 & 91.08 & 92.22 & 90.26 \\ MTU-Net(IRM) + DNN & 89.32 & 93.31 & 94.36 & 95.48 & 93.12 \\ MTU-Net(IRM) + bDNN & 89.47 & 92.39 & 92.30 & 93.52 & 91.92 \\ MTU-Net(IRM) + LSTM & 89.63 & 92.92 & 92.75 & 93.89 & 92.30 \\ Proposed mask-based VAD & 84.63 & 88.04 & 89.88 & 89.13 & 87.92 \\ \hline
**EER (\%)** & **\(\mathbf{-5}\) dB** & **0 dB** & **5 dB** & **10 dB** & **Average** \\ \hline Noisy + DNN & 23.16 & 17.28 & 15.74 & 14.34 & 17.63 \\ Noisy + bDNN & 23.64 & 18.62 & 20.40 & 18.687 & 20.34 \\ Noisy + LSTM & 24.17 & 19.07 & 19.36 & 17.72 & 20.08 \\ MTU-Net(IRM) + DNN & 21.42 & 16.70 & 15.89 & 14.06 & 17.02 \\ MTU-Net(IRM) + bDNN & 20.37 & 17.01 & 17.43 & 15.68 & 17.62 \\ MTU-Net(IRM) + LSTM & 20.25 & 16.66 & 16.78 & 15.30 & 17.25 \\ Proposed mask-based VAD & 22.70 & 19.36 & 18.17 & 18.71 & 19.73 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Comparison of the receiver operating characteristic curves (AUCs) (%) and equal error rates (EERs) (%) of seven different voice activity detection methods according to their different signal-to-noise ratios under matched noise conditions.