## 4 Evaluation  
### Evaluation Settings  
To evaluate the proposed system for domestic SED, numerical simulations were performed using the DESED dataset of the DCASE 2020 Task 4 database, which is an audio dataset for weakly supervised sound event detection in domestic environments. The database consists of real-recorded subsets of Audioset [43] and Freesound [44]. Some of the data are synthesized using the SINS database [45] as background sounds. There are 10 event classes: speech, dog, cat, alarm bell, dishes, frying, blender, running water, vacuum cleaner, and electric shaver. Each audio clip is a 10-s wav file with 44.1 (for the weakly-labeled and unlabeled data) or 16 kHz (for strongly-labeled data) sampling frequency. All audio files were resampled to a 16 kHz sampling frequency. The database consists of strongly-labeled data (2045 files, with event classes and time stamp information), weakly-labeled data (1578 files, with event classes information only), and unlabeled data (14412 files, without any annotation). The detailed information of the database configuration can be found in [34]. The audio clips were short-time-Fourier-transformed by a 1024-sample Hanning window with 75% overlap to match the number of input frames in the classifier for the baseline of the DCASE 2020 Task 4 [40]. To train the NMF frequency basis matrix, 200 iterations were performed to calculate both \(\mathbf{W}_{c}\) and \(\mathbf{H}_{c}\) matrices (Equations (9) and (10)) for each  
Figure 6: (**a**) A flow chart; and (**b**) an illustrative example of the event-wise post-processing. class. The number of the basis for each class was set to 13 (11 for the \"vacuum cleaner\" class) so that the dimension of the extracted feature was 128. In the case of iterative feature extraction, 50 iterations were performed for each clip to calculate \(\mathbf{H}_{clip}\) (Equation (11)). The singular value threshold (\(\delta\) in Equation (20)) was set to be a proportional to the maximum singular value as  
\[\delta=\gamma\max_{k,r}[\boldsymbol{\Sigma}]_{k,r}, \tag{27}\]  
where \(\gamma\) denotes a proportionality constant. The classifier was trained using Adam optimizer, and the learning rate was exponentially ramped up during 50 epochs to the maximum learning rate of 0.001. The classification and the consistency costs were categorical cross-entropy and mean-squared error, respectively, and the weight for the consistency costs, \(\beta\), was set to 2.0. The classifier was trained using the training dataset of the DCASE database for 200 epochs and evaluated using the validation dataset. The batch size was set to 24, and the moving-average weighting factor, \(\alpha\), of the mean-teacher model was 0.999. The performances of the two types of post-processing systems, with and without the event-wise post-processing, were evaluated. The post-processing system without the event-wise post-processing consisted of only the frame-wise thresholding, and the threshold value was set to 0.5. The system with the event-wise post-processing consisted of the frame-wise and event-wise thresholding. The event threshold value was set to 0.8 as it exhibited good overall performance, while the frame thresholds were set to the optimal values that exhibited the best performance among \(\{0.1,0.2,0.3,0.4,0.5,0.6,0.7\}\) for each class. In this evaluation, the event-based F1-score was utilized as the performance measure, which is defined by a geometric mean of precision \(P\) and recall \(R\) as  
\[F_{1}=\frac{2 PR}{P+R}, \tag{28}\]  
where precision and recall are defined as  
\[P=\frac{n_{TP}}{n_{TP}+n_{FP}}, \tag{29}\]  
and  
\[R=\frac{n_{TP}}{n_{TP}+n_{FN}}, \tag{30}\]  
where \(n_{FP}\), \(n_{FN}\), and \(n_{TP}\) denote the numbers of false answers (false positives), missing answers (false negatives), and correct answers (true positives). The answers were considered as \"correct answer\" if the onset error was smaller than 0.2 s, and the offset error was smaller than 0.2 s (or 20 % of the event duration).