# Sound Event Detection Using Derivative Features in Deep Neural Networks
## 4 Experimental Results
### Experimental Results
\begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{4}{c}{**DCASE 2018 Test Set**} \\ \hline \multicolumn{4}{c}{**Single Channel**} & \multicolumn{2}{c}{**Three Channels**} \\ \hline \multicolumn{4}{c}{**F-Score (\%)**} & **ER** & **F-Score (\%)** & **ER** \\ \hline
[weakly + unlabeled] & 12.79 & 1.44 & 14.48 & 1.42 \\ (DCASE 2018 baseline) & (14.06) & (1.54) & - & - \\ \hline
[weakly + unlabeled + strongly] & 17.57 & 2.42 & 18.85 & 2.41 \\ \hline
[strongly] & 14.99 & 2.41 & 16.62 & 2.51 \\ \hline
[weakly + strongly] & 15.25 & 2.42 & 17.83 & 2.37 \\ \hline Average & 15.15 & 2.17 & 16.95 & 2.18 \\ \hline Average relative improvement & - & - & 11.6\% & 0.5\% \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Sound event detection (SED) results for the basic CRNN using derivative features (DCASE 2018 test set). For the training of the mean-teacher model, we only used the [weakly + unlabeled + strongly] combination because it yielded the best performance in the basic CRNN. Instead of early stopping, the model was trained for 100 and 200 epochs (max epoch = 100 and max epoch = 200 in Tables 5 and 6, respectively), and the best model on the validation data was accordingly selected for the evaluation of the test data. The Adam optimizer was used for training with a learning rate of 0.0001 and a median filtering of length 5 was applied to the output of the classification layer. The SED results of the mean-teacher model are shown in Table 5. It can be seen that significant performance improvement is obtained by the mean-teacher model (cf. Table 3). Furthermore, the addition of a derivative to the static feature resulted in a 3% relative improvement in the F-score in the DCASE 2018 test set. In the DCASE 2019 test set, a 4.4% relative improvement was attained. In addition, a 2.5% relative improvement was observed when the strongly labeled training data set was used as the test data to demonstrate the effect of the derivative features in case the difference between the testing data and training data is small. It can be concluded that a consistent performance improvement was attained by using the derivative features in the mean-teacher model regardless of the test set: however, the improvement was rather diminished compared with the basic CRNN. We also compared the performance of our system with the baseline system of the DCASE 2019 [1]. Although the baseline system uses the same audio data and similar mean-teacher model, it showed an F-score of 23.7%, which is worse than our single channel result of 25.95%. As mentioned in the comparison with the baseline of DCASE 2018 in Table 3, the performance difference seems to come from some details in the implementation. However, by using the derivative features, we could further increase the F-score of our system to 27.36%, which is better than the result of the DCASE 2019 baseline system as expected. The SED results when the training epochs increased to 200 are shown in Table 6. For the DCASE 2018 test set, a 5% relative improvement was attained by using the derivative features. The results for the DCASE 2019 test set indicate a 7.5% improvement. When strongly labeled training data were used \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{4}{c}{**DCASE 2019 Test Set**} \\ \hline  & \multicolumn{2}{c}{**Single Channel**} & \multicolumn{2}{c}{**Three Channels**} \\ \hline  & **F-Score (\%)** & **ER** & **F-Score (\%)** & **ER** \\ \hline
[weakly + unlabeled] & 11.28 & 1.55 & 11.93 & 1.54 \\ \hline
[weakly + unlabeled+ strongly] & 13.80 & 2.99 & 14.63 & 2.92 \\ \hline
[strongly] & 12.85 & 3.07 & 13.11 & 3.09 \\ \hline
[weakly + strongly] & 13.39 & 2.98 & 14.41 & 2.91 \\ \hline Average & 12.83 & 2.65 & 13.52 & 2.62 \\ \hline Average relative improvement & - & - & 5.3\% & 1.1\% \\ \hline \hline \end{tabular}
\end{table} 
Table 4: SED results for the basic CRNN using derivative features (DCASE 2019 test set). Figure 5: Learning curve in F-score of the basic CRNN with weakly labeled training data. as test data, a 2.9% relative improvement was attained.