# Improved Multi-Model Classification Technique for Sound Event Detection in Urban Environments
## 5 Experimental and Performance Evaluation
### Results and Discussion
#### 5.2.1 ANN-Based Classifier  
In this section, we compare and analyze the results obtained in the experiment using the ANN classifier on the Dev-Set, which are tabulated in Table 4 below. The baseline model [1] utilizes MFCC for feature extraction, while the ANN-based classifier uses both MFCC and RAST-PLP as feature extraction methods. The results show that applying RASTA-PLP improved both the ER and the F1-Score as compared to the MFCC in the baseline model. For clarity, F-score and F1-score have the same meaning in this article.  
In Figure 5, the obtained ER results using the ANN-based classifier are compared with the baseline model. The lowest ER was achieved using ANN RASTA-PLP for the sound of squeaking brakes (0.95), followed by those of cars (0.47), heavy vehicles (0.67), people speaking (0.92), and people walking (0.58), while for sound events such as children the lowest ER achieved using ANN with the MFCC feature was 1.01.  
Figure 6 depicts the F1-score comparison of the ANN-based classifier with the baseline model. The overall F1-score using RASTA-PLP is not improved; however, when using ANN-based MFCC, the F1-score is improved compared to the baseline model. For example, for large vehicles, when using ANN-based MFCC the F1-score obtained is 27, which lower than the other two extraction features. For car sounds, the baseline F1-score is 74 as compared to other features (i.e., in the range of 70-80).  
In summary, when comparing the ER results obtained using the ANN-based classifier to the baseline model, the lowest ER was achieved using ANN RASTA-PLP as a feature input for the sound of squeaking brakes (0.95), followed by those of cars (0.47), heavy vehicles (0.67), and people speaking (0.92), while the lowest ER (1.01) was achieved for people walking (0.58) and sound events such as children by the ANN with MFCC features. As measured by the F1-score, squeaking brakes are detected well using RASTA-PLP, while using MFCC as the input feature works well for detecting sounds made by children as compared to the baseline model.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{
\begin{tabular}{c} **Sound Events** \\ **Class** \\ \end{tabular} } & \multicolumn{2}{c}{**Baseline [1]**} & \multicolumn{2}{c}{**ANN**} \\ \cline{2-7}  & \multicolumn{2}{c}{**MFCC**} & \multicolumn{2}{c}{**MFCC**} & \multicolumn{2}{c}{**RastA-PLP**} \\ \cline{2-7}  & **ER** & **F1-Score** & **ER** & **F1-Score** & **ER** & **F1-Score** \\ \hline \multirow{2}{*}{Brakes Squeaking} & 0.98 & 4 & 1.00 & 0 & 0.95 & 7 \\ \hline \multirow{2}{*}{Car} & 0.57 & 74 & 0.51 & 55 & 0.47 & 63 \\ \hline \multirow{2}{*}{Children} & 1.35 & 0 & 1.01 & 40 & 1.21 & 0 \\ \hline \multirow{2}{*}{Large Vehicle} & 0.9 & 51 & 0.80 & 27 & 0.67 & 38 \\ \hline \multirow{2}{*}{People Speaking} & 1.25 & 18 & 0.99 & 0 & 0.92 & 10 \\ \hline \multirow{2}{*}{People Walking} & 0.9 & 51 & 0.93 & 10 & 0.58 & 48 \\ \hline \end{tabular}
\end{table}
Table 4: Comparison of results using ANN classifier on Dev-Set.