## 5 Experimental Results
### Evaluation on an Embedded System
#### 5.5.3 Cascade Approach on an Embedded System  
Like the experiments on the computer, the embedded implementations use the Python 2.7 libraries LibROSA and scikit-learn for the feature extraction and the sound classification, respectively. The training of all six datasets (Root dataset plus the 5 datasets) required for the cascade approach required around 1.5 h on the Raspberry Pi. This training time is possible by specifying the audio length when loading the audio with LibROSA, which for our case is 5 s per audio file. Both the features extraction and classification are also performed on the Raspberry Pi. The ANN and k-NN classifier has been evaluated due to their relatively high accuracy. They are configured with the parameters present on Table 9.  
Table 15 summarizes the measured accuracy and timing of the classifiers when using the cascade approach on the Raspberry Pi. The results are the average of 10 executions. Despite the relatively high accuracy achieved by both classifiers for each dataset (or supercategory), the global accuracy when considering the combined accuracy of both classifiers (Table 16) decreases similarly to the accuracy depicted in Table 11. The achievable accuracy obtained by Equation (3) rounds to 54.4% for the ANN classifier and to 50.3% for the k-NN classifier. Both classifiers slightly decrease their accuracy in their embedded version. Some improvements in the dataset (or supercategory) _Warning_ would certainly lead to a higher accuracy since both classifiers significantly decrease their accuracy for this dataset. Although the execution time of both classifiers remains in the order of milliseconds, the ANN classifier has increased its timing significantly, becoming only around 3.5 times faster than the k-NN classifier. Notice that the k-NN and ANN classifiers are significantly slower than the other classifiers for the _Root_ dataset. There are several reasons for this:  
* The k-NN classifier is the most time demanding classifier, not only when embedded (Tables 4, 11 and 12 and in Figure 10). Similarly to other classifiers, the k-NN classifier becomes around 10 times slower when embedded. * The exception is the ANN classifier. The increment of the execution time of the ANN classifier when embedded, reflected in Table 15, becomes around 100 times slower when compared to Table 11.
* The k-NN and the ANN classifiers are evaluated with their optimal configuration. This fact increments the ANN's execution time like that shown in Figure 7 and in Figure 12.  
The main benefit of the cascade approach is the combination of classifiers. Whereas the ANN classifier outperforms the k-NN classifier for the Root dataset, the accuracy of the k-NN remains higher for the _Airport_ and the _Vehicles_ datasets. Thus, the ANN classifier can be used for all the datasets except the _Airport_ and the _Vehicles_ datasets, where the k-NN classifier could be applied due to its higher accuracy. Such combination reaches an accuracy of 55.7% but demands 95.7 ms. Note that the overall execution time of the cascade approach decreases if the Decision Tree classifier is used in stage 1 and datasets _Residence_, _Vehicles_ and _Warning_ in the second stage while the ANN classifier is used for the _Airport_ and _Construction_ datasets in the second stage. Such a solution would only require 8.6 ms, but the accuracy decreases to 35.7%. Both are examples of how the cascade approach enables multiple combinations, either targeting accuracy or a fast response. The evaluation of the ANN and k-NN classifiers for the non-cascade approach is summarized in Table 17. The achieved performance of both classifiers is similar to the cascade approach, while the execution time of both classifiers is slightly lower than the two-stages approach. The timing increases from a factor of 8.8 to 52.2 times for the k-NN and for the ANN classifiers respectively compared to the timing when both classifiers are not embedded (Table 12).