# Road Scanner: A Road State Scanning Approach Based on Machine Learning Techniques
## 6 Results and Discussion  
The presented results in the following tables (Tables 3-5) correspond to the average metrics (accuracy, F-score, and AUC) followed by the standard deviation, all computed from the four iterations of the cross-validation process. These table results are also summarized in Figure 12.  
Taking the accuracy results (Table 3), the best two classifiers were SVM--with an accuracy rate of 88.05%--and Random Forest with an accuracy rate of 87.79%. Similar results were observed for the F-score metric (Table 4). Their respective rates were 87.87% and 87.43%. For the AUC metric (Table 5), remarkable results were given by the Extra Trees classifier (AUC = 94.94%) and the Random Forest classifier (AUC = 93.89%). In addition, low standard deviation rates were observed for all metrics and all classifiers. For instance, the Random Forests classifier has outputted a standard deviation rate of 1.31, 1.30, and 0.22 for the three measured metrics (Accuracy, F-score, and AUC). Overall, the high average rates and the low standard deviation rates (among all metrics) confirm the good performance of the chosen classifiers. Moreover, execution times for all the models were also computed. Precisely both training and testing times were calculated as shown in Table 6. For the training part, we notice higher times for ANNs and the non-tree-based ensemble methods (e.g., bagging). For ANNs, this can be explained by the high number of epochs that were needed in the training process. Additionally, because ensemble methods integrate a large number of estimators (e.g., 100 estimators for bagging), the learning step may take a longer time, especially for the non-tree-based models. For the testing part, all models (except bagging) present great performances, which is suitable for the deployment of these models in real-time applications. Furthermore, for a better understanding of the results, we have also computed the confusion matrix of the SVM classifier, which has shown the best performance in terms of Accuracy and F-score. The confusion matrix is shown in Table 7.  
On the one hand, the model has succeeded to detect 1007 instances (TP) among the 1080 instances of the smooth class. On the other hand, 349 instances (TN) were correctly detected among the 460 instances of the potholed class. From these confusion matrix statistics, per-class metrics were computed as shown in Table 8. These per-class metrics are mainly Precision, Recall, and F-score. Taking the smooth class, the detection rates are significantly higher than those of the potholed class. For instance, the precision rate of the smooth class is 90.07% versus 82.70% for the potholed class. The difference is much larger for the Recall metric as the smooth class recorded 93.24% versus only 75.87% for the potholed class. Indeed, the classifiers seem to be accurate and precise when detecting a certain class since both precision rates are relatively high. However, the classifiers seem to have more difficulties in recalling the totality of the potholed-class instances. This confusion may be caused by the speed dependency problem [39]. Actually, when a vehicle crosses a pothole at various speeds, the captured signal by the accelerometer will have a variable amplitude which may confuse the trained classifiers. This issue may be overcome in the future by increasing the sampling rate when recording the potholed segments. Higher sample rates deliver more data per second, which enhances recognition rates. Another limitation of our work and which represents a challenge, is the accuracy and precision of the smartphone sensors themselves. It differs depending on the brand, the specifications, and the quality of the smartphone; therefore, appropriate preprocessing operations should be applied to filter the possible noises related to the sensing process. In addition to this, a car's suspension quality can also affect the sensing accuracy and precision.