# Comparative Analysis of Classifiers for Classification of Emergency Braking of Road Motor Vehicles
## 5 Experimental Results  
For the purpose of obtaining benchmark results for the considered EMC problem, we obtained the results of the simple threshold classifier on both the training set and testing set, and used these results as a benchmark. Later, we will compare the performance of the simple threshold classifier with the proposed learned classifiers on the same EMC problem. The performance of the simple threshold classifier is illustrated in Table 7.  
In spite of the high precision, the threshold classifier has a very low recall and, as a consequence, a very low F-score. The reason for such a poor performance is that the classifier fails to classify many of the samples (events) of class \"1\" (emergency braking). By employing a single feature, the classifier fails on many samples of class \"1\" due to the existence of overlapping (\"gray\") zones in the landscape of the classified cases. It is important to note that the threshold classifiers built on aR and mR miss different emergency braking samples. Moreover, the samples which were correctly classified by the mR-based threshold classifier were not classified correctly by the aR-based classifier, and vice versa. These facts illustrate that a simple thresholding of the rates of lifting the accelerator pedal would not result in a good quality of classification of emergency braking situations. Unlike the one-dimensional classifier, \(k\)-NN performed much better with respect to the metrics under consideration. As can be seen from Table 8, \(k\)-NN with \(k\) = 23 and the best feature combination of [mP, aR, mR] (obtained through exhaustive search) performs slightly better than that with the default value of the parameter (\(k\) = 5). We notice that the accuracy, F-score, and recall of 23-NN on the testing set are higher than the same metrics on the training set. We speculate that the reason for this result is that either the training set has many \"difficult\" cases to learn or the testing set has \"easier\" cases to predict (or, the combination of both). As a result of applying GA both (i) for tuning the hyperparameters of XGBoost and SVM classifiers and (ii) for configuring the combinations of features pertinent to the dynamics of the accelerator, in the best of multiple independent runs of GA, the value of the F-score for the XGBoost and SVM classifiers reached 0.924 and 0.925, respectively. The convergence of the fitness value (CV F-score) during 100 independent runs of GA for XGBoost and SVM classifiers is shown in Figure 11.  
The sample best-evolved chromosomes containing the optimal values of hyperparameters of SVM and XGBoost are shown in Tables 9 and 10, respectively. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Metric**} & \multicolumn{2}{c}{**Classifier Based on mR Feature**} & \multicolumn{2}{c}{**Classifier Based on aR Feature**} \\ \cline{2-5}  & **Training Set** & **Testing Set** & **Training Set** & **Testing Set** \\ \hline Accuracy & 0.769 & 0.791 & 0.888 & 0.873 \\ Precision & 1.000 & 1.000 & 0.964 & 0.980 \\ Recall & 0.461 & 0.513 & 0.671 & 0.610 \\ F-score & 0.631 & 0.677 & 0.791 & 0.752 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Performance of simple threshold classifiers on training and testing set. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Metric**} & \multicolumn{2}{c}{**5-NN Method**} & \multicolumn{2}{c}{**23-NN Method**} \\ \cline{2-5}  & **Training Set** & **Testing Set** & **Training Set** & **Testing Set** \\ \hline Accuracy & 0.944 & 0.938 & 0.926 & 0.95 \\ Precision & 0.937 & 0.867 & 0.918 & 0.896 \\ Recall & 0.933 & 0.951 & 0.908 & 0.951 \\ F-score & 0.935 & 0.907 & 0.913 & 0.923 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Performance of \(k\)-NN classifier on training and testing set. Similar to the results obtained via other nature-inspired optimization approaches (genetic programming, evolutionary strategies, neural networks, etc.