## 5 Discussion  
In this section, to evaluate the suitability of the previously analyzed classification models, overall accuracy is not the only metric to be considered. Although it is not an imbalanced sample, it is useful to measure the success of prediction in each of the classes (collision and avoidance cases) through a set of performance metrics (especially precision and recall). Finally, the analysis of the performance of these classifiers is also completed by means of the Receiver Operating Characteristic (ROC) curves. Mathematically, precision is defined as the ratio of true positives to the sum of true positives plus false positives, while recall (or sensitivity) is the result of dividing true positives by the sum of true positives and false negatives. In other words, precision  
Figure 16: Random Forest feature importance using MDI criterion. Figure 17: Random Forest feature importance using MDA criterion. \begin{table}
\begin{tabular}{c c} \hline \hline
**Model** & **Hyperparameters** \\ \hline
**SVM** & C: 1000; _Kernel function_: rbf; _gamma_ (_only for rbf_): \(1\times 10^{-3}\) \\ \hline
**KNN** & _K:_ [1,31]; _metric (p)_: 2 (Minkowski) \\ \hline
**Individual decision tree * & _Split criteria_: Gini-index \\ \hline
**Random Forest * & _Split criteria_: Gini-index, _number trees_: 24 (OOB rate stable  around 0.12) \\ \hline \hline \end{tabular}  
* _Minimum sample splits_ were left to two and _minimum sample leaf_ was left to one.  
\end{table} 
Table 8: Optimized hyperparameters. measures the ability of the classifier to identify the relevant data points and the recall to find all the relevant cases within the sample. From the point of view of future implementation in the decision algorithm of an AEB system, it is primary to measure the precision and recall for the \"Collision\" class. In the case of obtaining a very high recall, it would result in a lower precision. That is, the model would label a large proportion of trials as collisions, even when the event resulted in avoidance. If such a model were included in the algorithm of an AEB device, there would be a risk that the system would act conservatively by executing a more determined and abrupt braking maneuver in those cases where it is not necessary, generating avoidable wear on the braking system and reducing the risk of causing a rear-end collision by taking a more anticipatory action. Similarly, a very high precision would classify many cases as \"Avoidance\" and not as an accident, which would imply that the system would not act promptly in risky cases. Therefore, to avoid these two extreme scenarios, a trade-off between these two metrics should be maximized. An optimal blend of precision and recall is the F1-score, defined as the harmonic mean of both, giving equal weight to each. Of the four models, the ones with the most balanced precision and recall values are tree models and SVM (Figure 18). On the contrary, the KNN has lopsided values for these two metrics, considering that the precision for the collision cases is notably higher than the corresponding recall, which would imply a lack in the detection of a possible accident. The SVM tends to behave similarly in terms of these metrics and the F1-score, although, given the overall accuracy value (Table 9) of the individual decision tree, this is more recommendable. The results obtained can also be illustrated through the ROC curves in Figure 19. These curves show the trade-off between the True Positive Rate (sensitivity) and False Positive Rate (1-specificity). Note how in all models the average area under the curve (AUC) is higher than 0.95 applying cross-validation with k = 5. Likewise, the tree-based models present ROC curves farther away from the line of no-discrimination, resulting in values closer to perfect classification when varying the threshold used to predict the classes.