## 7 Experiments
### Experimental Comparison Considering All the Features  
For any prediction model, the averaged results of the 25 cross-validation repetitions are provided in Table 3 by considering four metrics: classification accuracy, macro-averaged precision, recall, and F1 score. The models are ordered by accuracy. First of all, let us note that: the ranking among the models is stable across all the four metrics, and the difference between accuracy and F1 scores is small. These two observations suggest that the slight imbalance of the dataset does not constitute a big issue in practical terms. In order to further analyze the results of Table 3, let us also consider the baseline classification rule ZeroR, which always outputs the most frequent dataset class [39]. By considering the data in Table 1, ZeroR has an accuracy of 0.36, larger than that of a totally random classifier (0.25). Interestingly, all the models involved in our comparison obtained better accuracy scores with respect to ZeroR, thus validating our general approach. Nevertheless, important differences in terms of effectiveness can be observed among the models for all the considered metrics. The best performing models are RF and SVM, which correctly classified, respectively, 72.5% and 71.7% of the dataset. They both outperformed the neural network models MLP\({}_{3}\), MLP\({}_{2}\), MLP\({}_{1}\), and the other tree-based model GBDT, that were able to reach an accuracy larger than 0.65. Further, the two kNN variants and the Bayesian models NB and QDA had an accuracy smaller than 0.60. In particular, the last one is better than ZeroR by only five percentage points, thus revealing that QDA is possibly overfitting the training set. In order to statistically analyze the differences between the best performing model RF and all the rest of the models, we conducted the non-parametric Mann-Whitney U test [70] on the accuracy scores obtained by the 25 repetitions of the cross-validation experiment. Therefore, by considering a significance level of 0.05, in Table 3, beside the accuracy scores of every model (except RF), we mark with the symbol \"=\" the models that are statistically equivalent to RF, and with the symbol \"\(\blackblackblack\)\" those models that are significantly outperformed by RF. The marks in Table 3 show that the only model that is not significantly outperformed by RF is SVM--with a small \(p\)-value of around 0.09,--while all the other models are significantly outperformed and registered a \(p\)-value smaller than \(10^{-5}\). Finally, in Figure 3, we provide the box-plots for the accuracy scores obtained by all the models in the 25 repetitions of the cross-validation experiment. Interestingly, the small size of the boxes show a good robustness for the proposed approach. Numerically, the larger standard deviation among the 25 accuracy scores is only 0.044 and it was observed for the two kNN models, while the most performing RF model registered a smaller standard deviation of 0.036. Moreover, let also note that the repetition with the best accuracy was obtained by the SVM model. \begin{table}
\begin{tabular}{l l l l l} \hline \hline
**Model** & **Accuracy** & **Precision** & **Recall** & **F1 Score** \\ \hline RF & 0.725 & 0.696 & 0.691 & 0.694 \\ \hline SVM & 0.717 = & 0.690 & 0.682 & 0.686 \\ \hline MLP\({}_{3}\) & 0.684 \(\blacktriangledown\) & 0.663 & 0.657 & 0.660 \\ \hline MLP\({}_{2}\) & 0.674 \(\blacktriangledown\) & 0.657 & 0.653 & 0.655 \\ \hline GBDT & 0.667 \(\blacktriangledown\) & 0.634 & 0.634 & 0.634 \\ \hline MLP\({}_{1}\) & 0.651 \(\blacktriangledown\) & 0.625 & 0.624 & 0.624 \\ \hline NB & 0.597 \(\blacktriangledown\) & 0.568 & 0.580 & 0.574 \\ \hline kNN\({}_{\text{U}}\) & 0.542 \(\blacktriangledown\) & 0.560 & 0.541 & 0.550 \\ \hline kNN\({}_{\text{D}}\) & 0.530 \(\blacktriangledown\) & 0.551 & 0.537 & 0.544 \\ \hline QDA & 0.415 \(\blacktriangledown\) & 0.442 & 0.318 & 0.369 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Averaged results for the experiment that considers all the features.