## 8 Models Training  
As a good practice, the CIDAD was split into training and test sets, with a training subset of 80%. For the evaluation of the models, a nested stratified k-fold cv was chosen with an internal and an external loop. Both loops were adjusted with k = 5. Additionally, to guarantee balanced samples, SMOTE was applied in the internal loop using five k-neighbour. The minority class is over-sampled at about a ratio of 0.4, and the majority class is under-sampled at a ratio of 0.5. This section outlines the process of hyperparameter tuning on the machine learning models over the remaining training data, computing ROC AUC scores. The results presented correspond to the average value of the inner cv, and the best parameters will be chosen to be used in the test stage. The first model to consider was logistic regression used with a liblinear solver. The penalty hyperparameter was adjusted to \(L1\) and \(L2\). To tune the C parameter, a sweep over the values of 0.001, 0.1, 1, 100, and 1000 was carried out. The best model was obtained with an \(L2\) penalty and a C value of 0.1 and an ROC AUC score of 0.984. The global mean over all iterations was 0.9845, with a standard deviation of 1.013 \(\times\) 10\({}^{-3}\). Normally, a better operation is expected at high values of C; however, as can be seen in Figure 3, the variation in the parameter C does not significantly affect the performance of the model with an appropriate choice of penalty. The tuned hyperparameters that offer the best results for LR, in all cases, are a C equal to 0.1 and a penalty \(L2\), as shown in Table 5. This table presents the parameters that obtain the best average score in each internal cv and its standard deviation, among all the possible combinations tested. The parameters selected here are those used in the evaluation of the model in the test split. Figure 2: Accuracy versus variable importance. To identify the optimal composition of hyperparameters for the Bernoulli naive Bayes model, a sweep for the values 0.01, 0.1, 0.5, 1, and 10 in the parameter \(\alpha\) was performed. The overall mean ROC AUC value was 0.9398, with a standard deviation of \(7.35\times 10^{-3}\). The best individual score for this model was 0.9596, with \(\alpha\) equal to 10. Figure 4 shows the average results obtained over the selected values of \(\alpha\) in the inner iterations. As a logistic regression, naive Bayes does not present a linear relationship between values of \(\alpha\) in each iteration. Figure 4: Naive Bayes mean scoring by iteration. Figure 3: Logistic regression mean score by iteration. An average of the values obtained in the internal iteration in each of the parameter combinations is calculated for the selection of the parameters that will be used in the model. The best mean in the inner cv is adopted and defines the parameter that will be used in the test split. Table 6 presents the results of this process for the NB model. To perform RF, the hyperparameters were tuned using \(sqrt\) and \(log2\) as max_features, and tested with 100, 200, and 300 trees. This model obtains a score of 1 on average over all iterations. Figure 5 presents the average AUC ROC results by iteration. The best performance found among all the models at the training stage is obtained by random forest. In tree-based models, the number of trees and features can affect the time and performance of the model during its execution. For this dataset, many combinations of parameters presented perfect scores; however, those that achieved the same performance with fewer trees were selected. Table 7 shows the chosen parameter combinations that perform perfectly for each inner iteration. On the other hand, the AdaBoost classifier employs a DT base estimator initialised with a max_depth = 1. In this model, tuning is performed on the values of 500, 1000, and