## 3 Materials and Methods
### Predictive Modeling
#### 3.3.4 Evaluation and Comparison of Models Obtained  
At this point we had to calculate the metrics of accuracy, sensitivity, specificity, area under the ROC curve (AUC), positive predictive value, negative predictive value. However, because the distribution of classes in the target variable is not balanced, we selected the best model based on sensitivity, area under the ROC curve (AUC), lower number of false-negatives and higher number of true-positives that we obtained from the confusion matrix. We show the structure of the confusion matrix for two classes in Figure 6.  
\begin{table}
\begin{tabular}{l l l l l l} \hline \hline
**Model** & **Hyper-Parameter** & **Value Range** & **Description** & **Default** & **Optimal** \\ \hline LR & penalty & “[”11”, “12”, “elastic-net”]” & “Penalty type” & 12 & 12 \\  & C & [0.001, 0.01, 0.1, 1, 10] & Inverse of regularization strength & 1.0 & 1 \\  & solver & [”1 bfgs”, “liblin-ear”, “newton-cg”, “newton-cholesky”, “sag”, “saga”] & Inverse of regularization strength & 1.0 & 1 \\  & & [”glgs”, “liblin-ear”, “newton-cg”, “newton-cholesky”, “sag”, “saga”] & Inverse of regularization strength & 1.0 & 1 \\ \hline RF & n\_estimators & [10, 17, 25, 33, 41, 48, 56, 64, 72, 80] & Number of decision trees in the random forest. & 100 & 64 \\  & max\_depth & [3,5,7] & Maximum depth of trees. & none & 7 \\  & Criterion & [”gini\", “entropy”] & Function that measures the quality of the division. & gini & entropy \\  & min\_samples\_split & [2,5] & Minimum number of samples required to split & 2 & 5 \\  & min\_samples\_leaf & [1,2] & Minimum number of samples required for a leaf node. & 1 & 2 \\  & max\_features & [”auto\", “sqrt\"] & Number of randomly selected features without replacement in the division. & auto & auto \\ \hline XGB & n\_estimators & [10, 17, 25, 33, 41, 48, 56, 64, 72, 80] & Number of decision trees in XGB. & 100 & 72 \\  & max\_depth & [3,5,7] & Maximum depth of trees. & 6 & 3 \\  & learning\_rate & [0.1] & Learning rate & 0.3 & 0.1 \\  & subsample & [0.6: 0.9] step 0.1 & Fraction of observations that will be random samples for each tree. & 1 & 0.9 \\  & colsample\_bytes & [0.6: 0.9] step 0.1 & Fraction of columns that will be random samples for each tree. & 1 & 0.6 \\ \hline GB & n\_estimators & [10, 17, 25, 33, 41, 48, 56, 64, 72, 80] & The number of sequential trees to model. & 100 & 48 \\  & max\_depth & [3,5,7] & Maximum depth of trees. & 3 & 3 \\  & min\_samples\_split & [500: 1000] step 5.05 & Minimum number of samples required at a node for splitting. & 2 & 949 \\  & min\_samples\_leaf & [20, 28, 37, 46, 55, 64, 73, 82, 91, 100] & Minimum number of samples required in a leaf node. & 1 & 55 \\  & max\_features & [”auto\", “sqrt\", “log2”] & Number of features to consider when looking for the best split. & None & log2 \\  & subsample & [0: 1] step 0.1 & Fraction of observations to select for each tree. & 1.0 & 0.3 \\  & learning\_rate & [0.01: 0.1] step 0.03 & Learning rate. & 0.1 & 0.09 \\ \hline DT-Cart & Criterion & [”gini\", “entropy”, “log\_loss”] & Function to measure the quality of a division. & gini & entropy \\  & max\_depth & [2:10] step 1 & Max Tree Depth & None & 6 \\  & min\_samples\_split & [1: 10] step 1 & Minimum number of samples required for splitting on an internal node. & 2 & 2 \\  & min\_samples\_leaf & [2: 10] step 1 & Minimum number of samples required in a leaf node. & 1 & 3 \\  & max\_features & [”auto\", “sqrt\", “log2”, “none”] & Number of features to consider when looking for the best split of the tree. & None & None \\ \hline \hline \end{tabular}
\end{table}
Table 4: Hyper-parameter settings.  
In Appendix A.3, we detailed the metrics for the confusion matrix.