## 5 Experimental Results
\begin{table}
\begin{tabular}{l l c c c c c c c c c} \hline \hline
**ID** & **ALGORITHM** & **FS** & **VARIABLES** & **BEST** & **WORST** & **MEAN** & **STD +/—** & **PRECISION_0** & **PRECISION_1** & **RECALL_0** & **RECALL_1** \\ \hline DT-S & Decision Tree & Stud-GA & 14 & 0.7322 & 0.6407 & 0.6934 & 0.0212 & 0.7419 & 0.7214 & 0.7468 & 0.7163 \\ DT-T & Decision Tree & Tournament-GA & 15 & 0.7525 & 0.6169 & 0.6862 & 0.0307 & 0.7548 & 0.75 & 0.7697 & 0.7343 \\ DT-RFFCV & Decision Tree & RFCV & 1 & 0.7186 & 0.6873 & 0.6821 & 0.0195 & 0.7200 & 0.7172 & 0.7248 & 0.7123 \\ DT & Decision Tree & No PS & 26 & 0.7153 & 0.6203 & 0.6799 & 0.0252 & 0.7533 & 0.6759 & 0.7062 & 0.7259 \\ XGB-S & XGBOOST & Stud-GA & 12 & 0.7424 & 0.6237 & 0.6989 & 0.026 & 0.7702 & 0.709 & 0.7607 & 0.7197 \\ XGB-T & XGBOOST & Tournament-GA & 11 & 0.7288 & 0.6542 & 0.6948 & 0.0223 & 0.7235 & 0.736 & 0.7885 & 0.6619 \\ XGB-RFFCV & XGBOOST & RFCV & 2 & 0.7186 & 0.6475 & 0.6850 & 0.0164 & 0.6746 & 0.7778 & 0.8028 & 0.6405 \\ XGB & XGBOOST & No PS & 26 & 0.7254 & 0.6068 & 0.6803 & 0.0295 & 0.7024 &  
The interpretation of the _FS_ column is:  
* **Stud-GA**: Evolutionary feature selection with _Stud_ selection operator. * **Tournament-GA**: Evolutionary feature selection with tournament selection operator. * **RFECV**: Feature selection with recursive feature elimination (RFE) with cross-validation (CV). * **No-FS**: No feature selection applied in the configuration  
As mentioned, ten classification algorithms were used, using the implementation available at the Scikit-learn python library [27]:  
* **Decision Tree (DT)**: Its objective is to create a model to predict the value of a target variable by learning simple decision rules from data characteristics. * **Gradient Boosting (GB)**: An additive model is created in a stepwise way, allowing the optimization of arbitrary differentiable loss functions. At each step, a regression tree is fitted to the negative gradient of the given loss function
* **Adaboost (ADB)**: A meta-estimator that starts by fitting a classifier on the original dataset and next fits additional copies of the classifier for the same dataset where the weights of the misclassified instances are adjusted in order to make the subsequent classifiers concentrate on the difficult cases. * **Bagging (BG)**: Fits base classifiers on random subsets of the original dataset and aggregates its individual predictions into a final prediction. * **Bernoulli Naive Bayes (BNB)**: This classifier is useful for discrete data and is designed to handle binary/boolean features. * **Extra Trees (ET)**: A meta estimator that fits random Decision Trees on multiple subsamples of the dataset and uses the average to improve predictive accuracy and control overfitting. * **Gaussian Naive Bayes (GNB)**: Another Naive Bayes model. This classifier is used when the values of the predictor are continuous
* **Logistic Regression (LR)**: This algorithm attempts to predict the probability that a given data entry will belong to a category. Just as linear regression assumes that the data follow a linear function, logistic regression models the data using the sigmoid function. * **Random Forest (RFC)**: This technique fits several Decision Tree classifiers on multiple subsamples of the dataset and uses averaging to improve predictive accuracy and control overfitting. * **XGBoost (XGB)**: A tree boosting system that stands out for its scalability and is widely used by data scientists. Appendix A.3, Table A3 provides a table with information on the parameters used for each model. Regarding evolutionary feature-selection methods, we focus their application on three models. The first one is Gradient Boosting [28] as it obtained consistently good results among the set of classifiers without feature selection. Next is XGBoost, [29], which is a state-of-the-art machine-learning algorithm that allows us to measure the goodness of results from the other algorithms.