## 3 Methodology
### Selection of Predictors
However, variable selection is unstable, with a small sample size compared to the number of variables to be studied, as many combinations of variables can fit the data similarly. Additionally, this method does not consider the possible causal relationships between the variables. The Boruta variable selection model was proposed in [49], which works as an algorithm surrounding Random Forest. This algorithm begins by adding randomness to the dataset. This creates shadow variables from the study variables, which are copied and permuted with each other. Afterwards, a random forest classifier acts on the dataset, where the relative importance of each of the variables is calculated. To evaluate this importance, the decrease in the average accuracy value (mean decrease accuracy) of each feature is used. A greater number of average accuracy values indicates more importance in the model. In each of the iterations, a feature is checked and compared to the importance of its shadow variables. If it is lower than its shadow counterparts, the feature is removed because it is considered to have little importance. This process ends when all the features are confirmed or rejected, or a limit of random forest execution is reached. This variable selection method is relevant because it selects variables that are important to the target variable. The recursive feature elimination method proposed in [50] is a backward feature selection method, which begins by building a model on the entire set of predictors and calculating the importance order of each predictor. The lowest-importance variables are eliminated, and the model is rebuilt by calculating the importance order of the remaining variables. Backward selection is frequently used with random forest models, as it tends not to exclude variables from the prediction equation and forces the trees to contain suboptimal divisions of the predictors using a random sample of predictors. Simulated annealing is a method that begins by randomly selecting a subset of entities and defining the operator with a maximum number of iterations. Afterwards, this method builds a model and calculates its predictive performance. Once calculated, it proceeds to include or exclude a small number of features randomly (from one to five) to recalculate the performance and check if it improves or not with the new modification. In the case of worsening, the solution can even be accepted, according to [51], although the acceptance probability decreases as the iteration of the algorithm increases. The genetic algorithm proposed in [52] is inspired by natural selection and biological evolution processes. Its main objective is to find the optimal set of predictor variables that maximizes model performance. It begins with an initial population of solutions, where each solution represents a set of variables. The performance of each solution is then evaluated using a fitness function that measures the quality of the selected variables. Solutions with higher fitness are more likely to be selected for the next generation. After the selection, genetic operators such as reproduction, crossover, and mutation are applied to generate a new population of solutions. Reproduction involves copying the selected solutions directly into the next generation. Crossover combines the characteristics of two solutions by exchanging parts of their chromosomes. Mutation introduces random changes into a solution to explore new possibilities. This process of selection, reproduction, crossover, and mutation is repeated for several generations until a convergence criterion is reached (e.g., a maximum number of iterations or insignificant improvement in the fitness of the solutions). The final solution obtained represents the optimal set of predictor variables for a given problem.