## 3 Materials and Methods
### Random Forest Model  
The random forest model is a supervised learning algorithm that can be employed for both regression and classification tasks [33; 34]. It constructs multiple decision trees during training and falls under bagging methods. Bagging, short for bootstrap aggregating, is an ensemble learning technique where multiple models of the same type are trained on different subsets of the initial dataset generated through bootstrapping. In the random forest model, a collection of decision trees is created. Each input data point is processed through all the trees, and the final output is determined by the model's unique ability to aggregate the tree results. This aggregation, whether by taking the mode of the classes for classification tasks or by averaging the predictions for regression tasks, is a crucial step in determining the final output. A vital feature of the random forest model is the randomness in selecting characteristics for each tree. This randomness reduces the correlation among the trees and enhances the overall robustness of the model (Table 1). The process can be broken down into several phases:  
* Creating Subsets: For each node in a tree, a random subset of traits is selected from the original dataset. This subset is usually much smaller than the total number of available features. * Choosing the Best Feature: From the randomly selected traits, the feature that best separates the data is chosen based on a quality measure, such as entropy or the Gini index. * Repeating the Process: This process is repeated for each node of every tree in the forest, ensuring that each tree is constructed using a different subset of characteristics. Before training the model, several important hyperparameters must be set. A hyperparameter search technique was applied to determine the parameters that optimize model predictions. This involved specifying a grid of parameters and the number of folds for cross-validation. The parameters considered were as follows:  
n_estimators: [100; 200; 300];  
max_depth: [None, 10; 20; 30];  
min_samples_split: [2; 5; 10];  
min_samples_leaf: [1; 2; 4]. All possible combinations of these parameters were evaluated. For each combination, the model was trained and assessed using cross-validation. The combination that yielded the best performance was selected as the optimal set of parameters. \begin{table}
\begin{tabular}{c c} \hline \hline
**Hyperparameters** & **Function** \\ \hline Estimators & This specifies the number of decision trees in the forest. More trees can improve accuracy but increase computational costs. \\ max\_features & This indicates the number of features to consider when splitting \\  & each node as a fixed value or a fraction of the total features. \\ max\_depth & This limits the depth of each tree to avoid overfitting. \\ min\_samples\_leaf & This defines the minimum number of samples needed to form a \\  & leaf, with higher values reducing overfitting. \\ min\_samples\_split & This specifies the minimum number of samples required to split a \\  & node, also helping to control overfitting. \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Hyperparameters of random forest. The \"fit\" method was employed to train the model on each combination of parameters, and the best combination was identified using a dedicated function. The model was trained and evaluated five times, using a different portion of the data as the test set. The optimal parameters identified were as follows:  
n_estimators: 300;  
max_depth: 20;  
min_samples_split: 5;  
min_samples_leaf: 2. The total number of iterations conducted while searching for the optimal parameters was 540, corresponding to the number of combinations specified in the grid. Below is a graphical representation of the results of the optimal parameter search where each point on the graph represents a combination of parameters evaluated during the search (Figure 1).