## 3 Materials and Methods
### Hyperparameter Optimization
Figure 2: Hyperparameter optimization process: this figure shows the process of the fine-tuning of hyperparameters using the GridSearchCV tool. The utilization of cross-validation helps to improve the model's performance by reducing variance through averaging performance estimates from multiple iterations of training and testing on different data subsets. It presents a better generalization by evaluating the model on multiple subsets of the data, aiding in assessing its generalization capabilities. This is especially important for detecting overfitting, as the model's performance is evaluated on various data partitions, resulting in a more robust estimate of its accuracy on unseen data, as is mentioned in the study guided by [PERSON] [21]. 3. Selection of Optimal Hyperparameters: Post evaluation, the combination of hyperparameters that produces the best performance is selected based on the minimal average error of the test folds. As mentioned before, GridSearchCV presents the advantages of increasing the model's precision and improving the generalization and potentially reduced bias in the estimation of model performance. Another advantage is that it facilitates the comparison between various model families because, although each model has its own sets of hyperparameters, the evaluation is performed based on the same metric to measure the error. On the other hand, its main disadvantage is the computational complexity that can arise for very large datasets or complex models. The optimization process in this research was conducted utilizing the GridSearchCV tool from the Scikit-learn library [22]. The models subjected to testing included RandomForest [23], GradientBoosting [24], and Support Vector Regression (SVR) [25]. In the case of SVR, a pipeline was established to assess the impact of data scaling, involving MinMax normalization and Z-score standardization, alongside hyperparameter tuning. The comprehensive list of hyperparameters is presented in Table 3.  
GridSearchCV is a process that can be time-consuming because it must test every combination of hyperparameters on the training sets. For this reason, [PERSON]'s formula [26] was used for sample size selection for quantitative variables when the population size is known to train the models with a smaller sample size and reduce execution time. The formula used is shown in Equation (4). \[n=\frac{NZ^{2}\sigma^{2}}{(N-1)E^{2}+Z^{2}\sigma^{2}} \tag{4}\]  
where:  
\(n\): = sample size. \(N\): = population size. \(Z\): = critical Z value; for a confidence level of 99%, z equals 2.58. \(\sigma^{2}\): = population variance. \(E\): = absolute precision level; 1% of the population's standard deviation. \begin{table}
\begin{tabular}{l c} \hline \hline
**Model** & **Hyperparameters** \\ \hline RandomForest & Max depth : 40–50 \\  & Max features: 10–25 \\ \hline SVR (kernel RBF) & Scaling: [StandardScaler, MinMaxScaler] \\  & Gamma: [0.001, 0.01, 0.1, 1, 10, 100] \\  & C: [0.001, 0.01, 0.1, 1, 10, 100] \\ \hline SVR (kernel poly) & Scaling: [StandardScaler, MinMaxScaler] \\  & Degree: [2, 3, 4, 5, 6, 7, 8, 9] \\  & C: [0.001, 0.01, 0.1, 1, 10, 100] \\ \hline GradientBoosting & n estimators: [1, 2, 5, 10, 20, 50, 100, 200, 300] \\  & Max depth: [5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19] \\  & Learning rate: [0.001, 0.01, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9] \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Evaluated models. The dataset, containing data from 2015 to 2022, comprises 577,031 records. The sample size selection formula determined that a sample size of 59,680 records would provide a 99% confidence level, representing 10.34% of the total records. For the validation sample, 10% of the complete dataset was chosen, resulting in 57,704 records. It is important to note that the training and validation sets are entirely exclusive.