## 4 Model Training and Evaluation Methods
### Hyperparameter Optimization with GA
This configuration enabled the GA to simultaneously search for both continuous variables in numerical ranges and categorical criteria with a limited number of options, and to perform an efficient search process within the defined population size and number of generations, instead of 4 \(\times\) 10 \(\times\) 10 \(\times\) 10 = 4000 theoretical \begin{table}
\begin{tabular}{l l l l} \hline \hline
**ML Models** & **Parameters and Their Ranges** & **ML Models** & **Parameters and Their Ranges** \\ \hline \multirow{8}{*}{RF} & n\_estimators: 10 values in increments of 10 in the & \multirow{4}{*}{RF} & n\_estimators: 10 values in increments of 10 in the & \multirow{4}{*}{RF} \\  & range [100, 1000] & & & \\  & criterion: squared\_error, friedman\_mse, & & & \\  & absolute\_error, poisson & & & \\  & max\_depth: 10 values in increments of 5 in the & & & \\  & range [5; 50] & & & \\  & min\_samples\_leaf: 10 values in increments of & & & \\  & 1 in the range [1; 10] & & & \\ \hline \multirow{8}{*}{K-NNs} & n\_neighbors: 10 odd numbers in the & \multirow{8}{*}{K-NNs} & kernel: linear, poly, rbf, sigmoid \\  & interval [3; 21] & & & max\_iter: 10 values in increments of 10 in the \\  & weights: uniform, distance & & & \\  & metric: minkowski, euclidean, & & & range [100, 1000] \\  & manhattan, chebyshev & & & \\  & leaf\_size: 10 values in increments of 10 in the & & & \\  & range [10; 100] & & & \\ \hline \multirow{8}{*}{AdaBoost} & n\_estimators: 10 values in increments of 10 in the & \multirow{8}{*}{HGBR} & learning\_rate: 10 values in the range [0.01, 2.0] \\  & range [100, 2000] & & & \\  & learning\_rate: 10 values in the range [0.01, 1.0] & & & \\  & loss: linear, square, exponential & & & \\ \hline \multirow{8}{*}{GB} & n\_estimators: 10 values in increments of 10 in the & \multirow{8}{*}{XGBoost} & n\_estimators: 10 values in increments of 10 in the \\  & range [100, 1000] & & & \\  & learning\_rate: 10 values in the range [0.01, 2.0] & & & \\ \cline{1-1}  & max\_depth: 10 values in the range [3; 40] & & & \\ \cline{1-1}  & loss: squared\_error, absolute\_error, & & & \\ \cline{1-1}  & huber, quantile & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 4: _Cont_. combinations in total, which can be seen in Table 4. Using hyperparameter optimization with GA, the population size was started at 100 individuals and the optimization process was carried out for 10 generations, resulting in a total of 1000 evaluated combinations. Although the total theoretical number of combinations was 4000, with the GA, instead of 4000 combinations, a total of 1000 individuals were evaluated, calculated as the sum of the product of the initial population, the number of generations and the population size. The best-performing combination identified by the GA was [criterion = absolute_error, max_depth = 10, max_leaf_nodes = 362, min_samples_leaf = 7]. The hyperparameter optimization of the RF model with the GA was designed to simultaneously explore a total of four different parameter sets. The n_estimators value, which determines the number of trees, was defined as starting from 100 and increasing in increments of 10 up to 1000; the criterion parameter was defined as a decision-tree splitting criterion with four possible values: squared_error, friedman_mse, absolute_error, and poisson. The max_depth parameter was set to have ten different values between 5 and 50, each in increments of five units (5, 10,..., 50). Finally, the min_samples_leaf parameter was defined as containing ten equal values from 1 to 10. This configuration, according to Table 4 while theoretically defining 10 \(\times\) 4 \(\times\) 10 \(\times\) 10 = 4000 possible combinations, allowed the GA to explore the entire solution space in an efficient evolutionary search process within the constraints of population and number of generations. Similar to the DT model, a total of 1000 individuals were evaluated by GA in the RF model. The best-performing combination identified by the GA was [n_estimators = 600, criterion = squared_error, max_depth = 95, min_samples_leaf = 2].