## 2 Materials and Methods
### Random Hyperparameter Search  
To find the optimal combination of hyperparameters, the RHS method was used. To develop this method for GPSR, the initial testing of each hyperparameter was required. The GPSR hyperparameter ranges defined after initial testing are listed in Table 3.  
In the case of GPSR, there were four parameters that were most-influential: the size of the population (population_size), the number of possible generations (number_of_generations), the tree depth (init_depth), and finally, the parsimony coefficient [23]. The population size and the number of possible generations are highly correlated hyperparameters. If the size of the population and number of generations is too large, it can lead to a very long execution time. The ranges shown in Table 3 proved to be optimal. Initially, init_depth was increased, but the GPSR execution showed that higher values of the init_depth hyperparameter can lead to longer execution times without any benefit to the estimation accuracy of the obtained SE. As the stated parsimony_coefficient is the most-crucial and most-sensitive hyperparameter to define, the range is defined by trial and error, so the range shown in Table 3 will prevent the occurrence of bloat phenomena, but will allow the stable growth of the population members.  
The initial assumption was that crossover or subtree mutation will be the most-influential genetic operation in the evolution process. However, an initial investigation showed that very high values of these two genetic operations can lead to local minimums, i.e., the fitness function value is constant, while the size of the population members grows rapidly. Therefore, the value of all four genetic operations was set to the 0.001-1 range. In this way, the sum of all possible genetic operations was nowhere near one, so some tournament selection winners entered the next generation unchanged.  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Hyperparameter Name** & **Range** \\ \hline population\_size & 1000–2000 \\ \hline number of generations & 100–200 \\ \hline tournament\_selection & 10–500 \\ \hline init\_depth & 3–15 \\ \hline crossover & 0.001–1.0 \\ \hline subtree\_mutation & 0.001–1.0 \\ \hline hoist\_mutation & 0.001–1.0 \\ \hline point\_mutation & 0.001–1.0 \\ \hline stopping\_criteria & 0–\(1\times 10^{-8}\) \\ \hline maximum\_samples & 0.99–1 \\ \hline constant\_range & –10,000–10,000 \\ \hline parsimony\_coefficient & 0–\(1\times 10^{-4}\) \\ \hline \hline \end{tabular}
\end{table}
Table 3: The range of GPSR hyperparameters.  
It should be noted that each GPSR execution is terminated after reaching the defined number of maximum generations. As described, the GPSR algorithm has two termination criteria, i.e., the maximum number of generations and the stopping criteria. However, the stopping criterion, i.e., the value of the fitness function, was set to an extremely small value to ensure that the execution of the GPSR algorithm is terminated after reaching the maximum number of generations.  
The maximum samples were set in the 0.99 to 1 range, so each time the fitness function was computed, almost the entire training dataset was used. If the value of the maximum samples was set to 1, then during the GPSR execution, the out-of-bag raw (OOB) fitness would not be shown. OOB fitness refers to the fitness of each population member in held-out samples. To see OOB fitness during execution, the maximum samples must be less than 1. To ensure that the population's growth is stable, a relatively large range of constants was provided to ensure that the small range of constants can lead to the large growth of members.