## 2 Materials and Methods
### Hyperparameter Tuning
#### 2.4.2 The Hyperparameter Space  
To begin with, we tuned the hyperparameters of KNN, including the distance metric, the value of k, and the distance weighting. The reasons why we tuned these hyperparameters are as follows. First, different distance metrics mean different ways to calculate the distance between the training samples and test point. Second, the KNN will be sensitive to noise if the value of k is too small. However, on the other hand, the accuracy will reduce if the value of k is too large, because it considers datapoints that are too far away [35]. Finally, distance weighting is used to determine in which the datapoints are treated [44].  
To sum up, Table 2 shows the hyperparameter space of KNN in detail.  
Next, we tuned the hyperparameters of CART, including the splitting criterion, the size of the tree, and the minimum node size. The reasons why we tuned these hyperparameters are as follows. First, using different splitting criterions such as mean squared error criterion or mean absolute error criterion might build different CART models, and those criterions will set different predicted values of the terminal nodes [31]. In addition, the size of the tree can affect the CART model's complexity [26], and it can prevent the tree from becoming a fully grown tree or maximal tree and stop the tree from overfitting [37]. Furthermore, the minimum node size refers to the minimum number of samples required to split an internal node, i.e., the tree will grow until a given minimum node size is reached; thus, the CART model can be pruned to an optimal size with this hyperparameter [37]. In practice, the minimum node size is often set to 10% of the number of training samples [38].  
In short, Table 3 shows the hyperparameter space of CART in detail.  
Furthermore, we tuned the hyperparameters of RF, including the number of trees in the forest, the size of the random subset of features, the splitting criterion, the size of the tree, and the minimum node size. The reasons why we tuned these hyperparameters are as follows.  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Hyperparameter** & **Hyperparameter Candidates** \\ \hline \hline \multirow{2}{*}{Distance metric} & Euclidean distance \\ \cline{2-3}  & Manhattan distance \\ \hline \multirow{4}{*}{The value of k} & minimal value: 1 \\ \cline{2-3}  & maximal value: 338 \\ \cline{1-1} \cline{2-3}  & (the number of training samples) \\ \cline{1-1} \cline{2-3}  & numbers of steps: 1 \\ \hline \multirow{2}{*}{Distance weighting} & uniform weights \\ \cline{1-1}  & (each neighbor is weighted equally) \\ \cline{1-1} \cline{2-3}  & weighting by the inverse of neighbors’ distance \\ \hline \hline \end{tabular}
\end{table}
Table 2: The hyperparameter space of KNN.  
In general, the larger the number of trees, the better the performance will be. However, the performance will stop improving when the number of trees increases beyond a critical number [31]. Although the size of the bootstrap sample is often set to the number of original training samples [16], it is important to tune the size of the random subset of features, because they can control the degree of the randomization and reduce the variance [41]. In practice, the size of the random subset of features is usually set to the square root of the number of features [26; 35] or the number of features divided by three [39]. In order to achieve the best performance, the hyperparameters of CART should be tuned, including the splitting criterion, the size of the tree, and the minimum node size [42].  
As mentioned above, Table 4 shows the hyperparameter space of RFs in detail.