## 2 Materials and Methods
### Random Forest
#### 2.2.3 Hyperparameter Optimization  
Hyperparameters are parameters that indicate the algorithm that will be used to build the machine learning model or to minimize the loss function [34]. It is essential to research within the range of possibilities in order to create an optimal machine learning model. Hyperparameter tuning is the process of setting optimal parameters to create an ideal model. One of the most crucial steps in creating an effective machine learning model is tuning the hyperparameter. In specific cases, such as tree-based machine learning models and deep learning models with multiple hyperparameters, its importance can be stated to be bigger [35]. Since the existence of various types of hyperparameters, such as categorical, discrete, and continuous hyperparameters, the hyperparameter tuning method varies by machine learning algorithm [36]. Hyperparameter optimization (HPO) is a technique for automatically optimizing hyperparameters, with the purpose of allowing users to utilize machine learning models to handle specific problems more efficiently by automatinghyperparameter tuning. Equation (5) is the goal of the hyperparameter optimization problem and can be expressed as follows [37]:  
\[x^{*}=\text{argmin}_{\times\in\mathbb{X}}f(x) \tag{5}\]  
The objective function to be minimized is \(f(x)\), and \(x^{*}\) is the construct of hyperparameters that generate the optimal value of \(f(x)\). In the search space \(\mathbb{X}\), there are hyperparameters for which \(x\) can have a value. HPO's goal is to tune hyperparameters so that the model performs optimally or nearly optimally within a given range of values [38]. For each objective function, the mathematical expression of function \(f\) is different. The accuracy, RMSE (Root Mean Square error), and F1-score of the model could all be used to evaluate the performance. To improve a machine learning model with HPO, the designer must first understand the main idea of tuning a machine learning model to a specific problem or dataset type. Normally, machine learning algorithms are divided into two categories: supervised learning and unsupervised learning. Since it is a supervised learning dataset labeled by classifying the occurrence of an accident into a binary classification, we proceeded with HPO, focusing on this point. Both input and output values exist in supervised learning algorithms. The Bayesian Optimization (BO) algorithm is a well-known iterative algorithm for solving HPO problems. BO calculates the future evaluation point based on the previous results and determines the next hyperparameter design using a surrogate model and an acquisition function. The object function of the surrogate model is to fit all of the existing observed points into it. After obtaining the predicted distribution of the probabilistic surrogate model, the acquisition function regulates the use of other points by balancing the trade-off between exploration and exploitation. Exploration entails sampling the instance in previously unexplored places. In contrast, exploitation involves sampling in the currently attractive area where the global optimum is most likely to occur, based on the posterior distribution [34]. By balancing the exploration and exploitation processes, the BO model finds the current area that is most likely optimal and avoids missing better configurations in the unexplored [39]. As surrogate models for HPO, we use two different types of BO algorithms in this paper: BO-TPE and BO-GP. The Gaussian Process (GP) is a general surrogate model for objective function modeling in BO [40; 41]. The prediction follows a normal distribution and can be expressed as Equation (6) as follows: ([PERSON] et al., 2013). \[p(y|x,\,\mathbb{S})=N\Big{(}y\big{|}\hat{\mu},\hat{\sigma}^{2}\Big{)} \tag{6}\]  
where the function \(f\) with a mean \(\mu\) and a covariance \(\sigma^{2}\) is a realization of a GP, \(\mathbb{S}\) is the space which configures the hyperparameters, and \(y=f(x)\) is the evaluation result of each hyperparameter value \(x\).