## 2 Materials and Methods
### Random Forest  
Random forest is a supervised ensemble algorithm that generates a large number of sample data obtained by the bootstrap sampling with the replacement method. It is a technique that creates multiple decision tree models by randomly designating independent variables to be used in the predictive model for multiple sample data and synthesizing the output to create final results. It provides better performance than bagging trees because it uses a method that removes the correlation of trees. In addition, because the number of decision tree models constituting the random forest is large, the overfitting problem is less compared to other techniques [29]. The gradient boosting (GB) model is based on boosting and a powerful ensemble model to perform classification or regression. It uses the gradient as a loss function that measures how efficiently the model coefficient fits the data. Since GB creates predictive models in the form of ensembles of several simple decision tree models, it is possible to inherit all the advantages of decision tree models, as well as to improve in several aspects, such as robustness and accuracy. In addition, it is a model with various advantages, such as the capability of handling large datasets without preprocessing, resistance to outliers, the capability of handling missing data, robustness to complex data, and resistance to overfitting. The process of the GB model starts with fitting the data with a simple decision tree model. When this simple decision tree has a certain level of error in terms of fit, and the error has the same correlation with the result value, then the GB model develops a different decision tree model from the errors or residuals of the previous tree [29; 30]. XGBoost  
\"Extreme Gradient Boosting\" is how XGBoost obtained its name. As the name suggests, The XGBoost model is a machine learning model that is a decision tree-based ensemble approach, especially as a boosting method; as the name implies, when Equation (1) has a dataset, D contains n examples and n characteristics. \[D=\{(x_{i},y_{i})\}(|D|=n,\ x_{i}\in\mathbb{R}^{m},\ y_{i}\in\mathbb{R}) \tag{1}\]  
Equation (2) defines a tree ensemble model that uses the ensemble \(K\) additive functions to estimate the output:  
\[\hat{y_{i}}=\phi(x_{i})=\sum_{k=1}^{K}f_{k}(x_{i}),\ where\ f_{k}\in\mathcal{F} \tag{2}\]  
where the set of regression trees is \(\mathcal{F}=\Big{\{}f(x)=w_{q(x)}\Big{\}}\big{(}q:\mathbb{R}^{m}\to T,w\in \mathbb{R}^{T}\big{)}\), each tree structure in which data are mapped to the corresponding leaf index is designated by the letter \(q\). \(T\) is the number of leaves in the tree, whereas \(f_{k}\) denotes the independent tree structure of \(q\) and the leaf weight \(w\). Each leaf of a regression tree has a continuous score, and \(w_{i}\) displays the \(i\)-th score. To categorize a leaf in the tree, the tree's decision rule (provided by \(q\)) is used to add the scores from the associated leaves (supplied by \(w\)) and then compute the prediction. It is following regularized objective to learn the function set utilized in the model. \[Obj=\sum_{i}l(y_{i},\ y_{i})+\sum_{k}\Omega(f_{k}) \tag{3}\]  
\[where\ \Omega(f_{t})=\gamma T+\frac{1}{2}\lambda\sum_{j}w_{j}^{2} \tag{4}\]  
As shown in Equation (3), the set of functions used in the model can be learned by following the normalized objective function. In this equation, \(l\) is a differentiable convex loss function that measures the difference between the prediction \(y_{i}\) and target \(y_{i}\). The \(\Omega\) in Equation (4) penalizes the model, so as to not make it too complex. To minimize the formula above, an extra regularization component inhibits overfitting by smoothing the final learning weights. As a result, a model with simple and predictable functions is more likely to be chosen by a regularized objective [31; 32]. To summarize, XGBoost is a distributed processing technique that utilizes a greedy approach to identify a classifier and appropriate parameters at a fast speed.