## 2 Materials and Methods
### Model Building  
The predictive models built for this study were based on the Support Vector Machine (SVM), Random Forest (RF), and eXtreme Gradient Boosting (XGBoost) machine learning algorithms. In our previous review of 65 studies, these algorithms were found to be the most widely used for building predictive models [30]. They represent a traditional model (SVM), an ensemble model (RF), and a high-scalability model (XGBoost). We utilized a free version of Google Colab 1.2 to perform comparative analyses.  
SVM is one of the most utilized traditional algorithms for both classification and regression predictive models [31,32]. As a supervised algorithm, SVM generates a hyperplane as a decision boundary for classification. Because our dataset has 2008 instances, it has 2008 data points. The objective function of SVM is as follows [7]:  
\[\arg\min_{w}\left\{\frac{1}{2008}\sum_{i=1}^{2008}\max\{0.1-y_{i}f(x_{i})\}+C _{w}v_{w}\right\}\]  
where \(C\) is the regularization parameter, and \(w\) is the normalization vector. These parameters are important for configuring SVM models. To obtain the optimal hyperplane for an SVM model, the decision function \(f(x)\) is defined as the function. This function measures the similarity between \(x_{i}\) and \(x_{j}\), which can be of various types in SVM models. Consequently, the choice of the kernel type is crucial to tuning the hyper-parameter. Common kernel types in the SVM of this study include linear, radial basis function (RBF), and sigmoid kernels.  
RF is commonly used for classification and regression purposes [33]. RF builds and trains multiple decision trees through a bagging method and combines the predictions made by the trees to obtain a more accurate and stable prediction [33,34]. RF is decision tree-based and has three essential components: a root node, decision nodes, and leaf nodes.  
Figure 2: Bayesian Optimization.  
In this study, we followed the hyper-parameters of RF as found in the scikit-learn library of Python 3.9 [7; 35]. For more accurate decision-making, we configured the tree depth maximum to obtain a deeper tree. The decision and leaf nodes consist of sample split and leaf parameters that can be configured to prevent the model from overfitting possibility. Moreover, to control the number of features considered when looking for the best split at each node, we configured the maximum features parameter. Unlike Decision Tree (DT), RF has a parameter that controls the number of trees in a forest as \(n\) estimators, which we configured to determine the optimal decision trees of the model.  
The other tree-based model is an XGBoost algorithm. This is a popular model that provides improved performance and speed through the use of boosting and gradient descent methods while combining decision trees [19; 36]. We assigned additional parameters to our XGBoost model, including a learning rate that determines the boosting process, a subsample for the ratio of the training instances, and a colsample bytree that represents the subsample ratio while constructing decision trees.  
Table 1 below lists the hyper-parameter configuration we tested to optimize the machine learning algorithms for our experimental setup, which is shown in Figure 3. The listed parameters are based on the algorithm's characteristics.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Algorithm** & **Hyper-Parameter** & **Values** \\ \hline \multirow{3}{*}{Support Vector Machine} & C & 0.1, 1.0, 10, 100, 1000 \\  & Gamma & 1, 0.1, 0.01, 0.001, 0.0001, auto, scale \\ \cline{1-1}  & Kernel & Linear, RBF, Sigmoid \\ \hline \hline \end{tabular}
\end{table}
Table 1: List of configurations for the hyper-parameter of the algorithms.  
Figure 3: Experimental flow chart [17].