## 6 Model Results
### Comparing with Other Machine Learning Models  
In order to fully demonstrate the capabilities of the BO-XGBoost algorithm, a well-known ensemble learning technique, this study also incorporates two widely used individual learning methods, namely KNN and Support Vector Regression (SVR). The procedure for establishing the model parameters is analogous to the one applied for BO-XGBoost. This involves setting preliminary values for these parameters, followed by the utilization of BayesSearchCV to identify the best possible values. SVRs are a type of machine learning model used for both classification and regression tasks. They work by finding the hyperplane (a subspace of one dimension less than the ambient space) that best separates the \begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**Folds**} & \multicolumn{3}{c}{**Performance Measures**} \\ \cline{2-4}  & **RMSE (MPa)** & **R\({}^{2}\)** & **MAE (MPa)** \\ \hline Fold 1 & 2.1534 & 0.9724 & 1.6802 \\ Fold 2 & 2.1328 & 0.9772 & 1.7275 \\ Fold 3 & 2.9256 & 0.9598 & 2.0709 \\ Fold 4 & 2.0121 & 0.9614 & 1.4167 \\ Fold 5 & 1.8548 & 0.9819 & 1.4967 \\ Average & 2.2158 & 0.9705 & 1.6802 \\ SD & 0.4142 & 0.0097 & 0.254 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Results derived from a five-fold cross-validation. Figure 6: Outcomes from a five-fold cross-validation process. classes of data, maximizing the margin between the closest points (the \"support vectors\") of different classes. The best hyperparameters to optimize in SVR are the regularization parameter \"C\", the kernel type, the gamma value if using the 'rbf' or 'poly' kernel, the degree of the polynomial for the 'poly' kernel, and the epsilon parameter. The penalty coefficient, often referred to as \"C\" in machine learning, especially in algorithms like SVM and SVR, is a regularization parameter that controls the trade-off between achieving a low training error and a low testing error, that is, the ability to generalize the model to unseen data. In the context of Support Vector Machines (SVM) and SVR, the kernel type refers to the function used to transform the input data into a higher-dimensional space, making it easier to classify or fit a regression line to the data. This process is known as the kernel trick. In the context of the Radial Basis Function (RBF) kernel and the Polynomial kernel, gamma defines how far the influence of a single training example reaches, which in turn affects the flexibility of the decision boundary. On other hand, KNN is a type of instance-based learning, or lazy learning, where the function is only approximated locally and all computation is deferred until prediction. The KNN works by finding a predefined number of training samples closest in distance to a new point and predicting the label from these. The label assigned to the new instance is computed using the mean of the labels of its nearest neighbors. Here are the most important parameters to tune in KNeighborsRegressor:  
* \(n\_\)neighbors: The number of neighbors to use for prediction. This is typically the '\(K\)' in KNN. A small value of '\(k\)' means that noise will have a higher influence on the result and a large value will make it computationally expensive. * weights: The weight function used in prediction. It can take the values of 'uniform', 'distance' or any other user-defined function. 'Uniform' means all points in each neighborhood are weighted equally, whereas 'distance' means points are weighted by the inverse of their distance. * algorithm: The algorithm used to compute the nearest neighbors. It can be 'ball_tree', 'kd_tree', 'brute' or 'auto'. If 'auto' is chosen, the algorithm tries to decide the most appropriate algorithm based on the values passed to the fit method. * leaf_size: Leaf size passed to BallTree or KDTree. This can affect the speed and memory usage of the construction and query, as well as the memory required to store the tree. * \(p\): Power parameter for the Minkowski metric.