## 5 Machine Learning Models
### Evaluation of Machine Learning Model Prediction Performances  
This section discusses models created to predict an output using five input variables, along with their corresponding results. Models created to predict an output using five input variables are discussed with their results in this section. The relevant prediction results are shown in detail in Table 8. The performance of the models in predicting training and testing outcomes was evaluated. The analysis results showed that the classified and regression tree (CART) and extreme gradient boosting (XGBoost) models exhibited the highest R\({}^{2}\) (0.99) and lowest RMSE (0.00) values for plane-strain ratio (PSR) prediction during training. The test results regarding prediction ability were R\({}^{2}\) 0.99 with XGBoost and R\({}^{2}\) 0.98 with CART. RMSE of 0.01 was obtained for XGBoost and 0.02 for CART. However, artificial neural networks (ANNs) and support vector regression (SVR) models demonstrated comparatively lower performance compared with other prediction models. The plane-strain ratio (PSR) prediction and finite element (FE) PSR relationships of the prediction models are illustrated in Figure 14. While the ideal fit is given as a dark blue line, the \(\pm\)20% error rate range is also indicated on the graphics. Among the prediction models, artificial neural networks (ANNs) and support vector regression (SVR) tend to make errors. Therefore, extreme gradient boosting (XGBoost) and classified and regression tree (CART) \begin{table}
\begin{tabular}{c c c} \hline \hline
**ML** & **Parameters** & **Value** \\ \hline \multirow{4}{*}{XGBoost} & Number of estimator & 200 \\  & Maximum depth & 5 \\  & Minimum child weight & 1 \\  & Learning rate & 0.1 \\ \hline \multirow{4}{*}{SVR} & Support vectors & 117 \\  & Cost & 100 \\  & Epsilon & 0.1 \\  & Gamma & 0.01 \\  & Kernel & Radial basis functions \\ \hline \multirow{4}{*}{CART} & Max. depth & None \\  & Min. samples leaf & 2 \\  & Min. samples split & 5 \\ \hline \multirow{2}{*}{ANN} & Hidden layer size & 2 \\  & Neurons & 20 + 20 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Optimized hyperparameters used to predict ML models. \begin{table}
\begin{tabular}{c c c} \hline \hline
**No** & **Parameter** & **Equation** \\ \hline
1 & Coefficient of determination (\(R^{2}\)) & \(R^{2}=\frac{\sum_{i=1}^{n}(y_{i}-y_{nom})^{2}-\sum_{i=1}^{n}(y_{i}-y_{i})^{2}}{ \sum_{i}-(y_{i}-y_{nom})^{2}}\) \\
2 & Mean absolute error (MAE) & \(MAE=\frac{1}{n}\sum\limits_{i=1}^{n}\left(\left|\hat{y}_{i}-y_{i}\right|\right)\) \\
3 & Root-mean-square error (RMSE) & \(RMSE=\sqrt{\frac{1}{N}\sum\limits_{i=1}^{n}\left(y_{i}-y_{i}\right)^{2}}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Performance metrics [62]. The comparison of 10 randomly selected plane-strain ratio (PSR) datapoints, shown in Table 9, with extreme gradient boosting (XGBoost), classified and regression tree (CART), artificial neural networks (ANNs), and support vector regression (SVR) prediction models is shown in Figure 15. It can be seen that the XGBoost and CART prediction models were more consistent with the finite element (FE) and PSR data compared with the other 2 methods (ANN and SVR). Permutation importance analysis was performed on extreme gradient boosting (XGBoost) and classified and regression tree (CART), where the highest performance was achieved. For plane-strain ratio (PSR) estimation, the relative percentage importance of the input parameters distance from the corner (xH), nail length (L/H), excavation depth (H), wall angle (\(\alpha\)), and slope angle (\(\beta\)) were listed as 88, 12, 5, 9, and 9, respectively, according to CART. According to the XGBoost prediction model, the importance percentages were 86%, 1%, 1%, 1%, and 10%, respectively. xH was significantly reflected in both models for PSR prediction, as shown in Figure 16. For the XGBoost model prediction, the \(\beta\) factor was the second-most-important parameter after xH.