## 4 Surrounding Rock Grade Identification Based on Optuna-XGBoost
### The Metrics for Evaluating Models  
When evaluating machine learning models, using graphical methods for assessment is merely an intuitive approach rather than a rigorous mathematical expression. Therefore, it is essential to select appropriate evaluation metrics to identify and choose the best-performing models. These evaluation metrics, as shown in Table 4, are commonly used to judge the quality of predictive models [40]. It is important to note that different evaluation metrics handle prediction errors differently. Hence, when choosing evaluation metrics, one must consider the characteristics of the data, as well as specific scenarios and business requirements.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Name** & **Title 2** & **Title 3** \\ \hline \hline \multirow{7}{*}{Accuracy} & \multirow{7}{*}{Accuracy =\(\frac{TP}{TP+FP+TN}\)} & True positive (\(TP\)) indicates number correctly predicted to be in the positive category \\  & & False positive (\(FP\)) indicates number of incorrectly predicted \\  & & positive classes \\  & & True negative (\(TN\)) indicates number of correctly predicted \\  & & negative categories \\  & & False negative (\(FN\)) indicates number of incorrectly predicted \\  & & negative classes \\  & & Accuracy represents proportion of correctly categorized samples to \\  & & all samples \\ \hline \multirow{2}{*}{Precision} & \multirow{2}{*}{Precision =\(\frac{TP}{TP+FP}\)} & The parameters are defined as above. Precision represents the proportion of the number of correctly predicted positives to the total number of positive predictions made. \\ \hline \multirow{2}{*}{Recall} & \multirow{2}{*}{Recall =\(\frac{TP}{TP+FN}\)} & The parameters are defined as above. Recall represents the proportion of the number of correctly predicted positives to the total number of actual positives. \\ \hline \multirow{2}{*}{F1} & \multirow{2}{*}{\(F1=2\times\frac{\text{Precision}\times\text{Recall}}{\text{Precision}+\text{Recall}}\)} & The parameters are defined as above. F1 represents the harmonic mean of precision and recall and is used to measure the accuracy of the model. \\ \hline \multirow{2}{*}{\
\begin{tabular}{c} Confusion matrix \\ \end{tabular} } & \multirow{2}{*}{
\begin{tabular}{c} Comprised of four main \\ components: \(TP\), \(FP\), \(TN\), and \(FN\) \\ \end{tabular} } & The parameters are defined as above. Confusion matrix is used to assess the performance of classification models and represents the relationship between the predicted results for each category and the actual categories. \\ \hline \multirow{7}{*}{ Kappa} & \multirow{2}{*}{\(Kappa=\frac{P_{a}-P_{c}}{1-P_{c}}\)} & \(P_{0}\) indicates proportion correctly predicted by the model \\  & & \(P_{e}\) indicates expected accuracy of the model’s stochastic predictions \\ \cline{1-1}  & & Kappa represents a statistical measure used to evaluate the performance of a classifier, especially in cases of imbalanced data. This measure takes into account the effect of random agreement, providing a performance assessment that goes beyond mere accuracy. The value 1 indicates perfect agreement. The value 0 indicates that the observed agreement is no better than the random prediction. Negative values indicate agreement that is worse than the random prediction. \\ \hline \hline \end{tabular}
\end{table}
Table 4: The metrics for evaluating models.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Algorithm** & **Hyperparameter** & **Search Space Best** & **Optimal Hyperparameter** \\ \hline \multirow{7}{*}{XGBoost} & n\_estimators & [5, 300] & 204 \\  & learning\_rate & [0.01, 1] & 0.2018 \\  & max\_depth & [3, 50] & 9 \\  & subsample & [0.7, 1.0] & 0.4001 \\  & colsample\_bytree & [0.7, 1.0] & 0.4221 \\  & reg\_alpha & [10\({}^{-8}\), 10] & 0.0024 \\  & reg\_lambda & [10\({}^{-8}\), 10] & 7.01 \(\times\) 10\({}^{-7}\) \\ \hline \hline \end{tabular}
\end{table}
Table 3: Hyperparameter optimization of XGBoost.