## 3 Results of Analysis of Ground Subsidence Risk Levels Using Machine Learning
### Results of Applying Machine Learning
\begin{table}
\begin{tabular}{c c c c c c c c c c c c c} \hline \hline
**Model** & \multicolumn{4}{c}{**RF**} & \multicolumn{4}{c}{**XGB**} & \multicolumn{4}{c}{**LGBM**} \\ \hline \multirow{2}{*}{**No.**} & **Train** & **Test** & **F1-Score** & **AUC** & **Train** & **Test** & **F1-Score** & **AUC** & **Train** & **Test** & **Score** & **AUC** \\  & **Score** & **Score** & **(Macro)** & **Score** & **Score** & **(Macro)** & **Score** & **(Macro)** & **Score** & **Score** & **(Macro)** \\ \hline
1 & 0.676 & 0.626 & 0.560 & 0.790 & 0.648 & 0.608 & 0.560 & 0.770 & 0.716 & 0.628 & 0.560 & 0.790 \\
2 & 0.683 & 0.593 & 0.550 & 0.790 & 0.705 & 0.608 & 0.580 & 0.800 & 0.656 & 0.593 & 0.560 & 0.800 \\
3 & 0.718 & 0.608 & 0.550 & 0.790 & 0.744 & 0.644 & 0.590 & 0.800 & 0.706 & 0.620 & 0.570 & 0.810 \\
4 & 0.662 & 0.624 & 0.560 & 0.790 & 0.679 & 0.585 & 0.510 & 0.770 & 0.656 & 0.582 & 0.520 & 0.790 \\
5 & 0.687 & 0.595 & 0.560 & 0.800 & 0.666 & 0.587 & 0.550 & 0.790 & 0.690 & 0.597 & 0.550 & 0.800 \\
6 & 0.699 & 0.585 & 0.540 & 0.800 & 0.712 & 0.612 & 0.560 & 0.800 & 0.729 & 0.624 & 0.570 & 0.820 \\
7 & 0.671 & 0.603 & 0.530 & 0.760 & 0.655 & 0.580 & 0.520 & 0.770 & 0.668 & 0.580 & 0.520 & 0.770 \\
8 & 0.645 & 0.543 & 0.460 & 0.750 & 0.615 & 0.553 & 0.500 & 0.740 & 0.655 & 0.545 & 0.490 & 0.750 \\
9 & 0.632 & 0.501 & 0.380 & 0.740 & 0.628 & 0.553 & 0.490 & 0.770 & 0.627 & 0.551 & 0.490 & 0.760 \\
10 & 0.682 & 0.597 & 0.520 & 0.770 & 0.660 & 0.578 & 0.520 & 0.760 & 0.669 & 0.578 & 0.510 & 0.770 \\
11 & 0.608 & 0.541 & 0.450 & 0.750 & 0.647 & 0.555 & 0.490 & 0.740 & 0.651 & 0.570 & 0.500 & 0.750 \\
12 & 0.636 & 0.511 & 0.390 & 0.750 & 0.615 & 0.532 & 0.460 & 0.760 & 0.676 & 0.568 & 0.490 & 0.750 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Results of machine learning model (SMOTE applied). \begin{table}
\begin{tabular}{c c c c c c c c c c c c} \hline \hline
**Model** & \multicolumn{4}{c}{**RF**} & \multicolumn{4}{c}{**XGB**} & \multicolumn{4}{c}{**LGBM**} \\ \hline \multirow{2}{*}{**No. **} & **Train** & **Test** & **F1-Score** & **AUC** & **Train** & **Test** & **F1-Score** & **AUC** & **Train** & **Test** & **Score** & **F1-Score** & **AUC** \\  & **Score** & **Score** & **(Macro)** & **Score** & **Score** & **(Macro)** & **Score** & **(Macro)** & **Score** & **Score** & **(Macro)** \\ \hline
1 & 0.742 & 0.670 & 0.450 & 0.780 & 0.725 & 0.668 & 0.480 & 0.770 & 0.765 & 0.676 & 0.480 & 0.800 \\
2 & 0.766 & 0.64 In addition, the tuning of the hyperparameters of each classifier was set to the hyperparameter that produces the optimal result using a trial-and-error method. Table 7 summarizes the main hyperparameters of the selected XGB model. The XGB model included a function to derive the importance of the input data employed in the process of solving the classification problem. Using this function, we selected the main influencing factors used to classify the ground subsidence risk levels. Figure 3 shows a graph that exhibits the importance of the factors used in the model, in which Y refers to the number of years used and DTR refers to the pipeline diameter. Density was the most importantly used factor in the classification of ground subsidence risk levels in the XGB model. The number of years used was found to be more important than the diameter of the pipeline. In pipelines used between 20 and 40 years, it was found to be relatively more important. \begin{table}
\begin{tabular}{c c} \hline \hline
**Model** & **Hyper Parameter** \\ \hline XGB & Estimators (300), learning rate (0.002), max depth (4) \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Summary of hyperparameters in the model. Figure 3: Importance of influencing factors in the XGB model. Figure 2: ROC Curves of XGB Model.