# Enhancing Predictive Accuracy of Landslide Susceptibility via Machine Learning Optimization
## 4 Results
### Validation and Comparison of Models  
The success rate curve (SRC) under the training dataset and the prediction rate curve (PRC) under the validation dataset were used to complete the model comparison and validation process in this study. As a quantitative indicator, the AUC can reflect the goodness-of-fit and accuracy of the landslide susceptibility model. In the training dataset, the AUC values of the four landslide susceptibility models were 0.746 for MLP, 0.648 for NB, 0.839 for the CDT, and 0.868 for the RF (Figure 12). In the validation dataset, the corresponding AUC values of the four landslide susceptibility models MLP, NB, CDT, and RF were 0.806, 0.639, 0.520, and 0.751, respectively (Figure 13).  
Figure 12: ROC curves of the models using the training dataset. **(a)** MLP model—Area Under the Curve (AUC) = 0.746, Standard Error = 0.044, 95% Confidence Interval: 0.659 to 0.820, \(p\) < 0.0001; **(b)** NB model—AUC = 0.648, Standard Error = 0.0498, 95% Confidence Interval: 0.556 to 0.732, \(p\) = 0.0030; **(c)** CDT model—AUC = 0.839, Standard Error = 0.0354, 95% Confidence Interval: 0.762 to 0.899, \(p\) < 0.0001; **(d)** RF model—AUC = 0.868, Standard Error = 0.0351, 95% Confidence Interval: 0.795 to 0.923, \(p\) < 0.0001. The solid blue lines represent the ROC curves of the models. The dashed lines indicate the 95% confidence intervals of the ROC curves, while the diagonal reference line (gray) denotes the line of no discrimination (AUC = 0.5), used as a baseline for comparison.  
At the same time, some statistical indicators (standard error, 95% confidence interval, z statistic, and significance level P) were introduced to compare and verify the performances of the models. The standard error values corresponding to the MLP, NB, CDT, and RF models under the training dataset were 0.0444, 0.0498, 0.0354, and 0.0351, respectively. For the validation dataset, the corresponding standard deviation values of the MLP, NB, CDT, and RF models were 0.0594, 0.0774, 0.0760, and 0.0671, respectively.  
The 95% CIs of the MLP, NB, CDT, and RF models were 0.659-0.820, 0.556-0.732, 0.762-0.899, and 0.795-0.923, respectively, under the training dataset. In the validation dataset, the 95% CIs of the MLP, NB, CDT, and RF models were 0.673-0.903, 0.494-0.768, 0.337-0.661, and 0.612-0.861, respectively.  
Figure 13: ROC curves of the models using the validation dataset. **(a)** MLP model—Area Under the Curve (AUC) = 0.806, Standard Error = 0.0594, 95% Confidence Interval: 0.673 to 0.903, \(p\) < 0.0001; **(b)** NB model—AUC = 0.639, Standard Error = 0.0774, 95% Confidence Interval: 0.494 to 0.768, \(p\) = 0.0723; **(c)** CDT model—AUC = 0.520, Standard Error = 0.0760, 95% Confidence Interval: 0.377 to 0.661, \(p\) = 0.7926; **(d)** RF model—AUC = 0.751, Standard Error = 0.0671, 95% Confidence Interval: 0.612 to 0.861, \(p\) = 0.0002. The solid green lines represent the ROC curves of the models. The dashed lines indicate the 95% confidence intervals of the ROC curves, while the diagonal reference line (gray) denotes the line of no discrimination (AUC = 0.5), used as a baseline for comparison.  
In the training dataset, the Z-values of the MLP, NB, CDT, and RF models were 5.532, 2.966, 9.570, and 10.503, respectively. Meanwhile, the Z-values of the MLP, NB, CDT, and RF models were 5.135, 1.797, 0.263, and 3.748, respectively, under the validation dataset.  
Except for the NB model, the \(p\)-values (significance level \(p\)) of the other three models (MLP, CDT, and RF) were all below 0.0001 under the training dataset. In the validation dataset, only the MLP model showed a small \(p\)-value (\(p<0.0001\)).