## 4 Analysis of Evaluation Results
### Evaluation Accuracy Analysis  
To comprehensively evaluate the performance of the model in this study area, we conducted an additional validation analysis by using the receiver operating characteristics (ROC) curve and the area under the curve (AUC) value based on the approach by [PERSON] et al. [57]. We used the AUC value as an indicator of the probabilistic model's quality, signifying its reliability in predicting the occurrence or non-occurrence of events. In general, when the AUC value was lower than 0.5, the model had no prediction ability; when the AUC value was 0.5-0.7, the model possessed lower prediction ability; when the AUC value was 0.7-0.9, the model possessed higher prediction ability; when the AUC values exceeded 0.9, the model had very high prediction ability. The ROC curve is plotted with the true positive rate as the horizontal coordinate and the true negative rate as the vertical coordinate, which satisfy Equations (6) and (7). \[TPR=\frac{TP}{TP+FR}, \tag{6}\]  
Figure 6: Landslide susceptibility map. (**I**): PSO-SVM (**II**): BO-RF (**III**): BO-CNN. \[FPR=\frac{FP}{TN+FP}, \tag{7}\]  
where \(TPR\) denotes the true positive rate; \(FPR\) represents the false positive rate; \(TP\) is the number of true positives; \(FP\) is the number of false positives; \(FR\) expresses the number of false negatives; and \(TN\) is the number of true negatives. The ROC curve and AUC value for this study were computed by using Python statistical software, which is depicted in Figure 7. Our calculations indicated that the ROC curves drawn by the PSO-SVM, BO-RF, and BO-CNN models were all close to the inflection point at the upper left corner, which indicated that all three models can effectively evaluate the landslide susceptibility. The AUC values of PSO-SVM, BO RF, and BO-CNN models also reached 0.9388, 0.9529, and 0.9535, which suggested a commendable level of accuracy for the employed model. Through reclassification in the Spatial Analyst tool in ArcGIS 10.2 software, the raster image elements and the number of landslides in the geologic landslide susceptibility zones of the study area were counted, and the results of landslide hazard susceptibility in the study area were analyzed using the FR method. As depicted in Figure 8, the prediction results of the PSO-SVM, BO-RF, and BO-CNN models were classified into five categories of susceptibility zones, namely, high, high, medium, low, and low, using the natural breakpoint method. The area of landslide susceptibility in the low susceptibility zones of the BO-CNN model accounted for 68.43% of the total area, and the area of the low susceptibility zones was larger than that of BO-RF and PSO-SVM, while the area of the high susceptibility zones accounted for 5.46% of the total area and the area of high susceptibility zones accounted for 5.46% of the total area. Furthermore, the area of the high susceptibility zone was smaller than that of BO-RF and PSO-SVM; the distribution of landslides in each susceptibility zone was relatively the same in each model, and the proportion of landslides in the high susceptibility zones reached 76.10%, 84.39%, and 81.37%, respectively. The frequency ratios of BO-CNN in the high susceptibility zones and higher susceptibility zones exceeded 1, and those in the middle susceptibility zones, lower susceptibility zones, and lower susceptibility zones were less than 1, with the value of the frequency ratio in  
Figure 7: ROC curve and AUC value. the high susceptibility zone being the highest. The frequency ratio of BO-CNN in the high susceptibility zone and the higher susceptibility zone exceeded 1. The frequency ratio of BO-CNN in the middle susceptibility zone, lower susceptibility zone, and low susceptibility zone was less than 1, in which the frequency ratio of the high susceptibility zone was the highest, with a value of 14.90, higher than that of BO-RF and PSO-SVM, whose frequency ratios in the high susceptibility zone were 10.35 and 11.21, respectively.