## 3 Materials and Methods
### Landslide Susceptibility Assessment
#### 3.3.3 Assessing Predictive Performance  
The Receiver Operating Characteristic (ROC) curve is a graphical representation used to evaluate the performance of a binary classification model. At each step, the True Positive Rate (TPR) and the False Positive Rate (FPR) were calculated, and they were plotted as the vertical and horizontal coordinates, respectively, to obtain the \"ROC curve\". It uses the True Positive Rate (TPR) on the vertical axis and the False Positive Rate (FPR) on the horizontal axis. Other performance metrics derived from the confusion matrix are presented in Equations (4)-(7). The components of the confusion matrix are defined as follows:  
* _TP_ (True Positives): the number of correctly predicted landslide instances;
* _FP_ (False Positives): the number of non-landslide instances incorrectly predicted as landslides;
* _TN_ (True Negatives): the number of correctly predicted non-landslide instances;\(FN\) (False Negatives): the number of landslide instances incorrectly predicted as non-landslides.  
\[Precision=\frac{TP}{TP+FP} \tag{4}\]  
\[Recall=\frac{TP}{TP+FN} \tag{5}\]  
\[Accuracy=\frac{TP+TN}{TP+TN+FP+FN} \tag{6}\]  
\[F_{1}=2.\frac{Precision\cdot Recall}{Precision+Recall} \tag{7}\]  
Five-fold cross-validation was used to evaluate model performance. The dataset was partitioned into five equal or nearly equal subsets, with each subset used once as the test set, while the remaining subsets were used for training [42]. To ensure representative fold composition, K-means clustering was applied to group data into distinct clusters based on feature similarity prior to partitioning [43; 44]. K-means-based 5-fold cross-validation is well-suited for landslide susceptibility assessment because it accounts for the inherent spatial heterogeneity of landslide occurrences; validating the model on spatially disjoint clusters tests its ability to generalize across different terrain characteristics, reflecting real-world scenarios where predictions are applied to new, unseen areas [14].  
Model performance was evaluated using two metrics: area under the receiver operating characteristic curve (AUROC) and accuracy. AUROC measures the ability of a classifier to distinguish between classes, with a value of 1 indicating perfect classification and 0.5 indicating random guessing. Accuracy represents the proportion of correctly classified instances among all samples and was chosen as a complementary metric to provide an intuitive measure of overall correctness. The calculation of accuracy is given in Equation (6).  
The cross-validation procedure was repeated 20 times to obtain stable performance estimates. The resulting data were used for subsequent statistical significance analyses. Statistical significance of performance differences was assessed to determine if observed variations were attributable to factors beyond random variability. The Mann-Whitney U test was used to compare performance between different sampling methods, while the Kruskal-Wallis one-way ANOVA test was applied to compare multiple susceptibility models. To control for false discoveries arising from multiple comparisons, the Benjamini-Hochberg procedure was used to control the false discovery rate (FDR).  
This evaluation framework is critical not only for quantifying model performance but also for ensuring that model improvements are statistically meaningful. It supports the reliability and reproducibility of the susceptibility assessment, making the findings applicable to real-world risk management and decision-making scenarios.  
The modeling and performance metric calculations were conducted using MATLAB R2023a. All statistical significance analyses were performed using IBM SPSS Statistics 27.