## 4 Experimental Results
### Cross-Validate the Prediction Patterns  
A cross-validation strategy was used to obtain and characterize the patterns as predictors. Iterative cross validation by the sequential exclusion of seven was tentatively selected out of the 70 active-landslide trigger zones in the DSP. It was computed to obtain the corresponding prediction-rate tables. The tables associate cumulative proportions of equal-area ranks of the prediction pattern with the corresponding cumulative proportions of validation occurrences. From the tables, cumulative curves are obtained, as shown in Figure 4. The illustration provides the prediction-rate curves from the iterative process using the five models. The horizontal axis shows the proportion of study area as cumulative equal-area ranks, each of 0.5% of the study area, i.e., \(\approx\)3230 pixels. The vertical axis shows the corresponding cumulative proportion of occurrence pixels in the cumulative class. The 697 occurrence pixels are distributed over the **70 at** landslides so that the proportion of occurrence corresponds to the pixel numbers in each.  
Note that the curves in Figure 4b represent proportions of seven occurrences, while the thick red curve represents the proportion of all 70 occurrences (curve **FZ_70 atm7_ulpACDIS**, where m indicates minus).  
The widespread distribution of the curves is indicative of the uncertainty affecting their aggregated thick red curve. What is predicted here is the likelihood of the \"next\" seven occurrences. In the iterations, they are assumed to be unknown. In all iterations, 63 occurrences are used to generate a prediction pattern that is then cross-validated as a predictor of the remaining seven occurrences. Note also that the proportions shown on the vertical axis are three orders of magnitude smaller than those on the horizontal axis (proportions of 697 pixels versus proportions of 646,091 pixels). As a reminder, the vertical axis was kept half the length of the horizontal axis.  
We can observe that the prediction-rate curves are not particularly good. For instance, the thick red curve in Figure 4a shows that the top 10% ranks (0 to 0.1) contain about 14% of the occurrences for FZ, LI, and LO, and 22% for LR and BP; the top 20% ranks contain around 50% of the occurrences for FZ, LR, and BP, and about 26% for LI and LO; the top 30% ranks contain 96%, 91%, and 67% for BP, LR, and FZ, respectively, and about 57% for LI and LO.  
These relative proportions represent the predictive capability of the modeling results: the proportion of predicted occurrences in the corresponding equal-area ranks. The shallow initial part of the curves is a sign of poor congruity of the setting of the landslide trigger areas used for cross validation. A good prediction pattern should provide an initially steep prediction-rate curve through cross validation in which most of the validation occurrences fall on higher ranks. The curves in Figure 4a show that the FZ, BP, and LR patterns of Figure 3 predict better than those for LI and LO. Figure 4b shows, in addition to the FZ prediction-rate curve identified with a thick red line (**FZ_70 atm7_ulpACDIS**), the curves for each of the 10 iterations of the FZ model.  
Note that a shallow initial curve is the one from the fourth iteration. Recall that in the iterations the cumulative proportions of occurrences in the diagram refers to the seven occurrences being cross-validated, while the proportions for the aggregated FZ curve refer to the 70 occurrences, validated into successive 10 groups of 7. What is being predicted here are the \"next\" 7 occurrences using the \"previous\" 63. This is the best approach when not having the time of the occurrences.