An advanced method to validate and compare susceptibility maps by investigating local-scale differences and highlighting the role of geomorphological features
Authors: Segoni Samuele , Xiao Ting , Chen Lixia , Yin Kunlong , Casagli Nicola 
Published: March 2020

&lt;p&gt;In landslide studies, comparing the outcomes obtained by different models is a very robust test for their predictive capability and quantitative indexes are often used to assess which model provides the best predictions. The literature about landslide susceptibility is rich of works where two or more susceptibility models are validated in terms of AUC (area under ROC curve), then the AUC values are compared and the model that provided the highest AUC value is considered the best one.&lt;/p&gt;&lt;p&gt;The main purpose of this work is to expand this classical approach, which is too simplistic as it neglects any geomorphological consideration, and to propose a new approach that shifts the comparison at the pixel scale, linking the local-scale differences encountered with specific features of the study area. The proposed advanced comparison approach can be summarized with the following steps:&lt;/p&gt;&lt;ol&gt;&lt;li&gt;The susceptibility maps obtained by different models are compared on a pixel-by-pixel basis to define pixels affected by underestimation (UE) and overestimation (OE) of susceptibility values.&lt;/li&gt;
&lt;li&gt;If present, systematic spatial patterns of UE and OE are identified.&lt;/li&gt;
&lt;li&gt;The patterns are cross-checked with all the explanatory variables used in the susceptibility assessments.&lt;/li&gt;
&lt;li&gt;The lithological and morphological features of the study area that are typically associated to underestimations and overestimations of susceptibility are identified and quantitatively characterized.&lt;/li&gt;
&lt;li&gt;The quantitative information provided by the previous steps is used to provide a geomorphological interpretation of the differences in the susceptibility values provided by the models, thus adding a more robust element to judge which of them should be used in hazard management, and how.&lt;/li&gt;
&lt;/ol&gt;&lt;p&gt;As a case of study, we used four susceptibility maps already defined with random forest (RF), index of entropy (IOE), frequency ratio (FR), and certainty factor (CF) in Wanzhou County (China). A classical validation procedure showed that RF provided the best outcomes, with a 0.801 AUC. After applying the advanced comparison procedure, we obtained deeper insights on the susceptibility models, explaining e.g. why and where RF performed significantly better than the other models and identifying systematic errors that could be associated to distinctive geomorphological features of the test site. Indeed, we discovered that RF is more able to exploit the very complex parameterization of the problem, with 13 parameters, sometimes interrelated each-other, with a total of 80 classes. Moreover, we found that the other models produced systematic errors in correspondence with some lithological units and in fluvial terraces. The area is characterized by 5 orders of relict fluvial terraces, clearly defined only in some small stretches, and the results obtained showed that landsliding has probably been one of the predominant geomorphological process responsible for their depletion.&lt;/p&gt;