# Performance Analysis of Logistic Model Tree-Based Ensemble Learning Algorithms for Landslide Susceptibility Mapping

Authors: Panpan Yang; Nianqin Wang; Youjin Guo; Xiao Ma; Chao Wang
Journal: Journal of Sensors
 (2022)

---

## Abstract

Landslide susceptibility prediction (LSP) is the key technology in landslide monitoring, warning, and evaluation. In recent years, a lot of research on LSP has focused on machine learning algorithms, and the ensemble learning algorithm is a new direction to build the optimal prediction. Logistic model tree (LMT) combines the advantages of decision tree and logistic regression, which is smaller and more robust than ordinary algorithms. The main aim of this study is to construct and test LMT-based random forest (RF) and selected ensemble learning algorithms including bagging and boosting algorithms to compare their performance. Firstly, taking the county of Ziyang, China, as the study area, through historical reports, aerial-photo interpretations, and field investigations, 690 inventory maps of landslide locations were constructed and randomly divided into the 70/30 ratio for a training and validation dataset. Secondly, considering geological conditions, and landslide-induced disease and its characteristics, 14 landslide-conditioning factors was selected. Thirdly, the variance-inflation factor (VIF) and tolerance (TOL) were used to analyze the 14 factors, and the prediction ability was calculated with information-gain technology. Ultimately, the receiver-operating-characteristic (ROC) curve was applied to verify and compare model performance. Results showed that the LMT-RF model (0.897) was superior to other models, and the performance of LMT single model (0.791) was the worst. Therefore, it can be inferred that the LMT-RF model is a promising model, and the outcome of this study will be useful to planners and scientists in landslide sensitivity studies in similar situations.


---

## Body

## 1. Introduction

At present, the rapid development of urbanization has put pressure on the geological environment, and geological disasters frequently occur. Landslides cause at least 17% of global deaths toll from natural disasters, and they are a type of geological disaster affected by the control of multiple environmental factors, seriously affecting the safety of human life and property [1–3]. To effectively avoid landslides and reduce subsequent losses, landslide-risk assessment and management have been the focus of much attention [4].Landslide-risk evaluation is a significant work [5, 6]. Landslides are monitored by many factors, which build the model with geological-environment variables, and landslide characteristics and their influencing factors, so it is difficult to effectively guarantee the quality of landslide-susceptibility research [7–9]. Therefore, improving the prediction ability of landslide-susceptibility research is an urgent problem.The landslide susceptibility evaluation model mainly includes deterministic and nondeterministic [10]. The deterministic model is based on the principle of slope instability and requires masses of known data, this method needs to be highly simplified and easy to analyze, and it is not suitable for the large-scale research of the LSP [11, 12]. The nondeterministic model is based on statistical analysis, with the maturity of GIS technology and rapid computer development; simple algorithms include information model [13], weight-of-evidence model [14], and the analytic-hierarchy process [15]. With the rise of data mining, some more sophisticated algorithms have gradually been used in landslide-susceptibility research, such as the decision-tree model [16], support-vector-machine model [17], and artificial neural networks [18]. These machine learning methods are useful for analyzing the problem of nonlinear geological spatial distribution and simulate the intricate relationship between landslide and factors, but the pursuit of better predictive ability is still the key to landslide-susceptibility research [19, 20].In recent years, ensemble learning algorithms has received extensive attention to process large amounts of high-dimensional data and improve the model prediction [21]. The integrated frameworks of bagging [22], AdaBoost [23], MultiBoost [24], random forest (RF) [25], rotating forest [26], and random subspace [27] are based on the C4.5 decision tree with minimal experience risk as to the base classifier, but it is easy to overfit the training dataset. Some scholars use support vector machines as the basic classifier of the ensemble learning structure, which can avoid overfitting the training dataset, but reduce the ability of the ensemble learning framework to interpret the results [28]. The LMT is an extended algorithm, and it combines the common decision tree with the logistic regression model in the same tree, which is useful for improving classification accuracy and interpretation ability [29]. Some scholars [30] proposed an integrated bagging model with LMT as the base classifier. Compared with the support-vector-machine and LMT, the hybrid model has higher classification accuracy and prediction ability. Therefore, to improve landslide-prediction performance, hybrid models of ensemble learning algorithms are needed for further research.Given the above literature review, the main purpose is to develop a novel model, LMT-based ensemble learning algorithms. This is a hybrid approach of LMT and ensemble learning algorithms for LSP. The method was applied to the landslide susceptibility study in Ziyang County, Shaanxi Province, China, for the first time. Several ensemble learning algorithms (RF, bagging, and boosting) were selected to combine with LMT, and their performance was compared and analyzed, including a single LMT model; the main differences between research and the literature in Ziyang County were obtained. At last, the results were verified by the ROC curve. The combined model of LMT and ensemble learning algorithms can effectively improve the predictive ability, while the single model is poor.


## 2. General Regional Situation

Ziyang County is a subordinate to the city of Ankang in the southern Shaanxi province. It lies in upper reaches of the Han River and northern foot of the Daba mountain. It is close to Hanbin district and the counties of Langao in the east, Zhenba in the west, Chengkou and Wanyuan of the city of Chongqing in the south, and Hanyin in the north.It has an area of about 2204 km2, which lies between longitude 108°06′ to 108°43′ and latitude 32°08′ to 32°49′. The study area has a subtropical continental monsoon climate, and the average temperature and rainfall are 15.1°C and1054 mm, respectively. The study area presents the “three mountains (Daba, Micang, and Phoenix) two valleys (Han and Ren river valleys), and one river (Haoping river channel)” topography contour.The study area spans the Yangtze platform and Qinling geosyncline that are bounded by the Raofeng–Maliuba fault, Qinling fold system in the north, and Daba mountain uplift fold belt in the south. Affected by earthquakes and regional neotectonic uplift movements, the crust is frequently intermittently uplifted, rivers are cut down, slopes and valleys are deep, stratum folds are strongly deformed, and joint fractures develop. At present, there are 721 geological disasters in Ziyang, including 690 landslides (Figure1).Figure 1
Landslide inventory map and location.


## 3. Spatial Database and Methods

### 3.1. Landslide-Conditioning Factors

Landslides are affected by many factors, and a comprehensive, scientific, and rational selection of Landslide-Conditioning factors is essential [31, 32]. According to previous experience combined with geological-environment conditions, landslide development characteristics, and landslide-induced factors, 14 factors were selected. An elevation map was obtained by the Geographic Data Cloud of Chinese Academy of Sciences (30×30m), and the digital elevation model (DEM) image was processed by ArcGIS software to obtain terrain relief, slope aspect, curvature, terrain roughness, slope angle, TWI and elevation. The geological map was used to extract lithology and fault distribution, and Euclidean distance analysis of the fault obtained the distance from the landslide point to each fault. Landsat-8 images were obtained from the same DEM and used in ArcGIS software to obtain land use and NDVI. To make the rainfall map by multiyear annual precipitation, the road-network and river-system maps were vectorized, and the distance from the landslide point to each element was obtained by Euclidean distance analysis. These landslide-conditioning factors are shown in Figure 2.Figure 2
Landslide-conditioning factors.
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)(k)(l)(m)(n)Elevation has a certain effect on slope deformation and failure [33]. There are differences in rainfall, rock and soil types, vegetation distribution, and human-activity intensity in different elevation ranges in the same area [34]. In this research, elevation range was 270–2512 m, which was divided into 5 levels by the natural break method (Figure 2(a)), namely, 270–639, 640–909, 910–1199, 1200–1564, and 1565–2512 m.Curvature is an important parameter to express the structure of a terrain surface [35]. The landslide sensitivity of convex and concave slopes is greater than that of flat slopes, and landslides often occur in concave areas with high pore-water concentrations [36, 37]. In this research, curvature ranged from –51 to 81 and was divided into 4 levels by the natural break method, namely, –51 to –12, –12 to –1, –1 to 25, and 25 to 81 (Figure 2(b)).The slope angle is a controlling factor that affects slope deformation and failure [38]. In this research, slope-angle range was 0°–90° and was divided into 5 levels by the natural break method, namely, 0°–10°, 10°–20°, 20°–30°, 30°–40°, and 40°–90° (Figure 2(c)).Slope aspect affects light intensity, and difference in light intensity has a positive influence on vegetation coverage, slope rock and soil, etc., which indirectly affect landslide size and distribution [39]. In this research, slope aspect was divided into 9 levels, namely, flat, north, northeast, east, southeast, south, southwest, west, and northwest (Figure 2(d)).Terrain relief can be used to measure slope-height difference. Different types of slopes have different types of geological hazards [37, 40]. In this research, terrain relief range was 0–661 m, which was divided into 5 levels, namely, 0–25, 25–41, 41–61, 61–266, and 266–661 m (Figure 2(e)).Terrain roughness is a macrotopographic factor reflecting terrain fluctuations and erosion; it is an important quantitative indicator for measuring the degree of surface erosion, which is affected by various surface processes [41]. In this research, the terrain-roughness range was 1–9.69, which was divided into 5 levels by the natural break method, namely, 1.00–1.07, 1.07–1.20, 1.20–1.45, 1.45–4.00, and 4.00–9.69 (Figure 2(f)).Lithology affects landslide development, and the physical and mechanical properties of different lithology are very different, which directly affects slope stability [42]. In this research, lithology was divided into 9 levels (Table 1, Figure 2(d)).Table 1
Study area lithology.
GroupLithologyGeological ages1Slope product, slope residual cohesive soil with a lot of gravel and rock debrisQuaternary2Quartz sandstone and shale with muddy sandstoneJurassic3Argillaceous limestone with yellow-green shale and dolomite limestoneTriassic4Gray shale and marl interlayerPermian5Siliceous shale, black shale, grey shale, quartz sandstoneSilurian6Carbonaceous slate and marl interlayer, phylliteOrdovician7Carbonaceous slate, carbonaceous shale, marl, and limestoneCambrian8Fuchsia banded shale, carbonaceous shale, and siliceous limestoneSinian Period9Peridotite, granite porphyry, diabase, gabbro, roughCambrianNDVI represents the vegetation coverage of slope and surrounding soil, which affects slope stability to a certain extent. Vegetation roots have a fixed effect on the soil, and it is beneficial to reduce the erosion effect of the slope surface [43, 44]. In this research, NDVI ranged from –018 to 0.83 and was divided into 5 levels, namely, –0.18 to 0.18, 0.18 to 0.38, 0.38 to 0.48, 0.48 to 0.55, and 0.55 to 0.83 (Figure 2(h)).TWI reflects the spatial distribution of soil-moisture content [45]. In this research, the TWI range was 1.0–35.16 and was divided into 5 levels by the natural break method, namely, 1.0–4.8, 4.8–6.5, 6.5–8.5, 8.5–12.0, and >12 (Figure 2(i)).Land use is an intensive expression of human activity that affects soil erosion, precipitation infiltration, and surface-structure characteristics; these factors are direct causes of landslides [46, 47]. In this research, land use was divided into 4 levels (Figure 2(k)).The fault has a certain influence on the slope stability; the closer to the fault, the worse the erosion and weathering resistance of the rock mass are, and the higher landslide probability is [48]. The distance to faults was divided into 5 levels by 500 m steps, namely, 0–500, 500–1000, 1000–1500, 1500–2000, and >2000 m (Figure 2(k)).The road expresses the intensity of human activities, and the free surface formed by road excavation creates favorable conditions for the occurrence of landslide disasters [49]. In this research, distance to roads was divided into 5 levels, namely, 0–200, 200–400, 400–600, 600–800, and >800 m (Figure 2(l)).The river affects the change of slope stress. The closer to the river, the more likely landslides will occur in strong erosion [50]. In this research, the distance to fault was divided into 5 levels by the step of 200 m, namely, 0-200, 200-400, 400-600, 600-800, and >800 (Figure 2(m)).Rainfall has great influence on landslide occurrence [51, 52]. In this research, the rainfall range was 1038–1161 and was divided into 5 levels by natural break method, namely, 1038–1072, 1072–1095, 1095–1112, 1112–1133, and 1133–1161 (Figure 2(n)).


### 3.2. Logistic Model Tree

The C4.5 algorithm and logistic regression function are combined to form the LMT, which has been a popular machine learning algorithm in recent years [53]. The LMT combines the advantages of the logistic-regression and decision tree. The decision tree can only give a certain identification type and provide the sample with probability values of various types. Compared with other standard decision trees, the LMT algorithm is smaller and more robust and has better classification performance [54]. The LMT selects the best segmentation attributes by the information-gain technology. The tree is recursively constructed from top to bottom, with each leaf node constructing an independent logistic-regression model and determining the corresponding category [55]. To prevent LMT overfitting, the classification and regression tree (CART) algorithm is applied for pruning [56, 57]. The information-gain ratio [58] is calculated by
(1)GainratioD=gainDsplitinfoD,where D is the basis for sample division, gain (D) is impurity reduction after sample division, and split info (D) is information entropy obtained when m samples are divided into n subsets.On the basis of the logitBoost algorithm, logistic-regression Equation (2) is obtained by least-squares fitting [59, 60], and the poster probability is calculated using linear-logistic-regression Equation (3).
(2)Lcx=∑i=1nβixi+β0,(3)PCx=expLCx∑c,=1CexpLC,x,where βi is the logistic coefficient and n is the number of landslide-influence factors.


### 3.3. Bagging Algorithm

Bagging is a method of generating multiple subsets from a training dataset using guided sampling [22]. The basic idea is used by the bootstrap sampling method (with put-back and repeat sampling) to train multiple base classifiers under the same base classifier. By clustering all base classifiers, the final model is determined, and the result is obtained by voting. The bagging algorithm usually requires the base classifier to satisfy unstable performance. Small differences of the training samples may cause huge changes in the learning model, mainly by reducing error variance to improve classification accuracy [61, 62]. Therefore, the more sensitive the base classifier, the better capability of the bagging algorithm.


### 3.4. Boosting Algorithm

Boosting is an algorithm that converts weak learners into keen learners, and it is widely used in statistical learning [63]. The principle is to learn multiple decision-tree classifiers (base classifiers) by changing training-sample distribution and linearly combining these base classifiers to improve model performance. Each time sample distribution is changed; sample weight that was wrongly classified by the previous classifier increases. In contrast, those correctly classified by the previous classifier decrease, so the misclassified sample receives considerably more attention in the next learning. When base learners are linearly combined, the classifier with a high error rate is given a smaller weight. In contrast, the classifier with a lower error rate is given more significant weight, and the final boosting model is obtained according to the rule. The algorithm can effectively improve the deviation and variance of classifier performance [64, 65].


### 3.5. Random Forest Algorithm

RF is formed by multiple decision trees, and it is a prediction model developed by the statistical-analysis principle. The RF has high generalization ability in dealing with high-dimensional and large datasets, and it has certain advantages compared with traditional methods [66]. The principle of the RF is to propose k samples from a training dataset by the bootstrap resampling method and then to obtain the k classification consequences by building k decision-tree models for k samples. At last, through voting on the classification result to get final prediction or classification results. The RF model randomly selects the sample data and features to avoid model overfitting [67]. Numerous studies have shown that RF algorithms have excellent performance in prediction accuracy and tolerance [68].


### 3.6. Performance Evaluation Method for Landslide Susceptibility

ROC curve is the most commonly used method for landslide susceptibility evaluation at present; this curve was originally derived from statistical decision theory [69, 70]. As an outcome evaluation method, it has the advantage of being unconstrained, can effectively test the specificity and sensitivity in the model, and has good accuracy in practical applications. In this curve, the horizontal and vertical axes in the coordinate system represent the sensitivity and specificity, respectively, which are the false positive rate (FPR) and the true positive rate (TPR). The value of the area under the curve (AUC) is usually between 0.5 and 1; the larger the value, the better the final prediction effect of the model.


## 3.1. Landslide-Conditioning Factors

Landslides are affected by many factors, and a comprehensive, scientific, and rational selection of Landslide-Conditioning factors is essential [31, 32]. According to previous experience combined with geological-environment conditions, landslide development characteristics, and landslide-induced factors, 14 factors were selected. An elevation map was obtained by the Geographic Data Cloud of Chinese Academy of Sciences (30×30m), and the digital elevation model (DEM) image was processed by ArcGIS software to obtain terrain relief, slope aspect, curvature, terrain roughness, slope angle, TWI and elevation. The geological map was used to extract lithology and fault distribution, and Euclidean distance analysis of the fault obtained the distance from the landslide point to each fault. Landsat-8 images were obtained from the same DEM and used in ArcGIS software to obtain land use and NDVI. To make the rainfall map by multiyear annual precipitation, the road-network and river-system maps were vectorized, and the distance from the landslide point to each element was obtained by Euclidean distance analysis. These landslide-conditioning factors are shown in Figure 2.Figure 2
Landslide-conditioning factors.
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)(k)(l)(m)(n)Elevation has a certain effect on slope deformation and failure [33]. There are differences in rainfall, rock and soil types, vegetation distribution, and human-activity intensity in different elevation ranges in the same area [34]. In this research, elevation range was 270–2512 m, which was divided into 5 levels by the natural break method (Figure 2(a)), namely, 270–639, 640–909, 910–1199, 1200–1564, and 1565–2512 m.Curvature is an important parameter to express the structure of a terrain surface [35]. The landslide sensitivity of convex and concave slopes is greater than that of flat slopes, and landslides often occur in concave areas with high pore-water concentrations [36, 37]. In this research, curvature ranged from –51 to 81 and was divided into 4 levels by the natural break method, namely, –51 to –12, –12 to –1, –1 to 25, and 25 to 81 (Figure 2(b)).The slope angle is a controlling factor that affects slope deformation and failure [38]. In this research, slope-angle range was 0°–90° and was divided into 5 levels by the natural break method, namely, 0°–10°, 10°–20°, 20°–30°, 30°–40°, and 40°–90° (Figure 2(c)).Slope aspect affects light intensity, and difference in light intensity has a positive influence on vegetation coverage, slope rock and soil, etc., which indirectly affect landslide size and distribution [39]. In this research, slope aspect was divided into 9 levels, namely, flat, north, northeast, east, southeast, south, southwest, west, and northwest (Figure 2(d)).Terrain relief can be used to measure slope-height difference. Different types of slopes have different types of geological hazards [37, 40]. In this research, terrain relief range was 0–661 m, which was divided into 5 levels, namely, 0–25, 25–41, 41–61, 61–266, and 266–661 m (Figure 2(e)).Terrain roughness is a macrotopographic factor reflecting terrain fluctuations and erosion; it is an important quantitative indicator for measuring the degree of surface erosion, which is affected by various surface processes [41]. In this research, the terrain-roughness range was 1–9.69, which was divided into 5 levels by the natural break method, namely, 1.00–1.07, 1.07–1.20, 1.20–1.45, 1.45–4.00, and 4.00–9.69 (Figure 2(f)).Lithology affects landslide development, and the physical and mechanical properties of different lithology are very different, which directly affects slope stability [42]. In this research, lithology was divided into 9 levels (Table 1, Figure 2(d)).Table 1
Study area lithology.
GroupLithologyGeological ages1Slope product, slope residual cohesive soil with a lot of gravel and rock debrisQuaternary2Quartz sandstone and shale with muddy sandstoneJurassic3Argillaceous limestone with yellow-green shale and dolomite limestoneTriassic4Gray shale and marl interlayerPermian5Siliceous shale, black shale, grey shale, quartz sandstoneSilurian6Carbonaceous slate and marl interlayer, phylliteOrdovician7Carbonaceous slate, carbonaceous shale, marl, and limestoneCambrian8Fuchsia banded shale, carbonaceous shale, and siliceous limestoneSinian Period9Peridotite, granite porphyry, diabase, gabbro, roughCambrianNDVI represents the vegetation coverage of slope and surrounding soil, which affects slope stability to a certain extent. Vegetation roots have a fixed effect on the soil, and it is beneficial to reduce the erosion effect of the slope surface [43, 44]. In this research, NDVI ranged from –018 to 0.83 and was divided into 5 levels, namely, –0.18 to 0.18, 0.18 to 0.38, 0.38 to 0.48, 0.48 to 0.55, and 0.55 to 0.83 (Figure 2(h)).TWI reflects the spatial distribution of soil-moisture content [45]. In this research, the TWI range was 1.0–35.16 and was divided into 5 levels by the natural break method, namely, 1.0–4.8, 4.8–6.5, 6.5–8.5, 8.5–12.0, and >12 (Figure 2(i)).Land use is an intensive expression of human activity that affects soil erosion, precipitation infiltration, and surface-structure characteristics; these factors are direct causes of landslides [46, 47]. In this research, land use was divided into 4 levels (Figure 2(k)).The fault has a certain influence on the slope stability; the closer to the fault, the worse the erosion and weathering resistance of the rock mass are, and the higher landslide probability is [48]. The distance to faults was divided into 5 levels by 500 m steps, namely, 0–500, 500–1000, 1000–1500, 1500–2000, and >2000 m (Figure 2(k)).The road expresses the intensity of human activities, and the free surface formed by road excavation creates favorable conditions for the occurrence of landslide disasters [49]. In this research, distance to roads was divided into 5 levels, namely, 0–200, 200–400, 400–600, 600–800, and >800 m (Figure 2(l)).The river affects the change of slope stress. The closer to the river, the more likely landslides will occur in strong erosion [50]. In this research, the distance to fault was divided into 5 levels by the step of 200 m, namely, 0-200, 200-400, 400-600, 600-800, and >800 (Figure 2(m)).Rainfall has great influence on landslide occurrence [51, 52]. In this research, the rainfall range was 1038–1161 and was divided into 5 levels by natural break method, namely, 1038–1072, 1072–1095, 1095–1112, 1112–1133, and 1133–1161 (Figure 2(n)).


## 3.2. Logistic Model Tree

The C4.5 algorithm and logistic regression function are combined to form the LMT, which has been a popular machine learning algorithm in recent years [53]. The LMT combines the advantages of the logistic-regression and decision tree. The decision tree can only give a certain identification type and provide the sample with probability values of various types. Compared with other standard decision trees, the LMT algorithm is smaller and more robust and has better classification performance [54]. The LMT selects the best segmentation attributes by the information-gain technology. The tree is recursively constructed from top to bottom, with each leaf node constructing an independent logistic-regression model and determining the corresponding category [55]. To prevent LMT overfitting, the classification and regression tree (CART) algorithm is applied for pruning [56, 57]. The information-gain ratio [58] is calculated by
(1)GainratioD=gainDsplitinfoD,where D is the basis for sample division, gain (D) is impurity reduction after sample division, and split info (D) is information entropy obtained when m samples are divided into n subsets.On the basis of the logitBoost algorithm, logistic-regression Equation (2) is obtained by least-squares fitting [59, 60], and the poster probability is calculated using linear-logistic-regression Equation (3).
(2)Lcx=∑i=1nβixi+β0,(3)PCx=expLCx∑c,=1CexpLC,x,where βi is the logistic coefficient and n is the number of landslide-influence factors.


## 3.3. Bagging Algorithm

Bagging is a method of generating multiple subsets from a training dataset using guided sampling [22]. The basic idea is used by the bootstrap sampling method (with put-back and repeat sampling) to train multiple base classifiers under the same base classifier. By clustering all base classifiers, the final model is determined, and the result is obtained by voting. The bagging algorithm usually requires the base classifier to satisfy unstable performance. Small differences of the training samples may cause huge changes in the learning model, mainly by reducing error variance to improve classification accuracy [61, 62]. Therefore, the more sensitive the base classifier, the better capability of the bagging algorithm.


## 3.4. Boosting Algorithm

Boosting is an algorithm that converts weak learners into keen learners, and it is widely used in statistical learning [63]. The principle is to learn multiple decision-tree classifiers (base classifiers) by changing training-sample distribution and linearly combining these base classifiers to improve model performance. Each time sample distribution is changed; sample weight that was wrongly classified by the previous classifier increases. In contrast, those correctly classified by the previous classifier decrease, so the misclassified sample receives considerably more attention in the next learning. When base learners are linearly combined, the classifier with a high error rate is given a smaller weight. In contrast, the classifier with a lower error rate is given more significant weight, and the final boosting model is obtained according to the rule. The algorithm can effectively improve the deviation and variance of classifier performance [64, 65].


## 3.5. Random Forest Algorithm

RF is formed by multiple decision trees, and it is a prediction model developed by the statistical-analysis principle. The RF has high generalization ability in dealing with high-dimensional and large datasets, and it has certain advantages compared with traditional methods [66]. The principle of the RF is to propose k samples from a training dataset by the bootstrap resampling method and then to obtain the k classification consequences by building k decision-tree models for k samples. At last, through voting on the classification result to get final prediction or classification results. The RF model randomly selects the sample data and features to avoid model overfitting [67]. Numerous studies have shown that RF algorithms have excellent performance in prediction accuracy and tolerance [68].


## 3.6. Performance Evaluation Method for Landslide Susceptibility

ROC curve is the most commonly used method for landslide susceptibility evaluation at present; this curve was originally derived from statistical decision theory [69, 70]. As an outcome evaluation method, it has the advantage of being unconstrained, can effectively test the specificity and sensitivity in the model, and has good accuracy in practical applications. In this curve, the horizontal and vertical axes in the coordinate system represent the sensitivity and specificity, respectively, which are the false positive rate (FPR) and the true positive rate (TPR). The value of the area under the curve (AUC) is usually between 0.5 and 1; the larger the value, the better the final prediction effect of the model.


## 4. Results and Analysis

### 4.1. Landslide-Conditioning-Factor Analysis

Landslide-conditioning factors all have particular influence on landslides, but in practice, there may be multicollinearity between them. If factors with higher collinearity are brought into the model, the running speed of the model could be slowed down, and the model could be complicated, which may affect the end results [71]. Therefore, before model analysis, conditioning-factor multicollinearity was analyzed by variance-inflation-factor (VIF) and tolerance (TOL) methods, and the results were calculated by SAPSS software. When these parameters met the critical value (VIF>10orTOL<0.1), these factors had multicollinearity. In Table 2, the maximal VIF value is 9.4 and minimal TOL is 0.11; the result that those factors have no multicollinearity.Table 2
VIF and TOL for conditioning factors.
FactorsTOLVIFFactorsTOLVIFElevation0.511.98NDVI0.731.38Curvature0.771.29TWI0.591.69Slope angle0.991.01Land use0.921.09Slope aspect0.268.75Distance to faults0.941.07Terrain relief0.356.92Distance to roads0.751.43Terrain roughness0.119.41Distance to rivers0.711.43Lithology0.821.21Rainfall0.563.27The predictive capability is significant to landslide-susceptibility research. In this study, we applied information-gain technology to calculate the predictive capability [72]. It can be seen from Table 3 that the average merit (AM) of all factors was positive, indicating that these factors promote landslide occurrence, and NDVI information gain was the highest (0.319) and then land use (0.215), elevation (0.097), rainfall (0.064), slope angle (0.059), terrain relief (0.039), TWI (0.033), curvature (0.023), distance to rivers (0.020), slope aspect and distance to roads (0.017), terrain roughness (0.011), and distance to faults (0.001); by contrast, lithology (AM=0) had no predictive ability. Therefore, to avoid interference with the model, lithology was removed from landslide-susceptibility research.Table 3
Predictive capability.
NumberConditioning factorAverage merit (AM)Standard deviation (SD)1Elevation0.097±0.0182Curvature0.023±0.0113Slope angel0.059±0.0154Slope aspect0.017±0.0125Terrain relief0.039±0.0116Terrain roughness0.011±0.0217Lithology0.000±0.0008NDVI0.319±0.0159TWI0.033±0.00810Land use0.215±0.01711Distance to faults0.001±0.01612Distance to roads0.017±0.02413Distance to rivers0.020±0.01314Rainfall0.064±0.010


### 4.2. Landslide-Susceptibility Research

The sensitivity study is the end output by model training and verification results. The steps are as follows: first, calculating the flammability index (LSI) of each evaluation unit by the probability-distribution functions of LMT-bagging, LMT-boosting, LMT-RF, and LMT models. Then, the LSI was reclassified by Natural Breaks (Jenks), and this method uses variance to statistically minimize and interclass differences [73]. Therefore, landslide-susceptibility research was classified into 5 levels (Figure 3).Figure 3
Landslide-susceptibility maps.
(a)(b)(c)(d)The distribution of landslide susceptibility is presented in Figure4. For landslide-susceptibility maps generated using the LMT-bagging model, the high and very high grades were 18.6% and 66.78%, the moderate grade was 7.99%, and the very low and low grades were 3.12% and 3.51%, respectively. According to the LMT-boosting model, the very high, high, and moderate grades were 61.21%, 22.00%, and 3.65%, respectively; the very low and low grades are 2.62% and 1.52%, respectively. Regarding the LMT-RF model, the very low and low grades were 2.12% and 2.88%, respectively; 5.66% pertained to the moderate grade, 15.8% pertained to the high grade, and 89.34% pertained to the very high grade. By using the LMT model to establish landslide-susceptibility maps, 60.03% was the very high grade, 2.81% the very low grade, 3.97% the low grade, and 21.11% the high grade.Figure 4
Area percentages.


### 4.3. Model Validation and Comparison

Model validation is the key to the research, and its results have certain scientific and practical significance [74]. Assessing the predictive power of the 4 models by the subject performance and ROC curves; their training dataset parameters are presented in Table 4 and Figure 5(a). The performance and area under curve (AUC) value of the LMT-RF model were the highest (0.897), and the LMT-bagging, LMT-bagging, and LMT models were 0.863, 0.797, and 0.791, respectively. By using the LMT with RF model to obtain the performance (Table 4, Figure 5(b)), 0.856 was the LMT-RF model, 0.831 was the LMT-bagging model, 0.804 was LMT-bagging model, and 0.759 was LMT model.Table 4
ROC curves in training and validation step.
VariablesAUCSE95% CITrainingValidationTrainingValidationTrainingValidationLMT-bagging model0.8630.8310.0190.0210.827 to 0.8990.794 to 0.874LMT-RF model0.8970.8560.0170.0200.865 to 0.9300.814 to 0.893LMT-boosting model0.8280.8040.0220.0230.785 to 0.8700.758 to 0.849LMT model0.7910.7590.0230.0250.746 to 0.8370.710 to 0.808Figure 5
(a) ROC curves in training-dataset. (b) ROC curves in the validation dataset.
(a)(b)The results of the training and validation datasets were obtained at 95% confidence intervals (CI). From the two stages above, the LMT-RF model showed the maximum AUC and the minimum confidence interval (SE), followed by the LMT-bagging model, LMT-boosting model, and LMT model. The results found that the mixed model outperforms the single model. Wilcoxon’s signed-rank test was used to analyze the ROC contrast, and model independence was compared and analyzed [75]. The conclusions indicated that whole models were independent, with the most significant difference in AUC between the LMT model and the LMT-RF model (Table 5).Table 5
Wilcoxon’s signed-rank test.
Pairwise comparisonz valuep valueSignificanceLMT vs. LMT-bagging model3.7170.001YesLMT vs. LMT-boosting model3.4220.001YesLMT vs. LMT-RF model5.014<0.001Yes


## 4.1. Landslide-Conditioning-Factor Analysis

Landslide-conditioning factors all have particular influence on landslides, but in practice, there may be multicollinearity between them. If factors with higher collinearity are brought into the model, the running speed of the model could be slowed down, and the model could be complicated, which may affect the end results [71]. Therefore, before model analysis, conditioning-factor multicollinearity was analyzed by variance-inflation-factor (VIF) and tolerance (TOL) methods, and the results were calculated by SAPSS software. When these parameters met the critical value (VIF>10orTOL<0.1), these factors had multicollinearity. In Table 2, the maximal VIF value is 9.4 and minimal TOL is 0.11; the result that those factors have no multicollinearity.Table 2
VIF and TOL for conditioning factors.
FactorsTOLVIFFactorsTOLVIFElevation0.511.98NDVI0.731.38Curvature0.771.29TWI0.591.69Slope angle0.991.01Land use0.921.09Slope aspect0.268.75Distance to faults0.941.07Terrain relief0.356.92Distance to roads0.751.43Terrain roughness0.119.41Distance to rivers0.711.43Lithology0.821.21Rainfall0.563.27The predictive capability is significant to landslide-susceptibility research. In this study, we applied information-gain technology to calculate the predictive capability [72]. It can be seen from Table 3 that the average merit (AM) of all factors was positive, indicating that these factors promote landslide occurrence, and NDVI information gain was the highest (0.319) and then land use (0.215), elevation (0.097), rainfall (0.064), slope angle (0.059), terrain relief (0.039), TWI (0.033), curvature (0.023), distance to rivers (0.020), slope aspect and distance to roads (0.017), terrain roughness (0.011), and distance to faults (0.001); by contrast, lithology (AM=0) had no predictive ability. Therefore, to avoid interference with the model, lithology was removed from landslide-susceptibility research.Table 3
Predictive capability.
NumberConditioning factorAverage merit (AM)Standard deviation (SD)1Elevation0.097±0.0182Curvature0.023±0.0113Slope angel0.059±0.0154Slope aspect0.017±0.0125Terrain relief0.039±0.0116Terrain roughness0.011±0.0217Lithology0.000±0.0008NDVI0.319±0.0159TWI0.033±0.00810Land use0.215±0.01711Distance to faults0.001±0.01612Distance to roads0.017±0.02413Distance to rivers0.020±0.01314Rainfall0.064±0.010


## 4.2. Landslide-Susceptibility Research

The sensitivity study is the end output by model training and verification results. The steps are as follows: first, calculating the flammability index (LSI) of each evaluation unit by the probability-distribution functions of LMT-bagging, LMT-boosting, LMT-RF, and LMT models. Then, the LSI was reclassified by Natural Breaks (Jenks), and this method uses variance to statistically minimize and interclass differences [73]. Therefore, landslide-susceptibility research was classified into 5 levels (Figure 3).Figure 3
Landslide-susceptibility maps.
(a)(b)(c)(d)The distribution of landslide susceptibility is presented in Figure4. For landslide-susceptibility maps generated using the LMT-bagging model, the high and very high grades were 18.6% and 66.78%, the moderate grade was 7.99%, and the very low and low grades were 3.12% and 3.51%, respectively. According to the LMT-boosting model, the very high, high, and moderate grades were 61.21%, 22.00%, and 3.65%, respectively; the very low and low grades are 2.62% and 1.52%, respectively. Regarding the LMT-RF model, the very low and low grades were 2.12% and 2.88%, respectively; 5.66% pertained to the moderate grade, 15.8% pertained to the high grade, and 89.34% pertained to the very high grade. By using the LMT model to establish landslide-susceptibility maps, 60.03% was the very high grade, 2.81% the very low grade, 3.97% the low grade, and 21.11% the high grade.Figure 4
Area percentages.


## 4.3. Model Validation and Comparison

Model validation is the key to the research, and its results have certain scientific and practical significance [74]. Assessing the predictive power of the 4 models by the subject performance and ROC curves; their training dataset parameters are presented in Table 4 and Figure 5(a). The performance and area under curve (AUC) value of the LMT-RF model were the highest (0.897), and the LMT-bagging, LMT-bagging, and LMT models were 0.863, 0.797, and 0.791, respectively. By using the LMT with RF model to obtain the performance (Table 4, Figure 5(b)), 0.856 was the LMT-RF model, 0.831 was the LMT-bagging model, 0.804 was LMT-bagging model, and 0.759 was LMT model.Table 4
ROC curves in training and validation step.
VariablesAUCSE95% CITrainingValidationTrainingValidationTrainingValidationLMT-bagging model0.8630.8310.0190.0210.827 to 0.8990.794 to 0.874LMT-RF model0.8970.8560.0170.0200.865 to 0.9300.814 to 0.893LMT-boosting model0.8280.8040.0220.0230.785 to 0.8700.758 to 0.849LMT model0.7910.7590.0230.0250.746 to 0.8370.710 to 0.808Figure 5
(a) ROC curves in training-dataset. (b) ROC curves in the validation dataset.
(a)(b)The results of the training and validation datasets were obtained at 95% confidence intervals (CI). From the two stages above, the LMT-RF model showed the maximum AUC and the minimum confidence interval (SE), followed by the LMT-bagging model, LMT-boosting model, and LMT model. The results found that the mixed model outperforms the single model. Wilcoxon’s signed-rank test was used to analyze the ROC contrast, and model independence was compared and analyzed [75]. The conclusions indicated that whole models were independent, with the most significant difference in AUC between the LMT model and the LMT-RF model (Table 5).Table 5
Wilcoxon’s signed-rank test.
Pairwise comparisonz valuep valueSignificanceLMT vs. LMT-bagging model3.7170.001YesLMT vs. LMT-boosting model3.4220.001YesLMT vs. LMT-RF model5.014<0.001Yes


## 5. Discussion

Landslides are the most important and threatening of natural disasters, with a wide distribution area and severe disaster losses [1, 76]. Therefore, it is essential for selecting a high-quality model in landslide sensitivity, which has important practical and guiding significance for disaster prevention and human engineering construction [30, 77]. For example, when human activities happened in high landslide-prone areas, we should take preventive measures to avoiding heavy casualties and property loss caused by landslides. In this study, we selected a new machine learning technology (LMT) as the base classifier and combined it with the bagging, boosting, and RF models to build an integrated model for LSP.At present, selecting landslide conditions which has no unified standard; therefore, on the basis of predecessors, combined with the geological environment and its characteristics, we choose 14 factors in the study area, and the multicollinearity test of all factor was carried out by using VF and TOL methods. Results showed that there was no multicollinearity for any factors. In order to effectively calculate the predictive ability of the conditioning factors, the importance of these factors was assessed by using information-gain techniques. NDVI, land use, and elevation have the greatest influence, and lithology influence on them may be ignored. Results showed that the NDVI value and the vegetation-coverage rate are positively correlated; plant roots could effectively enhance soil and rock stability to reduce landslide occurrence. For land use, woodland grassland is the great mass of the south, the north is mostly cultivated land, and human activities are extensive in settlements and near rivers, which can quickly induce landslides. For elevation, landslides are prone to occurring where elevation is lower than 1200 m, and there are nearby road and river regulators. This is because the lower the altitude, the closer it is to roads and rivers, and the higher the intensity of human activity, the easier it is to promote the occurrence of landslides. These conclusions are consistent with similar studies [78, 79].The performance of the model was obtained by computing the ACU of the training and validation datasets. Results showed that the LMT-RF model performed the best; the AUC values were 0.897 and 0.856. Second was the LMT-bagging model; the AUC values were 0.863 and 0.831. Third was the LMT-boosting model; the AUC values were 0.828 and 0.8041. Last was the LMT model; AUC values were 0.791 and 0.759. In addition, through Wilcoxon’s signed-rank test analysis, the AUC of the LMT model was significantly different from the LMT-RF model, which is consistent with ROC curve performance. It can be said that all integrated machine learning algorithms have good performance in LS modeling, which has also been confirmed by similar studies of other scholars [80].


## 6. Conclusions

In this research, four models (LMT-bagging, LMT-boosting, LMT-FR, and LMT) were used to analyze the LSP in the Ziyang County, Shaanxi Province, China. This paper selected the LMT model as the basic classifier, which is a hybrid machine learning algorithm based on logistic regression and decision trees, which is more robust than other decision trees. Statistical analysis and ROC curves were used to verify and compare the predictive power of the models. The results expressed that all landslide models have good predictive effect, but the three hybrid models have better prediction ability than the single LMT model. In the hybrid model, the LMT-FR model has the best performance, followed by the LMT-bagging and LMT-boosting models. It can be seen that the LMT-FR model is a promising prediction model; this research can be provided references for land use planning and landslide prevention in local or similar areas.



---

*Source: 8254356-2022-07-28.xml*