# Examination of Landslide Susceptibility Modeling Using Ensemble Learning and Factor Engineering
## 3 Methodology
### Heterogeneous Ensemble Strategy  
This study evaluates four homogeneous ensemble models, random forest, XGBoost, LightGBM, and CatBoost, each employing distinct modeling strategies that result in unique decision boundaries. As shown in Figure 5, random forest exhibits relatively smooth decision boundaries with clear separation lines, indicating stable performance across most regions, although occasional misclassifications may occur. XGBoost demonstrates smoother decision boundaries and superior classification performance, particularly in densely populated data regions; however, it may overfit in sparse data areas. LightGBM, characterizedby more complex decision boundaries, showcases high flexibility and sensitivity to detail, enabling the capture of intricate patterns, but also increasing susceptibility to overfitting at boundaries. CatBoost's decision boundaries resemble those of XGBoost, showing stable performance in dense regions, while effectively avoiding overfitting in sparse areas. Given the unique strengths and limitations of each model, this study proposes a heterogeneous ensemble approach. By integrating the predictions from these models, the goal is to construct an ensemble with overall superior performance. This approach is expected to deliver more accurate and robust classification results across varying data distribution conditions, combining the individual advantages of each model, while mitigating their weaknesses. Bagging, short for bootstrap aggregating, is an ensemble learning technique, designed to enhance predictive performance by reducing model variance. The core idea of bagging is to generate multiple subsets from the original dataset through random sampling with replacement, to train a base model on each subset, and combine their predictions using averaging (for regression tasks) or voting (for classification tasks). This process aims to improve the stability and accuracy of predictions by mitigating overfitting and enhancing generalization. In this study, a bagging strategy is employed to integrate the random forest, XGBoost, LightGBM, and CatBoost models, leveraging the strengths of each to enhance classification accuracy and stability. A soft voting strategy is utilized to make the final decision in classification tasks by combining the predicted probabilities from multiple base classifiers. In regard to soft voting, each classifier outputs a probability distribution reflecting the likelihood of a sample belonging to each class. These distributions are then weighted and averaged, typically with equal weights, although adjustments can be made based on individual classifier performance. The class with the highest combined probability is selected as the final prediction. The advantage of soft voting lies in its ability to utilize the full predictive probability information from each classifier, rather than relying solely on hard classification results. This approach not only improves classification accuracy and  
Figure 5: Differences in classification performance among various models; (**a**) random forest, (**b**) XGBoost, (**c**) LightGBM, (**d**) CatBoost, (**e**) bagging, and (**f**) stacking. stability, but also generates smoother decision boundaries, making it particularly effective for handling complex classification problems. Stacking, a sophisticated ensemble learning technique, aims to construct a more effective final model by integrating the predictions of multiple diverse base models. This method employs a \"meta-learner\" or \"second-level learner\" to aggregate the predictions from various base models, creating a novel strategy that has been widely adopted in predictive tasks, particularly in competitions and complex problem-solving scenarios. The core mechanism of stacking involves using the predictions from each base model as new inputs, referred to as meta-factors, which are then analyzed by a meta-learner to produce the final prediction.