# Leveraging Internet News-Based Data for Rockfall Hazard Susceptibility Assessment on Highways
## 3 Results
### Random Forest Model Performance in Rockfall Prediction  
The performance of the Random Forest model, as shown in Table 3, demonstrates strong predictive capabilities across various hyperparameter settings during the 5-fold cross-validation (showing average metrics across the five folds). The results indicate consistent performance, with overall accuracy ranging from 0.7690 to 0.8069, and a Kappa statistic between 0.5379 and 0.6138, highlighting the model's stability across different hyperparameter combinations. Sensitivity and precision were well-balanced, with sensitivity values ranging from 0.8207 to 0.8759, and precision values between 0.7473 and 0.7721, suggesting that the model effectively identifies rockfalls while maintaining high accuracy in its predictions. According to the results, the optimal hyperparameter combination was found to be \(\text{mtry}=1\) and \(\text{ntree}=100\). Using this optimal combination, a final Random Forest model was trained on the entire training dataset and subsequently tested on the test dataset. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline  & & & \multicolumn{6}{c}{**5-Fold Cross-Validation**} \\ \hline  & **mtry** & **nttree** & **Accuracy** & **Kappa** & **Sensitivity** & **Specificity** & **Precision** & **F1** \\ \hline  
\hline \multicolumn{8}{c}{**Test Dataset**} \\ \hline  & & **Accuracy** & **Kappa** & **Sensitivity** & **Specificity** & **Precision** & **F1** \\ \hline  & & 0.8514 & 0.7027 & 0.8378 & 0.8649 & 0.8611 & 0.8493 \\ \hline \end{tabular}
\end{table} 
Table 3: Performance metrics from 5-fold cross-validation and test dataset. On the test dataset, the model achieved an accuracy of 0.8514, demonstrating its strong ability to correctly classify instances in previously unseen data. The sensitivity on the test dataset was 0.8378, indicating the model's consistent effectiveness in identifying rockfalls. Precision was also high at 0.8611, underscoring the model's reliability in making accurate rockfall predictions. The F1 score on the test dataset was 0.8493, indicating that the model maintains a good balance between precision and recall. Additionally, the Kappa statistic on the test dataset was 0.7027, suggesting substantial agreement between the predicted and actual classifications, even after accounting for chance. The Receiver Operating Characteristic (ROC) curve depicted in Figure 3 illustrates the predictive performance of the Random Forest model against the test dataset. This curve provides a graphical representation of the model's diagnostic ability by plotting the True Positive Rate (sensitivity) against the False Positive Rate (1-specificity) across various threshold settings. The orange line signifies the ROC curve of the model, while the diagonal blue dashed line represents the performance of a random classifier, which has an Area Under the Curve (AUC) of 0.5. The ROC curve's proximity to the upper left corner and the larger area under the curve indicate better model performance. In this instance, the model achieved an AUC value of 0.9317, indicating excellent discriminative ability in identifying rockfall occurrences with minimal misclassifications. Such a high AUC value suggests that the model maintains a high true positive rate while keeping the false positive rate low, thereby confirming the robustness and reliability of the Random Forest model in effectively distinguishing between the rockfall and non-rockfall classes. After validating the machine learning model, a section of Provincial Highway 7, also known as the Northern Cross-Island Highway, which has numerous occurrences of rockfalls, was selected to create a detailed rockfall susceptibility map. A 200-m buffer was generated around the road to encompass the potential impact area. This buffer zone was then converted from a shapefile to a raster format with a 5-m resolution to ensure high spatial accuracy.