## 4 Empirical Assessment Using Real-World Data
### Results
#### 4.3.4 Model Interpretability  
The interpretability of predictive models is a crucial component, particularly in road safety analysis, where understanding the underlying factors influencing crash frequency is as important as the accuracy of the predictions themselves. The SHAP summary plots, shown in Figures 7 and 8, bridge the opaque complexity of machine learning models and the transparent rationale required for practical application and policy development. In SHAP summary plots, the horizontal axis denotes the magnitude of each feature's SHAP value in influencing the model output. Features that shift the prediction higher are located towards the right, while those contributing to a lower prediction appear on the left. The color intensity represents the feature value itself; warmer colors indicate higher feature values, while cooler colors indicate lower ones. This visual encoding helps distinguish which features, in their high-value state, act to increase the predicted crash frequency, and which ones in the same state serve to decrease it. \begin{table}
\begin{tabular}{l c c} \hline \hline
**Model** & **RMSE** & **MAE** \\ \hline Poisson Hurdle Model & 2.097 & 0.527 \\ Negative Binomial Hurdle Model & 1.336 & 0.511 \\ CatBoost (Single Stage) & 1.197 & 0.421 \\ Proposed Model & 0.978 & 0.346 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Comparative evaluation of full model performance. Figure 6: Training and validation losses for custom CatBoost regression model. In the binary classification stage, we examined the likelihood of a road segment having any crashes versus none. The SHAP summary plot shown in Figure 7 reveals that the AADT exerts the most significant influence, with higher traffic volumes generally increasing the probability of crashes. This aligns with established traffic safety research that correlates increased traffic density with higher crash potential. Conversely, features like median barriers tend to reduce the likelihood of crashes when present, highlighting their effectiveness as a safety feature. The road segment length also appears to be a key factor. The plot shows that longer road segments are associated with an increased probability of crash occurrence, which could reflect a greater exposure to risk over extended distances. Meanwhile, the slope has a varied impact, suggesting that while steeper slopes may contribute to higher crash risks, the relationship is not uniformly straightforward across all road segments. Figure 8: SHAP summary plot for regression stage. Figure 7: SHAP summary plot for classification stage. Moving to the regression stage, we focused on the number of crashes for road segments where crashes are expected to occur. In the SHAP summary plot shown in Figure 8, AADT continues to be a prominent factor, with higher values pushing the predicted number of crashes up. This suggests that traffic volume affects not only the presence of crashes but also their frequency. The influence of road geometry is further explained here. The slope and shoulder width features, for example, demonstrate a mix of positive and negative effects on crash frequency. A steep slope may increase the risk, but a wide shoulder could potentially mitigate it by allowing drivers more room for error. Finally, we report the SHAP partial dependence plots (PDPs) for the regression model. Interpreting SHAP PDP plots involves understanding the relationship between the target variable and one or two predictor variables, while keeping all other features constant. These plots help to understand the marginal effect of each feature across its entire range, irrespective of the presence or distribution of other features in the data. SHAP PDP plots of selected features are shown in Figure 9. The additional variable on the right side of each PDP plot indicates the color scale used to differentiate data points based on another feature's value. This color scale helps visualize how the interaction between the primary feature on the \(x\)-axis and this secondary feature can impact the prediction results.