## 3 Proposed Methodology
### Generalized Machine Learning Hurdle Model
* **Dataset Splitting**: Dividing the dataset into training and testing sets, typically with an 80:20 split, to enable unbiased model evaluation on unseen data. **Step 2: Data Transformation**  
Once the data are preprocessed, they are further transformed to create two distinct datasets tailored for the binary classification and regression stages. This step prepares the data to suit the requirements of each model within the hurdle framework:  
* **Binary Data Creation**: For the classification stage, a binary dataset is generated, where the target variable indicates the presence or absence of crashes. This binary outcome facilitates training the classifier to distinguish between crash-prone and non-crash segments. * **Non-Zero Data Creation**: For the regression stage, a subset of the data containing only non-zero crash counts is created. This dataset allows the regression model to focus solely on segments where crashes are predicted, enabling accurate frequency prediction. **Step 3: Classifier Development Stage**  
This stage involves developing a binary classification model to identify road segments with potential crash occurrences. Key actions include the following:  
* **Dataset Splitting:** Dividing the data into training and testing sets to evaluate the model's performance on unseen data. * **Model Selection:** Choosing a classification algorithm suited for high-dimensional and categorical data. * **Handling Class Imbalance:** Applying methods to balance the class distribution, increasing sensitivity to the minority class (crash cases). * **Classification Model Evaluation:** Assessing the classification model's effectiveness in identifying crash-prone segments using metrics like accuracy, precision, recall, and F1 score. **Step 4: Regressor Development Stage**  
For segments where a crash is predicted, the model advances to the regression stage to estimate crash frequency. Steps include the following:  
* **Dataset Splitting:** Similar to the classification stage, dividing data into training and testing sets for unbiased evaluation. * **Regression Model Application:** Selecting a suitable regression model with a loss function aligned with count-based data. * **Feature Selection:** Emphasizing key features influencing crash frequency, such as road geometry and traffic volume, to improve accuracy. * **Regression Model Evaluation:** Using metrics like Root-Mean-Square Error (RMSE) and Mean Absolute Error (MAE) to assess prediction accuracy. **Step 5: Full Model Output**After the classification and regression stages are complete, the final model output is calculated by combining the results from both models:  
* **Final Output Calculation:** The expected crash frequency is computed, integrating the predictions from the classifier and the regressor. * **Full Model Evaluation**: The combined model's performance is evaluated to ensure that it provides a reliable analysis of crash likelihood and frequency. This overall evaluation includes both the classification and regression results to confirm the model's robustness. **Step 6: Model Interpretability**  
Interpretability is crucial for ensuring that model predictions can inform practical safety interventions. This step involves using Explainable AI (XAI) techniques to make the model's predictions interpretable, offering insights into the contribution of each feature to crash predictions. * **Interpretation for Classification**: Interpretability techniques for the classification model indicate the impact of each feature on the likelihood of a crash occurring, helping to clarify which factors most strongly influence the outcome. * **Interpretation for Regression**: In the regression model, interpretability methods show how specific features affect the crash frequency predictions. For instance, traffic volume might be associated with increased crash frequency, while other factors could reduce it. * **Insights for Road Safety Planning**: By making the model predictions interpretable, stakeholders gain actionable insights into the factors driving crash risks.