## 4 Empirical Assessment Using Real-World Data
### Experimental Design
#### 4.2.1 Classification Stage  
The CatBoost model was introduced as a robust machine learning algorithm that excels in handling categorical features and preventing overfitting. For our current analysis of crash frequency data, we employed the CatBoost model, with a particular focus on its loss function capabilities. We utilized the customized loss function to tackle the class imbalance problem inherent in crash frequency data, where non-occurrences (zeroes)  
Figure 3: Flowchart of the proposed model assessment approach. significantly outnumber occurrences (crashes). This customization enables the model to pay closer attention to the minority class, thereby improving the sensitivity of crash prediction. Additionally, we evaluated the performance of the CatBoost model using its default loss functions for comparison purposes, in order to demonstrate the effectiveness of custom loss functions in improving model performance for imbalanced datasets. The performance was compared against benchmark models. In traditional hurdle models for zero-inflated crash frequency data, logistic regression (LR) is commonly employed in the first stage to distinguish zero counts from positive counts. Unlike linear regression, which is unsuitable for binary outcomes, LR is apt for categorization tasks, since it predicts probabilities that naturally lie between zero and one. The LR model, in our context, is aimed at discerning the binary response variable: whether or not a crash has occurred within road segments. The LR model formulates the relationship between the binary response variable \(y\), where \(y\in\{0,\ 1\}\), and the predictors \(\mathrm{X}=(x_{1},x_{2},\ldots,x_{m})\). The predictors comprise various road geometry characteristics and traffic metrics, such as road curvature, lane width, and traffic volume. The probability of a crash occurrence, denoted as \(\mathrm{P}(Y=1|X)\), is modeled by the logistic function:  
\[\mathrm{P}(Y=1|X)=\frac{1}{1+e^{-(\beta_{0}+\beta_{1}x_{1}+\cdots+\beta_{m}x_{ m})}} \tag{13}\]  
where \(\beta_{0}\) is the intercept, and \(\beta_{1},\ldots,\beta_{m}\) are the estimated coefficients that quantify the impact of each predictor. The logistic model equation for our current analysis is expressed as follows:  
\[\log\biggl{(}\frac{\mathrm{P}(Y=1|X)}{1-\mathrm{P}(Y=1|X)}\biggr{)}=\beta_{0}+ \beta_{1}x_{1}+\cdots+\beta_{m}x_{m} \tag{14}\]  
This log-odds or logit transformation ensures that the output probability is confined within the range \([0,1]\). The decision tree (DT) classifier is a machine learning model that can be employed to categorize road segments based on crash occurrences. The DT model is comparable to a flowchart with a tree structure, where each internal node represents a \"question\" on features of the data, each branch represents the outcome of that question, and each leaf node represents a class label (outcome). The Gini index is commonly used as a criterion for creating the decision rules at each node and can be mathematically expressed as follows:  
\[Gini(D)=1-\sum_{y=1}^{m}p_{y}^{2} \tag{15}\]  
where is \(D\) is the dataset, \(n\) is the number of classes, and \(p_{y}\) is the relative frequency of class \(y\) in \(D\). When a dataset \(D\) is split into two subsets \(D_{1}\) and \(D_{2}\), the Gini index for the split is computed using the following formula:  
\[Gini_{split}(D)=\frac{N_{1}}{N}Gini(D_{1})+\frac{N_{2}}{N}Gini(D_{2}) \tag{16}\]  
where \(N_{1}\) and \(N_{2}\) are the sizes of datasets \(D_{1}\) and \(D_{2}\), respectively, and \(N\) is the total number of instances in \(D\). The goal is to choose the split that minimizes the Gini index, indicating the most homogeneous branches. DTs have parameters that can be fine-tuned--such as the minimum number of samples required to be at a leaf node, or the maximum depth of the tree--to prevent overfitting and ensure the model's generalization to new data. Random forest (RF) serves as an advanced classification method to address the binary outcome of crash occurrences in road segments.