## 4 Model Design
### Model Performance Evaluation  
#### 4.2.1 Hourly Predictive Model  
1. Accuracy: Refers to the proportion of correct predictions in all samples, which can be expressed by the formula:  
\[Accuracy=\frac{True\ Positive+False\ Positive}{True\ Positive+True\ Negative+False\ Positive+False\ Negative} \tag{21}\]  
2. Sensitivity: Represents the proportion of positive examples that are correctly classified, which measures the recognition ability of the classifier to positive examples. In the model established in this paper, the ability of correctly predicting accidents is very important; therefore, sensitivity is used as one of the indicators to measure the effect of the model, which can be expressed as follows:  
\[Sensitivity=\frac{True\ Positive}{True\ Positive+False\ Negative} \tag{22}\]  
3. ROC (Receiver Operating Characteristic Curve):  
ROC curve is often used to evaluate the effect of binary classifiers. When the proportion of positive and negative samples in the test set changes, ROC does not change. In the accident dataset analyzed in this paper, there is an imbalance between positive  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Variable** & **Description** & **Type** \\ \hline Traffic Volume (veh/h) & To record traffic volume for the first lane & Continuous \\ for Lane 1 & & \\ \hline Traffic Volume (veh/h) & & \\ for Lane 2 & & \\ \hline Traffic Volume (veh/h) & & \\ for Lane 3 & & \\ \hline Vehicle speed & Average speed of vehicle(km/h) & Continuous \\ \hline \multirow{4}{*}{Weather information} & 0: overcast & \\  & 1: sunny & \\  & 2: cloudy & Categorical \\  & 3: rainy & \\  & 4: snowy & \\ \hline Holiday information & 0: today is not a legal holiday in China & \\  & 1: today is a legal holiday in China & Categorical \\ \hline Truck to car ratio & The fraction of number of trucks to the & \\  & number of passenger vehicles & Continuous \\ \hline Crash in daily prediction & The number of car accidents on that day & Numerical, Ordinal \\ \hline \multirow{4}{*}{Crash in hourly prediction} & 0: no crash on a freeway segment & \\  & within one hour & \\ \cline{1-1}  & 1: one or two crashes on a freeway segment & \\ \cline{1-1}  & within one hour & \\ \hline \hline \end{tabular}
\end{table}
Table 1: Overview of variables.  
and negative samples. There are more negative samples than positive samples, and the proportion of positive and negative samples in the test data may also change with the partition of the dataset. The ROC curve can ensure the effectiveness of evaluation indicators under such circumstances.  
4. AUC (Area Under Curve):  
AUC is the area under the ROC curve, which can be used to evaluate the effect of binary classifier. AUC takes into account the ranking quality of sample prediction. AUC stands for the level or measurement of distinction, while ROC is a likelihood graph. It reveals how well the algorithm can differ across groups. The method is more appropriate at classifying 0 classes as 0, and classifying 1 event as 1, the greater the AUC. The larger AUC is, the better the classifier effect is. The closer AUC value is to 1, the better the model effect is. For random classifiers, the AUC value should be 0.5.