# A Data-Driven Deep Learning Framework for Prediction of Traffic Crashes at Road Intersections
## 5 Performance Evaluation
### Performance Evaluation of RoadInTCP  
**A. Experimental Setup. ** In real life, traffic crashes occur relatively infrequently, meaning that only a fraction of intersections experience crashes within any given time. To address the issue of sample sparsity, we employ a negative sampling method, where each data of a traffic crash is considered a positive sample, followed by the random selection of intersections that did not experience traffic crashes within a certain time period to generate negative samples, with the number of negative samples being equal to that of the positive samples. By extracting the topological relationship features of intersections and additional thematic information, _RoadInTCP_ proposed in this paper predicts whether the target intersection has a traffic crash. If a traffic crash occurs, its true value is labeled as 1; otherwise, it is 0. In the evaluation of _RoadInTCP_, for the New York City dataset, it is randomly divided into 80% for training and 20% for testing; the _F1-score_ is used as the performance metric. The RN2 Vec model is implemented in C and the RoadInTCP model is implemented in the Python 3.10.8. All experiments are run on the Ubuntu 18.04 operating system with an Intel Core i5-8400 CPU and an NVIDIA GTX 1080 GPU. **B. Evaluation of Models. ** The performance of all evaluated models, performed on all five categories of the heterogeneous data described in Section 3.1 is reported in Table 2. As shown in Table 2, the performance of the Support Vector Machine (SVM) model is superior to that of the Logistic Regression (LR) model among conventional methods. This is attributed to the kernel function in SVM, which is designed to find the optimal margin sep  
Figure 9: Parameter analysis for RoadInTCP model in Phase I of avenue classification. aration, thereby exhibiting stronger learning capabilities for complex nonlinear functions. Moreover, we observe that the Random Forest (RF) model shows a better performance, slightly outperforming the Decision Tree (DT) model. This is due to the RF and DT models' proficiency in selecting more important features related to traffic crashes, coupled with reduced sensitivity to input noise, and the RF model's stronger nonlinear data fitting capability as an ensemble model. Among conventional methods and deep learning models, we find that all the deep learning models outperform the conventional methods, thereby showing the deep learning architectures' capability in modeling complex relationships. The performance of the Random Forest (RF) model is slightly better than that of the Multi-layer Perceptron (MLP) model. This may be because the MLP model is the simplest type of neural network with structure, whereas the RF model is an optimization algorithm that improves upon the Bagging ensemble algorithm, exhibiting stronger anti-interference capabilities and better model generalization. Among deep learning models, the Graph Convolutional Network (GCN) model and the Long Short-Term Memory (LSTM) model perform better than the MLP model. Compared to the MLP model, the GCN model can learn spatial information from the network, while the LSTM model can learn temporal feature information. In Table 2, we observe that the _RoadInTCP_ model robustly outperforms all the compared models due to the following three reasons. First, although the GCN model can capture the spatial structure information of the network, it is not suitable for road network structures. Phase I of _RoadInTCP_ designed in this paper can capture the intrinsic geographical attributes of the road network, thereby achieving better performance; second, in Phase II of _RoadInTCP_, we consider additional thematic information including traffic situation, weather, environmental and calendar features associated with intersections. Third, in Phase III, multi-scale temporal feature information is considered, including immediate features, short-term features, and long-term features.