# Spatial Performance Indicators for Traffic Flow Prediction
While some studies on traffic prediction have considered prediction accuracy across different forecast horizons, they have been primarily limited to multi-step prediction problems. Evaluating the short- and long-term predictions of a spatiotemporal prediction model is important to understand its robustness and adaptability in real-world applications [15]. However, current standard KPIs do not provide a comprehensive evaluation of these temporal aspects. Similarly, the evaluation of the spatial aspects of spatiotemporal methods in traffic prediction is currently lacking. In this study, we aimed to investigate various spatial metrics commonly utilized in the domain of Geospatial Information Systems (GIS) and network graph analysis, including Global Moran's I [16], Getis-Ord General G [17], and Adapted PageRank Algorithm Modified (APAM) [18]. These metrics were chosen because of their simplistic nature, which makes them easy to interpret. Traffic prediction methods use various input data and algorithms, resulting in different levels of spatial characteristics. We consider these metrics to be the spatial KPIs utilized to measure the spatial context present in the traffic flow prediction method. We recognize that evaluating both spatial and temporal aspects is important. However, we will focus only on spatial aspects, deferring the evaluation of temporal aspects to future research. Our experiments primarily focused on spatiotemporal prediction methods that utilize spatial information, such as the structure of the traffic network based on a graph adjacency matrix. We compared the results of spatial KPIs that evaluate different traffic flow prediction methods on the PeMSD8 dataset [19], measured on freeway traffic networks in California. We considered four traffic flow prediction methods: (1) STSGCN [7], (2) ASTGCN [6], (3) DDGCRN [20], and (4) STAEformer [21]. These methods were chosen for their diverse approaches to incorporating spatial information in traffic prediction while having similar average RMSE. We specifically chose spatiotemporal traffic prediction that can effectively capture the intricate localized spatiotemporal correlations present in the traffic network, which aligns with the recent research in the field. To provide insight into the capabilities of spatial KPIs under different scenarios, we created synthetic scenarios by modifying the traffic flow prediction results to exhibit various clusters in the form of hotspots and coldspots. This allows us to demonstrate the capabilities of spatial KPIs to distinguish different scenarios when the standard accuracy KPIs fail to do so. The remainder of this paper is organized as follows. In Section 2, we briefly describe the traffic network graph and the traffic prediction problem. In Section 3, we explain the materials and methods, including (1) different spatial KPIs such as Global Moran's I, Getis-Ord General G, and Adapted PageRank Algorithm Modified (APAM) value for evaluating the spatial aspects of spatiotemporal prediction models and (2) experimental setups such as datasets, clustering scenarios for the case study, and different traffic prediction methods to generate traffic prediction results. In Section 4, the results of the evaluations for all clustering scenarios and traffic prediction methods utilizing spatial KPIs are presented. In Section 5, we summarize the main findings and discuss the implications of spatial KPIs for traffic management systems. The conclusion and future work are summarized in Section 6.