applied sciences Article  
Hierarchical Spatial-Temporal Neural Network with Attention Mechanism for Traffic Flow Forecasting [PERSON], [PERSON],*(r) and [PERSON]  
Footnote 1: College of Merchant Ship, Shanghai Maritime University, Shanghai 201306, China; [EMAIL_ADDRESS]  
Footnote 2: College of Information Engineering, Shanghai Maritime University, Shanghai 201306, China  
Footnote 3: footnotemark:  
Footnote 4: footnotemark:  
Footnote 5: footnotemark:  
Footnote 6: footnotemark:  
Footnote 7: footnotemark:  
Footnote 8:  
which exhibit multi-scale characteristics encompassing global and local spatial terms, as well as long and short-term temporal relationships. [PERSON] proposed T-GCN [3], a model that combines GCN with GRU. Instead of using LSTM, T-GCN employs GRU with fewer parameters for time dependence extraction, along with GCN for capturing spatial dependency and extracting spatial-temporal correlations for traffic forecasting. Building upon T-GCN, [PERSON] introduced A3 TGCN [4], which integrates an attention mechanism to dynamically adjust the significance of time points in the time series. Furthermore, [PERSON] proposed STTN [5], comprising stacked spatial-temporal blocks and prediction layers. Each spatial-temporal block consists of a spatial transformer and a temporal transformer, which jointly extract spatial-temporal correlations. Although the attention mechanism proves useful in modeling spatial-temporal correlations, the transformer structure results in an excessive number of parameters. In contrast, ASTTN [6] leverages a self-attention mechanism to model dynamic spatial-temporal correlations in graphs. It employs local spatial-temporal graphs to capture spatial-temporal features, focusing attention on one-hop spatial neighbors. Restricting attention to spatial neighbors and capturing long-range spatial dependencies becomes reasonable as the number of layers increases, given the typically weak long-range spatial-temporal attention.  
Inspired by the recent success of the transformer model in graph domains, this paper presents the HSTAN (Hierarchical Spatial-Temporal Self-Attentive Networks) model. HSTAN employs local multi-headed self-attention within each spatial-temporal block to directly model spatial-temporal correlations on graphs. Moreover, it utilizes multiple layers of spatial-temporal blocks to capture deep spatial-temporal features. By effectively representing spatial-temporal correlation through the combination of local attention and hierarchical structure, HSTAN overcomes the challenges associated with training models with a large number of parameters. The main contributions of this paper can be summarized as follows:  
1. We propose the STA-Blocks structure, designed to encapsulate local spatial-temporal correlations. This structure utilizes a temporal attention layer to capture temporal correlations, while local spatial correlations within a one-hop radius are captured through a spatial attention layer. Both of these layers operate on the basis of a multi-headed self-attention mechanism.
2. We utilize a hierarchical structure of stacked Spatial-Temporal Attention Blocks (STA-Blocks) to methodically extract and integrate spatial-temporal correlations. This hierarchical structure enhances the receptive field of local multi-headed self-attention, thereby achieving a level of performance equivalent to full attention.
3. We have evaluated the predictive performance of our proposed model using two real-world traffic datasets. Our model has demonstrated a significant improvement in the accuracy of traffic forecasting when compared to the baseline models. Additionally,  
Figure 1: Complex Spatial-Temporal Correlation. Except for the traditional temporal correlations (dotted line) and spatial dependency (solid line), the spatial-temporal correlations (solid line with arrow) will also affect traffic conditions.  
we have carried out ablation studies to assess the impact of our model's components on its overall performance.