## 3 Methodology
### STA-Blocks
#### 3.4.2 Spatial Attention Layer  
In a traffic network, the traffic conditions between different locations exhibit interactive and dynamic behavior, referred to as spatial dependency. The main objective of the spatial attention mechanism is to identify the features of neighboring nodes and capture the interdependency among them. This layer utilizes GAT [20], a form of multi-headed self-attention mechanism, as shown in the right part of Figure 2, to capture the spatial dependency. The GAT is expressed in Equations (10)-(13).  
\[e_{ij}=a\bigg{(}W\overset{\rightarrow}{h}_{i},W\overset{\rightarrow}{h}_{j} \bigg{)} \tag{10}\]  
Let \(h\) represent the output of the temporal attention layer, where \(h=\bigg{\{}\overset{\rightarrow}{h}_{1},\overset{\rightarrow}{h}_{2},\ldots.\overset{\rightarrow}{h}_{N}\bigg{\}}\), and \(\overset{\rightarrow}{h}_{i}\in R^{F}\). Here, \(N\) denotes the number of nodes, and \(F\) indicates the number of features. Moreover, \(W\) symbolizes the shared parameter matrix utilized for linearly transforming the features of each node, and it belongs to the space of \(W\in\mathbb{R}^{F^{\prime}\times F}\).  
\[\alpha_{ij}=softmax\big{(}j\big{(}e_{ij}\big{)}\big{)}=\frac{exp(e_{ij})}{\sum_ {k\in Ni}exp(e_{ik})} \tag{11}\]  
where \(\alpha_{ij}\) means the importance of node \(j\)'s features to node \(i\), as \(\alpha_{ij}\in\mathbb{R}^{N\times N}\).  
\[\alpha_{ij}=\frac{exp(LeakyReLU\bigg{(}\overset{\rightarrow}{a}^{T}\bigg{[} W\overset{\rightarrow}{h}_{i}\bigg{|}\bigg{|}W\overset{\rightarrow}{h}_{j} \bigg{]}\bigg{)})}{\sum_{k\in Ni}exp(LeakyReLU\bigg{(}\overset{\rightarrow}{ a}^{T}\bigg{[}W\overset{\rightarrow}{h}_{i}\bigg{|}\bigg{|}W\overset{ \rightarrow}{h}_{j}\bigg{]}\bigg{)})} \tag{12}\]  
where \(\overset{\rightarrow}{a}\) represents a single-layer FFN, parameterized by a weight vector \(\overset{\rightarrow}{a}\in\mathbb{R}^{2F^{\prime}}\) and non-linearized by applying the \(LeakyRelu\) function with a negative input slope, \(\alpha=0.2\).  
\[\overset{\rightarrow}{h}_{j}^{l+1}=\sigma\Bigg{\{}\sum_{j\in N_{i}}\alpha_{ ij}W\overset{\rightarrow}{h}_{j}\Bigg{\}} \tag{13}\]  
where \(\alpha_{ij}\) represents the correlation coefficient between nodes, which \(W\overset{\rightarrow}{h}_{j}\) signifies the weight of the feature vectors propagated from the previous layer.  
To enhance the generalization of the spatial attention layer and promote training stability, we incorporate \(K\) groups of self-attention mechanisms with identical dimensions across the input. These mechanisms are joined together using a splicing function, and dimension alignment is achieved through the mean operation. By employing multiple heads, we foster stronger spatial dependency between nodes, enabling a more rational assignment of weight coefficients. This is particularly beneficial for key nodes and leads to improved overall generalization. The mathematical representation of the multi-headed spatial attention mechanism can be found in Equation (14).  
\[\overset{\rightarrow}{h}_{j}^{\prime}=||\overset{\rightarrow}{h}_{i=1}^{ks }\sigma\bigg{(}\sum_{j\in N_{i}}a_{ij}^{k}W\overset{\rightarrow}{h}_{j}^{ \rightarrow}\bigg{)} \tag{14}\]  
where _hs_ represents the number of heads on GAT. The normalized coefficient, \(a_{ij}^{k}\), is calculated by the \(k_{\text{th}}\) group of attention mechanism. \(W^{k}\) denotes the parameter matrix of the \(k_{\text{th}}\) group. The symbol \(\parallel\) corresponds to the operation of the _Concat_ function.