## 3 The Proposed Traffic Flow-Prediction Model
### The Attention Mechanism Layer  
The attention mechanism layer consists of the MH-SSA layer (multi-position attention layer), the MT-SSA layer (multi-time attention layer), and the BiAtten layer (the spatiotemporal bidirectional attention layer). Figure 2 depicts the configuration of the MH-SSA layer and the MT-SSA layer. (1) The multi-position attention module  
A location attention module is introduced in the encoder to extract the spatial correlation between potential local and global regions in the traffic flow history data. The location attention module weights all the location features based on similarities between regions  
Figure 2: The structure of the location and temporal attention modules. (**a**) The location attention module. (**b**) The temporal attention module. in order to focus on both the local spatial correlation and global spatial correlations at the same time. As shown in Figure 2a, given a historical dataset \(A\in R^{C\times U\times V}\), it is passed through a convolutional layer so as to generate two new feature maps \(B\) and \(C\), i.e., \(\{B,C\}\in R^{C\times U\times V}\). Further, they are reshaped into \(R^{C\times N}\), and \(N=U\times V\) is the amount of city regions. Then, matrix \(C\) is multiplied by the transpose of \(B\), and the spatial positional attention map \(S\in R^{N\times N}\) is calculated via the softmax function,  
\[s_{ij}=\frac{\exp(B_{i}\cdot C_{j})}{\sum_{i=1}^{N}\exp(B_{i}\cdot C_{j})} \tag{6}\]  
where \(S_{ij}\) is the impact of position \(i\) on position \(j\). The greater the similarity of the feature representations, the greater the correlation between the two locations. In the meantime, feature \(A\) is input into the convolution layer, and a new feature map \(D\) is generated and reshaped into \(R\). The transpositions of \(D\) are multiplied by \(S\) and then the result is reshaped into \(R\) as well. Afterwards, the reshaped result is multiplied with the scaling parameter \(\alpha\) and summed up piecewise to obtain the final output,  
\[E_{j}=\alpha\sum_{i=1}^{N}\left(s_{ji}D_{i}\right)+A_{j} \tag{7}\]  
where \(\alpha\) is the learning parameter with initial value 0 [28]. From Equation (7), it can be obtained that feature \(E\) is the weighted sum of the global spatial correlation features and local spatial correlation, and similar regional spatial correlations are mutually enhanced accordingly. Then, \(E\) is used as a value matrix to participate in the calculation of the attention matrix. The multi-temporal attention module  
A temporal attention module is designed to capture the long-term and short-term temporal periodical correlation from historical traffic flow data. The module emphasizes the interrelated long-term time features to improve the short-term time-dependent feature representation, shown in Figure 2b. Unlike the location-attention module, the temporal attention graph \(X\in R^{C\times C}\) is calculated from the history traffic flow data \(A\in R^{C\times U\times V}\). \(A\) is reshaped to \(A\in R^{C\times N}\), \(N=U\times V\), then the time attention map \(X\in R^{C\times C}\) is obtained via the softmax layer:  
\[x_{ij}=\frac{\exp(A_{i}\cdot A_{j})}{\sum_{i=1}^{C}\exp(A_{i}\cdot A_{j})} \tag{8}\]  
where \(x_{ij}\) represents the impact of the \(i^{\text{th}}\) timestamp on the \(j^{\text{th}}\) timestamp. Subsequently, matrix multiplication is performed between \(X\) and the transpose of \(A\) to capture the periodicity in the long term, reshaping the result into \(R^{C\times U\times V}\). Then the standard parameter \(\eta\) is applied with the obtained results and the final output \(K\in R^{C\times U\times V}\) can be obtained with the element-wise summation of the original data \(A\):  
\[K_{j}=\eta\sum_{i=1}^{C}\left(x_{ji}A_{i}\right)+A_{j} \tag{9}\]  
where \(\eta\) is the learnable weight with initial value 0. Equation (9) indicates that the final feature of each timestamp is a weighted summation of the periodic features of all timestamps with the correlation features of short-term timestamps.