## 4 Our Proposed ADDGCN System
### Multi-Head Attention Mechanism  
To effectively describe the enduring interdependence of traffic data, we develop a multi-head temporal attention layer [42] to model the global contextual information of time series. This layer, as shown in Figure 5, comprises a position embedding layer, a multi-head attention mechanism and a feed-forward network layer. The specific structure is illustrated in the figure. By utilizing the temporal attention mechanism, our model is more capable of learning longer-term temporal trends and discovering the worldwide traffic data information in the temporal dimension.  
Figure 5: Structural diagram of the multi-head temporal attention layer.  
After undergoing processing by the DS-DGC module, two time subsequences \(X_{event}^{\prime\prime}\) and \(X_{odd}^{\prime\prime}\) are obtained. Then, these subsequences are rearranged and combined in the order of their time indices to form a new time series, denoted as \(X_{input}\in R^{C\times N\times T}\), which will be utilized as the multi-head temporal attention layer's input. Considering the fact that the multi-head attention mechanism may not accurately account for the relative position in the sequence during the dot product operation, it is important to address this issue. At each time step, a position vector is added to the input data to overcome this issue:  
\[\hat{X}_{input}=X_{input}+e_{t} \tag{9}\]  
where the \(e_{t}\) of the position code is defined as:  
\[e_{t}=\begin{cases}\sin\Bigl{(}t/10000^{2i/d_{model}}\Bigr{)},&t=0,2,4\ldots \\ \\ \cos\Bigl{(}t/10000^{2i/d_{model}}\Bigr{)},&t=1,3,5\ldots\end{cases} \tag{10}\]  
First, the encoded input sequence \(\hat{X}_{input}\in R^{C\times N\times T}\) is used to generate key matrix \(K\), value matrix \(V\), and the query matrix \(Q\) for the self-attention mechanism. The formulas are as follows:  
\[Q=\hat{X}_{input}\ W_{q}^{T},K=\hat{X}_{input}\ W_{k}^{T},V=\hat{X}_{input} \ W_{v}^{T} \tag{11}\]  
where \(W_{q}^{T}\), \(W_{k}^{T}\) and \(W_{v}^{T}\) are the learnable projection matrices. Next, the query matrix \(Q\) is multiplied by the transpose of the key matrix \(K\), and the result is normalized to obtain the weight coefficient in the time dimension. Finally, the weight coefficient is applied to the value matrix \(V\) to obtain the weighted results of various time steps. This is the formula:  
\[Attention(Q,K,V)=soft\max\Bigl{(}\frac{QK^{T}}{\sqrt{d_{k}}}\Bigr{)}V \tag{12}\]  
where \(\sqrt{d_{k}}\) denotes the weight scaling factor. To obtain the weighting of each variable, the attention scores are normalized using the softmax function.  
The multi-head attention mechanism improves the model's capacity to capture global temporal correlations [43], allowing it to focus on multiple subspace characteristics simultaneously. In the multi-head attention mechanism, multiple sets of self-attention operations are performed on the original input sequence, and their results are concatenated. Finally, the concatenated sequence is linearly transformed using the projection matrix \(W^{0}\) to obtain the results of the multi-head attention mechanism. The equation is depicted as follows:  
\[\begin{split} MultiHead(Q,K,V)=Concat(head_{1},head_{2},\cdots, head_{n})W^{0},\\ head_{i}=Attention\Bigl{(}\hat{X}_{input}W_{i}^{Q},\hat{X}_{ input}W_{i}^{X},\hat{X}_{input}W_{i}^{V}\Bigr{)}.\end{split} \tag{13}\]  
where \(W^{O}\) is the linear transformation projection matrix. The feed-forward network layer receives the output of the multi-head attention layer and performs further processing on it. Residual links and layer normalization are used after each sublayer to optimize the model's performance.