## 3 Methodology
### Dual Branch Spatio-Temporal Graph Neural Network with an Attention Mechanism  
In this section, we provide a detailed description of the architecture of DBSTGNN-Att, which was named dual branch spatio-temporal graph neural network with an attention mechanism. DBSTGNN-Att comprised multiple ST-Blocks (two ST-Blocks were used in this paper). Each ST-Block contained two temporal blocks and a spatial block. Matrix P was fed into the temporal block, while the attributed KNN graph was fed into the spatial block. This setup allowed for the extraction of spatio-temporal features from the cellular network traffic. The prediction results Q were ultimately generated through the projection layer. Figure 3 illustrates the structure of DBSGTNN-Att, where the parameter \(L\) represents the historical time-series length. The dimension of the matrix \(S\) obtained by splicing the multi-modal feature matrix with the attributed KNN matrix was \((L+1)\times K\), where \(K\) is the number of similar nodes extracted and \(L\) is the length of the historical time series. 2.1 Time Feature Extraction Based on Temporal Dual Branch-Gated Convolution with an Attention Mechanism  
The self-attention mechanism is one of the attention mechanisms. The weight assigned to each input item depends on the interaction between the input items, that is, the voting within the input item determines which input items should be focused on. Compared with the strong attention mechanism and the soft attention mechanism, the self-attention mechanism has the advantage of parallel computing when dealing with very long inputs and neutralizes the degree of information neglect and the running time of the strong attention mechanism and the soft attention mechanism. Specifically, the self-attention mechanism captures the dependencies between sequences by the query matrix \(Q\), the key matrix \(K\), and the value matrix \(V\). By comparing the query matrix \(Q\) of the time series with the key matrix \(K\), and matching the value matrix \(V\) with the value corresponding to the key matrix \(K\), the self-attention can be calculated. By normalizing the time dependence of each time step, the _SoftMax_ function can obtain the attention probability matrix. In order to prevent significant differences in the probability distribution of the matrix obtained by the _SoftMax_ function after the dot product operation, it is necessary to apply the \(\sqrt{d}\) scaling  
Figure 3: Structure diagram of the DBSTGNN-Att model. factor for normalization. Finally, the output is processed by the linear layer and the _ReLU_ activation function to obtain the representation of the time feature. In this paper, we used the self-attention mechanism. Before the temporal dual branched convolution layer, several self-attention layers were augmented to learn feature weights adaptively. One input can generate three matrices: \(Q\) (query), \(K\) (key), and \(V\) (value). The self-attention mechanism is shown in Equations (2) and (3). \[Attention(Q,K,V)=softmax\big{(}QK^{T}/\sqrt{d_{k}}\big{)}V \tag{2}\]  
\[ \begin{cases}Q=W_{q}S\\ K=W_{k}S\\ V=W_{v}S\end{cases}  \tag{3}\]  
where \(d_{k}\) is the vector dimension of \(K\). \(S\) is the time series of the input. \(W_{q}\), \(W_{k}\), and \(W_{v}\) are the weight matrices for calculating the query, key, and value, respectively. The self-attention mechanism structure diagram is shown in Figure 4.  
Despite the prevalent usage of RNN and its derivative LSTM for time-series prediction, the computational complexity associated with this type of recurrent neural network infrastructure is considerably high. Conversely, CNN possesses several advantages, such as rapid training speed, uncomplicated structure, and the absence of dependency constraints. Consequently, we introduced the temporal dual branch-gated convolution, augmented with a self-attention mechanism, aiming to effectively discern the temporal features that aid in capturing cellular network traffic.