## 3 DST-LEO Satellite-Traffic Prediction Model
### Model Architecture
#### 3.4.4 Attention Mechanism  
In this paper, the authors introduced temporal attention and spatial attention to enable the model to learn the importance of traffic information at each moment. This was used to process the low Earth orbit satellite traffic data involving time and space. Its goal is to extract important features by capturing the temporal and spatial dependencies in the data. The design steps of the attention mechanism are as follows:  
The input data are \(x\) with the shape of (B, T, N, F), where B is the batch_size, T is the length of the time series, N is the number of nodes, and F is the feature dimension. spatial_neighbors is the adjacency matrix.  
Temporal attention calculation: For each time lag i (ranging from 1 to num_lags. Here, the setting of the time step needs to consider the actual situation and the data collection frequency. In Section 4.1, as the collection frequency of the Abilene dataset was 5 min, it is reasonable to set the time step to 24 (2 h), and as the collection frequency of the traffic data in the internal low Earth orbit constellation business simulation system is 5 s, it is reasonable to set the time step to 30 (1 min). An overly long time step will lead to high memory usage, introduce noise, and even cause model overfitting, while an overly short time step may limit the learning ability of the model. The historical data and the current data are extracted for the lagged data lagged_x = x[;-;-;,:] and the current data current_x = x[;i,:,:,:], both of which have the shape of (B, T-i, N, F). Next, the historical data and the current data are weighted using the temporal attention weights. The formula is as follows, and the resulting data also have the shape of (B, T-i, N, F). \":\" indicates the selection of all elements in that dimension. For example, x[; ; ; ; ; ; ] means selecting all elements of the tensor x. \":i\" means taking the elements from the i-th position (including the i-th position) of that dimension to the end of that dimension, while \":-i\" means taking the elements from the starting position of that dimension to the i-th position from the end (excluding the i-th position from the end).  
\[attention^{(i)}=\sum_{f=1}^{F}W_{temp}^{(i-1)}\cdot lagged\_x_{;;\ldots,f}\cdot current \_x_{;;\ldots,f} \tag{9}\]  
The attention scores of all time lags are stored and concatenated together. The formula is as follows, and the resulting tensor has the shape of (B, T-num_lags, N, num_lags).  
\[\text{temporal\_attentions}=\left[\text{attention}^{(1)},\text{attention}^{(2) },\cdots,\text{attention}^{(\text{num\_lags})}\right] \tag{10}\]  
\[\text{temporal\_attention}=\text{concat}(\text{temporal\_attentions},\text{ dim}=1) \tag{11}\]  
Spatial attention calculation: Expand the data x to the shape of (B, T, N, 1, F) so that it can be multiplied with the spatial adjacency matrix. Weight the input data and the spatial adjacency matrix using the spatial attention weights Wspatial. The formula is as follows, and the shape of spatial_attention is (B, T, N, 1).  
\[\text{spatial\_attention}=\sum_{f=1}^{F}W_{spatial}^{(\text{f})}\cdot\cdot \cdot\cdot\text{x}_{;\ldots,f}\cdot\text{spatial\_neighbors} \tag{12}\]Expand the weighted spatial_attention to the shape of (B, T-num_lags, N, num_lags), then add it to the temporal attention, and perform a softmax operation to calculate the attention weights.  
\[\text{combined\_attention}=\text{temporal\_attention}+\text{spatial\_attention} \tag{13}\]  
\[\text{attention\_weights}=\text{softmax}(\text{combined\_attention},\text{ dim}=1) \tag{14}\]  
Expand the input data x to the shape of (B, T-num_lags, N, num_lags, F), and then use the attention weights to perform a weighted sum on the input data. The formula is as follows, and the resulting data has the shape of (B, N, F), representing the weighted features of each node.  
\[\text{output}=\sum_{\text{t}=1}^{\text{T-num\_lags}}\sum_{i=1}^{\text{num\_lags}} \text{attention\_weights}_{\text{,t,:}}\text{:}\text{x\_t,:} \tag{15}\]