# Research on Satellite Network Traffic Prediction Based on Improved GRU Neural Network
## 4 Traffic Prediction Method of the AT-GRU Satellite Network  
### Design of Coding Unit Based on Attention Mechanism  
To capture the long-term dependence on satellite traffic data, the encoder structure is based on the GRU unit [18; 19]. GRU is a variant network form derived from the recurrent neural network RNN which incorporates a gating unit into the basic structure of RNN and controls the flow of information through the \"gate\" structure. It can encode the input sequence as a feature representation, and combined with the attention mechanism [20; 21; 22],  
Figure 2: Traffic prediction model of AT-GRU satellite network.  
the purpose of the encoding stage is to learn the spatial relationship between input variables. The encoding stage is shown in Figure 3.  
Given the input sequence \(Y\), GRU neural network is used to learn the nonlinear mapping function between the input sequence and the hidden state \(h_{t}\) of the encoder at time \(t\):  
\[h_{t}=f_{e}(h_{t-1},Y), \tag{2}\]  
where \(f_{e}(\cdot)\) stands for GRU unit. The GRU unit update process is summarized as follows:  
\[r_{t}=\sigma(W_{r}[h_{t-1},Y]+b_{r}), \tag{3}\]  
\[z_{t}=\sigma(W_{z}[h_{t-1},Y]+b_{z}), \tag{4}\]  
\[\widetilde{h}_{t}=\tanh\bigl{(}W_{\widetilde{h}}[h_{t-1},Y]+b_{\widetilde{h} }\bigr{)}, \tag{5}\]  
\[h_{t}=(1-z_{t}).h_{t-1}+z_{t}.\widetilde{h}_{t}, \tag{6}\]  
where \(r_{t}\) is the reset gate, and how many previous states can be controlled by the reset gate that need to be memorized and stored; \(z_{t}\) is the update gate, which is used to control how cells store information; \(W_{r}\), \(W_{z}\), \(W_{\widetilde{h}}\) represent the hidden layer weight; \(b_{r}\), \(b_{z}\), \(b_{\widetilde{h}}\) represent offset values, which are all parameters that need to be optimized for learning; : indicates point multiplication relation; \([h_{t-1},Y]\) indicates the concatenation of the currently input network traffic data and the previous hidden state; \(\sigma\) means activation function.  
Combining the attention mechanism with GRU neural network, the spatial relationship between input variables can be adaptively learned. Given the Kth sequence \(Y^{k}\) of input data, the attention mechanism in the first stage can be expressed as  
\[e_{t}^{k}=v_{e}^{T}\tanh\Bigl{(}U_{e}\Bigl{[}h_{t-1},Y^{k}\Bigr{]}\Bigr{)}, \tag{7}\]  
\[\alpha_{t}^{k}=\frac{\exp\Bigl{(}e_{t}^{k}\Bigr{)}}{\sum\limits_{i=1}^{n}\exp \bigl{(}e_{t}^{i}\bigr{)}}, \tag{8}\]  
where \(h_{t-1}\) represents the historical hidden state of the encoder, \(v\) and \(U\) are all parameters to be learned, and \(e_{t}^{k}\) can be obtained by the tanh transformation. Attention weight (determined by the historical hidden state of the encoder and the current input value) is introduced to describe the importance of the kth input feature. Formula (8) ensures that all attention weights add up to \(1\).  
Figure 3: Coding stage.  
At this point, Formula (2) can be updated to:  
\[h_{t}=f_{e}(h_{t-1},\widetilde{x}_{t}). \tag{9}\]  
By designing an encoder with an attention mechanism, we can pay attention to the importance of different input sequences, instead of treating all input sequences uniformly.