## 4 The Proposed G-TIAG
### Design of Feature Representation Layers
#### 4.3.2 Design of GRU Structure  
Different attacks differ in their topological changes over a specific time interval. To effectively extract the variability between TIAGs in adjacent time intervals, we use the GRU as the basis for temporal feature extraction. GRUs use a threshold mechanism to remember as much long-term information as possible. They can capture time-dependent features by capturing the current network information and preserving the changing trend of historical network information. Compared with LSTM, GRU units have fewer parameters and lower complexity, and compared with RNN, GRU can solve the gradient disappearance and gradient explosion problems. We combine GRU with a self-attention capable of capturing critical information to achieve the representation of the changes and evolutionary patterns of TIAGs in adjacent time intervals.  
We design this part to consist of a GRU and a self-attention. First, the output vector \(X\) of GCN is input to the GRU unit to obtain the state vector \(h_{t}\) in each time step \(L\). Using the hidden state at the previous moment \(t-1\) and the input vector at the current moment \(t\), the feature vector \(t\) is obtained using the following equation.  
\[z_{t}=\sigma_{s}(x_{t}U^{z}+h_{t-1}W^{z}+b^{z}) \tag{8}\]\[r_{t}=\sigma_{s}(x_{t}U^{t}+h_{t-1}W^{r}+b^{r}) \tag{9}\]  
\[\widetilde{h}_{t}=\tanh\Bigl{(}x_{t}U^{h}+(r_{t}\odot h_{t-1})W^{h}+b^{h}\Bigr{)} \tag{10}\]  
\[h_{t}=(1-z_{t})\odot h_{t-1}+z_{t}\odot\widetilde{h}_{t} \tag{11}\]  
where \(z_{t}\) is the output of the update gate to control the effect of the feature \(h_{t-1}\) at moment \(t-1\) on the current moment \(t\). \(r_{t}\) is the output of the reset gate to control the extent to which features \(h_{t-1}\) at the moment \(t-1\) are ignored. U, W is the training weight matrix, \(\sigma_{s}\) is the Sigmoid activation function and \(b^{z}\), \(b^{r}\), \(b^{h}\) are the bias terms. In experiments, we find that GRU has difficulty processing longer vectors, and the hidden representation of longer vectors appears in the next moment. Therefore, we design to add self-attention after the GRU to capture adequate information. The output vector \(h_{t}\) of the GRU is sent to self-attention to calculate the attention values of neighboring nodes and assign different weights to each neighboring node. First, for an input vector \(h_{i}\), we use Equation (12) to obtain the \(e_{i}\), which is responsible for measuring the importance of \(h_{i}\). \(W_{(1)}\)and\(W_{(2)}\) are the hyperparameters, and \(b_{(1)}\) and \(b_{(2)}\) are the bias terms. Then, we use Equation (13) to obtain \(A_{attr(i)}\), which denotes the probability of selecting the ith input vector. The \(n\) value is the number of input vectors.  
\[e_{i}=W_{(2)}\Bigl{(}W_{(1)}h_{i}+b_{(1)}\Bigr{)}+b_{(2)} \tag{12}\]  
\[A_{attr(i)}=\frac{exp(e_{i})}{\sum_{k=1}^{n}exp(e_{k})} \tag{13}\]  
Finally, the final representation is obtained by using the weight sum method according to the input vector \(h_{i}\) and \(A_{attr(i)}\), as shown in equation (14).  
\[Z_{t}=\sum_{i=1}^{n}A_{attr(i)}\;\times\;h_{i} \tag{14}\]  
With all the above designs, G-TIAG can capture spatial features of TIAGs at each moment and mine the difference of TIAGs at adjacent moments. It will be verified in the experimental results in the next section.