## 3 FDIA Detection Model Based on GAN-GRU and Transformer
### FDIA Detection Model Based on the Transformer Neural Network
#### 3.2.2 Multi-Head Attention  
The scaled dot-product attention within multi-head attention represents a distinctive form of attention mechanism. In practical applications, the attention function for a set of queries is computed concurrently, and these computations are aggregated into a matrix \(\mathbf{Q}\). The encoding of keys and values into matrices \(\mathbf{K}\) and \(\mathbf{V}\) follows a similar approach. The computation of the output matrix is then formulated as follows:  
\[\text{Attention}(\mathbf{Q},\mathbf{K},\mathbf{V})=\text{softmax}\bigg{(}\frac{\mathbf{Q}\mathbf{ K}^{T}}{\sqrt{d_{k}}}\bigg{)}\mathbf{V}, \tag{16}\]  
where \(d_{k}\) represents queries and keys of dimension.  
In contrast to the isolated application of dimensional keys, values, and queries to a single attention function, we determined that projecting queries, keys, and values linearly to the \(\mathbf{Q}\), \(\mathbf{K}\), and \(\mathbf{V}\) dimensions, respectively, yields considerable benefits. Consequently, for each projected version of the query, key, and value, we concurrently execute the attention function, generating results encompassing \(\mathbf{Q}\), \(\mathbf{K}\), and \(\mathbf{V}\) dimensional output values. These output values are subsequently concatenated and subjected to further projection to generate the ultimate output values, as elucidated in Figure 4.  
The incorporation of a multi-head attention mechanism empowers the model with the capacity to collectively attend to distinct subspaces associated with diverse positions. However, for an individual attention head, this effect is mitigated through averaging, as computed below:  
\[\text{MultiHead}(\mathbf{Q},\mathbf{K},\mathbf{V})=\text{Concat}(\text{head}_{1},\dots, \text{head}_{\text{h}})\mathbf{W}^{\mathbf{O}}, \tag{17}\]\[\text{head}_{i}=\text{Attention}\Big{(}\mathbf{Q}\mathbf{W}_{i}^{\mathbf{Q}},\mathbf{K}\mathbf{W}_{i}^{ \mathbf{K}},\mathbf{V}\mathbf{W}_{i}^{\mathbf{V}}\Big{)}, \tag{18}\]  
where \(\text{MultiHead}(\mathbf{Q},\mathbf{K},\mathbf{V})\) represents the result of the calculation of the multi-head attention, and \(\text{h}=8\) represents the number of parallel attention layers in the model. \(\mathbf{W}_{i}^{\mathbf{Q}}\), \(\mathbf{W}_{i}^{\mathbf{K}}\), \(\mathbf{W}_{i}^{\mathbf{V}}\), and \(\mathbf{W}^{\mathbf{O}}\) represent the projection parameter matrices corresponding to \(\mathbf{Q}\), \(\mathbf{K}\), and \(\mathbf{V}\), along with the computed results.  
Lastly, the feature vectors are aligned to the same dimensions for classification tasks by means of a linear layer. Each dataset, equipped with labels 0, 1, 2, and 3, undergoes processing to yield a conclusive output probability distribution. This distribution serves as the basis for predicting the classification outcomes concerning manipulated load data. To enhance interpretability, a normalized confusion matrix representing the distribution outcomes can be generated. Through the self-attention mechanism and parallel computing capabilities inherent to the Transformer, our methodology effectively harnesses sequence data insights and extracts elevated-level feature representations. This proficiency enables the precise classification of altered load data within power systems.