# ForecastNet Wind Power Prediction Based on Spatio-Temporal Distribution
## 2 Materials and Methods
### Wind Power Prediction Model
#### 2.2.2 Graph Attention  
In order to establish spatial and temporal correlations between wind turbines in different geographic locations, an attention-based spatio-temporal graph network [38] is introduced into the time series prediction model. In order to improve the training efficiency and prediction accuracy of the model, the feature information needs to be filtered, and the information with low importance is ignored [39]. Graph attention networks process information by calculating the weights of the information and weighting the information according to certain weights to aggregate the information. Specifically, the attention score of the information is calculated using query  
Figure 3: Results of K-sums clustering. vector (Query), key-value vector (Key), and value vector (Value) [40]. The formula is shown in Equation (5):  
\[Attention(Q,K,V)=\sum\limits_{i}^{L}<Q,K_{i}>V_{i}, \tag{5}\]  
where \(Q\) is the feature vector of the current node, \(K\) is the feature vector of the neighboring node, and \(V\) is the feature vector of the neighboring node after mapping the weight value \(W\). An attention score is obtained by performing an inner product calculation on the feature vector of a node itself and the feature vectors of the neighboring nodes, and the score is weighted with the node's feature vectors after a normalization operation. Specifically:  
1. The central target node and neighbor node attention scores are calculated. The formula for calculating the value of attention on node \(i\) based on the features of node \(j\) is shown in Equation (6), where \(e_{ij}\) is the attention score between node \(i\) and node \(j\), \(a\) represents the correlation calculation function between nodes, \(h_{j}\) is the output vector of node \(i\), and \(W\) is the weight. \[e_{ij}=a(Wh_{i},Wh_{j}). \tag{6}\]
2. The weight scores are activated using the activation function. \[e_{ij}=LeakyReLU(a^{T}[Wh_{i}||Wh_{j}]). \tag{7}\]  
The calculation formula is shown in Equation (7), where \(||\) is the vector vertical splicing operation, which splices mapped column vectors; \(LeakyReLU\) is the activation function; \(a\) is the vector to be learnt; and \(a^{T}\) is the transpose of vector \(a\). 3. Weight normalization  
The sum of all the weights should be \(1\); therefore, the attention score is normalized. The weights are normalized for all neighboring node attention values of node \(i\) using the softmax function. The calculation formula is shown in Equation (8). \[a_{ij}=\frac{\exp(LeakyReLU(e_{ij}))}{\sum\limits_{k\in N_{i}}\exp(LeakyReLU( e_{ik}))}. \tag{8}\]
4. Information aggregation  
The graph attention layer adds the node's feature information and neighbor node's feature information according to a certain weight coefficient, performs feature extraction to form a new node to represent the feature, and outputs the new node feature as a result. The calculation formula is shown in Equation (9):  
\[h_{i}^{\prime}=\sigma(\sum\limits_{j\in N_{i}}a_{ij}Wh_{j}), \tag{9}\]  
where \(h_{i}^{\prime}\) is the new node features, \(\sigma(\cdot)\) is the activation function, and \(a_{ij}\) is the effect of node \(i\) features on the node \(j\) attention score. The features of the new node obtained according to the parameter \(W\) will be different in dimension from the features of the original node, and the parameter \(W\) will map the information of the original node to the new space. 5. Multi-head attention mechanisms  
The multi-head attention mechanism introduces multiple attention mechanisms to aggregate information, and each attention mechanism is able to focus on different features to enhance the expression ability of the attention layer [41].