## 4 STGPT2 UGAN Design
### Generator (Gen) Design  
In order to model spatio-temporal correlations and produce wind speed series in the future \(Q\) timesteps, the generator with STGPT-2 as the backbone is devised. Formed of three structures (e.g., the feature embedding layer, STB, and TGPT-2), STGPT-2 is designed as a pipeline structure, as shown in Figure 2a. In the following, we provide a detailed introduction to the three structures and how they collaborate to mine spatio-temporal features and produce predicted wind speed series. _(1) Feature embedding layer_: Considering that deep neural networks cannot directly deal with original wind speed representations, we must transform the original representations \(\mathcal{X}\in\mathbb{R}^{P\times N\times K}\) of historical \(P\) timesteps into higher dimensions. In particular, the two FC layers are leveraged to convert the dimensions from \(K\) to \(D\), represented as \(\mathcal{H}=\left(H^{t_{1}},H^{t_{2}},\ldots,H^{t_{p}}\right)\in\mathbb{R}^{P \times N\times D}\). _(2) STB_: To capture spatio-temporal correlations in wind speed representations, the STB is proposed. In particular, the wind speed embeddings of historical \(P\) timesteps \(\mathcal{H}\in\mathbb{R}^{P\times N\times D}\) are acquired through the feature embedding layer. Then, we design the STB to mine spatio-temporal features from these embeddings. As described in Figure 2a, the STB consists of \(B\) stacked STA modules. Given \(\mathcal{H}\) as the input of \(B\) stacked STA modules, the \(B\)th STA module produces the final spatio-temporal representations, represented as \(\mathcal{HST}^{B}\in\mathbb{R}^{P\times N\times D}\). Each STA module contains spatial attention and temporal attention. The former is first adopted to model spatial dependencies. The latter is then leveraged to extract temporal dependencies. _Spatial Attention_: In order to mine spatial dependencies, we use spatial attention based on a single-layer graph attention network, as depicted in Figure 2b. Such a model has the ability to dynamically allocate weights to turbines in a turbine network founded on correlations between turbines. Figure 1: Overview of the proposed STGPT2 UGAN. \(\mathcal{X}\) is the input of \(Gen.\)\(\mathcal{Y}_{1},\mathcal{Y}_{2},\mathcal{Y}_{N}\in\overline{\mathcal{Y}}_{WP}\) are the predicted wind speed sequences generated by \(Gen.\)\(\mathcal{X}_{WP}\) is the wind speed sequence in the past \(P\) timesteps. \(\overline{\mathcal{X}}_{WP}\) is the real wind speed sequence in the future \(Q\) timesteps. \(\overline{\mathcal{Y}}_{WP}\) is the predicted wind speed sequence in the future \(Q\) timesteps. \(Dis(R_{E})\) represents the probability that an input sample comes from the real sample set \(R_{E}\). \(Dis(F_{A})\) denotes the probability that an input sample comes from the fake sample set \(F_{A}\). In the \(b\)th STA module, let the wind speed representations \(\mathcal{HST}^{b-1}\in\mathbb{R}^{P\times N\times D}\) of historical \(P\) timesteps generated by the \(b-1\)th STA module be the input of spatial attention, in which the representations at the timestep \(t_{k}\) are denoted as \(HST_{t_{k}}^{b-1}\in\mathbb{R}^{N\times D}\). The spatial representations generated by spatial attention are \(\mathcal{HS}^{b}\in\mathbb{R}^{P\times N\times D}\), in which \(HS_{t_{k}}^{b}\in\mathbb{R}^{N\times D}\) denotes the representations at timestep \(t_{k}\). Specifically, at timestep \(t_{k}\), the \(z\)th head operation of the single-layer graph attention network is calculated as follows:  
\[HS_{z,t_{k}}^{b}=\text{Softmax}\left(\alpha Q_{z}^{S}\Big{(}K_{z}^{S}\Big{)}^{ \top}\right)V_{z}^{S}, \tag{2}\]  
where \(HS_{z,t_{k}}^{b}\in\mathbb{R}^{N\times(\frac{D}{N_{z}})}\) indicates the \(z\)th head's spatial representations at timestep \(t_{k}\).