## 4 STGPT2 UGAN Design
### Adversarial Training Strategy  
Most LLM-based time series models strive to pursue a supervised training strategy, i.e., minimizing the MSE loss. Unfortunately, minimizing the MSE loss can only align the true values and the predicted values: it cannot learn the distribution of the true data. This results in limited prediction precision. To tackle the problem, we decided to use an adversarial training strategy, which contains supervised training and unsupervised training.  
To train the parameters of the model, a loss function is needed to measure the prediction accuracy of the model. During the training phase, _Gen_ and _Dis_ optimize the output as they compete with each other; _Gen_ makes attempts to maximize the probability that its output is true judged by the discriminator, while _Dis_ tries to minimize the probability that the generator output is true. _Gen_ and _Dis_ are trained alternately. In each step of the training, one of them is fixed while the other performs the optimization of parameters. As a whole, _Dis_ and _Gen_ play a two-player minimax game, and the whole objective function to be optimized is denoted as follows:  
\[\begin{split}\min_{\textit{Gen}}\max_{\textit{Dis}}V(\textit{ Dis},\textit{Gen})=& E_{z\sim P(R_{E})}[\log(\textit{Dis}(z\mid\mathcal{X})]\\ &+E[\log(1-\textit{Dis}(\textit{Gen}(\mathcal{X})))],\end{split} \tag{12}\]  
where \(E\) denotes the empirical estimation of the expected probability value. \(z\sim P(R_{E})\) indicates that the training data \(z\) adheres to the real sample's distribution \(P(R_{E})\). \(\mathcal{X}\) denotes the wind speed representations of historical \(P\) timesteps. \(\textit{Gen}(\mathcal{X})\) is the generator's output. \(\textit{Dis}(z\mid\mathcal{X})\) indicates the probability produced by _Dis_. In summary, _Dis_ endeavors to minimize \(\log(1-\textit{Dis}(\textit{Gen}(\mathcal{X})))\) using supervised training.  
The objective of _Gen_ is to precisely assign the real sample's distribution and mislead _Dis_ as far as possible. The _Gen_'s loss function is expressed as follows:  
\[L_{\textit{Gen}}=\alpha L_{\textit{F2R}}+L_{\textit{mse}}+L_{2}, \tag{13}\]  
where \(a\) signifies the adversarial factor that controls the degree of adversariality between _Gen_ and _Dis_. \(a\) is set as 0.01. \(L_{\textit{F2R}}\) stands for the loss that _Dis_ mistakenly evaluates predicted sequences (fake samples) as real. It tries to make the predicted wind speed sequences generated by _Gen_ closer to the real values, defined as  
\[L_{\textit{F2R}}=-\log(\textit{Dis}(\textit{Gen}(\mathcal{X}))). \tag{14}\]  
\(L_{\textit{mse}}\) stands for the Mean Squared Error (MSE) function, which can be denoted as  
\[L_{\textit{mse}}=\frac{\sum_{t_{z}=1}^{Q}\sum_{s=1}^{N}\left(\vec{x}_{WP}^{t_{ z},s}-g_{WP}^{t_{z},s}\right)^{2}}{Q\times N}, \tag{15}\]  
where \(\vec{x}_{WP}^{t_{z},s}\in\overline{\mathcal{X}}_{WP}\) and \(\vec{y}_{WP}^{t_{z},s}\in\overline{\mathcal{Y}}_{WP}\) denote real values and predicted values on the turbine \(s\) at timestep \(t_{z}\). The purpose is to converge the predicted values to the real values.  
\(L_{2}\) is the L2 norm, which is adopted to alleviate model overfitting.  
_Dis_'s intention is to assess whether a sample derives from the real distribution by the unsupervised training. The entire loss function (adversarial loss) of _Dis_ is expressed as \[L_{Dis}=L_{F}+L_{R}, \tag{16}\]  
where \(L_{F}\) denotes the loss function that the forecasted results from \(Gen\) are false, defined as  
\[L_{F}=-\log(1-Dis(Gen(\mathcal{X}))). \tag{17}\]  
\(L_{R}\) is the loss function that the real value is true, denoted as  
\[L_{R}=-\log(Dis(z\mid\mathcal{X})). \tag{18}\]  
During the training phase, \(Gen\) and \(Dis\) play against each other until they reach Nash equilibrium. At this time, the sample generated by \(Gen\) is almost indistinguishable from the real sample, the probability determined by \(Dis\) infinitely approaches to \(1/2\), and the optimal solution is obtained as the \(Dis\) finds it hard to distinguish fake samples.