## 5 Experiments
### Experimental Results
#### 5.2.4 Effect of Hyperparameters  
Figures 4 and 5 show the average RMSE and MAE of STGPT2 UGAN on wind speed prediction in the next 12 h through the DSWE 1 and NREL datasets. When one hyperparameter is adjusted, the other hyperparameters maintain their optimal values (e.g., \(L_{SA}=1\), \(L_{TA}=1\), \(M=4\), \(d=8\), \(B=2\), and \(L_{r}=1\times 10^{-3}\)). Bayesian optimization is employed to obtain the optimal hyperparameters [34]. In particular, we first specify the range of values for each hyperparameter. Then, we design an objective function to measure the prediction accuracy of the hyperparameter groups. Furthermore, the Gaussian process is chosen as a suitable prior scheme to illustrate the relationship between hyperparameters and the objective function. We leverage the Gaussian process sampling to determine the following hyperparameter group for assessment. For each hyperparameter group, its prediction accuracy is recorded and evaluated through the objective function. According to the evaluated hyperparameter groups, we iteratively update the Gaussian process, intending to accurately explore the prediction precision of the remaining hyperparameter groups. The above process is repeated until the objective function achieves convergence. From Figures 4c-e and 5c-e, we observe that more sophisticated structures are more likely to result in underfitting, while simpler structures are more prone to overfitting. Figures 4a,b and 5a,b illustrate that fewer layers reach the highest prediction precision. The potential reason is that excess layers bring about cumulative errors. From Figures 4f and 5f, we observe that a smaller learning rate is prone to getting stuck in local optima, while a larger learning rate is more likely to skip the global optimum. In view of this, it is vital to opt for a suitable learning rate.  
\begin{table}
\begin{tabular}{l c c c c c c c c c c} \hline \hline
**Dataset** & **Model** & \multicolumn{3}{c}{**1-4 h**} & \multicolumn{3}{c}{**5-8 h**} & \multicolumn{3}{c}{**9-12 h**} & \multicolumn{3}{c}{**Average**} & \multicolumn{1}{c}{**TT**} & \multicolumn{1}{c}{**GPUM**} \\  & **RMSE** & **MAE** & **RMSE** & **MAE** & **RMSE** & **MAE** & **RMSE** & **MAE** & **RMSE** & **MAE** \\ \hline \multirow{6}{*}{DSWE 1} & Basic & 15.21 & 10.86 & 15.21 & 10.86 & 15.21 & 10.86 & 15.21 & 10.86 & **5.27 min** & **63.29 M** \\  & +TA & 1.94 & 1.47 & 2.52 & 2.00 & 3.01 & 2.43 & 2.49 & 1.97 & 10.49 min & 205.69 M \\  & +SA & 1.79 & 1.33 & 2.36 & 1.86 & 2.89 & 2.30 & 2.35 & 1.83 & 14.22 min & 272.95 M \\  & +TGPT2 & 1.68 & 1.29 & 2.26 & 1.75 & 2.80 & 2.14 & 2.25 & 1.73 & 16.51 min & 286.34 M \\  & STGPT2 UGAN & **1.59** & **1.20** & **2.17** & **1.66** & **2.62** & **2.07** & **2.13** & **1.64** & 18.63 min & 314.42 M \\ \hline \multirow{6}{*}{NREL} & Basic & 21.15 & 14.72 & 21.15 & 14.72 & 21.15 & 14.72 & 21.15 & 14.72 & **4.39 min** & **56.57 M** \\  & +TA & 3.46 & 2.69 & 4.42 & 3.48 & 5.40 & 4.43 & 4.43 & 3.53 & 9.37 min & 192.14 M \\ \cline{1-1}  & +SA & 3.36 & 2.60 & 4.29 & 3.38 & 5.31 & 4.30 & 4.32 & 3.43 & 11.26 min & 225.14 M \\ \cline{1-1}  & +TGPT2 & 3.31 & 2.57 & 4.16 & 3.32 & 5.21 & 4.20 & 4.23 & 3.36 & 12.58 min & 257.25 M \\ \cline{1-1}  & STGPT2 UGAN & **3.21** & **2.45** & **4.03** & **3.19** & **5.08** & **4.02** & **4.11** & **3.22** & 13.32 min & 279.42 M \\ \hline \hline \end{tabular}
\end{table}
Table 3: Ablation study of STLIMUGAN for wind speed prediction on DSWE 1 and NREL datasets. Minute is represented as min. M stands for megabit (Mb).