# KAN-Transformer Model for UltraShort-Term Wind Power Prediction Based on EWMA Data Processing
## 3 Experimental Analysis
### Optimization Analysis of Transformer Model Based on EWMA and KAN  
The KAN network is a novel deep-learning algorithm that captures nonlinear relationships by introducing learnable activation functions at the network's edges [26,27]. It has significant advantages in handling complex function fitting and nonlinear patterns, and it has been demonstrated to be an effective method for improving accuracy in the field of time series forecasting [28].  
To verify that the KAN network can effectively improve the prediction accuracy of the Transformer model, we replace the final linear layer of the Transformer model with a KAN layer. This modification improves the model's ability to capture nonlinear relationships. The comparison of prediction curves between the Transformer model and the KAN-Transformer model is shown in Figure 13, and the comparison of test set prediction evaluation metrics is shown in Table 3.  
Data from 3 days are selected for plotting in Figure 13, and Table 3 shows the results of calculations for the entire test set data. In Figure 13, the orange curve represents the prediction of the Transformer model, while the green curve represents the prediction of the KAN-Transformer model. Table 3 indicates that the KAN-Transformer model reduces MAE and RMSE by 0.41 MW and 0.49 MW, respectively, and increases R\({}^{2}\) by 0.18% compared to the Transformer model. This demonstrates that the KAN layer's ability to self-learn non-linear functions results in higher prediction accuracy than the Transformer model's linear mapping, highlighting the importance of capturing non-linear relationships.  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Hyperparameter** & **Parameter Values** \\ \hline Batch\_size & 32 \\ Train\_epochs & 15 \\ Kenel\_size & 3 \\ D\_model & 512 \\ H\_heads & 8 \\ Encoder\_layers & 2 \\ Decoder\_layers & 1 \\ Dropout & 0.05 \\ Learning\_rate & Adaptive Optimization \\ \hline \hline \end{tabular}
\end{table}
Table 2: Hyperparameter configuration.  
EWMA, with its exponential weighting average characteristic, helps to smooth time series data and can effectively address issues related to noise and outliers in the data [29; 30], which is particularly important in applications such as time series forecasting. In the context of wind power forecasting, noise from external variables like wind speed and wind direction can directly impact the accuracy of power predictions. By applying EWMA smoothing, we can preprocess these external variables, providing the model with more stable feature inputs. This paper first introduces the KAN network based on the Transformer time series forecasting model and then combines it with EWMA to suppress noise in the data. To validate that the combination of the two achieves the highest prediction accuracy, we compare the KAN-Transformer with the EWMA-KAN-Transformer model. The comparison of prediction curves between the KAN-Transformer model and the EWMA-KAN-Transformer model is shown in Figure 14, and the evaluation metrics for the test set are compared in Table 4.  
Data from 3 days is selected in Figure 14 for plotting, and Table 4 shows the results calculated from the entire test set data. In Table 4, the KAN-Transformer model has a reduction in MAE and RMSE by 0.08 MW and 0.35 MW, respectively, and an increase in R\({}^{2}\) by 0.12% compared to the EWMA-KAN-Transformer model. This confirms that applying EWMA for denoising when introducing multiple variable features improves the model's prediction accuracy for future wind power values. From Tables 3 and 4, it can be seen that the EWMA-KAN Transformer model reduces MAE and RMSE by 0.49 MW and 0.84 MW, respectively, and increases R\({}^{2}\) by 0.3% compared to the Transformer model.