## 4 Result and Discussion
### Regression Model Performance
For Polynomial Regression and Support Vector Regression, the Med AE is notably reduced with cycling 12 and cycling 24, albeit to a lesser extent than Linear Regression. K-Nearest Neighbor Regression displays the absolute consistency for Med AE at 0 with cycling 24 and cycling 12, underscoring the benefit of this preprocessing in reducing the typical error in predictions. Cycling 12 and cycling 24 enhance Linear Regression performance, as indicated by substantially lower MSE and MAE values compared to the original and positional methods. Specifically, the cycling 24 method appears to be most effective in capturing full daily patterns within the data, which is beneficial for time series datasets with pronounced diurnal or seasonal variations. For Polynomial Regression, a slight improvement in MSE with cycling 24 compared to cycling 12 is noted, indicating some benefit from considering the entire daily cycle. In the case of SVR, the improvements in MAE suggest that both cycling methods can reduce the typical prediction error, although the extent of improvement is less than that observed in Linear Regression. The KNN model exhibits the most remarkable improvement in MAE with cycling 24, emphasizing the value of an entire 24 h cycle in capturing the inherent temporal dynamics of the dataset. This improvement is significant, as it highlights that integrating cyclical temporal information into preprocessing can substantially enhance predictive accuracy for models sensitive to time-based patterns. The RMSE analysis aligns with these findings, where Linear Regression benefits significantly from the cycling 12 and cycling 24 methods, revealing a solid fit to the data when temporal cycles are considered. KNN Regression also improves with these cycling-based preprocessing methods, validating the value of the cycling layer. Overall, the results from the comparative analysis suggest that cycling 12 and cycling 24 preprocessing methods redefine the paradigm in regression analysis by incorporating cyclical patterns, leading to more accurate predictions in strong cyclical patterns in time series data. While Figure 6 indicates that Linear Regression achieves the lowest MSE in this specific dataset, which does not necessarily imply that it is the optimal choice for all scenarios. The proposed cycling layer integrated with self-attention mechanisms demonstrates superior performance in metrics, such as the Mean Absolute Error (MAE) and Root Mean Squared  
Figure 9: MedAE for four different regressions. Error (RMSE), which are more indicative of real-world performance when dealing with complex and non-linear time series data. Moreover, our approach captures long-term cyclical patterns that Linear Regression cannot model effectively, as evidenced by its stable performance across different datasets and extended time horizons. This adaptability and robustness make our method more suitable for applications that require handling dynamic, non-linear data patterns, such as environmental monitoring, financial forecasting, and anomaly detection in industrial settings. To further assess the efficacy of the cycling layer self-attention autoencoder preprocessing, we conducted comparative experiments involving four regression models in conjunction with the existing sequence model. In this experiment, the same data split was used to maintain consistency, except that both LSTM and GRU models included 12 h and 24 h time factors as focal points. In Table 4, it can be seen that the cycling layer preprocessing with Linear Regression under the 24 h time factor significantly leads in MAE performance. For MSE, the cycling layer preprocessing with KNN regression achieves the best performance. Other regression methods also outperform LSTM and GRU models to varying degrees, which indirectly proves the advantage of using cycling layer self-attention preprocessing in this data environment. Comparing other research from the same dataset, it can still show the novelty. [PERSON] et al.