# Transformer-Based Hybrid Forecasting Model for Multivariate Renewable Energy
## 3 H-Transformer
### Experimental Results  
Table 3 shows the results with both RMSE and MAE metrics for single and hybrid approaches on the testing set for all datasets. The error on solar datasets corresponds to power generation in KWh, while the wind datasets' loss represents the wind speed in m/s. Since the metrics are different, Table 3 provides a comparison between models in the same dataset, not between datasets. Overall, the hybrid approaches achieved better results than respective single models. The proposed H-Transformer attained the best RMSE metrics in 3 out of 5 time series. In MAE terms, the proposal reached the best values in 4 out of 5 time series. Table 4 shows the attained CV(RMSE) metric for all approaches on the testing set of all datasets using the results from Table 3. CV(RMSE) permits a comparison of the model's performance between datasets, where lower values mean that the model outputs a better forecast. The H-Transformer achieved better performance on Wind datasets, and had a CV(RMSE) less than 25% in 4 out of 5 series. Overall, the hybrid approaches attained better results than their single counterparts. The proposed hybrid system, the H-Transformer, outperformed all single models when analyzing the RMSE and achieved the best MAE in all datasets compared to other hybrid systems. Figure 3 shows that in most of the time series, the H-Transformer attained a remarkable gain from other models. For instance, in all datasets, the proposed hybrid system improved the SARIMA accuracy, reaching an accuracy superior to the single Transformer model. This result corroborates the hypotheses that support the proposition of the H-Transformer. Similarly, Figure 4 shows the superior result in terms of MAE of the proposed hybrid system. For instance, the H-Transfomer attained a percentage gain greater than 30% in  
four out of five data sets compared to the SARIMA model. Even in the cases where other models outperformed the H-Transformer, the percentage difference is considerably low. Figure 5 shows the forecasting for the last 100 actual values (blue line) of the five renewable energy time series used in the study. For all data sets, the comparison is performed between the forecasts of the SARIMA model(red line) and the proposed H-Transformer (green line). Figure 4: The percentage difference (Equation (11)) between H-Transformer and literature methods in terms of MAE. Figure 3: The percentage difference (Equation (11)) between H-Transformer and literature methods in terms of RMSE. \begin{table}
\begin{tabular}{c c c c c c c c c c c} \hline \hline \multicolumn{2}{c}{**Model Info**} & \multicolumn{2}{c}{_Solar1_} & \multicolumn{2}{c}{_Solar2_} & \multicolumn{2}{c}{_Wind1_} & \multicolumn{2}{c}{_Wind2_} & \multicolumn{2}{c}{_Wind3_} \\ \hline
**Approach** & **Model** & **RMSE** & **MAE** & **RMSE** & **MAE** & **RMSE** & **MAE** & **RMSE** & **MAE** & **RMSE** & **MAE** \\ \hline \multirow{4}{*}{Single} & SARIMA [23] & 975 & 756 & 1674 & 1451 & 0.810 & 0.659 & 1.115 & 0.930 & 0.795 & 0.632 \\  & RNN [33] & 807 & 548 & 1568 & 1258 & 0.710 & 0.543 & 0.907 & 0.699 & 0.697 & 0.529 \\  & LSTM [34] & 817 & 570 & 1455 & 1201 & 0.714 & 0.550 & 0.960 & 0.738 & 0.701 & 0.528 \\  & GRU [35] & 793 & 526 & 1531 & 1176 & 0.680 & 0.511 & 0.849 & 0.656 & 0.681 & **0.503** \\  & Transformer [24] & 872 & 594 & 2438 & 1914 & 0.952 & 0.720 & 0.907 & 0.697 & 0.851 & 0.601 \\ \hline \multirow{4}{*}{Hybrid} & SARIMA + RNN & 803 & 577 & **1252** & 939 & 0.637 & 0.482 & 0.847 & 0.653 & 0.673 & 0.539 \\  & SARIMA + LSTM & 807 & 580 & 1460 & 1097 & **0.611** & 0.438 & 0.846 & 0.656 & 0.671 & 0.538 \\  & SARIMA + GRU & 871 & 608 & 1300 & 971 & 0.645 & 0.466 & 0.849 & 0.660 & 0.671 & 0.537 \\  & H-Transformer & **766** & **516** & 1307 & **934** & 0.623 & **0.422** & **0.838** & **0.645** & **0.665** & 0.533 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: RMSE and MAE performance metrics of the H-Transformer, single and hybrid approaches for renewable time series. The best value for each time series is highlighted in bold.