# Transformer-Based Hybrid Forecasting Model for Multivariate Renewable Energy
## 5 Conclusions  
Due to the wide variation of the solar photovoltaic and the volatile nature of the wind speed, the necessity for developing accurate systems to forecast renewable energy generation has been increasing. Numerous studies have considered different methods and metrics to improve such approaches' performance. However, issues regarding model nature, parameter selection, weather, and environmental conditions affect the performance in renewable time series forecasting. This work proposed a hybrid system that combines the traditional linear SARIMA model with a state-of-the-art Transformer model using the residual series. The proposal, H-Transfomer, was developed for renewable energy forecast, aiming to deal with the abovementioned issues. The well-known Bayesian optimization was employed to select the best hyperparameters and the number of lags of the Transformer. In the nonlinear modeling of the residual series, exogenous data were considered in an unprecedented way to improve the accuracy of the Transformer model. The proposed hybrid system was compared with single and hybrid approaches in the literature using five renewable energy time series. The results in terms of two well-known evaluation metrics were used to compare each model on the test set: RMSE and MAE. The hybrid system, H-Transformer, obtained the best overall result, showing that the proposal can be an interesting tool to attain accurate forecasts in the renewable energy scenario. This research, however, is subject to limitations:  
* The optimization step requires multiple executions of deep neural networks, which require high computational power. Powerful machines can execute the optimization step much faster with many more combinations, possibly outputting better results;
* There are not many open multivariate renewable energy datasets available, which limits the comparison and evaluation of the models;
* The proposed hybrid system employs a linear combination of statistical and ML models. Some works show that nonlinear combinations can be more accurate than linear ones [29] in the forecasting task;
* Selecting the perfect combination of models is challenging due to the no-free-lunch theorem [40] in search and optimization problems. Multiple variables and parameters influence the performance of each model differently, which makes the choice of models and combinations a challenging task itself. Alternative versions of the Transformer and attention-based models have been created specifically for the time series forecasting field and produced promising results [41]. Hybrid systems improved the performance of simple Transformers for renewable energy forecast, so testing hybrid systems with those alternative Transformers is expected to output even better results. Improvements on hybrid systems are also an attractive approach for future works. [PERSON] et al. [42] proposed a three-phase hybrid system for forecasting energy consumption for smart meters. Their system outperformed previously tested models. A similar hybrid system could be developed using Transformers for renewable energy forecast. Future works may include a more comprehensive number of exogenous variables and running a feature selection on each dataset. Finally, the optimization step can be improved by increasing the number of hyperparameters and their ranges and using a more suitable fitness function [43]. As concluded by [PERSON] et al. [27], evolutionary algorithms have performed well on the optimization task of renewable energy forecasting models. **Author Contributions:** Conceptualization, G.A.G.P. and P.S.G.d.M.N. ; methodology, P.S.G.d.M.N. ; software, G.A.G.P. ; validation, J.J.J. and P.S.G.d.M.N. ; formal analysis, P.S.G.d.M.N. and J.J.J. ; investigation, G.A.G.P. and J.K.; resources, G.A.G.P., P.S.G.d.M.N. and J.J.J. ; data curation, G.A.G.P. and P.S.G.d.M.N. ; writing--original draft preparation, G.A.G.P. ; writing--review and editing, P.S.G.d.M.N., J.J.J. and J.K.; visualization, G.A.G.P. and J.K.; supervision, P.S.G.d.M.N. ; project administration, J.J.J.J.