# Transformer-Based Hybrid Forecasting Model for Multivariate Renewable Energy
## 2 Related Works  
In 2012, [PERSON] and [PERSON] [25] released a review paper on the significant prediction and forecasting techniques for the energy consumption of buildings. The authors specifically contrasted physical models to machine learning and statistical models. According to the authors, machine learning-based models provided the most accuracy and adaptability, especially when compared to statistical models. Optimized applications is one of the areas where further research is suggested.  
In 2017, [PERSON] and [PERSON] [26] investigated the application of artificial intelligence-based models as well as comprehensive models for predicting and forecasting building energy demand. The authors broke down how artificial intelligence as a whole was used to anticipate the energy consumption of buildings. The majority of AI-based projects used hourly data applied to an entire building's capacity. The authors also looked into how general methodologies were used to predict the energy consumption of buildings. It was highlighted that similar assemblies had been widely used in domains other than building energy, with the results demonstrating their superior performance when compared to traditional prediction models.  
In 2020, [PERSON] et al. [27] review the state-of-the-art PV solar power forecasting, listing the best accuracy found with different models, forecast horizons, feature selection, and many other variables that can affect the performance of a forecasting model. Finally, they suggest using hybrid artificial neural networks and optimization for future works.  
In the context of time series forecasting, hybrid systems combine the strengths of both statistical techniques and Machine Learning (ML) for modeling the linear and nonlinear components of a time series. Many authors have successfully developed such hybrid models, combining the linear and nonlinear components in various ways. [PERSON] [18] assumes that a time series \(Z_{t}\) is composed by a linear relationship between linear (\(L_{t}\)) and nonlinear (\(N_{t}\)) temporal patterns (Equation (1)).  
\[Z_{t}=L_{t}+N_{t}. \tag{1}\]  
[PERSON] [18] proposes an architecture composed of three phases: (i) generation of the time series forecasting (\(\hat{L}_{t}\)) using the statistical linear model ARIMA, (ii) forecast (\(\hat{N}_{t}\)) of the residual series \(E_{t}\), calculated by Equation (2), using a Multilayer Perceptron neural network (MLP), and finally, (iii) combination of (i) and (ii) outputs using a simple sum (Equation (3)) resulting in \(\hat{Z}_{t}\).  
\[E_{t}=Z_{t}-\hat{L}_{t}. \tag{2}\]  
\[\hat{Z}_{t}=\hat{L}_{t}+\hat{N}_{t}. \tag{3}\]  
Several publications [19; 20; 21; 22] evaluated and compared multiple forecasting models for short-term PV energy generation and wind speed, including hybrid models and many variations of neural network techniques, highlighting their merits and drawbacks. However, residual series modeling is challenging because it presents heteroscedastic behavior, random fluctuations, and nonlinearity.  
The current paper aims to contribute by evaluating the hybridization of SARIMA with the Transformer neural network using exogenous variables to model a renewable energy data series.