Forecasting of PV plant output using hybrid wavelet‐based LSTM‐DNN structure model

IET Renewable Power GenerationVolume 13, Issue 7 p. 1087-1095 Research ArticleFree Access Forecasting of PV plant output using hybrid wavelet-based LSTM-DNN structure model Juan Ospina, Corresponding Author Juan Ospina jjospina@fsu.edu Department of Electrical and Computer Engineering, FAMU-FSU College of Engineering, 2525 Pottsdamer St, Tallahassee, FL, 32310 USA Center for Advanced Power Systems (CAPS), 2000 Levy Ave, Tallahassee, FL, 32310 USASearch for more papers by this authorAlvi Newaz, Alvi Newaz orcid.org/0000-0001-7317-5352 Department of Electrical and Computer Engineering, FAMU-FSU College of Engineering, 2525 Pottsdamer St, Tallahassee, FL, 32310 USA Center for Advanced Power Systems (CAPS), 2000 Levy Ave, Tallahassee, FL, 32310 USASearch for more papers by this authorM. Omar Faruque, M. Omar Faruque Department of Electrical and Computer Engineering, FAMU-FSU College of Engineering, 2525 Pottsdamer St, Tallahassee, FL, 32310 USA Center for Advanced Power Systems (CAPS), 2000 Levy Ave, Tallahassee, FL, 32310 USASearch for more papers by this author Juan Ospina, Corresponding Author Juan Ospina jjospina@fsu.edu Department of Electrical and Computer Engineering, FAMU-FSU College of Engineering, 2525 Pottsdamer St, Tallahassee, FL, 32310 USA Center for Advanced Power Systems (CAPS), 2000 Levy Ave, Tallahassee, FL, 32310 USASearch for more papers by this authorAlvi Newaz, Alvi Newaz orcid.org/0000-0001-7317-5352 Department of Electrical and Computer Engineering, FAMU-FSU College of Engineering, 2525 Pottsdamer St, Tallahassee, FL, 32310 USA Center for Advanced Power Systems (CAPS), 2000 Levy Ave, Tallahassee, FL, 32310 USASearch for more papers by this authorM. Omar Faruque, M. Omar Faruque Department of Electrical and Computer Engineering, FAMU-FSU College of Engineering, 2525 Pottsdamer St, Tallahassee, FL, 32310 USA Center for Advanced Power Systems (CAPS), 2000 Levy Ave, Tallahassee, FL, 32310 USASearch for more papers by this author First published: 14 February 2019 https://doi.org/10.1049/iet-rpg.2018.5779Citations: 60AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract This paper proposes a novel forecasting model designed to accurately forecast the PV power output for both large-scale and small-scale PV systems. The proposed model uses available temperature data, approximate and detailed coefficients obtained from the decomposed PV power time series using the stationary wavelet transform (SWT), and statistical features extracted from the historical PV data. The model is comprised of four long–short–term memory (LSTM) recurrent neural networks (RNN) designed to perform multi-step forecasting on the individual approximate and detailed coefficients decomposed by the SWT and a final deep neural network (DNN) designed to perform the next time step PV power forecast. The DNN makes use of the reconstructed values estimated by the four LSTM networks together with temperature data and statistical features to predict the final forecasted value of the next time step PV power. 30-min resolution data from a 12.6 MW PV system located in the state of Florida are used for testing and evaluating the proposed method against several models found in the literature. The results obtained suggest that the proposed model improved the forecasting accuracy significantly in the metrics used to compare with other models while reducing the number of features needed to perform the forecasting operation. 1 Introduction As the cost of green energy sources decreases and other environmental issues arise due to the use of fossil fuels, new ways of producing renewable energy are being encouraged by governments and organisations in charge of promoting responsible use of our natural resources [1]. One of the most proliferating means of generating clean and renewable energy nowadays is the use of solar power. In the USA, the Sunshot Vision Study has reported that by 2030 almost 14% of the energy consumed in the USA will be generated from solar power only and by 2050 these numbers will nearly double to 27% [2]. Another example of this support is the Executive Order S21-09 in California which is calling for a 33% of its entire energy generation to come from renewable energy resources by the year 2020. Nonetheless, one major challenge for the integration of PV power in the electrical system and the overall energy market is its stochastic nature. The power generated from large-scale PV plants has the drawback of being very intermittent, affecting directly the ability to bid in the energy market and the consideration of PV power as a reliable energy-generation resource. For this reason, accurate forecasting of the available PV power in medium- to short-term scales could be a game changer in the integration of PV systems, by providing a more reliable estimation of the power being generated. Typically, solar power enters into the picture as a non-dispatchable component, which cannot be accounted as a reliable resource for entering the energy market and any considerable forecast inaccuracy in its generation could cause substantial economic losses and power reliability issues in the whole electrical power system. For that reason, accurate models for solar power forecasting have the potential of being a paradigm shift in the overall energy market through medium- and short-term forecasting by allowing optimal control systems to use its information for unit commitment, economic dispatch applications, energy transactions, unit maintenance, and energy management decision-making [3]. This paper presents a novel hybrid wavelet-based LSTM-DNN structure model designed to accurately forecast the PV power generated by a PV plant in the next 30-min. The proposed model only makes use of available historical PV power data and estimated temperature to accurately predict the next time step PV power generation of the system. It is important to note that the model can be categorised as an adaptive model since its forecasting horizon can be modified and adapted to perform multi-step forecast (e.g. 30 min, 1 h, … n hours) and is only dependent on the lowest time resolution of the data available. In other words, the proposed model is not locked-in to a particular forecasting horizon, since it can be adapted to any time resolution as long as it is equal or greater than the time resolution of the available historical data. This paper is organised as follows: Section 2 gives an overview of the current solar and PV power forecasting techniques. Section 3 reviews the stationary wavelet transform (SWT) and describes how it is used in the proposed forecasting model. Section 4 provides a description of the proposed model and describes the methodology used for designing the forecasting model. Section 5 shows the training and validation of the results together with a description of the metrics used for evaluating the performance of the model, and Section 6 presents the conclusions. 2 Overview of solar forecasting techniques This section focuses on reviewing some of the novel solar irradiance and PV power forecasting models which have been proposed in the last few years. Some relevant models related to power consumption and load forecasting applications are also mentioned for completeness. Authors in [4] present a solar energy forecasting model designed to forecast the solar energy generation in of solar farms. The model consists of forecasting energy values based on numerical weather predictions and convolutional neural networks (CNN). The output of the CNNs is then processed using Gaussian process regression in order to obtain the proper output based on the values given by solar farms. Similarly, in [5], authors present an adaptive framework based on a combination of data analytics approaches and machine-learning techniques to perform a solar energy forecast for a day-ahead prediction horizon. The model presented here consists on a nine-step process designed to extract relevant data, detect synoptic events, perform correlation analysis, process these features using an adaptive artificial neural network (ANN) and Monte–Carlo uncertainty analysis, and finally aggregate and fine-tune the data in order to present it to the user. In [6], researchers present an analogue ensemble approach designed to forecast regional PV power with an hourly resolution by utilising data such as weather forecasts, power measurements, and astronomical date. The primary strategy implemented here consists of a clustering and blending strategy applied to improve the accuracy of the solar PV forecast. Authors claim a significant reduction in the normalised mean square error (NRMSE), of around 13.8–61.21%, when compared to three baselines: North American Mesoscale Forecast System, the Global Forecast System, and the Short-Range Ensemble Forecast. Researchers in [7] propose a novel approach of estimating the PV maximum generation by relying on the measurements of the DC voltage, current, and cell temperature of a specific PV array. The PV plant power is forecasted by indirectly estimating the irradiance of the system using these three estimators. According to the authors, this approach can accurately reconstruct the PV power generation of the PV array even during curtailment periods and can outperform other pyranometer-based methods that use simple sensing systems. In [8], authors present four different PV power forecasting models: a radial basis function neural network (RBFNN), k-nearest neighbours (kNN), weighted kNN (WkNN), and least square support vector machine (LS-SVM). PV power forecast is performed in different sites including Braedstrup (Denmark), Catania (Italy), and San Diego (USA). The main contribution of this paper is the proposal of a pre-process strategy for the historical PV power data, in which a data mining approach is performed with the objective of finding days that are 'similar' to the required forecast days in specific measures. If this pre-processing procedure is done properly, authors claim that solar PV power forecasts can be generated with relatively high accuracy. In [9], authors propose a novel Takagi-Sugeno (T-S) fuzzy model-based PV power short-term forecasting. Here, researchers use a fuzzy C-mean algorithm and the recursive least squares methods to identify the antecedent and consequent parameters of the system. The model is tested using data from a 433 kW PV plant located at St. Lucia campus of the Queensland University in Australia. A similar approach is taken in [10], where researchers propose a type-1 and interval type-2 Takagi-Sugeno-Kang (TSK) fuzzy systems for modelling and predicting the PV power output of a PV plant. In [11], Sheng et al. present a short-term solar power forecasting model based on weighted Gaussian process regression. The main contribution of this paper is the introduction of an innovative method that employs the weighted Gaussian process regression approach to reduce the weight that high potential outliers can have in the data to be analysed. Yang et al. [12] present a weather-based hybrid forecasting algorithm designed to predict the day-ahead PV power output using three stages: a classification, a training, and a forecasting stage. According to the authors, this procedure improves results obtained from classic SVR and ANN models. In [13], Asrari et al. present a hybrid gradient-descent and meta-heuristic approach designed to improve the accuracy and computational burden of an ANN model by selecting the optimal set of parameters for PV power prediction. LSTM RNNs have also been proposed in the literature for forecasting the power generation of a PV plant. In [14], authors present an LSTM model designed to predict the PV power output of plant located in Hae-Nam, Korea. Similar approaches are used in [15, 16], where authors in [15] make use of multi-layer perceptrons (MLP), deep belief networks, autoencoders, and LSTMs to forecast the PV power generated from 21 different facilities, while authors in [16] present a deep LSTM models designed to capture the temporal changes in the PV power data. The use of LSTMs was also explored by Qing et al. in [17], where authors perform a day-ahead solar irradiance forecast based on a prediction structure that forecasts multiple outputs simultaneously with trained LSTM networks. The primary goal of the proposed model is to predict the next day irradiance values based only on features such as month, day of the month, hour of the day, temperature, dew point, humidity, visibility, wind speed, and weather type. It is important to mention that LSTMs have also been widely used in other forecasting applications, such as power consumption and short-term load forecasting, and present a similar structure as the ones used in solar power applications. For example in [18, 19], Kong et al. present two different load forecasting models based on LSTM networks that are designed to forecast aggregated residential and meter-level power consumption based on appliance learning. In [19], authors explore the difficulty of conducting single-meter load forecasting and demonstrate how the aggregation of individual forecasts provide a more accurate result when compared to directly forecasting the aggregated system load while using LSTMs. A similar approach is taken in [18], where the approach was modified to include measurements from single appliances and thus improve the overall forecasting results. Authors in [20] take a similar approach and use LSTMs to forecast aggregated load data but their main focus was centred in the development of a feature selection process, based on genetic algorithms, tailored to find the best features and optimal hyper-parameters of the LSTMs to improve the aggregated load forecast. The idea of applying wavelet analysis, more specifically the Discrete Wavelet Transform (DWT), has also been explored by many researchers in the literature of PV power forecasting. For instance in [21], authors propose a hybrid algorithm that uses a combination of a data filtering technique based on the wavelet transform (WT) using wavelet db4 together with a soft computing model based on fuzzy ARTMAP (FA) that is optimised using the firefly algorithm. Similarly, in [22], authors use the DWT with the purpose of denoising the original solar irradiance signal and then feed the resultant coefficients into a neural network or SVM model to predict the solar irradiance profile. In [23], Capizzi et al. proposed a novel wavelet recurrent neural network (WRNN) designed to take advantage of the correlation between solar radiation and variations of wind speed, humidity, and temperature. The main contribution made in [23] comes from the fact that the novel WRNN is capable of performing the prediction in the wavelet domain and performs the inverse transform before outputting the predicted signal. A novel forecasting model using a bias-compensated random forest is proposed in [24] to predict the PV power generation in a microgrid located at University of California, San Diego (UCSD). This paper also makes use of the SWT instead of the DWT due to its shift-invariant property. In [25], the power output of a photovoltaic system is forecasted using the DWT and an ANN. This model has some similarities with the model proposed here, in which the historical PV power signal is decomposed using WT decomposition. However, this model differs from the proposed one due to its use of the DWT instead of SWT, all components of the decomposed signal are fed into a single neural network and its need of additional features such as irradiance and historical temperatures. The models presented in [25] are used, among others, for evaluating the performance of the proposed model. Alternatively, there are many other methods used for solar and PV power forecasting that differ primarily in the data used as input and the overall statistical model for predicting the desired outcome. In [26], authors present an extensive comparison of both sophisticated and simple forecasting strategies tested in over 32 PV plants of different sizes over the span of one year. Here, researchers compare four state-of-the-art black-box forecasting methodologies such as k-Nearest Neighbours (kNN), Neural Networks (NN), Support Vector Regressors (SVR), and Quantile Random Forests with the objective of demonstrating their advantages and disadvantages using the same input data. Alternatively, the review papers [27, 28] present thorough reviews of most of the models found in the current literature. These models are presented and characterised by model type as clear-sky models, regressive models, remote sensing models, and artificial intelligence-based models. As seen, solar power forecasting is becoming an essential part of the integration of renewable energy resources into the modern grid, and its accuracy has the potential of making these resources more attractive for the overall energy market. Thus, a novel PV power forecasting model is proposed here, which has a significant increase in accuracy when compared to other comparable models with minor increases in complexity and computational costs. Specifically, this paper presents the following research contributions: A novel PV power forecasting model, designed to perform short-term forecasts (30 min), that demonstrates several advantages when compared to other comparable models including: (a) fewer measured features required to forecast the PV power output (only the historical PV power signal and predicted temperature are used as inputs to the model), (b) significant increase in forecasting performance, outperforming all five other models tested here, and (c) minor complexity and computational cost increase, making it suitable for real-time applications, such as optimal control algorithms that require PV forecasts in a short-term resolution in order to execute control actions in real-time. A novel forecasting model that uses a combination of the SWT, four LSTM (long–short–term memory) Networks, and a Deep Neural Network (DNN) to accurately forecast the PV power for the next time step. The model is defined in a way that it can be easily adapted to perform multi-step forecasts, i.e. (). A real-world test of the model is conducted by performing forecasts using real data obtained from a 12.6 MW PV plant located in the state of Florida and publicly available data from other databases facilitated by the National Renewable Energy Laboratory (NREL). 3 Wavelet transform The highly fluctuating nature and non-stationary behaviour of the original PV power signal are good indications of how the use of the WT can be useful for recognising the frequency and temporal patterns in order to improve the forecast error differences. The basic definition of the WT gives us the capability of representing the PV power signal as a set of wavelets, which in turn can be used for obtaining information about the time and frequency domains of the signal. The WT is most commonly defined in two categories: the continuous WT (CWT) and the discrete WT (DWT) [25]. The CWT is defined by (1), where the * denotes the complex conjugate of the set of wavelets chosen, and s and are the scale and translation dimensions of the signal. The set of wavelets is defined by (2), where the set is generated from a mother wavelet, , and specific values for the translation and scale of the signal, including an energy normalisation factor, . In other words, just as the Fourier Transform (FT) decomposes the respective signal into sine and cosine signals, the CWT decomposes the non-stationary signal into a series of wavelets with a different combination of scales and translations [29]. (1) (2) However, the high computational complexity of calculation and high redundancy of the CWT make it unpractical for the desired application. According to (1), in the CWT, the wavelet transform is constantly being calculated by shifting a continuously scalable function over a signal and obtaining the correlation between these two signals. In consequence, the obtained wavelet coefficients will be highly redundant and, for most functions, the CWT will have no analytical solution, and its solution must be calculated numerically. In order to address these issues, the DWT is introduced. The DWT analyses the signal at different frequency bands with various resolutions by decomposing the original signal into detailed and an approximation coefficient values. To achieve this, the DWT makes use of two set of functions called the scaling function, which is associated with a low-pass filter, and the wavelet function, associated with a high-pass filter. The DWT can be defined by (3), where j and k are integers and and are the discretised scales and translation values. Subsequently, the scaling function and wavelet function of the DWT can be described by (4) and (5). (3) (4) (5) Using these equations, the signal can be expressed as (6), where and are the coefficients obtained by taking the inner products of the signal, , with the scaling and wavelet function, respectively, as seen in (7) and (8). (6) (7) (8) The coefficients in (4) are known as the scaling filter and the coefficients in (5) are called the wavelet filter. These two sets of coefficients are associated with the decomposition or filtering process in which the signal is decomposed into different frequency bands. As seen in Fig. 1, the decomposition process of the signal begins with the filtering of the signal via a high-pass and a low-pass filter, followed by the subsampling of the resultant signal by 2. This decomposition procedure, commonly known as sub-band coding, halves the time resolution, by only taking half of the samples, while doubling the frequency resolution, by spanning half of the previous signal frequency band. Fig. 1 depicts the signal decomposition/reconstruction procedure. As observed, the low-pass filter removes the higher frequency components of the signal and outputs the approximate coefficients while the high-pass filter outputs the detailed coefficients of the signal. Fig. 1Open in figure viewerPowerPoint DWT signal decomposition procedure 3.1 Stationary wavelet transform (SWT) Nonetheless, the DWT presents a significant potential problem for the analysis of the signal due to the fact that it is a shift-variant transform [30]. These shift-variant results arise from the use of the sub-sampling operation on the DWT algorithm and result in wavelet coefficients that are highly dependent on their location on the sub-sampling lattice. In other words, as the analysis of the signal becomes more certain in the frequency components, the analysis of the time domain becomes less certain. Small shifts in the original signal have the potential of causing large changes in the wavelet coefficients and large changes in the reconstructed waveform [30]. To avoid this issue, the sub-sampling operation of the signal is removed from the filtering operation in the DWT and only the up-sampled operation is kept at each level of the decomposition. This modification in the DWT is commonly known as the SWT. The use of the stationary data resulted in more skilful forecasts when using the LSTM networks to predict the individual decomposed coefficients of the signal. Fig. 2 depicts the scheme used for the SWT. Fig. 2Open in figure viewerPowerPoint SWT signal decomposition filter procedure The mother wavelet or wavelet function chose to perform the SWT decomposition process was the nearly symmetrical wavelet, symlet 2. This wavelet was the one that gave the best forecasting performance when compared with the Daubechies family, Coiflet family, Morlet, and Haar wavelets [31]. The symlet wavelets have the characteristic of being orthogonal and near symmetric, ensuring minimal phase distortion. These symmetrical wavelets can be identified based on the order N, which indicates the number of vanishing moments of the wavelet for a given support width of . Fig. 3 shows the corresponding scaling function and wavelet function of the symlet with , most commonly known as symlet 2. A more thorough mathematical description of the symmetrical wavelet equations and coefficients can be found in [32]. Fig. 3Open in figure viewerPowerPoint Scaling () and wavelet () function of symlet 2 4 Methodology: hybrid LSTM-DNN model The essential concept behind the operation of the proposed hybrid wavelet-based LSTM-DNN model consists in the use of a DNN trained with features obtained from the statistical analysis of the power signal, the predicted temperature value, and a fabricated forecasted value obtained by the reconstruction of the forecasted output coefficients given by individual LSTMs. The LSTM networks are trained to perform multi-step forecast of the individual approximate and detailed coefficients of the SWT decomposed power signal for the next 24 h. These values are then reconstructed into a forecasted power signal using the inverse SWT (ISWT), and the respective value is used as a feature to the DNN. The main idea behind using the SWT decomposition in the model is based on the ability to capture and forecast only the most relevant information, in the time and frequency domain, from the historical signal. By decomposing the historical PV power signal into four different SWT coefficients (three detailed and one approximated), LSTMs are able to improve the overall signal forecast based on their ability to predict these individual components instead of forecasting the aggregated PV power signal. In our case, only four different SWT coefficients are chosen due to the fact that these were the ones that contain the most useful information for the forecasting process of the analysed signal. According to the tests conducted, it was observed that adding more coefficients, such as the approximate 1 and 2, or decomposing the signal further increased the complexity and training time significantly while adding almost zero or negative effects on the forecast results. The number of decomposition levels depends primarily in the particular signal being analysed, that is why, if the proposed model is used for another application, such as load forecasting, tests should be made in order to find the optimal decomposition levels for both the approximate and detailed decomposition levels. Fig. 4 shows the overall design of the proposed hybrid wavelet-based LSTM-DNN model. As observed in the figure, the training and validation of the proposed model can be divided into four major steps: Step 1: Data extraction, sliding window processing, and SWT decomposition. Step 2: Training and validation of LSTM RNNs models and ISWT reconstruction. Step 3: Statistical features extraction. Step 4: DNN training and validation via feature selection. Fig. 4Open in figure viewerPowerPoint Block diagram hybrid wavelet-based LSTM-DNN model 4.1 Data extraction, sliding window processing, and SWT decomposition Before applying the selected sliding window technique and performing the SWT decomposition, the historical PV power data is pre-processed with the objective of removing any outlier or incorrect measurements present in the data. After this pre-processing step is completed, a sliding window technique that divides the data into blocks of 24 h (or 48 values in 30-min resolution) is used to process data for later decomposing each block using the SWT. A sliding window technique is a data processing method designed to divide a datastream x into several chunks of data blocks, k. In our particular case, . The current block window of size 48 is then decomposed into approximate and detailed coefficients by passing the block through the SWT decomposition. As mentioned before, the mother wavelet used in the decomposition procedure is the symmetrical wavelet 2 (symlet2). After several tests using different wavelets, this was the wavelet that gave the best performance in terms of less error. Similarly, it was discovered that the optimal level for this particular PV signal was the decomposition level 3. This means that the coefficients used for this particular forecasting application are the detailed 1, detailed 2, detailed 3, and approximate 3. Other applications could benefit from the use of a different mother wavelet and a different decomposition level. 4.2 Training and validation of LSTM RNNs models and ISWT reconstruction An LSTM network can be defined as a type of RNN structured to solve the vanishing gradient problem present in regular RNNs. LSTMs are explicitly designed to avoid the long-term dependency problem by helping to preserve a more constant error allowing the RNNs to continue to learn over many time steps. An LSTM contains special blocks called memory blocks, which in part contain memory cells with recurrent connections, that allow storing the temporal state of the network, and other units called gates that control the flow of information in the network. Fig. 5 depicts the general architecture of a LSTM block. As seen in the figure, the LSTM block has the ability to add or remove information to the cell state, , by updating its value according to the resultant operations obtained from the 'forget gate layer' and the 'input gate layer'. The output of the block, , is computed using a filtered version (passed through ) of the current state of the cell () and the output computed from the inputs (). Succinctly, the LSTM networ