Randomised learning‐based hybrid ensemble model for probabilistic forecasting of PV power generation

IET Generation, Transmission & DistributionVolume 14, Issue 24 p. 5909-5917 ArticleFree Access Randomised learning-based hybrid ensemble model for probabilistic forecasting of PV power generation Wei Liu, Wei Liu orcid.org/0000-0002-9770-8923 School of Electrical and Electronic Engineering, Nanyang Technological University, SingaporeSearch for more papers by this authorYan Xu, Corresponding Author Yan Xu eeyanxu@gmail.com School of Electrical and Electronic Engineering, Nanyang Technological University, SingaporeSearch for more papers by this author Wei Liu, Wei Liu orcid.org/0000-0002-9770-8923 School of Electrical and Electronic Engineering, Nanyang Technological University, SingaporeSearch for more papers by this authorYan Xu, Corresponding Author Yan Xu eeyanxu@gmail.com School of Electrical and Electronic Engineering, Nanyang Technological University, SingaporeSearch for more papers by this author First published: 14 October 2020 https://doi.org/10.1049/iet-gtd.2020.0625Citations: 8AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinked InRedditWechat Abstract Probabilistic forecasting of solar photovoltaic (PV) generation is critical for stochastic or robust optimisation-based power system dispatch. This study proposes a randomised learning-based hybrid ensemble (RLHE) model to construct the prediction intervals of probabilistic PV forecasting. Three different randomised learning algorithms, namely extreme learning machine, randomised vector functional link, and stochastic configuration network, are ensembled as a hybrid forecasting model. Besides, bootstrap is used as the ensemble learning framework to increase the diversity of training samples. For each algorithm, a decision-making rule is designed to evaluate the credibility of the individual outputs and the incredible ones are discarded at the output aggregation step. The weight coefficients of the aggregated outputs of the three algorithms are then optimised to compute the final point forecast results. Based on the point forecast results, the prediction intervals are constructed considering both model misspecification uncertainty and data noise uncertainty. The variance in model misspecification uncertainty is directly calculated with the individual outputs and the variance in data noise uncertainty is separately trained with an RLHE model. The proposed method is tested with an open dataset and compared with several benchmarking approaches. 1 Introduction Driven by environmental and energy-saving incentives, increasing photovoltaic (PV) power is being integrated into the power grid. By the end of 2018, over 480 GW of PV has been installed and this figure is expected to increase further to 1.2 TW in the following 5 years [1]. However, the inherent volatility of PV power brings various technical challenges to the grid, e.g. voltage fluctuation [2] and frequency deviation. Therefore, it is necessary to develop accurate PV generation forecast models to address such challenges. PV generation can be either directly forecasted with historical PV generation data or indirectly calculated with approximate models whose inputs are relevant factors (i.e. solar irradiance). In the literature, some methods have been developed for the forecast of PV generation and solar irradiance, most of which only focus on point forecast. For example, artificial neural networks (ANNs) [3], Markov chains [4] and support vector machines (SVMs) [5] are applied to develop point forecast models. Using the persistent model as a baseline, Long et al. [6] studied the effectiveness of four data-driven algorithms: ANN, SVM and the k-nearest neighbour (k-NN). In [7], ANN, SVM, k-NN and decision tree regression are studied for solar irradiance forecast using aerosols measurements. In [8], a self-organising map (SOM), a learning vector quantisation (LVQ) network, the SVM and the fuzzy inference system are combined to forecast hourly PV generation. Those point forecast methods only provide the expected value of PV generation and the risk of the prediction cannot be evaluated. In this regard, the results of the point forecast are not sufficient for power system dispatch under uncertainties. For instance, in [9], the probability distribution function (PDF) of renewable energy generation is used as the input of stochastic optimisation-based multi-energy dispatch. In [10], the intervals of the renewable power output are used for robust optimisation-based energy management. To provide more comprehensive information to decision-makers, probabilistic forecast models can be developed to forecast the PDF of PV generation. A recent review of the state-of-the-art of probabilistic forecast of PV generation is carried out in [11]. Compared to the point forecast, the probabilistic forecast provides more applicable results because the uncertainties of point forecasting are also evaluated. Depending on the presumptions on the distribution function of forecast errors, probabilistic forecast models can be classified into the parametric model and non-parametric model. In parametric models, the forecast error of PV generation is assumed to follow a known distribution function. In [12–14], the forecast errors are assumed to be Gaussian distributed. Other than Gaussian distribution, Laplace distribution is also evaluated in [15] to model the forecast errors. In these approaches, point values are firstly forecasted and then their uncertainties are evaluated. Based on the point values and uncertainties, different levels of prediction intervals (PIs) can be generated. Non-parametric forecast models do not rely on any presumptions on the error distribution. Two probabilistic forecasting models of PV generation are proposed in [16, 17] based on pair copula construction and conditional probability combined with non-parametric kernel density estimation, respectively. A higher-order Markov chain is applied in [18] for a 15-min ahead forecast of PV generation, where the Gaussian mixture method is used to estimate the PDF. In [19], 18 quantiles with nominal levels from 0.05 to 0.95 at increments of 0.05 are forecasted using a randomised learning algorithm named extreme learning machines (ELMs). In [20], an improved Markov chain is applied to forecast the quantiles of PV generation. In [21], the forecast uncertainty is calculated from an ensemble method based on the dropout technique of deep learning. Despite the studies above, the accurate probabilistic forecast of PV generation remains challenging due to the inherent uncertainties involved. Hence, it is necessary to explore solutions to improve forecasting accuracy. As an effective method, ensemble learning [22] has been used to enhance the performance of a single learner. For instance, ensemble ELM is deployed to improve load forecasting accuracy in [23]. In [24], ensemble deep learning is applied to forecast the low-voltage load. Therefore, it is reasonable to develop an ensemble learning model to increase the accuracy of the PV generation forecast. Although ensemble learning can improve the performance of a basic learner, it suffers from the magnified computational burden because a large number of basic learners need to be trained. Hence, the learning speed of basic learners becomes critically important. In this sense, traditional backpropagation-based ANNs may not be appropriate for ensemble learning because of their computational burden. Randomised learning algorithms are perfect alternatives to traditional ANNs for ensemble learning due to their excellent training speed and inherent randomness. Taking advantage of the fast learning speed of randomised learning and the diverse hypothesis property of ensemble learning, this paper presents a randomised learning-based hybrid ensemble (RLHE) model for the probabilistic forecasting of PV generation. The proposed method is based on three randomised learning algorithms, namely ELM, randomised vector function link (RVFL) and stochastic configuration network (SCN). The major contributions of this paper are summarised as follows: (i) The proposed RLHE model considers multiple diversities to improve forecasting accuracy. The diversity of training samples is increased by bootstrap sampling and the diversity of training models is increased by three different learning algorithms. (ii) The aggregation of ensemble outputs is optimised by two steps. Firstly, a reliable decision-making rule is applied to filter out the inaccurate outputs from individual models at the aggregation step. Then, the weight coefficients of the outputs of the three algorithms are also optimised. (iii) To improve the reliability of constructed PIs, both model misspecification uncertainty and data noise uncertainty are modelled given the heteroscedasticity of PV generation data. The variance in model misspecification uncertainty is directly computed and the variance in data noise uncertainty is separated trained by an RLHE model. The two types of variances are then summed as the total variance. The remaining of this paper is organised as follows. Section 2 presents the basic principles of three randomised learning algorithms. Section 3 describes the overall framework of the proposed method. Section 4 presents the numerical results of the proposed method and existing approaches as benchmarks. Finally, Section 5 gives the conclusions of this paper. 2 Randomised learning algorithms Free of the gradient descent process, randomised learning algorithms have a much faster learning speed and strong generalisation capability. This paper will apply three randomised learning algorithms: (i) ELM, (ii) RVFL and (iii) SCN. 2.1 Extreme learning machine Huang et al. [25] proposed ELM for single-hidden layer feedforward networks (SLFNs) and highlighted that the training speed of ELM is much higher than conventional SLFNs. The key idea of ELM is to randomly assign the input weights and the typical structure of ELM with one output is shown in Fig. 1. Fig. 1Open in figure viewerPowerPoint Schematic structure of ELM Suppose the given dataset is , where X and T are the inputs and targets, respectively. For a single ELM, mathematical expression of output can be presented as ((1)) where Y is the output of ELM, is the randomised input weight matrix, b is the bias of hidden neurons, denotes activation function and is the output weight vector. The objective of the ELM learning process is to find the optimal that minimises the training error. This can be easily analytically solved by matrix operation and the optimal is computed as ((2)) where is the Moore–Penrose generalised inverse matrix of . 2.2 Randomised vector functional link As illustrated in Fig. 2, the structure of RVFL is similar to that of ELM except that there is a direct link from the input layer to the output layer [26]. Fig. 2Open in figure viewerPowerPoint Schematic structure of RVFL In RVFL, the input weights connecting the input and hidden layers are randomly assigned as in ELM. Then, the original input nodes and hidden nodes are concatenated and the concatenated nodes are connected to the output node with output weights, shown as follows: ((3)) where Y is the output of RVFL, denotes the horizontal concatenation of matrixes and ; is the randomised input weight matrix, b is the bias of hidden neurons, denotes activation function and is the output weight vector. As in ELM, the optimal output weight matrix of RVFL can be easily analytically solved as ((4)) where is the Moore–Penrose generalised inverse matrix of . 2.3 Stochastic configuration network SCN is a novel randomised learning algorithm SLFNs proposed in [27] and has the same structure as ELM. As a novel randomised learning algorithm, SCN is different from ELM and RVFL in the sense that the hidden nodes are incrementally added in accordance with a supervisory mechanism. Suppose an SCN with hidden nodes has been constructed, shown as follows: ((5)) where and are the input and output weights of the jth hidden node, respectively; denotes the activation function and is the bias at the jth hidden node. If the current residual error does not reach a predefined tolerance level, a new hidden node with and should be added to the current model and the output weights is analytically evaluated. As a result, the new model with L hidden nodes is constructed with decreased residual error, shown as follows: ((6)) When randomly assigning and , the following supervisory mechanism (inequality constraint) should be satisfied: ((7)) where denotes the residual error with nodes, r is a given constant and , is a sequence satisfying and , is one upper bound of the norm of the activation function . To summarise, the two main parts of SCN are (i) Configuration of hidden parameters: Adding a new hidden node to the current learner model, the input weights and biases are randomly assigned but have to be constrained by (7). (ii) Determination of output weights: Analytically determining the output weights through solving a least-squares problem with Moore–Penrose generalised inverse operation as (2). 3 Proposed RLHE model 3.1 Overall framework of RLHE A large number of primary models need to be trained for ensemble learning, thus the computational speed of each model becomes critically important. Using backpropagation to update the weights by iteration, traditional ANNs suffer from the computational burden and are therefore not suitable for ensemble learning. On the contrary, the fast learning speed of randomised learning algorithms, making them perfectly appropriate for ensemble learning. Besides, to increase the diversity of forecasting models and improve the forecasting accuracy, different types of algorithms should be applied. Therefore, this paper applies three different randomised learning algorithms described in the previous section as the primary models. The overall framework of the proposed RLHE method is illustrated in Fig. 3, including the following steps: Step 1 : Bootstrap, also named as bagging, is implemented for M times with the original training dataset. Therefore, M bagging training datasets are generated with replacement. Step 2 : Each bagging dataset is imported to three algorithms (ELM, RVFL and SCN) for training and thus sub-models are trained in total. Step 3 : Credibility evaluation is performed for each algorithm and incredible (i.e. potentially inaccurate) outputs are discarded. The rest credible outputs are averaged as the output of each algorithm. The three outputs are then aggregated by linear combination and the weight coefficients are pre-optimised using GA with the training dataset. The aggregated output is used as the final point forecast result. Step 4 : With the point forecast result and individual outputs, the variance in model uncertainty is directly calculated and the variance in data noise uncertainty is separately trained with an RLHE model. Then, model variance and data noise variance are summed as the total forecast variance. Step 5 : Based on the point forecast result and total uncertainty quantification, the PIs under different confidence levels can be accordingly constructed. Fig. 3Open in figure viewerPowerPoint Overall framework of the proposed RLHE method 3.2 Bagging ensemble method Ensemble learning combines several weaker learners as a superior model with better generalisation ability. As one of the common ensemble learning methods, bagging has been successfully applied for classification and regression [28]. The general idea of bagging is to build a sampling distribution by uniform sampling with replacements from the original data. The procedure of bagging is as follows: Obtain the original dataset with N training samples . Randomly select samples from the original dataset with replacement and construct a new training dataset. Train the learning model with the training dataset obtained from Step 2. Repeat Steps 2 and 3 for M times to obtain M replicates of learning models. As described above, M sub-models will be trained given M duplicates of training datasets for each randomised learning algorithm. Therefore, M outputs will be generated given each input and they are aggregated to generate the final output, formulated as ((8)) where denotes the mathematical function aggregating the M forecast results; is the individual output of the mth sub-model of randomised learning algorithm t; t represents the label of randomised learning algorithms. 3.3 Point forecast construction 3.3.1 Credibility evaluation Due to the randomness involved in the input weights and biases, the output of single ELM/RVFL/SCN can be unstable. In other words, individual outputs with large errors may be generated by individual models. Simply aggregating them by average summation does not eliminate the effects of such outputs with extreme errors and therefore impairs the overall accuracy. To make the final output more accurate and reliable, it is important to design proper aggregating shown in (8) so that individual forecasts with large errors can be removed. As such, this paper applies a reliable decision-making rule proposed in [29] to evaluate the credibility of individual output. An individual output is deemed as credible if it is close to the median of the ensemble outputs. On the contrary, if an individual output is at the edge of the ensemble outputs, then it is seen as incredible. The details of credibility evaluation are described below. Suppose that M outputs given input are obtained as , then the credibility of individual output can be evaluated as ((9)) where is the median value of original outputs for the algorithm t. With the above rule, credible outputs can be reserved and incredible ones are discarded. Then, the average values of the credible outputs are computed as the final output of each learning algorithm, shown as follows: ((10)) where q is the number of credible ones among M outputs. 3.3.2 Weight coefficients optimisation Based on the credibility evaluation and averaging described in the above section, the outputs of three learning algorithms have been produced. Next, it is important to find a proper way to aggregate these outputs into one final forecast result. Liner combination is applied for aggregation, shown as follows: ((11)) ((12)) where is the final point forecast value and is the weight coefficient of the randomised learning algorithm t. To minimise the accumulated residential error of all the training samples, the weight coefficients of three learning algorithms are optimised. In this paper, the optimisation problem is formulated as below and is solved using GA: ((13)) where N is the number of training samples and is the actual output given input . Once the weights are optimised in offline, they will be used in online. 3.4 Prediction intervals formulation Based on the point forecast results obtained in the previous section, the actual value considering forecast error can be formulated as ((14)) where is the point forecast value obtained with (11) and is the forecast error. As introduced in [30], model misspecification and data noise both introduce prediction uncertainty. As a result, forecast error can be formulated as the sum of model misspecification error and data noise error, shown as follows: ((15)) where and are the error caused by model estimation and data noise, respectively. Assuming that model misspecification error and data noise error are independent and both follow a Gaussian distribution, the total variance can be calculated as the sum of variance in model uncertainty , and variance in data uncertainty , shown as follows: ((16)) Combining the point forecast value and total variance, the PI of confidence level can be formulated as ((17)) where and are, respectively, the lower and upper bounds, calculated as ((18)) ((19)) where is the critical value of the standard Gaussian distribution. 3.4.1 Model uncertainty variance As mentioned above, models are trained and, therefore, the model misspecification variance can be calculated as ((20)) where is the output of the rth model of the algorithm t. 3.4.2 Data noise uncertainty variance The heteroscedasticity characteristic in data noise suggests that quantifying the data uncertainty is difficult because only one PV generation value is available at each time step. Given the input variable , the conditional variance of measured data [31] can be computed as ((21)) In this paper, a separate RLHE regression model is built to model the data noise variance. Based on the point forecast results and actual PV generation, the squared error of each training sample can be calculated as . Then, a new training dataset can be formed by reserving the inputs and replacing the targets with the squared errors, given as ((22)) The above new training dataset shown as (22) is used to train a separate RLHE model, which follows the similar training process of the RLHE model described in Section 3.3. 4 Case study 4.1 Dataset description The proposed method is tested with an open dataset. In this dataset, PV generation data are collected at a one-minute time resolution. The PV site has a total capacity of 433 kW and locates at St. Lucia Campus, University of Queensland, Australia [32]. It is worth pointing out that in this study, all the PV generation data are normalised by the site capacity. In this paper, the data recorded between December 2015 and November 2017 are used for validation. Note that only the PV generations between 7.00 am and 5.59 pm each day are considered. As an example, Fig. 4 shows the 7-day profiles of the normalised PV generation in summer (December 2016) and winter (June 2017). Fig. 4Open in figure viewerPowerPoint Normalised PV generation of 7 successive days in (a) December 2016, (b) June 2017 Compared to the PV generation in winter, Fig. 4 shows that the PV generation in summer is significantly higher and exhibit much more severe intermittence. In other words, the patterns of PV generation among different seasons are significantly different. In this regard, the whole year is divided into four seasons and the splitting of training and testing datasets is presented in Table 1. Table 1. Splitting of training and testing datasets Season Training dataset Testing dataset summer 2015 Dec–2016 Feb 2016 Dec–2017 Feb fall 2016 Mar–2016 May 2017 Mar–2017 May winter 2016 Jun–2016 Aug 2017 Jun–2017 Aug spring 2016 Sep–2016 Nov 2017 Sep–2017 Nov 4.2 Evaluation metrics 4.2.1 Evaluation criterion for point forecast The root of mean squared error (RMSE) and mean absolute error (MAE) are both applied to measure the accuracy of point forecast, respectively, defined as ((23)) ((24)) where is the number of samples in the training dataset; and are the point forecast of PV generation and true PV generation, respectively. 4.2.2 Performance criterion for probabilistic forecast To assess the performance of probabilistic forecast results, reliability and sharpness [33] are both evaluated via average coverage error (ACE) and interval score (IS), respectively. Reliability refers to the probability that the actual target falls within the corresponding PI under a given confidence level. With reliable PIs, the actual PI coverage probability (PICP) of constructed PIs on the full testing dataset should be close to the PI nominal confidence (PINC) . PICP is defined as the probability that the actual target falls within the constructed PI, expressed as ((25)) where is the number of samples that the constructed PI covers the actual PV generation; is the number of testing data samples. To quantify the reliability of constructed PI given PINC, ACE is computed as ((26)) To obtain reliable PIs, the ACE should be as close to zero as possible. In other words, smaller absolute ACE indicates more reliable PIs. Even with high reliability, PIs that are too wide are not useful in practice. To prevent too wide PIs, sharpness is used to measure the degree of concentration of probability distribution. The width of PI shown in (17) is defined as ((27)) The IS can be applied to comprehensively evaluate the overall skill of PV generation PIs to assess the sharpness. For an individual sample on the testing dataset, the IS of the PI with PINC is defined as ((28)) The overall IS on the whole testing dataset is defined as the average value of individual ISs, expressed as ((29)) As is shown in (28), the IS rewards narrow PI and gives a penalty if the actual target goes beyond the estimated PI. From (28), one can see that the IS is always negative and a larger interval score (closer to zero) suggests better forecast performance. 4.3 Tuning of model parameters The model parameters need to be properly tuned at the offline training stage as they are critical to the performance of the proposed model. Therefore, grid search with k-fold cross-validation ( in this paper) is used to determine the major optimal model parameters, including the number of hidden nodes of ELM/RVFL/SCN and the number of bagging duplicates M. Taking winter as examples, 80% of the whole training dataset is used for training and the rest 20% is used for validation. 4.3.1 Number of hidden nodes The number of hidden nodes is important for randomised learning algorithms. The relationship between the validation error and the number of hidden nodes of three algorithms is illustrated in Fig. 5. It is shown that as the number of hidden nodes increases, the forecasting error gradually decreases and then remains unchanged. As a result, the number of hidden nodes for all three algorithms is selected as 100. Fig. 5Open in figure viewerPowerPoint Validation RMSE of different number of hidden nodes 4.3.2 Number of bagging duplicates The number of duplicates is also important to the proposed framework. The relationship between the validation error and the number of bagging duplicates is presented in Fig. 6. One can see that as the number of bagging duplicate increases, the validation error gradually decreases and then remains stable. On the other hand, larger bagging duplicate will lead to longer training time. Therefore, the number of bagging duplicates is selected as 100, above which value the error will no longer decrease. Note that in this testing, the average summation is used for aggregation and the number of bagging duplicates for the three algorithms is set as the same. Fig. 6Open in figure viewerPowerPoint Validation RMSE of different number of bagging duplicates 4.4 Point forecast results To verify the effectiveness of the proposed RLHE model for point forecast, two back propagation-based neuron networks, including ANN [3] and long short-term memory (LSTM) [34], are implemented as the benchmarks. For a fair comparison, both the ANN and LSTM have one hidden layer and their numbers of hidden nodes are also optimally tuned using a grid search with five-fold cross-validation. Besides, single ELM (SELM) [19], bagging ELM (BELM) [23] and the average of ensemble randomised learning (AveERL) are also carried out. The difference between BELM and AveERL is that AveERL takes into account the algorithm diversity and applies three different algorithms, while BELM only uses ELM for the ensemble. AveERL is applied to highlight the effectiveness of credibility evaluation and weight coefficients optimisation of the proposed method. The point forecast results are summarised in Table 2. For each method, the forecast errors (RMSE and MAE) of different seasons are separately given as well as the average errors. The minimum errors of each season are highlighted with bold text. In general, one can see that the proposed RLHE approach outperforms the other methods in all the cases, verifying the effectiveness and high reliability of the proposed method. Table 2. Performance comparison of RMSE (%) and MAE (%) for point forecast Seasons Metrics Methods ANN [3] LSTM [34] SELM [19] BELM [23] AveERL Proposed RLHE summer RMSE, % 12.44 12.25 12.25 12.07 12.03 11.94 MAE, % 8.48 8.32 8.41 8.27 8.22 7.97 autumn RMSE % 11.75 11.27 11.42 11.21 11.17 11.08 MAE, % 7.63 7.37 7.43 7.29 7.28 7.10 winter RMSE, % 6.65 6.18 6.29 6.10 6.05 5.90 MAE, % 4.05 3.49 3.59 3.41 3.40 3.05 spring RMSE, % 11.66 11.52 11.55 11.30 11.29 11.21 MAE, % 7.16 7.17 7.04 6.88 6.85 6.63 average RMSE, % 10.62 10.30 10.38 10.17 10.14 10.03 MAE, % 6.83 6.59 6.62 6.46 6.44 6.19 It can be seen from Table 2 that LSTM outperforms ANN and SELM, indicating the superiority of LSTM for the time series forecast. Due to the better generalisation property of SELM, it produces smaller errors than ANN. Comparing SELM and BELM, the latter generates better results. For example, the RMSE and MAE of BELM in winter are, respectively, 3.0 and 5.0% lower than those of SELM. Averagely speaking, RMSE and MAE can be, respectively, reduced by 2.0 and 2.4%, indicating that ensem