PV power intra‐day predictions using PDE models of polynomial networks based on operational calculus

IET Renewable Power GenerationVolume 14, Issue 8 p. 1405-1412 Research ArticleFree Access PV power intra-day predictions using PDE models of polynomial networks based on operational calculus Ladislav Zjavka, Corresponding Author Ladislav Zjavka lzjavka@gmail.com orcid.org/0000-0002-2881-4319 Department of Computer Science, Faculty of Electrical Engineering and Computer Science, VŠB-Technical University of Ostrava, 17. listopadu 15/2172 Ostrava, Czech RepublicSearch for more papers by this author Ladislav Zjavka, Corresponding Author Ladislav Zjavka lzjavka@gmail.com orcid.org/0000-0002-2881-4319 Department of Computer Science, Faculty of Electrical Engineering and Computer Science, VŠB-Technical University of Ostrava, 17. listopadu 15/2172 Ostrava, Czech RepublicSearch for more papers by this author First published: 07 May 2020 https://doi.org/10.1049/iet-rpg.2019.1208Citations: 8AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract Precise daily forecasts of photo-voltaic (PV) power production are necessary for its planning, utilisation and integration into the electrical grid. PV power is conditioned by the current amount of specific solar radiation components. Numerical weather prediction systems are usually run every 6 h and provide only rough local prognoses of cloudiness with a delay. Statistical methods can predict PV power, considering a specific plant situation. Their intra-day models are usually more precise if rely only on the latest data observations and power measurements. Differential polynomial neural network (D-PNN) is a novel neuro-computing technique based on analogies with brain pulse signal processing. D-PNN decomposes the general partial differential equation (PDE), being able to describe the local atmospheric dynamics, into specific sub-PDEs in its nodes. These are converted using adapted procedures of operational calculus to obtain the Laplace images of unknown node functions, which are inverse L-transformed to obtain the originals. D-PNN can select from dozens of input variables to produce applicable sum PDE components which can extend, one by one, its composite models towards the optima. The PDE models are developed with historical spatial data from the estimated optimal numbers of the last days for each 1–9-h inputs-output time-shift to predict clear sky index in the trained time-horizon. 1 Introduction The chaotic nature of atmospheric processes and fluctuations resulting in specific local conditions (e.g. cloud structure and levels amount, temperature, relative humidity, visibility) pose a challenge in forecasting photo-voltaic power (PVP). The solar radiation variability is caused by many uncertain factors possible to describe by partial differential equations (PDEs). PVP prediction methods can be broadly classified into three major approaches [1]: numerical weather prediction (NWP) models using physical consideration; statistical regression or soft-computing models; hybrid or ensemble models of combined methods. Stochastic PVP supplies are difficult to predict using only deterministic NWP models which are unable to account for the detailed physical processes at a scale smaller than the grid used in the numerical simulations. NWP systems simplify the atmospheric circulation, solving sets of primitive physical equations for the ideal gas flow. Their solutions using actual time-step and parameter changes (of cell size) only provide approximate estimates, where errors enlarge exponentially in larger time horizons and grid-scales [2]. Spatial and temporal resolution of NWP models determines the extent of weather phenomena that can be directly recognised. NWP systems apply downscaling to outputs of their models, using coarse spatial resolution, to recognise detailed sub-grid scale features such as cloud structures and topography at the surface level [3]: dynamical downscaling uses outputs from global NWP systems and the physical equations to derive a regional numerical model in higher spatial resolution; statistical downscaling searches for a statistical relationship among observations between large-scale atmospheric variables and a local variable at a particular site), which is subsequently used on NWP data. NWP models can combine their outputs with satellite or ground-based sky image methods which analyse and detect particular cloud structures in a given period. Physical equations can describe PVP systems capability to convert solar radiation into electrical power, taking into account the specific characteristics, e.g. cell temperature, shading characteristics and load resistance. Higher temperatures of PVP plants lead to lower efficiency in solar radiation conversion [4]. Regression or artificial intelligence (AI) methods do not attempt to describe the physical processes of atmosphere or PVP systems. They can process the amount of time-series data to model or analyse local atmospheric phenomena or PV plant particular operations [5]. Statistical techniques can use historical observations to predict PVP few-hours’ ahead or combine them with NWP data in daily predictions to specify meso-scale forecasts for local idiosyncratic conditions via post-processing [6]. Input solar radiation and output PVP time-series over the last few days can be used to elicit daily prediction models. These can process 24-h local forecasts of the meso-scale ‘Aladin’ NWP model to calculate PVP at corresponding times. However, this conversion and refinement of daily solar radiation forecast series is primarily influenced by the accuracy of processed NWP data [7]. Besides this, artificial neural network (ANN) iterative training may provide slightly different prediction results, and thus the final solution is usually the average of several model runs [8]. Analogue clustering and blending strategies can be applied to similar spatial historical meteorological data with earth declination angle to improve the accuracy of regional NWP models for 24-h PVP ensemble forecasting [9]. Most of the existing AI methodologies show some drawbacks such as over-fitting or dependence on particular PVP system designs. AI can use extra information, e.g. cloud motion vectors or spectral parameters, which characterise specific variances and classes of cloud structures in several atmospheric levels, to improve the models’ performance. Additional statistical properties can be extracted from auto-correlated data to capture periodical cycles [10]. Hybrid methods combine several different techniques or models’ unique features to overcome their single negative performance [11]. They usually require input data pre-processing. Spatial averaging and temporal interpolation of NWP outputs can be used with a clear sky model to better account for a typical irradiance diurnal cycle. Day-ahead numerical forecasts combined with a clear sky model can be analysed and evaluated to assess the changing size of training data sets or the number of iterations [6]. An ensemble distribution model of forecast errors for different cloud situations can determine a specific prediction interval to represent inhomogeneous cloud situations [12]. Self-organising map (SOM) can combine meteorological observations and forecasts of solar radiation, relative humidity and temperature to classify local weather in the prediction day into four preliminary types (cloudy, foggy, sunny and rainy). After the weather classification, ANN [13] or support vector machine (SVM) uses the same data in the training and testing to do 24-h predictions for the current recognised conditions [4]. A hybrid forecast method can combine SOM and exponential smoothing in the processing of satellite images to obtain the cloud cover index [14]. This parameter can be used together with the atmospheric transmission factor, determined according to NWP outputs [15]. Cloud imagery and satellite-based models can recognise the current type of cloud structures. Extrapolation of their motion leads to a forecast of solar radiation local conditions. Their errors increase under low sun elevations and high spatial variabilities of structures [1]. Probabilistic methods allow for consideration of unavoidable uncertainties in solar radiation. A competitive ensemble of several different base predictors can be used to obtain the optimal characteristics of sharpness and reliability in the predictive distribution. Sharpness is related to the concentration of the probabilities, while reliability corresponds to the probabilistic correctness of the forecasts [16]. An ensemble combines PVP forecasts obtained by several sophisticated AI methodologies and outperforms the single algorithms alone. Although second-order regressive methods may still be regarded as accurate enough for most operations [17]. Deep learning has the ability for automatic feature extraction. It can tolerate high noise data and integrate seemingly unrelated data sources to provide 24-h probabilistic or deterministic PVP forecasts. Long short-term memory mechanism is intended to find hourly and seasonal patterns across longer daily data periods [18]. An integrated hybrid method can combine a genetic algorithm (GA) and particle swarm optimisation (PSO) to optimise adaptive neuro-fuzzy inference systems-based forecasting model. Binary GA with Gaussian process regression (GPR) model can determine important input parameters that significantly influence the amount of PVP output for day-ahead PVP forecasts [19]. Spatio–temporal dependencies of automatically selected input data can be exploited to take into account local meteorological conditions. These are described by the estimated coefficients of an intra-day statistical forecasting model. Spatial densities of considered real-world PV plants can be used as additional model parameters. Stationarisation process overcomes weaknesses of the clear sky normalisation especially for early and late hours when solar irradiation is low [20]. The previous differential polynomial neural network (D-PNN) concept, based on similarity theory, allows complex representation of the local weather dynamics [21]. Conventional and soft-computing techniques usually significantly reduce the number of input variables leading to their model over-simplification. They can predict in the horizon of a few hours only. The proposed novel D-PNN develops gradually a multi-layer structure of polynomial neural network (PNN), starting from one node, to decompose the general linear PDE into a set of two-variable node converted PDEs. Resulting pure rational terms represent the Laplace images of unknown node functions, according to operational calculus (OC). The inverse Laplace transformation (L-transform) of selected rational terms, based on OC, is necessary to obtain the originals, the sum of which gives the complete PDE model. D-PNN selects the best-fit 2-inputs in all its nodes to produce applicable sub-PDE components, one by one, which can be included in the sum model to better approximate the desired output. A continual test using external complement allows only those components selection and parameters updates which minimise both the training and testing errors. [22]. The phase representation of real data, used in complex neural networks (CNNs), is applied in the node inverse L-transform as the Laplace images are expressed with complex numbers (Section 2). The main contribution of the paper: D-PNN develops gradually a PNN to decompose the general PDE into node converted sub-PDEs using OC. The PNN structure is dynamically changed, its nodes can be added, removed or the 2-inputs modified. Step by step expansion of the sum PDE model (starting empty) leads to the optimal problem representation according to Goedel's incompleteness theorem. Progressive production of rational composite terms (CTs) using the products of selected neurons (simple fractions) of back-connected blocks in previous layers. Applicable sum PDE components (rational terms) are selected and their parameters pre-optimised in the actual node block, one by one, in each iteration step. Group method of data handling (GMDH) polynomial parameters of a node block are shared by all the selected active neurons (of back-connected blocks) used to form the CT products. The optimal linear or second-order polynomial types and their sum members are selected from in each added/modified node block to produce its output and compose applicable rational sum terms (of a determined order), which correspond to the converts of specific sub-PDEs. Additional D-PNN assistant models are used initially to pre-estimate the optimal numbers of training days, according to the minimal testing error calculated for the latest hours [23]. D-PNN is trained with an increased time-shift between inputs and PVP output data to elicit its sum PDE models. These are applied to the latest morning input data to predict PV power in the trained time-horizon (Section 3). The presented D-PNN intra-day statistical predictions better detail daily PVP progress than those of regression or AI techniques and the converted cloudiness forecasts of regional NWP models [7] (Sections 4 and 5). 2 Decomposition of the general PDE D-PNN develops the two-variable node PNN structure to decompose the n-variable PDE into specific sub-PDEs of a determined order. These are converted using OC into pure rational terms corresponding to the Laplace images of unknown node functions. The sum of the selected inverse L-transformed rational terms gives a complete PDE model of the searched separable n-variable function. 2.1 Novel polynomial PDE substitution based on OC The general linear PDE (1) can describe an unknown separable u function of n-inputs, possible to be expressed in convergent sum series (2) of partial uk function solutions of two-variable sub-PDEs (3) [24] (1) (2) where u(x) = f(x1, x2, …, xn) is unknown n-variable separable function, a, b, c (c1, c2, …, cn), d (d11, d12, …), … are parameters and uk are partial sum functions of u(3) where F(x1, x2, u, p, q, r, s, t) is an unknown function described by the PDE. GMDH develops gradually, adding layer by layer, the PNN structure to decompose the complexity of the Kolmogorov–Gabor general polynomial (4). Its connections can model non-linear inter-relationships between inputs - output variables of an unknown system. A simple two-variable polynomial transfer function is used to calculate the neurons’ outputs in PNN nodes (5), from which the best ones are chosen, in consideration of selection constraints, to be applied in the next layer etc. [22]. The D-PNN computing approach is different from GMDH, though it decomposes the general PDE (1) similarly to the GMDH polynomial expansion (4). The D-PNN total output is the sum of selected node sub-PDE solutions, contrary to GMDH using the best node function output in the last layer (4) where n is the number of input variables X (x1, x2, …, xn) and A (a1, a2, …, an), … are polynomial parameters (5) A polynomial OC conversion of the f(t) function nth derivatives in an ordinary differential equation (ODE) is based on the proposition of their L-transform in consideration of the initial conditions (6) (6) where f(t), f′(t), …, f(n)(t) are originals continuous in the interval 〈0 + , ∞〉 and p, t are complex and real variables, respectively. The OC conversion of an ODE leads to algebraic (6), from which the L-transform image F(p). expressed with the complex number p, is separated in the form of a pure rational function (7) (7) where B, C, Ak are coefficients of element.fractions, a, b are polynom.parameters and α1, α2, …, αk are simple real roots of Q(p). The resulting pure rational fraction, which represents the L-transform F(p) of the function f(t), can be reduced to elementary sum fractions (7). The inverse L-transform using OC definitions is applied to them to obtain the original function f(t) of a real variable t (8) described by the ODE (8) where p, t are complex and real variables, respectively, and α1, α2, …, αn are simple real roots of Q. Two-variable pure rational terms (8), which represent the Laplace images of unknown uksum functions (2), are composed of the GMDH polynomials (6) in blocks of PNN nodes (Fig. 1). They convert specific second-order sub-PDEs, analogous to the OC ODE conversion (7). The derivatives of two-variable PDEs (3) correspond to the GMDH polynomial members (5). The inverse L-transform is applied to the selected rational terms in PNN nodes according to OC (8). The sum of the originals of two-variable node uk functions gives a complete separable u model function (2) [23] (9) where yi are node converted PDE rational terms and a, b are polynomial parameters. Fig. 1Open in figure viewerPowerPoint Blocks can produce simple neurons (/) or CTs – products of neurons (*) from GMDH fractions in PNN 2-input nodes Each node block (Fig. 1) can solve specific sub-PDEs forming simple neurons, e.g. (10), which can be included in the D-PNN total output sum to better approximate the target output (10) where yi are specific second-order PDE solutions of unknown node functions, φ = arctg(x2/x1) is the phase of two-input variables x1, x2 and ai, bi are parameters. The block node output (Fig. 1) is calculated using the GMDH polynomials (5). The designed sigmoidal transformation of squared variables (10) improves the polynomial ability to approximate periodic functions [24] (11) Eulers's notation of a complex number c in polar coordinates (11), used in CNN, corresponds to the f(t) expression of OC (8). The radius r (i.e. amplitude) can represent a rational term while the angle (i.e. phase) φ = arctg(x2/x1) of two real-valued input variables x1, x2 can be used in the inverse L-transform of F(p). 2.2 Backward production of composite sum PDE components in D-PNN tree-like structures D-PNN develops gradually the PNN structure, inserting node by node in the last added layer. Initially, one node block (Fig. 1) searches the most valuable two-input combinations to produce an applicable neuron, i.e. sub-PDE solution (10), which can be inserted into the empty sum model. Next blocks are added in PNN nodes, one by one, to extend the sum model by neurons which can minimise gradually the output errors until an improvement is possible. After that, the next layer is added to apply the block node GMDH outputs from the previous one and continue in this iteration procedure (Fig. 2). The step by step model expansion is based on Goedel's incompleteness theorem which states if the model complexity is gradually increased, objective functions of problem definition pass into the minima [25]. The D-PNN structure is dynamically developed in each iteration cycle, allowing changes in the number of layers and the selected nodes. Some of the node blocks can be inactivated to not produce neurons which can be deleted from or added new ones in the sum model. Fig. 2Open in figure viewerPowerPoint D-PNN develops gradually a PNN structure increasing/changing the number of its nodes and layers. Blocks select the optimal 2-inputs in each layer node to produce acceptable neurons + CTs included in a sum PDE solution Multi-layer PNN forms composite polynomial functions (12). Node blocks (Fig. 1) in the second and next layers can produce CTs in addition to the simple neurons (10). CTs are the products of selected neurons, i.e. node converted sub-PDEs, of back-connected blocks in the actual and previous layers (Fig. 2). CTs detail sub-PDE solutions of the unknown uk node composite function in the product of the external and internal function images in accordance with the composite function partial derivation rules (13) (12) (13) Neurons in each next layer can produce more composite partial models, using output functions and PDE converts of the previous ones. Biological neurons seem to use analogously dynamic pulses to code the composite output functions and transform sub-solutions in the amplitudes and phases. This production process is terminated in an additive cognitive neuron to sum partial model solutions of some neurons in the complete derivative model (14) where Qij, Pij are output and reduced GMDH polynomials of n and n−1th degree, respectively and yij is a CT using the products of selected neurons of blocks in the previous layers, i.e. a specific second-order sub-PDE solution. The first layer blocks can use its own node two-inputs only to form simple neurons (10), i.e. sub-PDE solutions. The second and next layer node blocks can form CTs, in addition to the neurons, to detail the particular sub-PDE models in the product of its own and selected neurons of back-connected blocks in the previous layers. The third layer blocks, for example, can produce CTs using selected neurons of 2 and 4 blocks in the second and first layers (14). CTs are the products of converted sub-PDEs in the starting block (i.e. the external function L-image) and those in blocks of the previous layers (the internal function L-images) (14). The number of possible CTs in blocks doubles with each back-joined layer (Fig. 2). All the D-PNN blocks in each layer form equivalent neurons and CTs, i.e. node sub-PDE components, from which one can be selected to be included in the total output sum. The D-PNN output Y is divided by the number of active neurons + CTs (15) to simplify the optimisation of the parameters and weights, using the Gradient method [24] (15) where k is the number of active neurons and CTs (sub-PDE sum solutions) produced in blocks of PNN nodes. All input combination couples of a node block are tested in each iteration step to produce an applicable neuron or CT, which can be included in the sum model to better approximate the desired output. This 1-block iteration algorithm skips from the actual to the next block, one by one, to minimise the training error in consideration of a continual test. the external complement of GMDH allows only selection and parameters updates of those PDE components which decrease also the testing error, calculated on a minor reserved part of the data set (not used in the training). This approach, using appropriate restriction criteria, usually leads in successive steps to the optimal model which can be applied to unseen input data in prediction [25]. The root mean squared error (RMSE) is calculated (16) for the actual sum PDE model in each iteration step of the D-PNN training and testing (16) where yi is produced output for the ith training vector, is desired output and M is the number of training data samples. 3 Intra-day multi-step PV power prediction PV power was predicted for a plant (Table 1), located in Starojícka Lhota, Czech Rep., using its local historical data measurements of temperature and solar radiation together with spatial data of wind speed and direction from three wind farms in Maletín, Drahany and Veselí nad Moravou. This data set was extended with available meteorological observations (average temperature, relative humidity, see level pressure, wind speed and azimuth, sky conditions) from two nearby airport weather-stations in Brno-Turany and Ostrava-Mošnov (Fig. 3). Detailed time-series improve the accuracy of short-term prediction models although 10 min. measurements of the PVP plant and wind farms were averaged to be related to half-hour series of the airport weather observations [26]. The selection from 2 to 3 input time-series (lags) of the data variables, including 1 variable to indicate the time, improves the model performance. D-PNN was trained with data samples of 22 columns, from the estimated optimal number of the last days to the increased inputs-output 1, 2, …, 9-h shift for each 8:00 am – 5:00 pm prediction hour. The resulting PDE models were applied to the latest morning input data from 7:30 am to predict PV power in the trained time-horizon 8:00 am – 5:00 pm (Fig. 4). The rest of the intermediate points for PVP predictions at 8:30, 9:30, …, 16:30 were calculated using linear interpolation. Table 1. PV plant specification PVP plant PV panels Type Efficiency, % Max. power, MWp h Number of panels Starojícka Lhota monocrystalline 17.7 1.1 210 Fig. 3Open in figure viewerPowerPoint Map showing the PVP plant location and sites from where measured and observation data were obtained Fig. 4Open in figure viewerPowerPoint Intra-day CSI prediction 1–9-h ahead using separate PDE models (red-below) developed with training data from the estimated optimal daily periods for each increasing inputs-output time-shift (blue-above) D-PNN assistant test models were initially developed, using data from the last 2, 3, …, x days, to pre-estimate the optimal numbers of the last training days. The output of the assistant model is continually compared with the reserved last 6-h measurements of the desired CSI to obtain the lowest testing error which indicates the optimal length of a daily training period used to elicit all the intra-day prediction models. Training using the optimal number of the last days, with data patterns similar to those in the testing hours, usually leads to models applicable to the latest unseen input data in prediction (Fig. 4). Periods of a sort of less settled weather over several days tend to prevail till they are broken up by an overnight change in conditions (17) where PVPclear is clear sky power and t is time. Clear sky index (CSI) kt, the ratio of the actual PV power to the PV power under clear sky conditions at a time t (17), was used as the model output. PVP and solar radiation data series were normalised using CSI which gives the relative values aside from solar diurnal cycle [3]. The PVP output was restored from predicted kt values according to the prediction hours (17). The course of the clear sky cycles (at each time) was estimated from the maxima of seasonal data. The daily average ratios of PVP series (Fig. 5) indicate the relative complexity of data patterns. Small changes in their values denote a smooth simple PVP curve in clear weather (Fig. 6), contrary to ramp PVP changes in variable cloudiness (Fig. 7). The PVP series ratios are dimensionless and characterise daily data patterns regardless of real power values (Fig. 5). Fig. 5Open in figure viewerPowerPoint Daily average PV power and series ratio, 12–25 May 2011 Fig. 6Open in figure viewerPowerPoint St. Lhota, 14 May 2011 – variable cloudiness (previous sunny period), RMSE: D-PNN = 96.4, Persistent = 94.7, RNN = 131.9, SMLT = 104.8 kWp 30 min Fig. 7Open in figure viewerPowerPoint St. Lhota, 15 May 2011 – overcast (low PVP, catching and sunny periods follow), RMSE: D-PNN = 38.1, Persistent = 111.4, RNN = 19.6, SMLT = 66.5 kWp 30 min Figs. 6-9 show intra-day 9-h PVP predictions in the monitored 2-week period from 12 to 25 May 2011. The applied persistent models, using the optimal lengths of daily periods (Fig. 10), average CSI over the last 1, …, x days to calculate the best PVP predictions at the corresponding hours 8 am–5 pm. They represent the minimal error benchmark. Fig. 8Open in figure viewerPowerPoint St. Lhota, 19 May 2011 – almost clearly, RMSE: D-PNN = 44.3, Persistent = 46.2, RNN = 127.7, SMLT = 55.2 kWp 30 min Fig. 9Open in figure viewerPowerPoint St. Lhota, 23 May 2011 – variable cloudiness (catching and sunny periods follow), RMSE: D-PNN = 59.7, Persistent = 77.3, RNN = 84.9, SMLT = 67.0 kWp 30 min Fig. 10Open in figure viewerPowerPoint Daily initialisation time of the intra-day D-PNN, persistent and SMLT prediction models Recurrent neural network (RNN), using four lagged inputs of CSI time-series, was used to predict PVP at the next time steps. Its multi-step iterative approach employs predicted values at the previous times as the model inputs in the successive time steps. RNN consists of 1 hidden layer of 20 neurons with the sigmoidal activation function and the same context layer. Matlab – Statistics and Machine Learning Tool-box (SMLT), using data of the same 23 inputs-PVP output variables as D-PNN, was used analogously to predict the next day CSI (Fig. 4) converted to daily PVP estimations (17). Matlab SMLT for regression includes these methods [27]: Linear Regression – interactions, robust, stepwise. Regression Trees – fine, medium, coarse. Support vector machine – with linear, quadratic, cubic or Gaussian kernel. GPR – rational quadratic, exponential, squared exponential, matern. Ensemble boosted or bagged tree (EBT). The optimal 24-h SMLT prediction models were selected according to their minimal test errors calculated with the last day 6-h afternoon data. The optimal training periods (Fig. 10), used to elicit the RNN and SMLT prediction models, were initially pre-estimated by their additional test models examining data from the previous 1, …, x days, analogous to D-PNN (Fig. 4). The models of GPR, SVM and EBT obtained the lowest testing RMSEs with only slight differences. RNN using the same spatial meteorological and wind data is supposed to produce similar prediction results. Fig. 10 shows the estimated numbers of training days used by D-PNN, SMLT and persistent benchmark, i.e. initialisation time of all the intra-day prediction models for each inputs to output time-shift 1–9 h (Fig. 4). The optimal daily calc