# Integrated Machine Learning and Enhanced Statistical Approach-Based Wind Power Forecasting in Australian Tasmania Wind Farm

Authors: Fang Yao; Wei Liu; Xingyong Zhao; Li Song
Journal: Complexity
 (2020)

---

## Abstract

This paper develops an integrated machine learning and enhanced statistical approach for wind power interval forecasting. A time-series wind power forecasting model is formulated as the theoretical basis of our method. The proposed model takes into account two important characteristics of wind speed: the nonlinearity and the time-changing distribution. Based on the proposed model, six machine learning regression algorithms are employed to forecast the prediction interval of the wind power output. The six methods are tested using real wind speed data collected at a wind station in Australia. For wind speed forecasting, the long short-term memory (LSTM) network algorithm outperforms other five algorithms. In terms of the prediction interval, the five nonlinear algorithms show superior performances. The case studies demonstrate that combined with an appropriate nonlinear machine learning regression algorithm, the proposed methodology is effective in wind power interval forecasting.


---

## Body

## 1. Introduction

Wind power is rapidly expanding its market share around the world. However, the intermittency and uncertainty of wind make it a challenge to integrate wind power into the power system. The wind power forecasting system can greatly help the integration process since system operators rely on accurate wind power forecasts to design operational plans and assess system security [1, 2]. Servo mechanism is the foundation of wind turbines, and precise wind power forecasting can improve the accuracy of parameter estimation and control of wind turbine servo systems [3–5]. Predictions of wind power outputs are traditionally provided in the form of point forecasts. The advantage of point forecasts is that they can be easily understood. The single value is expected to tell everything about future power generation. Nowadays, the majority of the research efforts on wind power forecasting is still focused on point forecasting. The reviews of the state of the art in wind power forecasting can be found in [6, 7]. A book on physical approaches to short-term wind power forecasting also partly discusses the state of the art in wind power forecasting [8].However, even if both meteorological and power conversion processes are well understood and modelled, there will always be inherent and inevitable uncertainty in wind power forecasts. The uncertainty comes from the incomplete knowledge of the physical processes that influence future events [9]. The uncertainty of wind power forecasts mainly depends on the predictability of the current meteorological status and the level of the predicted wind speed [10]. To assist with the management of the forecasting uncertainty, extensive research studies have been conducted to develop wind power forecasting methods. Different regression methods have been introduced in [10–12]. These approaches use probabilistic forecasts generated by different quantile regression methods to provide the complete information of future wind production. A multiscale reliable wind power forecasting (WPF) method was developed by Yan et al. in [13]. This method provides the expected future value and the associated uncertainty by a multi-to-multi mapping network and stacked denoising autoencode. Wang et al. developed a short-term wind prediction method with the convolutional neural network (CNN) based on information of neighbouring wind farms [14]. One popular approach is to use ensemble-based probabilistic forecasting methodologies, which enable better wind power management and trading purposes [15, 16]. In [17, 18], statistical analysis has been conducted to study the distribution of wind power forecasting errors. Because wind power is stochastic in nature, errors will always exist in wind power forecasts. Therefore, besides predicting the expected value of the future wind power, it is also important to estimate its forecasting errors.A key weakness of the above studies lies in that they fail to establish proper statistical models for interval forecasting of wind power and also fail to take into account the time-changing effect of the error distribution. Generally speaking, a prediction interval is a stochastic interval, which contains the true value of wind power with a preassigned probability. Because the prediction interval can quantify the uncertainty of the forecasted wind power, it can be employed to evaluate the risks of the decisions made by market participants. Existing methods discussed above cannot effectively handle wind power interval forecasting since they mainly focus on predicting the expected point value of wind power.There are two main challenges for providing accurate interval forecasting of wind power: (i) the expected value of wind power should be accurately predicted. This is difficult since wind power is a nonlinear time series and is therefore highly volatile. The nonlinear research system has high complexity in the fundamental research, and lots of nonlinear control problems unceasingly emerge in real fact [19–23]; (ii) the probability distribution of forecasting errors should also be accurately estimated. This is even more difficult since the error distribution can be time-changing. In this paper, a novel approach is proposed to forecast the prediction interval of wind power. A statistical model is first formulated to properly model the time series of wind speed. Based on the proposed model, a number of different machine learning algorithms are introduced to predict the expected value of wind speed and the parameters of forecasting error distribution. Prediction intervals of wind speed are then constructed based on the predicted wind speed value and error distribution. The wind speed prediction interval is finally transformed into the wind power prediction interval with the wind turbine power curve. Comprehensive studies are performed to compare the performances of six machine learning algorithms in wind power interval forecasting.The main contributions of this paper are as follows:(1)
A comprehensive statistical model is introduced, which forms the theoretical basis for wind power interval forecasting.(2)
Different machine learning regression methods are incorporated into the proposed model. The comparison of different regression algorithms in wind power forecasting is presented.(3)
The proposed integrated statistical machine learning approach can highlight the essential information of the available data.The rest of the paper is organized as follows: in Section2, a statistical model for the wind speed time series is formulated. We also introduce the Lagrange multiplier (LM) test to verify that the forecasting errors of wind power have a time-changing distribution. In Section 3, the basic concept of machine learning and six machine learning algorithms for wind power forecasting are introduced. Afterwards, comprehensive case studies are performed in Section 4. Section 5 finally concludes the paper.


## 2. The Statistical Model of the Wind Speed Time Series

To forecast the power output of a wind turbine, a widely used approach is to predict the wind speed first and then transform the predicted wind speed into wind power with the power curve. Therefore, in this section, a statistical model of wind speed is first formulated. We will also briefly explain how to integrate the proposed model with nonlinear regression techniques to forecast the prediction intervals of wind speed. The wind speed time series can usually be assumed to be generated by the following stochastic process:(1)Yt=fYt−1,Yt−2,…,X⟶t+εt,where Yt denotes the random wind speed and yt is the observed value of Yt at time t. X⟶t∈Rm is an m-dimensional explanatory vector. Each element Xt,i of X⟶t represents an explanatory variable which can influence Yt, for example, the temperature and humidity. The current value of Yt can be determined by its lagged values Yt−1,Yt−2,… and the explanatory vector X⟶t. Note that the mapping f• from Yt−1,Yt−2,…,X⟶t to Yt can be any linear or nonlinear function. Most existing methods essentially forecast wind speed by estimating mapping f•; the forecasted value f^• of f• can be called the point forecast of wind speed. According to (1), the wind speed Yt contains two components: f• is a deterministic component, and εt is a random component, which is also known as noise. Statistical and engineering models are an approximation to reality, not reality, so they always have some degree of errors. Nowadays, there are lots of research studies about error tracking and control [24–29]. Precise prediction and reducing the error are the prerequisite for all further control works. Detailed statistical studies [30] show that εt can be assumed to follow a normal distribution. We therefore have(2)εt∼Nμ,σ2.Becausef• is a deterministic function, we should be able to approximate it with arbitrary accuracy by employing a powerful nonlinear machine learning technique (e.g., neural network). Most existing wind speed forecasting methods mainly focus on estimating f• and selecting its estimated value as the predicted wind speed. On the contrary, because of the uncertainty introduced by noise εt, errors will always exist in wind speed forecasts. Therefore, estimating μ and σ2 is essential for estimating the uncertainty of Yt. In models (1) and (2), parameters μ and σ2 are assumed to be constant. In practice, the model parameters can usually be time-changing. We therefore introduce the following time-changing distribution model of wind speed:(3)Yt=fYt−1,Yt−2,…,X⟶t+εt,εt=μt+σt⋅vt,vt∼N0,1,X⟶t′=Xt,1,Xt,2,…,Xt,m,ut=gεt−1,εt−2,…,X⇀t,σt=hεt−1,εt−2,…,X⇀t.Similar tof•, mappings g• and h• can also be either linear or nonlinear. According to model (3), the uncertainty of wind speed is time-changing. The mean and variance of noise εt are determined by the previous noises and the explanatory vector. Note that model (3) is a generalization to the traditional ARCH (AutoRegressive Conditional Heteroskedasticity) model; since by setting ut≡0 and assuming f• and h• are linear functions, model (3) will be identical to the ARCH model. To more strictly justify our model, the Lagrange multiplier (LM) test can be employed to verify that the wind speed has a time-changing distribution. In the case study, we will test whether the actual wind speed data of Australia have a time-changing distribution by performing the LM test.Based on statistical model (3) of wind speed, we can construct the prediction interval, which contains the true value of wind speed with any preassigned probability. The definition of the prediction interval can be given as follows.Definition 1.
Given a time seriesYt which is generated with model (3), an α-level prediction interval (PI) of Yt is a stochastic interval Lt,Ut calculated from Yt such that PYt∈Lt,Ut=1−α.
Because noiseεt is usually assumed to be normally distributed, the α-level prediction interval can therefore be calculated as(4)Lt=ft•+μt−z1−a/2×σt,(5)Ut=ft•+μt+z1−a/2×σt,where ft• represents the value of the deterministic component f• at time t, α is the confidence level, and z1−a/2 is the critical value of the standard normal distribution. Based on (4) and (5), to calculate the prediction interval, we should first obtain three quantities: the wind speed forecast ft•, the mean μ, and the variance σ2 of the noise. In practice, traditional time-series models, such as ARIMA and GARCH, usually perform poorly on short-term wind speed forecasting since they are linear models and therefore cannot handle the complex nonlinear patterns of wind speed data. To give accurate wind speed forecasts, the three mappings f•, g•, and h• in model (3) should be accurately estimated with nonlinear techniques. In this paper, we introduce six different machine learning methods to estimate f•, g•, and h•. To apply machine learning methods to estimate g• and h•, an unsolved problem is how to obtain the estimates of mean μt and variance σt2 of the noise. In this paper, the moving window method is employed. Given the noise series εt, the estimates of μt and σt2 can be calculated as(6)μ^t=12n+1∑s=t−nt+nεs,(7)σ^t2=12n∑s=t−nt+nεs−μ^s2.
By combining a machine learning method with proposed model (3), the main procedure of wind power interval forecasting is given as follows:(1)
Given the historical wind speed dataYt and the explanatory vector data X⟶t for time period 0,T, employ a machine learning technique to estimate function Yt=fYt−1,Yt−2,…,X⟶t. Denote the estimate of f• as f^•.(2)
Calculate the forecasting errorset=Yt−f^Yt−1,Yt−2,…,X⟶t for period 0,T. Note that et can be considered as the estimate of noise εt.(3)
Based on error serieset, calculate the estimates of μt and σt2 with equations (6) and (7).(4)
Based on error serieset and mean and variance estimate series μ^t and σ^t2, employ a machine learning technique to estimate functions μ^t=g^et−1,et−2,X⟶t and σ^t=h^et−1,et−2,X⟶t, and use them as the estimates of g• and h•.(5)
To forecast the wind speed att, first employ f^•, g^•, and h^• to calculate f^t•, μ^t, and σ^t2; then, calculate the wind speed prediction interval with equations (4) and (5).(6)
Transform the wind speed prediction interval into the wind power prediction interval with the wind turbine power curve, which will be discussed in the following sections.


## 3. Machine Learning Methods for Wind Power Interval Forecasting

In this section, we first provide a brief introduction to machine learning, which is an important research area in forecasting. Six machine learning algorithms used in this paper are then presented. The power curve for converting wind speed into wind power is introduced. We finally discuss how to evaluate the performance of wind power interval forecasting methods.


### 3.1. Introduction to Machine Learning

Machine learning is science that studies how to use the computer to simulate or realize human learning activities. It is one of the most intelligent and leading-edge research fields in artificial intelligence. Machine learning techniques are essential to the renewable energy integration such as PV and wind power [31, 32].Machine learning can be divided into supervised learning and unsupervised learning [33–35]. As can be seen from Figure 1, supervised learning can be classified into classification and regression, and unsupervised learning can be classified into clustering and correlation.Figure 1
Composition structure of machine learning.Regression [36] is a process to estimate a functional mapping between a data vector and a target variable. Regression aims at determining a continuous target variable, which is usually named as the dependent variable, while the data item itself is usually called independent variables, explanatory variables, or predictors. For example, in wind speed forecasting, the predictors can be historical wind speed, temperature, and humidity, while the independent variable is the future wind speed. Regression usually estimates the mapping based on a training dataset in which the independent variables of all data items have been given. Regression is therefore a supervised learning problem in the sense that the estimation of the mapping is supervised by the training data. Regression is also an important research area of statistics. The most important statistical method is linear regression, which assumes that the independent variable is determined by a linear function of predictors. In recent years, the machine learning society has proposed many other regression methods, such as deep learning. In this paper, we will introduce six different machine learning regression techniques and integrate them with the proposed statistical model to perform wind power interval forecasting.


### 3.2. Machine Learning Regression Algorithms Employed in This Paper

#### 3.2.1. Linear Regression

Linear regression is a traditional and widely used statistical technique for regression. It is selected as the baseline technique in this paper and will be compared with five nonlinear techniques. Linear regression models the relationship between the dependent variableyi and the vector of predictors xi. Linear regression assumes that the independent variable y is linearly dependent on the predictors x plus a noise term εi. The model can be written as(8)yi=β1xi1+⋯+βpxip+εi=xi′βT+εi,i=1,2,…,n,where xi′βT is the inner product between vectors xi and β. And these n equations can be written in the vector form as(9)y=βTX+ε,where(10)y=y1y2⋮yn,X=x1′x2′⋮xn′,βT=β1⋮βp,ε=ε1ε2⋮εn.ε is usually assumed to follow a normal distribution with a zero mean and variance σ2. We therefore have(11)ε∼N0,σ2,and β is a p-dimensional parameter vector, which specifies how much each component of X contributes to the output y [37].


#### 3.2.2. Multilayer Perceptron Network

A multilayer perceptron (MLP) network is a feedforward artificial neural network model that maps sets of input data onto a set of appropriate outputs. Based on the standard linear perceptron, MLP uses three or more layer nodes with nonlinear activation functions. An MLP network consists of a set of source nodes as the input layer, one or more hidden layers of computation nodes, and an output layer of nodes.Figure2 shows the signal flow process of a feedforward neural network. A MLP network has two stages: a forward pass and a backward pass. The forward pass includes presenting a sample input to the network and letting activations flow until they reach the output layer [38, 39].Figure 2
Diagram of a multilayer perceptron network.


#### 3.2.3. Long Short-Term Memory (LSTM) Network-Based Deep Learning Method

The concept of deep learning was first proposed by Hinton et al. in 2006. Deep learning is a branch of machine learning. It is essentially a special artificial neural network. Deep learning utilizes a multilayer network structure and applies appropriate nonlinear transformation functions to each hidden node to achieve the purpose of high-level abstraction of the data. The traditional feedforward artificial neural network usually contains only one hidden layer, but there are many hidden layer structures in deep learning. Therefore, deep learning adopts a training mechanism completely different from the traditional artificial neural network in order to solve the problem of the deep neural network in training [40].LSTM is a time-cycle neural network, which can effectively solve the gradient explosion and gradient disappearance problems compared with the traditional cycle neural network. LSTM is composed of a set of cyclic subnets called memory blocks. Each memory block is composed of the input gate, forgetting gate, and output gate. Figure3 shows the LSTM structure [41].Figure 3
LSTM storage unit with door.In general, the LSTM recursive neural network is composed of the following components: input gateit has the corresponding weight matrix wxi, whi, wci, and bi; forgetting gate ft has the corresponding weight matrix whf, wcf, and bf; and output door ot with the corresponding weight matrix wxo, who, wco, and bo. The function of the input gate is to record the new information to the cell state selectively. The function of the forget gate is to selectively forget the status information in the cell; the function of the output gate is to export certain information from the cell. The detailed workflow of LSTM is shown in the following [42]:(12)it=σωxixt+whiht−1+wcict−1+bi,ft=σwxfxt+whfht−1+wcfct−1+bf,ot=σwxoxt+whoht−1+wcoct−1+bo,ct=ftct−1+ittanhwxcxt+whiht−1+whcct−1+bc,where σ is the logistic sigmoid function with the output in [0, 1] and tanh represents the hyperbolic tangent function with the output in [−1, 1].


#### 3.2.4. Lazy IBK

Lazy IBK is one of the widely used lazy learning methods. Lazy learning methods defer the decision of how to assign the dependent variable until a new query explanatory vector is inputted. When the query explanatory vector is received, a set of similar data records is retrieved from the available training dataset and is used to assign the dependent variable to the new instance [43]. In order to choose the similar data records, lazy methods employ a distance measure that will give nearby data records higher relevance. Lazy methods choose the k data records that are nearest to the query instance. The dependent variable of the new instance is determined based on the k-nearest instances.Lazy learning algorithms have three basic steps:(i)
Defer: lazy learning algorithms store all training data and defer processing until a new query is given.(ii)
Reply: a local learning approach developed by Bottou and Vapnik in 1992 is a popular method to determine the dependent variables for news queries [44]. In the Bottou and Vapnik learning approach, instances are defined as points in the space, and a similarity function is defined on all pairs of these instances.(iii)
Flush: after solving a query, the answer and any intermediate results are discarded.


#### 3.2.5. Regression Tree

A regression tree is one of the widely used decision tree algorithms. A decision tree is a data-mining tool designed to extract useful information from large datasets and use the information to help decision-making processes. A regression tree consists of a set of nodes that can assign the value of the dependent variable to an explanatory vector. Regression tree constructs a tree style decision rule set and divides the training data into the leaf nodes of the decision tree according to the numerical or categorical values of explanatory variables. The regression rules of each leaf node are derived from a mathematical process that minimizes the regression errors of the leaf nodes [45].


#### 3.2.6. Decision Table

Similar to the regression tree, decision table also determines the value of the dependent variable with a set of decision rules [46]. However, the decision table arranges decision rules as a table, rather than a tree. A decision table usually consists of a number of parallel decision rules. Similar to the regression tree, the training data will be divided into several groups, each of which will be represented by a decision rule. For a given explanatory vector (input), an appropriate decision rule will be first selected based on the values of its explanatory variables. The dependent variable for this input will be assigned as the average of the dependent variables of all training data vectors in the corresponding group. The dependent variable can also be determined by performing linear regression on the corresponding group of training data. Empirical studies show that the decision table has a similar performance to regression trees.


### 3.3. Converting Wind Speed to Wind Power

An elementary method is used in this paper to convert the predicted wind speed to the predicted wind power output of a wind turbine or wind farm. The predicted wind speed is provided by one of the six machine learning regression methods discussed above. The wind speed is then input into the certified wind turbine power curve and transformed into the wind power.The Vestas V90-3.0 MW wind turbine is selected for the case studies in this paper. Vestas V90-3.0 MW is a pitch regulated upwind wind turbine with active yaw and a three-blade rotor. It has a rotor diameter of 90 m with a generator rated at 3.0 MW. Vestas V90-3.0 MW is widely used in Australia wind power plants and has a proven high efficiency.The typical power curve of Vestas V90-3.0 MW, 60 Hz, 106.7 dB(A) is shown in Figure4. It can be clearly observed that the wind power output pu is proportional to u3 for small wind speed u. Moreover, the power curve is steep for medium wind speeds and flat for large wind speeds. The cut-in speed is 3.5 m/s, and the cut-out speed is 25 m/s [47].Figure 4
Power curve for Vestas V90-3.0 MW, 60 Hz, 106.7.


### 3.4. Performance Evaluation

Before proposing the case study results, several criteria are introduced for performance evaluation. GivenT historical wind power values pt, 1 ≤ t ≤ T, of a time series pt which are converted from T historical wind speed observations and the corresponding forecasted power values pt∗, 1 ≤ t ≤ T; mean absolute percentage error (MAPE) is defined as(13)MAPE=1T∑t=1Tpt−pt∗pt.MAPE is a widely used criterion for time-series forecasting. It will also be employed to evaluate the proposed method in the case studies.Another two criteria are presented to evaluate interval forecasting. GivenT wind power values pt, 1 ≤ t ≤ T, of a time series Py and the corresponding forecasted α-level prediction intervals lt,ut, 1 ≤ t ≤ T, the empirical confidence α^ [48] and the absolute coverage error (ACE) are defined as(14)α^=frequencypt∈lt,utT,ACE=α−α∧,where α^ is the number of observations, which fall into the forecasted prediction interval (PI), divided by the sample size. It should be as close to α as possible.


## 3.1. Introduction to Machine Learning

Machine learning is science that studies how to use the computer to simulate or realize human learning activities. It is one of the most intelligent and leading-edge research fields in artificial intelligence. Machine learning techniques are essential to the renewable energy integration such as PV and wind power [31, 32].Machine learning can be divided into supervised learning and unsupervised learning [33–35]. As can be seen from Figure 1, supervised learning can be classified into classification and regression, and unsupervised learning can be classified into clustering and correlation.Figure 1
Composition structure of machine learning.Regression [36] is a process to estimate a functional mapping between a data vector and a target variable. Regression aims at determining a continuous target variable, which is usually named as the dependent variable, while the data item itself is usually called independent variables, explanatory variables, or predictors. For example, in wind speed forecasting, the predictors can be historical wind speed, temperature, and humidity, while the independent variable is the future wind speed. Regression usually estimates the mapping based on a training dataset in which the independent variables of all data items have been given. Regression is therefore a supervised learning problem in the sense that the estimation of the mapping is supervised by the training data. Regression is also an important research area of statistics. The most important statistical method is linear regression, which assumes that the independent variable is determined by a linear function of predictors. In recent years, the machine learning society has proposed many other regression methods, such as deep learning. In this paper, we will introduce six different machine learning regression techniques and integrate them with the proposed statistical model to perform wind power interval forecasting.


## 3.2. Machine Learning Regression Algorithms Employed in This Paper

### 3.2.1. Linear Regression

Linear regression is a traditional and widely used statistical technique for regression. It is selected as the baseline technique in this paper and will be compared with five nonlinear techniques. Linear regression models the relationship between the dependent variableyi and the vector of predictors xi. Linear regression assumes that the independent variable y is linearly dependent on the predictors x plus a noise term εi. The model can be written as(8)yi=β1xi1+⋯+βpxip+εi=xi′βT+εi,i=1,2,…,n,where xi′βT is the inner product between vectors xi and β. And these n equations can be written in the vector form as(9)y=βTX+ε,where(10)y=y1y2⋮yn,X=x1′x2′⋮xn′,βT=β1⋮βp,ε=ε1ε2⋮εn.ε is usually assumed to follow a normal distribution with a zero mean and variance σ2. We therefore have(11)ε∼N0,σ2,and β is a p-dimensional parameter vector, which specifies how much each component of X contributes to the output y [37].


### 3.2.2. Multilayer Perceptron Network

A multilayer perceptron (MLP) network is a feedforward artificial neural network model that maps sets of input data onto a set of appropriate outputs. Based on the standard linear perceptron, MLP uses three or more layer nodes with nonlinear activation functions. An MLP network consists of a set of source nodes as the input layer, one or more hidden layers of computation nodes, and an output layer of nodes.Figure2 shows the signal flow process of a feedforward neural network. A MLP network has two stages: a forward pass and a backward pass. The forward pass includes presenting a sample input to the network and letting activations flow until they reach the output layer [38, 39].Figure 2
Diagram of a multilayer perceptron network.


### 3.2.3. Long Short-Term Memory (LSTM) Network-Based Deep Learning Method

The concept of deep learning was first proposed by Hinton et al. in 2006. Deep learning is a branch of machine learning. It is essentially a special artificial neural network. Deep learning utilizes a multilayer network structure and applies appropriate nonlinear transformation functions to each hidden node to achieve the purpose of high-level abstraction of the data. The traditional feedforward artificial neural network usually contains only one hidden layer, but there are many hidden layer structures in deep learning. Therefore, deep learning adopts a training mechanism completely different from the traditional artificial neural network in order to solve the problem of the deep neural network in training [40].LSTM is a time-cycle neural network, which can effectively solve the gradient explosion and gradient disappearance problems compared with the traditional cycle neural network. LSTM is composed of a set of cyclic subnets called memory blocks. Each memory block is composed of the input gate, forgetting gate, and output gate. Figure3 shows the LSTM structure [41].Figure 3
LSTM storage unit with door.In general, the LSTM recursive neural network is composed of the following components: input gateit has the corresponding weight matrix wxi, whi, wci, and bi; forgetting gate ft has the corresponding weight matrix whf, wcf, and bf; and output door ot with the corresponding weight matrix wxo, who, wco, and bo. The function of the input gate is to record the new information to the cell state selectively. The function of the forget gate is to selectively forget the status information in the cell; the function of the output gate is to export certain information from the cell. The detailed workflow of LSTM is shown in the following [42]:(12)it=σωxixt+whiht−1+wcict−1+bi,ft=σwxfxt+whfht−1+wcfct−1+bf,ot=σwxoxt+whoht−1+wcoct−1+bo,ct=ftct−1+ittanhwxcxt+whiht−1+whcct−1+bc,where σ is the logistic sigmoid function with the output in [0, 1] and tanh represents the hyperbolic tangent function with the output in [−1, 1].


### 3.2.4. Lazy IBK

Lazy IBK is one of the widely used lazy learning methods. Lazy learning methods defer the decision of how to assign the dependent variable until a new query explanatory vector is inputted. When the query explanatory vector is received, a set of similar data records is retrieved from the available training dataset and is used to assign the dependent variable to the new instance [43]. In order to choose the similar data records, lazy methods employ a distance measure that will give nearby data records higher relevance. Lazy methods choose the k data records that are nearest to the query instance. The dependent variable of the new instance is determined based on the k-nearest instances.Lazy learning algorithms have three basic steps:(i)
Defer: lazy learning algorithms store all training data and defer processing until a new query is given.(ii)
Reply: a local learning approach developed by Bottou and Vapnik in 1992 is a popular method to determine the dependent variables for news queries [44]. In the Bottou and Vapnik learning approach, instances are defined as points in the space, and a similarity function is defined on all pairs of these instances.(iii)
Flush: after solving a query, the answer and any intermediate results are discarded.


### 3.2.5. Regression Tree

A regression tree is one of the widely used decision tree algorithms. A decision tree is a data-mining tool designed to extract useful information from large datasets and use the information to help decision-making processes. A regression tree consists of a set of nodes that can assign the value of the dependent variable to an explanatory vector. Regression tree constructs a tree style decision rule set and divides the training data into the leaf nodes of the decision tree according to the numerical or categorical values of explanatory variables. The regression rules of each leaf node are derived from a mathematical process that minimizes the regression errors of the leaf nodes [45].


### 3.2.6. Decision Table

Similar to the regression tree, decision table also determines the value of the dependent variable with a set of decision rules [46]. However, the decision table arranges decision rules as a table, rather than a tree. A decision table usually consists of a number of parallel decision rules. Similar to the regression tree, the training data will be divided into several groups, each of which will be represented by a decision rule. For a given explanatory vector (input), an appropriate decision rule will be first selected based on the values of its explanatory variables. The dependent variable for this input will be assigned as the average of the dependent variables of all training data vectors in the corresponding group. The dependent variable can also be determined by performing linear regression on the corresponding group of training data. Empirical studies show that the decision table has a similar performance to regression trees.


## 3.2.1. Linear Regression

Linear regression is a traditional and widely used statistical technique for regression. It is selected as the baseline technique in this paper and will be compared with five nonlinear techniques. Linear regression models the relationship between the dependent variableyi and the vector of predictors xi. Linear regression assumes that the independent variable y is linearly dependent on the predictors x plus a noise term εi. The model can be written as(8)yi=β1xi1+⋯+βpxip+εi=xi′βT+εi,i=1,2,…,n,where xi′βT is the inner product between vectors xi and β. And these n equations can be written in the vector form as(9)y=βTX+ε,where(10)y=y1y2⋮yn,X=x1′x2′⋮xn′,βT=β1⋮βp,ε=ε1ε2⋮εn.ε is usually assumed to follow a normal distribution with a zero mean and variance σ2. We therefore have(11)ε∼N0,σ2,and β is a p-dimensional parameter vector, which specifies how much each component of X contributes to the output y [37].


## 3.2.2. Multilayer Perceptron Network

A multilayer perceptron (MLP) network is a feedforward artificial neural network model that maps sets of input data onto a set of appropriate outputs. Based on the standard linear perceptron, MLP uses three or more layer nodes with nonlinear activation functions. An MLP network consists of a set of source nodes as the input layer, one or more hidden layers of computation nodes, and an output layer of nodes.Figure2 shows the signal flow process of a feedforward neural network. A MLP network has two stages: a forward pass and a backward pass. The forward pass includes presenting a sample input to the network and letting activations flow until they reach the output layer [38, 39].Figure 2
Diagram of a multilayer perceptron network.


## 3.2.3. Long Short-Term Memory (LSTM) Network-Based Deep Learning Method

The concept of deep learning was first proposed by Hinton et al. in 2006. Deep learning is a branch of machine learning. It is essentially a special artificial neural network. Deep learning utilizes a multilayer network structure and applies appropriate nonlinear transformation functions to each hidden node to achieve the purpose of high-level abstraction of the data. The traditional feedforward artificial neural network usually contains only one hidden layer, but there are many hidden layer structures in deep learning. Therefore, deep learning adopts a training mechanism completely different from the traditional artificial neural network in order to solve the problem of the deep neural network in training [40].LSTM is a time-cycle neural network, which can effectively solve the gradient explosion and gradient disappearance problems compared with the traditional cycle neural network. LSTM is composed of a set of cyclic subnets called memory blocks. Each memory block is composed of the input gate, forgetting gate, and output gate. Figure3 shows the LSTM structure [41].Figure 3
LSTM storage unit with door.In general, the LSTM recursive neural network is composed of the following components: input gateit has the corresponding weight matrix wxi, whi, wci, and bi; forgetting gate ft has the corresponding weight matrix whf, wcf, and bf; and output door ot with the corresponding weight matrix wxo, who, wco, and bo. The function of the input gate is to record the new information to the cell state selectively. The function of the forget gate is to selectively forget the status information in the cell; the function of the output gate is to export certain information from the cell. The detailed workflow of LSTM is shown in the following [42]:(12)it=σωxixt+whiht−1+wcict−1+bi,ft=σwxfxt+whfht−1+wcfct−1+bf,ot=σwxoxt+whoht−1+wcoct−1+bo,ct=ftct−1+ittanhwxcxt+whiht−1+whcct−1+bc,where σ is the logistic sigmoid function with the output in [0, 1] and tanh represents the hyperbolic tangent function with the output in [−1, 1].


## 3.2.4. Lazy IBK

Lazy IBK is one of the widely used lazy learning methods. Lazy learning methods defer the decision of how to assign the dependent variable until a new query explanatory vector is inputted. When the query explanatory vector is received, a set of similar data records is retrieved from the available training dataset and is used to assign the dependent variable to the new instance [43]. In order to choose the similar data records, lazy methods employ a distance measure that will give nearby data records higher relevance. Lazy methods choose the k data records that are nearest to the query instance. The dependent variable of the new instance is determined based on the k-nearest instances.Lazy learning algorithms have three basic steps:(i)
Defer: lazy learning algorithms store all training data and defer processing until a new query is given.(ii)
Reply: a local learning approach developed by Bottou and Vapnik in 1992 is a popular method to determine the dependent variables for news queries [44]. In the Bottou and Vapnik learning approach, instances are defined as points in the space, and a similarity function is defined on all pairs of these instances.(iii)
Flush: after solving a query, the answer and any intermediate results are discarded.


## 3.2.5. Regression Tree

A regression tree is one of the widely used decision tree algorithms. A decision tree is a data-mining tool designed to extract useful information from large datasets and use the information to help decision-making processes. A regression tree consists of a set of nodes that can assign the value of the dependent variable to an explanatory vector. Regression tree constructs a tree style decision rule set and divides the training data into the leaf nodes of the decision tree according to the numerical or categorical values of explanatory variables. The regression rules of each leaf node are derived from a mathematical process that minimizes the regression errors of the leaf nodes [45].


## 3.2.6. Decision Table

Similar to the regression tree, decision table also determines the value of the dependent variable with a set of decision rules [46]. However, the decision table arranges decision rules as a table, rather than a tree. A decision table usually consists of a number of parallel decision rules. Similar to the regression tree, the training data will be divided into several groups, each of which will be represented by a decision rule. For a given explanatory vector (input), an appropriate decision rule will be first selected based on the values of its explanatory variables. The dependent variable for this input will be assigned as the average of the dependent variables of all training data vectors in the corresponding group. The dependent variable can also be determined by performing linear regression on the corresponding group of training data. Empirical studies show that the decision table has a similar performance to regression trees.


## 3.3. Converting Wind Speed to Wind Power

An elementary method is used in this paper to convert the predicted wind speed to the predicted wind power output of a wind turbine or wind farm. The predicted wind speed is provided by one of the six machine learning regression methods discussed above. The wind speed is then input into the certified wind turbine power curve and transformed into the wind power.The Vestas V90-3.0 MW wind turbine is selected for the case studies in this paper. Vestas V90-3.0 MW is a pitch regulated upwind wind turbine with active yaw and a three-blade rotor. It has a rotor diameter of 90 m with a generator rated at 3.0 MW. Vestas V90-3.0 MW is widely used in Australia wind power plants and has a proven high efficiency.The typical power curve of Vestas V90-3.0 MW, 60 Hz, 106.7 dB(A) is shown in Figure4. It can be clearly observed that the wind power output pu is proportional to u3 for small wind speed u. Moreover, the power curve is steep for medium wind speeds and flat for large wind speeds. The cut-in speed is 3.5 m/s, and the cut-out speed is 25 m/s [47].Figure 4
Power curve for Vestas V90-3.0 MW, 60 Hz, 106.7.


## 3.4. Performance Evaluation

Before proposing the case study results, several criteria are introduced for performance evaluation. GivenT historical wind power values pt, 1 ≤ t ≤ T, of a time series pt which are converted from T historical wind speed observations and the corresponding forecasted power values pt∗, 1 ≤ t ≤ T; mean absolute percentage error (MAPE) is defined as(13)MAPE=1T∑t=1Tpt−pt∗pt.MAPE is a widely used criterion for time-series forecasting. It will also be employed to evaluate the proposed method in the case studies.Another two criteria are presented to evaluate interval forecasting. GivenT wind power values pt, 1 ≤ t ≤ T, of a time series Py and the corresponding forecasted α-level prediction intervals lt,ut, 1 ≤ t ≤ T, the empirical confidence α^ [48] and the absolute coverage error (ACE) are defined as(14)α^=frequencypt∈lt,utT,ACE=α−α∧,where α^ is the number of observations, which fall into the forecasted prediction interval (PI), divided by the sample size. It should be as close to α as possible.


## 4. Case Studies

### 4.1. The Setting of Case Studies

In the experiments, the wind power forecasting model has been evaluated using the wind speed data from the Devonport Airport Wind Station, Tasmania, Australia. The data were provided by the Australian Bureau of Meteorology. The training and testing data have the following four numerical features: wind speed, wind direction, humidity, and temperature. The training data are from 1st February 2018 to 1st March 2018, while the testing data are from 1st February 2019 to 1st March 2019.To empirically prove the validity of our model, we will first verify that the wind speed data exhibit time-changing distribution effect by performing the Lagrange multiplier test [49, 50]. The results of the LM test with 95% significance level on the data from 1st February 2019 to 1st March 2019 are given in Table 1.Table 1
The results of the Lagrange multiplier test.
DatasetOrderP valueLM statisticsCritical valueFeb 2008 to Mar 2008101913.63.8415Feb 2008 to Mar 2008501964.611.0705Feb 2008 to Mar 20081001969.318.307Feb 2009 to Mar 2009102898.93.8415Feb 2009 to Mar 2009503057.211.0705Feb 2009 to Mar 2009100307718.307As illustrated in Table1, setting the significance level as 0.05, P value of the LM test is zero in all six cases. Moreover, the LM statistics are significantly greater than the critical value of the LM test in all occasions. These two facts strongly indicate that the wind speed data have strong effect of time-changing distribution. In the test, an order of 10 means that the variance σt2 is correlated with its lagged values up to at least σt−102. In other words, the wind speed at 10 time units before time t can still influence the uncertainty of the wind speed at time t.


### 4.2. Results of Wind Speed Forecasting

Wind speed forecasting is the first step of wind power forecasting. Six regression methods are first employed to perform one-hour-ahead wind speed forecasting in this paper. The performances of six algorithms are shown in Table2.Table 2
Prediction errors of different methods.
Regression methodsMAPELinear regression12.81Multilayer perceptron12.32LSTM8.10Lazy IBK10.46Decision table15.10Regression tree11.26As illustrated in Table2, the MAPEs of LSTM and lazy IBK are smaller than other methods. Moreover, the MAPE of LSTM is under 10%, which is sufficiently good considering the very high volatility of wind speed. The results indicate that these two nonlinear machine learning regression methods perform well in wind speed forecasting.The forecasting errors of three methods are graphically shown in Figure5. In Figure 5, the visual inspection suggests that the forecasting errors of the three algorithms have a normal distribution. It is very important to know the type of the error distribution to ensure that the proposed statistical model has a valid assumption. To empirically prove that the wind speed forecasting errors are normally distributed, the forecasting errors of all six methods are checked for normality by performing the Kolmogorov–Smirnov normality test. The test results also show that all the six forecasting methods have normally distributed errors. These results again verify the validity of the assumptions of our model.Figure 5
Distributions of the errors of (a) linear regression, (b) LSTM, and (c) regression tree.
(a)(b)(c)


### 4.3. Results of Wind Power Interval Forecasting

The wind speed forecasts given by the six machine learning regression algorithms are then converted into wind power forecasts as discussed in Section3. Similarly, mean absolute percentage error (MAPE) is used to evaluate the performances of different methods. From Table 3, it is observed that, for wind power forecasting, the MAPE of LSTM is still lower than other five algorithms.Table 3
The MAPE of different methods for wind power forecasting.
Regression methodsMAPE (%)Linear regression37.62Multilayer perceptron42.48LSTM19.24Lazy IBK28.09Decision table35.58Regression tree30.05Based on Tables2 and 3, the LSTM method is selected as the wind speed point forecasting method (the estimator of f•). The procedure discussed in Section 2 is then employed to give the prediction intervals of wind power. We will employ all six regression methods to estimate g• and h• and then compare their performances in wind power interval forecasting.In Table4, for 95% and 99% confidence levels, the ACEs of different regression methods are presented. As seen in Table 4, the ACEs of five nonlinear methods are similar regardless of the confidence level. On the contrary, all the five nonlinear regression algorithms outperform linear regression. This is a clear proof that strong nonlinearity exists in the wind power data.Table 4
Performances of different methods on wind power interval forecasting.
Regression methodsACE for 95% confidenceACE for 99% confidenceLinear regression5.373.34Multilayer perceptron3.190.39LSTM3.020.16Lazy IBK3.160.38Decision table3.160.43Regression tree3.20.39The 95% level and 99% level prediction intervals given by different methods are illustrated in Figures6 and 7. As illustrated, the prediction intervals given by all the five nonlinear machine learning algorithms perfectly contain the true values of wind power. These results clearly prove the effectiveness of the proposed statistical model. Moreover, the results also show that nonlinear machine learning regression methods are suitable candidates in wind power interval forecasting. Compared with other machine learning methods, LSTM performed best in wind power interval forecasting. LSTM is a deep learning neural network algorithm. The improvement of the structure level of the deep learning neural network will make the information abstraction ability of the deep learning model stronger. Therefore, its ability to extract and learn complex information from large amounts of data is also stronger. The accuracy of wind power interval forecasting will be improved accordingly. Multilayer perceptron (MLP) can be categorized as the feedforward neural network. In the traditional feedforward neural network such as MLP, the input layer, the hidden layer, and the output layer in the network are fully connected, but the nodes within each layer are disconnected. This structure results in the inability of the traditional feedforward neural network to deal with the problem of correlation between inputs. Compared with the feedforward neural network, circular neural network introduces directional circulation. At this point, the nodes between hidden layers in the network are no longer disconnected but connected. And the input of the hidden layer includes not only the output of the input layer but also the output of the hidden layer at the last moment. As a conclusion, LSTM can perform better than MLP.Figure 6
95% level prediction intervals forecasted by six machine learning regression methods.
(a)(b)(c)(d)(e)(f)Figure 7
99% level prediction intervals forecasted by six machine learning regression methods.
(a)(b)(c)(d)(e)(f)


## 4.1. The Setting of Case Studies

In the experiments, the wind power forecasting model has been evaluated using the wind speed data from the Devonport Airport Wind Station, Tasmania, Australia. The data were provided by the Australian Bureau of Meteorology. The training and testing data have the following four numerical features: wind speed, wind direction, humidity, and temperature. The training data are from 1st February 2018 to 1st March 2018, while the testing data are from 1st February 2019 to 1st March 2019.To empirically prove the validity of our model, we will first verify that the wind speed data exhibit time-changing distribution effect by performing the Lagrange multiplier test [49, 50]. The results of the LM test with 95% significance level on the data from 1st February 2019 to 1st March 2019 are given in Table 1.Table 1
The results of the Lagrange multiplier test.
DatasetOrderP valueLM statisticsCritical valueFeb 2008 to Mar 2008101913.63.8415Feb 2008 to Mar 2008501964.611.0705Feb 2008 to Mar 20081001969.318.307Feb 2009 to Mar 2009102898.93.8415Feb 2009 to Mar 2009503057.211.0705Feb 2009 to Mar 2009100307718.307As illustrated in Table1, setting the significance level as 0.05, P value of the LM test is zero in all six cases. Moreover, the LM statistics are significantly greater than the critical value of the LM test in all occasions. These two facts strongly indicate that the wind speed data have strong effect of time-changing distribution. In the test, an order of 10 means that the variance σt2 is correlated with its lagged values up to at least σt−102. In other words, the wind speed at 10 time units before time t can still influence the uncertainty of the wind speed at time t.


## 4.2. Results of Wind Speed Forecasting

Wind speed forecasting is the first step of wind power forecasting. Six regression methods are first employed to perform one-hour-ahead wind speed forecasting in this paper. The performances of six algorithms are shown in Table2.Table 2
Prediction errors of different methods.
Regression methodsMAPELinear regression12.81Multilayer perceptron12.32LSTM8.10Lazy IBK10.46Decision table15.10Regression tree11.26As illustrated in Table2, the MAPEs of LSTM and lazy IBK are smaller than other methods. Moreover, the MAPE of LSTM is under 10%, which is sufficiently good considering the very high volatility of wind speed. The results indicate that these two nonlinear machine learning regression methods perform well in wind speed forecasting.The forecasting errors of three methods are graphically shown in Figure5. In Figure 5, the visual inspection suggests that the forecasting errors of the three algorithms have a normal distribution. It is very important to know the type of the error distribution to ensure that the proposed statistical model has a valid assumption. To empirically prove that the wind speed forecasting errors are normally distributed, the forecasting errors of all six methods are checked for normality by performing the Kolmogorov–Smirnov normality test. The test results also show that all the six forecasting methods have normally distributed errors. These results again verify the validity of the assumptions of our model.Figure 5
Distributions of the errors of (a) linear regression, (b) LSTM, and (c) regression tree.
(a)(b)(c)


## 4.3. Results of Wind Power Interval Forecasting

The wind speed forecasts given by the six machine learning regression algorithms are then converted into wind power forecasts as discussed in Section3. Similarly, mean absolute percentage error (MAPE) is used to evaluate the performances of different methods. From Table 3, it is observed that, for wind power forecasting, the MAPE of LSTM is still lower than other five algorithms.Table 3
The MAPE of different methods for wind power forecasting.
Regression methodsMAPE (%)Linear regression37.62Multilayer perceptron42.48LSTM19.24Lazy IBK28.09Decision table35.58Regression tree30.05Based on Tables2 and 3, the LSTM method is selected as the wind speed point forecasting method (the estimator of f•). The procedure discussed in Section 2 is then employed to give the prediction intervals of wind power. We will employ all six regression methods to estimate g• and h• and then compare their performances in wind power interval forecasting.In Table4, for 95% and 99% confidence levels, the ACEs of different regression methods are presented. As seen in Table 4, the ACEs of five nonlinear methods are similar regardless of the confidence level. On the contrary, all the five nonlinear regression algorithms outperform linear regression. This is a clear proof that strong nonlinearity exists in the wind power data.Table 4
Performances of different methods on wind power interval forecasting.
Regression methodsACE for 95% confidenceACE for 99% confidenceLinear regression5.373.34Multilayer perceptron3.190.39LSTM3.020.16Lazy IBK3.160.38Decision table3.160.43Regression tree3.20.39The 95% level and 99% level prediction intervals given by different methods are illustrated in Figures6 and 7. As illustrated, the prediction intervals given by all the five nonlinear machine learning algorithms perfectly contain the true values of wind power. These results clearly prove the effectiveness of the proposed statistical model. Moreover, the results also show that nonlinear machine learning regression methods are suitable candidates in wind power interval forecasting. Compared with other machine learning methods, LSTM performed best in wind power interval forecasting. LSTM is a deep learning neural network algorithm. The improvement of the structure level of the deep learning neural network will make the information abstraction ability of the deep learning model stronger. Therefore, its ability to extract and learn complex information from large amounts of data is also stronger. The accuracy of wind power interval forecasting will be improved accordingly. Multilayer perceptron (MLP) can be categorized as the feedforward neural network. In the traditional feedforward neural network such as MLP, the input layer, the hidden layer, and the output layer in the network are fully connected, but the nodes within each layer are disconnected. This structure results in the inability of the traditional feedforward neural network to deal with the problem of correlation between inputs. Compared with the feedforward neural network, circular neural network introduces directional circulation. At this point, the nodes between hidden layers in the network are no longer disconnected but connected. And the input of the hidden layer includes not only the output of the input layer but also the output of the hidden layer at the last moment. As a conclusion, LSTM can perform better than MLP.Figure 6
95% level prediction intervals forecasted by six machine learning regression methods.
(a)(b)(c)(d)(e)(f)Figure 7
99% level prediction intervals forecasted by six machine learning regression methods.
(a)(b)(c)(d)(e)(f)


## 5. Conclusion

This research work develops a novel comprehensive integrated statistical machine learning strategy for wind power forecasting in Australian wind farm, including exploration of the statistical characteristics of the data by statistical tools and developing the forecasting model by different statistical machine learning methods. Accurate wind power interval forecasting is essential for efficient planning and operation of power systems. Wind energy is characterised by its nonlinearity and intermittency, which pose significant challenges for wind power forecasting. Traditional linear time-series models cannot appropriately handle these challenges and therefore cannot achieve satisfactory performances. In this paper, we propose a machine learning-based statistical approach, which can handle nonlinear time series with time-changing distributions, thus is suitable for wind power interval forecasting.Compared with other relevant references, this research work shows that classical regression techniques are not suitable for complicated applications such as wind power interval forecasting. It is inappropriate simply to use linear assumptions for these problems. In addition, other research works only using complicated machine learning approaches failed to balance the important information of the historical data. Experimental results show that LSTM is the most suited candidate for wind power forecasting. Moreover, the effectiveness and accuracy of the proposed model in wind power interval forecasting are also proven with the case studies.



---

*Source: 9250937-2020-09-16.xml*