# Support Vector Regression Method for Wind Speed Prediction Incorporating Probability Prior Knowledge

Authors: Jiqiang Chen; Xiaoping Xue; Minghu Ha; Daren Yu; Litao Ma
Journal: Mathematical Problems in Engineering
 (2014)

---

## Abstract

Prior knowledge, such as wind speed probability distribution based on historical data and the wind speed fluctuation between the maximal value and the minimal value in a certain period of time, provides much more information about the wind speed, so it is necessary to incorporate it into the wind speed prediction. First, a method of estimating wind speed probability distribution based on historical data is proposed based on Bernoulli’s law of large numbers. Second, in order to describe the wind speed fluctuation between the maximal value and the minimal value in a certain period of time, the probability distribution estimated by the proposed method is incorporated into the training data and the testing data. Third, a support vector regression model for wind speed prediction is proposed based on standard support vector regression. At last, experiments predicting the wind speed in a certain wind farm show that the proposed method is feasible and effective and the model’s running time and prediction errors can meet the needs of wind speed prediction.


---

## Body

## 1. Introduction

Wind power is a clean, renewable energy that will play an increasingly important role in the future electricity supply [1]. Unfortunately, due to the stochastic and nonstationary nature of wind, the wind power is variable and uncontrollable. It is difficult to maintain the balance between the supply and the demand of electricity, which is required by the electricity system [2]. Wind speed prediction is a key point in the management of wind farms because it is directly related to the power produced by each of the farm’s turbines, so it is usually the base of wind power forecasts, and it is necessary to increase the accuracy of the wind speed prediction for the effective use of wind energy.At present, there are mainly two kinds of wind speed prediction methods. One is based on the physical model, and the other is based on historical data. The prediction methods based on physical model often use the numerical weather prediction (NWP) data for wind speed prediction [3, 4]. Wind speed prediction methods based on NWP do not focus on the speed of a farm’s turbines but on the speed of a region. Thus, it needs to solve the problem of how the wind speed of a region is mapped to the wind speed of a certain wind generator. Wind speed prediction methods based on historical data predict the wind speed by using correlations among the initial data. In 2008, Louka et al. [5] improved wind speed forecasts for wind power prediction using Kalman filtering. In 2012, Cao et al. [6] presented a comparative analysis of the wind speed prediction accuracy of univariate and multivariate ARIMA models with their recurrent neural network counterparts. In 2013, Woods et al. [7] developed a method to produce synthetic time series of wind power at several locations based on a measured time series of wind speed from a reference site, and so on.In the 1990s, Vapnik et al. [8, 9] proposed support vector machines (SVMs), including support vector classifications (SVCs) and support vector regressions (SVRs). SVMs focus on the statistical learning problems for small size samples by solving a convex quadratic optimization problem and can solve the local minimization problem which cannot be avoided by the neural network algorithm. SVMs use a kernel function to map the data in original space to a high dimensional feature space and then solve the nonlinear decision problem in high dimensional space. Thus, SVMs can successfully solve the problem of dimension disaster and have good generalization ability. However, the standard SVMs focus on historical data and cannot incorporate prior knowledge into learning process, which may causes the generalization ability of the standard SVMs to decrease. Therefore, in 2009, Guan et al. [10] proposed a modified method that incorporated prior knowledge into cancer classification based on gene expression data to improve accuracy. In 2011, Zhang et al. [11] proposed a fully Bayesian methodology for generalized kernel mixed models, which are extensions of generalized linear mixed models in the feature space induced by a reproducing kernel. In 2012, Liu and Xue [12] focused on designing a new class of kernels to incorporate the prior information into the training process of support vector regressions. Currently, SVMs have received extensive attention and are attracting more and more scholars to study from different views [13–22].In 2011, Zhou et al. [23] presented a systematic study on fine tuning of LS-SVM model parameters for one-step ahead wind speed prediction, and Ortiz-García et al. [24] proposed an improvement to an existing wind speed prediction system using banks of regression support vector machines for a final regression step in the prediction system.However, for the problem of wind speed prediction in practice, there is much prior knowledge. For example, the wind speed has a certain probability distribution in a season or in a day, and the probability distribution can be estimated with historical wind speed data. As the probability distribution can provide much more information about the wind speed, it is necessary to incorporate it into the wind speed prediction. Also, in a wind farm, the output wind speedv at a fixed time t is the mean value v- of many measured values vk(k=0,1,…,l) during a certain period of time Δt. Assume that vmax⁡=max⁡k{vk} and vmin⁡=min⁡k{vk}, then the larger the vmax⁡-vmin⁡ is, the more the fluctuation of wind speed during the period of time Δt is. Conversely, the smaller the vmax⁡-vmin⁡ is, the less the fluctuation of wind speed during the period of time Δt is. Nevertheless, the mean value v- does not provide this prior knowledge at all. Therefore, in order to decrease the wind speed prediction errors, it is necessary to find a way to incorporate this prior knowledge in the wind speed prediction. However, the present methods for wind speed prediction often used the historical wind speed data directly to predict the wind speed, instead of dredging information from the data, and the prediction errors are difficult to decrease. Therefore, in order to decrease the prediction errors of the wind speed at a fixed time, the probability distribution of historical wind speed data is estimated and incorporated into the training data and testing data to provide the information about the wind speed fluctuation. Then a support vector regression model for wind speed prediction is proposed combined with the standard SVR.This paper is structured as follows. Section2 is the preliminaries. Section 3 is the method of estimating the probability distribution of the historical wind speed data. Section 4 is to incorporate the prior knowledge about the wind speed fluctuation into the training data and testing data and then establish the ε-support vector regression method for wind speed prediction incorporating probability prior knowledge (PPK-ε-SVR). Section 5 includes two experiments with the historical wind speed data coming from a wind farm in Gansu province, and Section 6 draws the conclusions.


## 2. Preliminaries

In this section, we briefly review some relevant knowledge of probability theory and the standard support vector regression often used in applications.Let(Ω,ℱ) be a measurable space, P a function defined on ℱ, and 0≤P(A)≤1 for any A∈ℱ. We call P(A) the probability of event A occurring, and P(A) indicates the possibility of event A occurring. Assume that X(ω) is a random variable on Ω, F(x)=P{X(ω)<x}(x∈R) is the distribution function, E(X) is the expectation, and D(X) is the variance of random variable X, respectively.Definition 1 (see [25]).
LetF(x) be the distribution function of random variable X(ω). If there exists a nonnegative and integrable function f(x) satisfying
(1)F(x)=P{X<x}=∫-∞xf(t)dt,∀x∈R,
then we call f(x) the probability density of continuous random variable X(ω).Definition 2 (see [25]).
Let{Xn} be a sequence of random variables. {Xn} is said to be independent if for any finite random variables X1,X2,…,Xk(k≤n) are independent.Definition 3 (see [25]).
Let the sequence of random variables{Xn} be independent. {Xn} is said to be independent and identically distributed if they have the same distribution function F(x).Definition 4 (see [25]).
Let{Xn} be a sequence of random variables and let a be a constant. If, for any ε>0, we have
(2)lim⁡n→+∞P{|Xn-a|≥ε}=0
or
(3)lim⁡n→+∞P{|Xn-a|<ε}=1,
then the sequence {Xn} is said to converge in probability to a. Denoted by Xn→Pa.Theorem 5 (Bernoulli’s law of large numbers [25]).
LetnA be the number of event A occurring in n independent duplicate experiments, and let p be the probability of event A occurring. Then, for any ε>0, we have
(4)lim⁡n→+∞P{|nAn-p|<ε}=1;
that is,
(5)nAn→Pp.Remark 6.
Bernoulli’s law of large numbers proves the frequency stability in theory. In other words, the frequencynA/n converges in probability to p. Then for a sufficiently large n, the frequency nA/n almost equals the probability p. Hence, if the number of experiments is very large, the frequency nA/n can be treated as the probability p in practice.


### 2.1. Regression Problem

Suppose that(6)T={(x1,y1),(x2,y2),…,(xl,yl)}
is a given training set, where xi∈Rn,yi∈R,i=1,2,…,l. The regression problem is to find a real valued function y=g(x) on Rn according to data (6) to predict the output y for any given input x.Remark 7.
For the above regression problem, if the real valued functiony=g(x) is linear, then we call it a linear repression problem. If the real valued function y=g(x) is nonlinear, then we call it a nonlinear repression problem.


### 2.2.ε-Support Vector Regression

Support vector regression (SVR) differs from conventional regression in that it maps the training data (6) into a high dimensional reproducing kernel Hilbert space. In ε-SVR, the goal is to find a function y=f(x) which has at most ε deviation from the actually obtained targets yi for all the training data and at the same time as flat as possible. And ε-SVR uses an ε-insensitive loss function
(7)|ξε|={0,if|ξ|<ε|ξ|-ε,otherwise
to solve the optimal problem
(8)min⁡ω,b,ξ,ξ*12∥ω∥2+C∑i=1l(ξi+ξi*)s.t.yi-f(xi)≤ε+ξi,f(xi)-yi≤ε+ξi*,ξi,ξi*≥0,i=1,…,l,
where f(xi)=ωTϕ(xi)+b and data (6) are mapped to a higher dimensional characteristic space by the function ϕ(x) and the constant C>0 determines the tradeoff between the flatness of f(x) and the amount up to which deviations larger than ε are tolerated. Similar to support vector classification, ω may be a huge vector variable, and then we solve the dual problem
(9)min⁡α(*)∈R2l12∑i,j=1l(αi*-αi)(αj*-αj)K(xi,xj)+ε∑i=1l(αi*+αi)-∑i=1lyi(αi*-αi)s.t.∑i=1l(αi-αi*)=0,0≤αi(*)≤C,i=1,…,l,
where K(xi,xj)=ϕ(xi)T·ϕ(xj) is a kernel function.As a result, SVR has a sparse representation of solutions and hence is relatively fast in training and testing. SVR is the most common application form of SVM and has been popular for regression and function estimation problems in the past decades.


## 2.1. Regression Problem

Suppose that(6)T={(x1,y1),(x2,y2),…,(xl,yl)}
is a given training set, where xi∈Rn,yi∈R,i=1,2,…,l. The regression problem is to find a real valued function y=g(x) on Rn according to data (6) to predict the output y for any given input x.Remark 7.
For the above regression problem, if the real valued functiony=g(x) is linear, then we call it a linear repression problem. If the real valued function y=g(x) is nonlinear, then we call it a nonlinear repression problem.


## 2.2.ε-Support Vector Regression

Support vector regression (SVR) differs from conventional regression in that it maps the training data (6) into a high dimensional reproducing kernel Hilbert space. In ε-SVR, the goal is to find a function y=f(x) which has at most ε deviation from the actually obtained targets yi for all the training data and at the same time as flat as possible. And ε-SVR uses an ε-insensitive loss function
(7)|ξε|={0,if|ξ|<ε|ξ|-ε,otherwise
to solve the optimal problem
(8)min⁡ω,b,ξ,ξ*12∥ω∥2+C∑i=1l(ξi+ξi*)s.t.yi-f(xi)≤ε+ξi,f(xi)-yi≤ε+ξi*,ξi,ξi*≥0,i=1,…,l,
where f(xi)=ωTϕ(xi)+b and data (6) are mapped to a higher dimensional characteristic space by the function ϕ(x) and the constant C>0 determines the tradeoff between the flatness of f(x) and the amount up to which deviations larger than ε are tolerated. Similar to support vector classification, ω may be a huge vector variable, and then we solve the dual problem
(9)min⁡α(*)∈R2l12∑i,j=1l(αi*-αi)(αj*-αj)K(xi,xj)+ε∑i=1l(αi*+αi)-∑i=1lyi(αi*-αi)s.t.∑i=1l(αi-αi*)=0,0≤αi(*)≤C,i=1,…,l,
where K(xi,xj)=ϕ(xi)T·ϕ(xj) is a kernel function.As a result, SVR has a sparse representation of solutions and hence is relatively fast in training and testing. SVR is the most common application form of SVM and has been popular for regression and function estimation problems in the past decades.


## 3. Method of Estimating the Probability Distribution of Historical Wind Speed Data

In practical problems, there are a lot of historical wind speed data. In order to dredge more information from them, in this part, we will give a method to estimate the probability distribution of historical wind speed data.


### 3.1. Method

Assume thatf(x) in Figure 1 is the probability density of independent and identically distributed random variables Xk(k=1,2,…,l) and the number and the frequency of Xk falling into interval [xi-1,xi)(i=1,2,…,n,n≤l) are ui and ui/l, respectively. By Definition 1, we can obtain that the probability pi (namely, the area of trapezoid with curved edge ABCD) of Xk falling into interval [xi-1,xi) is
(10)pi=∫xi-1xif(x)dx.
By Theorem 5, we have
(11)pi≈uil,i=1,2,…,n.
With the infinitesimal method, the area of trapezoid with curved edge ABCD can be approximately substituted by the area of trapezoid ABCD, namely,
(12)∫xi-1xif(x)dx≈12[f(xi-1)+f(xi)](xi-xi-1).Figure 1
The probability density function.By formulas (10), (11), and (12) we have
(13)f(xi-1)+f(xi)≈2pixi-xi-1≈2uil(xi-xi-1).
Similarly, for f(xi-1), f(xi) and f(xi+1), we have
(14)f(xi)+f(xi+1)≈2ui+1l(xi+1-xi),f(xi-1)+f(xi+1)≈2(ui+1+ui)l(xi+1-xi-1).From formulas (13) and (14) we can obtain that
(15)f(xi)≈1l(ui+1xi+1-xi-ui+1+uixi+1-xi-1+uixi-xi-1),i=1,2,…,n.


### 3.2. Algorithm

Suppose thatv(t) is the wind speed at a fixed point A. Then v(t) can be seen as a random variable defined on time set T. The l samples data v(tk)(k=1,2,…,l) can be seen as a sequence of sample data coming from l independent random variables Xk(k=1,2,…,l). Supposing that the probability density function of wind speed v(t) is f(x), then the steps of estimating the probability density f(x) are as follows.Step 1.
Denotex0=min⁡k{v(tk)},xn=max⁡k{v(tk)}. Insert n-1 points xj(j=1,2,…,n-1) between x0 and xn equidistantly, and interval [x0,xn] is divided into n intervals [xi-1,xi)(i=1,2,…,n) with the same length.Step 2.
Count the numberui(i=1,2,…,n) of samples v(tk)(k=1,2,…,l) falling into interval [xi-1,xi), and we have ∑i=1nui=l. By formula (11), the probability pi of samples v(tk) falling into interval [xi-1,xi) is pi≈ui/n(i=1,2,…,n).Step 3.
Calculatef(xi)(i=1,2,…,n) with formula (15).Step 4.
Connecting the points(xi,f(xi)) in turn with smooth curves, the probability density f(x) of wind speed sample data v(tk)(k=1,2,…,n) can be obtained.


### 3.3. Experiments

In order to verify the effectiveness of the above method, in this subsection, sample data coming from normal distribution and exponential distribution are used to make experiments, respectively.Experiment 1.
Suppose thatX is a standard normal random variable; then its probability density is
(16)f(x)=12πexp⁡[-x22],-∞<x<+∞,
and the graph of f(x) is shown in Figure (10).
Suppose that the sample data(17)x1,x2,…,x10000
are independent and identically distributed coming from the probability density f(x) and xi∈[-3,3](i=1,2,…,10000). Insert 99 points in interval [-3,3] equidistantly, and interval [-3,3] is divided into n=100 small intervals with the same length. With the above proposed method, we can obtain function g(x) shown in Figure 2 which is the estimation of the standard normal density f(x).Figure 2
Graphs of standard normal densities.Experiment 2.
Supposing thatX is a random variable following an exponential distribution, then the probability density is
(18)f(x)={12exp⁡[-x2],x>0,0,x≤0,
and the graph of f(x) is shown in Figure 3. With the similar steps in the above experiment, suppose that the sample data x1,x2,…,x10000 are independent and identically distributed coming from the probability densityf(x) and xi∈[0,10](i=1,2,…,10000). Insert 99 points in interval [0,10] equidistantly, and interval [0,10] is divided into n=100 small intervals with the same length. With the proposed method above, we can obtain function g(x) which is the estimation of exponential density f(x) (see Figure 3).Figure 3
Graphs of exponential densities.


### 3.4. Results Analysis

From Figure2 we can see that the estimation density g(x) obtained with the sample data is not as smooth as the standard normal density f(x), but the rough shapes of the two functions are almost the same. And from Figure 3 the similar conclusions can be drawn. That is to say, the method of estimating the probability density based on Bernoulli’s law of large numbers and infinitesimal method is effective, and it lays a foundation for establishing the PPK-ε-SVR.


## 3.1. Method

Assume thatf(x) in Figure 1 is the probability density of independent and identically distributed random variables Xk(k=1,2,…,l) and the number and the frequency of Xk falling into interval [xi-1,xi)(i=1,2,…,n,n≤l) are ui and ui/l, respectively. By Definition 1, we can obtain that the probability pi (namely, the area of trapezoid with curved edge ABCD) of Xk falling into interval [xi-1,xi) is
(10)pi=∫xi-1xif(x)dx.
By Theorem 5, we have
(11)pi≈uil,i=1,2,…,n.
With the infinitesimal method, the area of trapezoid with curved edge ABCD can be approximately substituted by the area of trapezoid ABCD, namely,
(12)∫xi-1xif(x)dx≈12[f(xi-1)+f(xi)](xi-xi-1).Figure 1
The probability density function.By formulas (10), (11), and (12) we have
(13)f(xi-1)+f(xi)≈2pixi-xi-1≈2uil(xi-xi-1).
Similarly, for f(xi-1), f(xi) and f(xi+1), we have
(14)f(xi)+f(xi+1)≈2ui+1l(xi+1-xi),f(xi-1)+f(xi+1)≈2(ui+1+ui)l(xi+1-xi-1).From formulas (13) and (14) we can obtain that
(15)f(xi)≈1l(ui+1xi+1-xi-ui+1+uixi+1-xi-1+uixi-xi-1),i=1,2,…,n.


## 3.2. Algorithm

Suppose thatv(t) is the wind speed at a fixed point A. Then v(t) can be seen as a random variable defined on time set T. The l samples data v(tk)(k=1,2,…,l) can be seen as a sequence of sample data coming from l independent random variables Xk(k=1,2,…,l). Supposing that the probability density function of wind speed v(t) is f(x), then the steps of estimating the probability density f(x) are as follows.Step 1.
Denotex0=min⁡k{v(tk)},xn=max⁡k{v(tk)}. Insert n-1 points xj(j=1,2,…,n-1) between x0 and xn equidistantly, and interval [x0,xn] is divided into n intervals [xi-1,xi)(i=1,2,…,n) with the same length.Step 2.
Count the numberui(i=1,2,…,n) of samples v(tk)(k=1,2,…,l) falling into interval [xi-1,xi), and we have ∑i=1nui=l. By formula (11), the probability pi of samples v(tk) falling into interval [xi-1,xi) is pi≈ui/n(i=1,2,…,n).Step 3.
Calculatef(xi)(i=1,2,…,n) with formula (15).Step 4.
Connecting the points(xi,f(xi)) in turn with smooth curves, the probability density f(x) of wind speed sample data v(tk)(k=1,2,…,n) can be obtained.


## 3.3. Experiments

In order to verify the effectiveness of the above method, in this subsection, sample data coming from normal distribution and exponential distribution are used to make experiments, respectively.Experiment 1.
Suppose thatX is a standard normal random variable; then its probability density is
(16)f(x)=12πexp⁡[-x22],-∞<x<+∞,
and the graph of f(x) is shown in Figure (10).
Suppose that the sample data(17)x1,x2,…,x10000
are independent and identically distributed coming from the probability density f(x) and xi∈[-3,3](i=1,2,…,10000). Insert 99 points in interval [-3,3] equidistantly, and interval [-3,3] is divided into n=100 small intervals with the same length. With the above proposed method, we can obtain function g(x) shown in Figure 2 which is the estimation of the standard normal density f(x).Figure 2
Graphs of standard normal densities.Experiment 2.
Supposing thatX is a random variable following an exponential distribution, then the probability density is
(18)f(x)={12exp⁡[-x2],x>0,0,x≤0,
and the graph of f(x) is shown in Figure 3. With the similar steps in the above experiment, suppose that the sample data x1,x2,…,x10000 are independent and identically distributed coming from the probability densityf(x) and xi∈[0,10](i=1,2,…,10000). Insert 99 points in interval [0,10] equidistantly, and interval [0,10] is divided into n=100 small intervals with the same length. With the proposed method above, we can obtain function g(x) which is the estimation of exponential density f(x) (see Figure 3).Figure 3
Graphs of exponential densities.


## 3.4. Results Analysis

From Figure2 we can see that the estimation density g(x) obtained with the sample data is not as smooth as the standard normal density f(x), but the rough shapes of the two functions are almost the same. And from Figure 3 the similar conclusions can be drawn. That is to say, the method of estimating the probability density based on Bernoulli’s law of large numbers and infinitesimal method is effective, and it lays a foundation for establishing the PPK-ε-SVR.


## 4. Support Vector Regression Method Incorporating Probability Prior Knowledge

In this section, we aim to predict the wind speedv(t) at a fixed point A.


### 4.1. Incorporating the Prior Knowledge about the Wind Speed Fluctuation into the Training Data and Testing Data

Supposed thatf(x) is the probability density estimated with the above method, t0 is the initial time, and v0 is the wind speed at t0, denoted by (t0,v0). In practice, the wind speed at a fixed point A is often measured many times t0k(k=1,2,…,l) for every certain period of time Δt (where t0<t01<⋯<t0l≤t0+Δt), and the mean value v-=(1/l)∑k=1lv0k is output as the predicted wind speed at the fixed point A. In other words, the wind speed v1 at t1=t0+Δt is the mean value v- of the measured values v0k from t0 to t1, denoted by (t1,v1), where v1=v-. Of course, the mean value v- can represent the wind speed v1 at t1 in a sense, but in some cases the mean value v- is quite different from v1. For example, if the measured wind speed v0k(k=1,2,…,l) is the same value during a certain period of time Δt, then “the mean value v- is the wind speed v1” holds with a high probability. Conversely, if the measured wind speed v0k fluctuates wildly during a certain period of time Δt, then “the mean value v- is the wind speed v1” holds with a very low probability.Hence, in order to incorporate this prior knowledge into the wind speed prediction, the training datum(t1,v1) is converted into (t1,1-P1,v1), where(19)P1=P{vmin⁡≤v1≤vmax⁡}=∫vmin⁡vmax⁡f(x)dx,(20)vmin⁡=min⁡k{v0k},vmax⁡=max⁡k{v0k}.In fact, from formulas (19) and (20) we can see that the larger the vmax⁡-vmin⁡ is, the larger the P1 is and, furthermore, the smaller the 1-P1 is. That is to say, the possibility of “wind speed at t1 is v1” is very small. On the other hand, the large vmax⁡-vmin⁡ illustrates that the wind speed from t0 to t1 fluctuates wildly and “the mean value v- is the wind speed v1” holds with a low probability (namely, the possibility of “wind speed at t1 is v1” is very small), which is in accord with the information provided by 1-P1. Thus, the probability 1-P1 provides the fluctuation about the wind speed during a certain period of time Δt. Therefore, datum (t1,1-P1,v1) contains the prior knowledge provided by the historical data.Letti(i=1,2,…,l) be the l times at a fixed point A, and let
(21)T={(t1,1-P1,v1),(t2,1-P2,v2),…,(tl,1-Pl,vl)}
be a training set. Denote xi=(ti,1-Pi); then training set (21) can be rewritten by
(22)T={(x1,v1),(x2,v2),…,(xl,vl)},
where xi∈R×[0,1],vi∈[0,+∞),i=1,2,…,l. The problem of wind speed prediction is to find a real valued function v=g(x) on R×[0,1] according to training set (22) to predict the wind speed v for any given input x.


### 4.2.ε-Support Vector Regression Method for Wind Speed Prediction Incorporating Probability Prior Knowledge

For the above problem of wind speed prediction,ε-support vector regression method incorporating probability prior knowledge (PPK-ε-SVR) can be constructed as follows.Step 1.
Obtain the training set(23)T={(x1,v1),(x2,v2),…,(xl,vl)},
where xi∈R×[0,1],vi∈[0,+∞),i=1,2,…,l.Step 2.
Select a proper kernel functionK(x,x′), ε>0 and punishment parameter C>0.Step 3.
Constructing and solving the convex quadratic programming problems:(24)min⁡α(*)∈R2l12∑i,j=1l(αi*-αi)(αj*-αj)K(xi,xj)+ε∑i=1l(αi*+αi)-∑i=1lvi(αi*-αi),s.t.∑i=1l(αi-αi*)=0,0≤αi(*)≤C,i=1,…,l,
we can obtain the optimal solution
(25)α-(*)=(α-1,α-1*,…,α-l,α-l*)T.Step 4.
Choose the componentα-j or α-k* of α-(*) in interval (0,C). If α-j is chosen, then
(26)b-=vi-∑i=1l(α-i*-α-i)K(xi,xj)+ε.
If  α-k* is chosen, then
(27)b-=vk-∑i=1l(α-i*-α-i)K(xi,xk)-ε.Step 5.
Construct the decision function withb-(28)v=g(x)=∑i=1l(α-i*-α-i)K(xi,x)+b-.Remark 8.
Solving regression problems with support vector regression, the kernel function can be selected according to prior knowledge, such as the characteristics of the problem, or the training set. More details about selecting kernel function with prior knowledge will be investigated in another paper.


## 4.1. Incorporating the Prior Knowledge about the Wind Speed Fluctuation into the Training Data and Testing Data

Supposed thatf(x) is the probability density estimated with the above method, t0 is the initial time, and v0 is the wind speed at t0, denoted by (t0,v0). In practice, the wind speed at a fixed point A is often measured many times t0k(k=1,2,…,l) for every certain period of time Δt (where t0<t01<⋯<t0l≤t0+Δt), and the mean value v-=(1/l)∑k=1lv0k is output as the predicted wind speed at the fixed point A. In other words, the wind speed v1 at t1=t0+Δt is the mean value v- of the measured values v0k from t0 to t1, denoted by (t1,v1), where v1=v-. Of course, the mean value v- can represent the wind speed v1 at t1 in a sense, but in some cases the mean value v- is quite different from v1. For example, if the measured wind speed v0k(k=1,2,…,l) is the same value during a certain period of time Δt, then “the mean value v- is the wind speed v1” holds with a high probability. Conversely, if the measured wind speed v0k fluctuates wildly during a certain period of time Δt, then “the mean value v- is the wind speed v1” holds with a very low probability.Hence, in order to incorporate this prior knowledge into the wind speed prediction, the training datum(t1,v1) is converted into (t1,1-P1,v1), where(19)P1=P{vmin⁡≤v1≤vmax⁡}=∫vmin⁡vmax⁡f(x)dx,(20)vmin⁡=min⁡k{v0k},vmax⁡=max⁡k{v0k}.In fact, from formulas (19) and (20) we can see that the larger the vmax⁡-vmin⁡ is, the larger the P1 is and, furthermore, the smaller the 1-P1 is. That is to say, the possibility of “wind speed at t1 is v1” is very small. On the other hand, the large vmax⁡-vmin⁡ illustrates that the wind speed from t0 to t1 fluctuates wildly and “the mean value v- is the wind speed v1” holds with a low probability (namely, the possibility of “wind speed at t1 is v1” is very small), which is in accord with the information provided by 1-P1. Thus, the probability 1-P1 provides the fluctuation about the wind speed during a certain period of time Δt. Therefore, datum (t1,1-P1,v1) contains the prior knowledge provided by the historical data.Letti(i=1,2,…,l) be the l times at a fixed point A, and let
(21)T={(t1,1-P1,v1),(t2,1-P2,v2),…,(tl,1-Pl,vl)}
be a training set. Denote xi=(ti,1-Pi); then training set (21) can be rewritten by
(22)T={(x1,v1),(x2,v2),…,(xl,vl)},
where xi∈R×[0,1],vi∈[0,+∞),i=1,2,…,l. The problem of wind speed prediction is to find a real valued function v=g(x) on R×[0,1] according to training set (22) to predict the wind speed v for any given input x.


## 4.2.ε-Support Vector Regression Method for Wind Speed Prediction Incorporating Probability Prior Knowledge

For the above problem of wind speed prediction,ε-support vector regression method incorporating probability prior knowledge (PPK-ε-SVR) can be constructed as follows.Step 1.
Obtain the training set(23)T={(x1,v1),(x2,v2),…,(xl,vl)},
where xi∈R×[0,1],vi∈[0,+∞),i=1,2,…,l.Step 2.
Select a proper kernel functionK(x,x′), ε>0 and punishment parameter C>0.Step 3.
Constructing and solving the convex quadratic programming problems:(24)min⁡α(*)∈R2l12∑i,j=1l(αi*-αi)(αj*-αj)K(xi,xj)+ε∑i=1l(αi*+αi)-∑i=1lvi(αi*-αi),s.t.∑i=1l(αi-αi*)=0,0≤αi(*)≤C,i=1,…,l,
we can obtain the optimal solution
(25)α-(*)=(α-1,α-1*,…,α-l,α-l*)T.Step 4.
Choose the componentα-j or α-k* of α-(*) in interval (0,C). If α-j is chosen, then
(26)b-=vi-∑i=1l(α-i*-α-i)K(xi,xj)+ε.
If  α-k* is chosen, then
(27)b-=vk-∑i=1l(α-i*-α-i)K(xi,xk)-ε.Step 5.
Construct the decision function withb-(28)v=g(x)=∑i=1l(α-i*-α-i)K(xi,x)+b-.Remark 8.
Solving regression problems with support vector regression, the kernel function can be selected according to prior knowledge, such as the characteristics of the problem, or the training set. More details about selecting kernel function with prior knowledge will be investigated in another paper.


## 5. Experiments

In this part, we take a wind farm in Gansu province as an example. For a fixed pointA, in order to predict the wind speed v(t) (m/s) at t (Hours), we recorded the wind speed from November 2006 to April 2008 and found that the wind speed had a periodicity with the change of seasons, months, or days. Therefore, the probability distribution P of wind speed in the period of the previous year (month or day) can be incorporated into the wind speed prediction of the corresponding period in this year (month or day). In this wind farm, the wind speed v is output every ten minutes, so there are 144 sets of data in a day and more than four thousand sets of data in a month. As SVR focuses on the statistical learning problems for small size samples and the wind speed had a periodicity with the change of days, the experiment is aimed at the short-term wind speed prediction, and we choose 144 sets of data to carry out the experiment. Here, without loss of generality, we take the wind speed prediction on 1 April 2008 as an example.


### 5.1. Estimation of the Probability Density of Historical Wind Speed Data

Supposed(ti,vi)  (i=1,2,…,144) indicates that the wind speed at time ti is vi. Wind speed was measured ten times v(tik)(k=1,2,…,10) during every ten minutes, and the mean value v-i=(1/10)∑k=110v(tik) is output as the wind speed vi+1 at time ti+1/6. By the proposed method in Section 3, the probability density f(x) of the wind speed data on 1 April 2008 was estimated and the graph of f(x) is shown in Figure 4.Figure 4
Probability density of wind speed data on 1 April 2008.


### 5.2. Incorporating the Prior Knowledge about the Wind Speed Fluctuation into the Training Data and Testing Data

Denote thatvimax⁡=max⁡k{v(tik)} and vimin⁡=min⁡k{v(tik)}. By the estimated probability density f(x) of the wind speed data on 1 April 2008 and formula (19), the probability Pi=P{vimin⁡≤vi≤vimax⁡} can be calculated. The wind speed sample data on 1 April 2008 are
(29)(t1,v1),(t2,v2),…,(t144,v144),
and data (29) are converted into
(30)(x1,v1),(x2,v2),…,(x144,v144)
by the method in Section 4.1, where xi=(ti,1-Pi)  (i=1,…,144). And the data (x1,v1),(x2,v2),…,(x143,v143) in data (30) is the training data to train a model to predict the wind speed v144 for the given x144.Similarly, if we want to predict the wind speedv1′ for the given x1′ on 2 April 2008, we can estimate the probability density of wind speed (ti,vi)(i=2,3,…,144) on 1 April 2008 and use the data (xi,vi)  (i=2,3,…,144) as the training data.


### 5.3.ε-Support Vector Regression Method for Wind Speed Prediction Incorporating Probability Prior Knowledge

In the experiment, a grid search method based on 5-fold cross-validation is chosen to determine model parameters, whereσ∈{2-10,2-9,…,21}, C∈{2-6,2-5,…,26}, and the kernel function is the radial basis function (RBF). In order to predict the wind speed v144 for the given x144 with training data (x1,v1),(x2,v2),…,(x143,v143) in data (30), we make experiment with PPK-ε-SVR and standard ε-SVR, respectively; the results are shown in Tables 1 and 2, respectively. The wind speeds of training data, normalized wind speeds of testing data, and wind speeds of testing data with PPK-ε-SVR are shown in Figures 5(a), 5(b), and 5(c), respectively. The wind speeds of training data, normalized wind speeds of testing data, and wind speeds of testing data with standard ε-SVR are shown in Figures 6(a), 6(b), and 6(c), respectively.Table 1
Experiment results with PPK-ε-SVR.
Data
Number in the data set
ε
σ
C
Mean squared errors
Training data
143
0.01
0.25
8.00
0.0217
Testing data
1
0.0035Table 2
Experiment results with standardε-SVR.
Data
Number in the data set
ε
σ
C
Mean squared errors
Training data
143
0.01
0.25
4.00
0.0307
Testing data
1
0.0747Initial wind speeds and predicted wind speeds with PPK-ε-SVR.
(a)
Wind speeds of training data
(b)
Normalized wind speeds of testing data
(c)
Wind speeds of testing dataInitial wind speeds and predicted wind speeds with standardε-SVR.
(a)
Wind speeds of training data
(b)
Normalized wind speeds of testing data
(c)
Wind speeds of testing dataSimilar to the steps of predicting wind speedv144 for the given x144, we make experiment 50 times to predict the wind speed vj for the given xj(j=1,2,…,50) (namely, the former 50 wind speeds monitored on 2 April 2008), the average mean squared errors are shown in Table 3, and the numbers after ± are the standard deviations.Table 3
Average mean squared errors.
Methods
Experiment times
Average mean squared errors
PPK-ε-SVR
50
0.0304
±
0.0367
Standardε-SVR
50
0.0402
±
0.0408


### 5.4. Result Analysis

From Tables1 and 2, we can see that the mean squared errors of training data and testing data with PPK-ε-SVR are all smaller than the corresponding ones with standard ε-SVR. Comparing Figure 5(a) with Figure 6(a) and Figure 5(c) with Figure 6(c), we can find that the predicted wind speeds of training data and testing data with PPK-ε-SVR are more close to the initial wind speeds than those with standard ε-SVR. This illustrates that the prediction error of PPK-ε-SVR is smaller than that of standard ε-SVR in predicting the wind speed v144 for the given x144. From Figure 5(b) we can obtain that the difference between normalized initial wind speed and normalized predicted wind speed is 0.766-0.707=0.059 which is equal to the square root of mean squared error 0.0035 in Table 1, which shows the effectiveness of the PPK-ε-SVR.From Table3, we can see that the average mean squared error (namely, the average prediction error) of PPK-ε-SVR is smaller than that of standard ε-SVR, which illustrates that PPK-ε-SVR method is more accurate than standard ε-SVR. What is more, the standard deviation of PPK-ε-SVR is also smaller than that of standard ε-SVR, which illustrates that the PPK-ε-SVR is more stable than the standard ε-SVR. And also the running time of PPK-ε-SVR is less than one minute, which shows that the model’s running time can meet the needs of wind speed prediction in application.


## 5.1. Estimation of the Probability Density of Historical Wind Speed Data

Supposed(ti,vi)  (i=1,2,…,144) indicates that the wind speed at time ti is vi. Wind speed was measured ten times v(tik)(k=1,2,…,10) during every ten minutes, and the mean value v-i=(1/10)∑k=110v(tik) is output as the wind speed vi+1 at time ti+1/6. By the proposed method in Section 3, the probability density f(x) of the wind speed data on 1 April 2008 was estimated and the graph of f(x) is shown in Figure 4.Figure 4
Probability density of wind speed data on 1 April 2008.


## 5.2. Incorporating the Prior Knowledge about the Wind Speed Fluctuation into the Training Data and Testing Data

Denote thatvimax⁡=max⁡k{v(tik)} and vimin⁡=min⁡k{v(tik)}. By the estimated probability density f(x) of the wind speed data on 1 April 2008 and formula (19), the probability Pi=P{vimin⁡≤vi≤vimax⁡} can be calculated. The wind speed sample data on 1 April 2008 are
(29)(t1,v1),(t2,v2),…,(t144,v144),
and data (29) are converted into
(30)(x1,v1),(x2,v2),…,(x144,v144)
by the method in Section 4.1, where xi=(ti,1-Pi)  (i=1,…,144). And the data (x1,v1),(x2,v2),…,(x143,v143) in data (30) is the training data to train a model to predict the wind speed v144 for the given x144.Similarly, if we want to predict the wind speedv1′ for the given x1′ on 2 April 2008, we can estimate the probability density of wind speed (ti,vi)(i=2,3,…,144) on 1 April 2008 and use the data (xi,vi)  (i=2,3,…,144) as the training data.


## 5.3.ε-Support Vector Regression Method for Wind Speed Prediction Incorporating Probability Prior Knowledge

In the experiment, a grid search method based on 5-fold cross-validation is chosen to determine model parameters, whereσ∈{2-10,2-9,…,21}, C∈{2-6,2-5,…,26}, and the kernel function is the radial basis function (RBF). In order to predict the wind speed v144 for the given x144 with training data (x1,v1),(x2,v2),…,(x143,v143) in data (30), we make experiment with PPK-ε-SVR and standard ε-SVR, respectively; the results are shown in Tables 1 and 2, respectively. The wind speeds of training data, normalized wind speeds of testing data, and wind speeds of testing data with PPK-ε-SVR are shown in Figures 5(a), 5(b), and 5(c), respectively. The wind speeds of training data, normalized wind speeds of testing data, and wind speeds of testing data with standard ε-SVR are shown in Figures 6(a), 6(b), and 6(c), respectively.Table 1
Experiment results with PPK-ε-SVR.
Data
Number in the data set
ε
σ
C
Mean squared errors
Training data
143
0.01
0.25
8.00
0.0217
Testing data
1
0.0035Table 2
Experiment results with standardε-SVR.
Data
Number in the data set
ε
σ
C
Mean squared errors
Training data
143
0.01
0.25
4.00
0.0307
Testing data
1
0.0747Initial wind speeds and predicted wind speeds with PPK-ε-SVR.
(a)
Wind speeds of training data
(b)
Normalized wind speeds of testing data
(c)
Wind speeds of testing dataInitial wind speeds and predicted wind speeds with standardε-SVR.
(a)
Wind speeds of training data
(b)
Normalized wind speeds of testing data
(c)
Wind speeds of testing dataSimilar to the steps of predicting wind speedv144 for the given x144, we make experiment 50 times to predict the wind speed vj for the given xj(j=1,2,…,50) (namely, the former 50 wind speeds monitored on 2 April 2008), the average mean squared errors are shown in Table 3, and the numbers after ± are the standard deviations.Table 3
Average mean squared errors.
Methods
Experiment times
Average mean squared errors
PPK-ε-SVR
50
0.0304
±
0.0367
Standardε-SVR
50
0.0402
±
0.0408


## 5.4. Result Analysis

From Tables1 and 2, we can see that the mean squared errors of training data and testing data with PPK-ε-SVR are all smaller than the corresponding ones with standard ε-SVR. Comparing Figure 5(a) with Figure 6(a) and Figure 5(c) with Figure 6(c), we can find that the predicted wind speeds of training data and testing data with PPK-ε-SVR are more close to the initial wind speeds than those with standard ε-SVR. This illustrates that the prediction error of PPK-ε-SVR is smaller than that of standard ε-SVR in predicting the wind speed v144 for the given x144. From Figure 5(b) we can obtain that the difference between normalized initial wind speed and normalized predicted wind speed is 0.766-0.707=0.059 which is equal to the square root of mean squared error 0.0035 in Table 1, which shows the effectiveness of the PPK-ε-SVR.From Table3, we can see that the average mean squared error (namely, the average prediction error) of PPK-ε-SVR is smaller than that of standard ε-SVR, which illustrates that PPK-ε-SVR method is more accurate than standard ε-SVR. What is more, the standard deviation of PPK-ε-SVR is also smaller than that of standard ε-SVR, which illustrates that the PPK-ε-SVR is more stable than the standard ε-SVR. And also the running time of PPK-ε-SVR is less than one minute, which shows that the model’s running time can meet the needs of wind speed prediction in application.


## 6. Conclusions

In this paper, a method of estimating the probability density of historical wind speed data is proposed, and the estimated probability density is used to describe the wind speed fluctuation between the maximal value and the minimal value in a certain period of time. Then the prior knowledge provided by historical wind speed data is incorporated into the training data and the testing data. Then, based on standardε-SVR, a kind of support vector regression for wind speed prediction incorporating probability prior knowledge is proposed. The comparing experiments show that the proposed PPK-ε-SVR is feasible and effective and the model’s running time can meet the needs of wind speed prediction in application. And, how to incorporate prior knowledge into selecting the kernel function to decrease the prediction error further is our future study.



---

*Source: 410489-2014-03-04.xml*