# Crude Oil Price Prediction Based on a Dynamic Correcting Support Vector Regression Machine

Authors: Li Shu-rong; Ge Yu-lei
Journal: Abstract and Applied Analysis
 (2013)

---

## Abstract

A new accurate method on predicting crude oil price is presented, which is based onε-support vector regression (ε-SVR) machine with dynamic correction factor correcting forecasting errors. We also propose the hybrid RNA genetic algorithm (HRGA) with the position displacement idea of bare bones particle swarm optimization (PSO) changing the mutation operator. The validity of the algorithm is tested by using three benchmark functions. From the comparison of the results obtained by using HRGA and standard RNA genetic algorithm (RGA), respectively, the accuracy of HRGA is much better than that of RGA. In the end, to make the forecasting result more accurate, the HRGA is applied to the optimize parameters of ε-SVR. The predicting result is very good. The method proposed in this paper can be easily used to predict crude oil price in our life.


---

## Body

## 1. Introduction

In recent years, crude oil prices have experienced four jumps and two slumps. The fluctuation of crude oil price adds more changes to the development of world economy. Grasping the change of oil price can provide guidance for economic development [1]. Therefore, it is very important to predict the crude oil price accurately.The predicting methods can be divided into two aspects. One is from the qualitative angle [2]; the other is from quantitative angle, such as econometric model and statistical model [3, 4]. And the latter method is adopted by most scholars. But it is a difficult job to predict crude oil price, since the price is nonlinear and nonstationary time series [5]. The traditional predicting methods such as AR(p) model, MA(q) model, and ARMA(p,q) model, base on linear model. They are only suitable for linear prediction and cannot be applied to model and predict nonlinear time series [6]. Wang got the predicting model by using time series and artificial neural network in 2005 [7], Xie proposed a new method for crude oil price forecasting based on support vector machine (SVM) in 2006 [8], Mohammad proposed a hybrid artificial intelligence model for crude oil price forecasting by means of feed-forward neural networks and genetic algorithm in 2007 [9], and Guo proposed a hybrid time series model on the base of GMTD model in 2010 [10]. The experimental results tell us that the prediction accuracy of these methods is better than traditional models. But the results is still existing biggish errors especially when the crude oil price is fluctuating violently.Neural network technique provides a favorable tool for nonlinear time series forecasting. But the predictive ability of conventional neural network is low, because of the problems such as the local minimum, over learning, and the lacking of theoretical direction for selecting the hidden layer nodes. The SVM was proposed in the 1990s [11]; it can get the optimal results on the basis of the current information. The basic idea of SVM is that it fits the sample capacity of functions on the basis of regulating the upper bound of the minimum VC dimension, which also means the numbers of support vector. Compared with neural network [12, 13], SVM has strong generalization ability of learning small samples and with the inferior dependence on quantity. But the prediction performance of SVM is very sensible to parameter selection. On the other hand, the research on parameter optimization of SVM is very few at the moment. The parameters are usually determined on experience or trial method. In this way, if the parameters are not suitably chosen, the SVM will lead to poor prediction performance. So, it is important to find one good method to get the optimal parameters of SVM.In this paper, anε-support vector regression machine with dynamic correction factor is proposed. And a novel hybrid RNA genetic algorithm (HRGA) is proposed to obtain the optimal parameters for a SVM. The HRGA is from the development of biological science and technology; the structure and information of RNA molecular are known profoundly. To improve the optimal performance of genetic algorithm, one genetic algorithm which bases on coding and biological molecular operation has been widely concerned [14]. This method improves the search efficiency and optimization performance through coding the individuals into biological molecules by use of bases [15, 16]. The appropriate mutation operator can improve the population diversity and prevent premature. While the mutation operator of classical RNA genetic algorithm (RGA) is fixed, so we need to find a suitable method to determine the mutation operator. In 2003, Kennedy did some improvement on particle swarm optimization (PSO) and proposed the bare bones particle swarm algorithm [17].In the proposed HRGA, the position displacement idea of bare bones PSO is applied to change the mutation operator. The nucleotide base encoding, RNA recoding operation, and protein folding operation are reserved in the new algorithm. Thus, the strong global search capability is kept. At the same time, to make sure of the directivity of local searching, the optimal experience of the whole population and the historical experience of individuals are used. The convergence speed and solution precision are improved. Furthermore, to test the validity of HRGA, three benchmark functions are adopted. The mean value of optimum of HRGA is smaller than that of traditional RNA genetic algorithm.Once the support vector regression machine is designed optimally, it can be used to predict crude oil price. Dynamic correction factor is brought in to improve the predictive effect and can strengthen the robustness of systems. In order to test the performance of the proposed predicting method, we provided the predicting results by using a back propagation neural network and a traditional support vector regression machine which are also improved with dynamic correction factor [7, 8]. The results show that our predicting method obtains greater accuracy than that of the other two in this paper.The paper is organized as follows. Section2 discusses the support vector regression machine with dynamic correction factor. Section 3 presents HRGA based on bare bones PSO, and some testing examples are applied to verify the effectiveness of the algorithm. Section 4 applies the dynamic correcting ε-SVR to predict the crude oil price. Section 5 concludes the paper.


## 2. Support Vector Regression Machine with Dynamic Correction Factor

Consider the training sample set(xi,yi), i=1,2,…,n, xi∈Rn, as the input variable and yi∈R as the output variable.The basic idea of SVM is to find a nonlinear mappingϕ from input space to output space [18–20]. Data x is mapped to a high-dimensional characteristic space F on the basis of the nonlinear mapping. The estimating function of linear regression in characteristic space F is as follows:
(1)f(x)=[ω×φ(x)]+b,ϕ:Rn⟶F,ω∈F,
where b denotes threshold value.Function approximation problem is equal to the following function:(2)Rreg(f)=Remp(f)+λ∥ω∥2=∑i=1sC(ei)+λ∥ω∥2,
where Rreg(f) denotes the objective function, Remp(f) denotes the empirical risk function, s denotes the sample quantity, λ denotes adjusting constant, and C denotes the error penalty factor. ∥ω∥2 reflects the complexity of f in the high-dimensional characteristic space.Since linearε insensitive loss function has better sparsity, we can get the following loss function:
(3)|y-f(x)|ε=max{0,|y-f(x)-ε|}.The empirical risk function is as follows:(4)Rempε(f)=1n∑i=1n|y-f(x)|ε.According to the statistical theory, we bring in two groups of nonnegative slack variable{ξi}i=1n and {ξi*}i=1n. Then, the question can be converted to the following nonlinear ε-support vector regression machine (ε-SVR) problem:
(5)min(ξi,ξi*){12∥ω∥2+C∑i=1n(ξi+ξi*)},yi-ω·ϕ(x)-b≤ε+ξi*,ω·ϕ(x)+b-yi≤ε+ξi,ξi,ξi*≥0,
where ε denotes the insensitive loss function. C is used to balance the complex item and the training error of the model.We bring into Lagrange multipliersαi and αi*, then the convex quadratic programming problem above can be changed into the below dual problem:
(6)max(αi,αi*)[-12∑i,j=1n(αi*-αi)(αj*-αj)K(Xi,Xj)+∑i=1nαi*(yi-ε)-∑i=1nαi(yi-ε){-12∑i,j=1n(αi*-αi)(αj*-αj)K(Xi,Xj)}],∑i=1n(αi-αi*)=0,0≤αi,αi*≤C,i=1,2,…,n,
where K(Xi,Xj) denotes the inner product kernel satisfying Mercer theorem.We can get theε-SVR function through solving the above dual problem:
(7)f(x)=∑i=1n(αi-αi*)K(Xi,X)+b.Whenε-SVR is used on prediction, it may have a certain error since the data fluctuates violently such as the crude oil price. To reduce the error in some certain as possible as we can, we bring in the dynamic correction factor ε*. The main idea of the dynamic correction factor is that we use the error of back step with multiplying ε* to revise the current predicting results. Thus, we can reduce the current predicting error. The dynamic correcting SVR can be defined as follows:
(8)Yd(i+1)=Y1(i+1)+ε*[Y(i)-Yd(i)],
where Y denotes the real results, Yd denotes the final prediction results, Y1 denotes the initial predicting results, ε* denotes the dynamic correction factor, and i denotes the prediction steps.In order to make the predicting results more accurate, the optimal value ofε* and the parameters of ε-SVR involving C,δ (the variable in gauss kernel function) should be designed (in (8)). To this end, an HRGA is studied below to optimize the following problem:
(9)min(ε*,C,σ)∑in[Yd(i)-Y(i)]2.


## 3. HRGA Based on Bare Bones PSO

Assuming that population size isN, the dimension of particle is m. The position of particle i on generation t is Xi(t)=(xi1(t),…,xij(t),…,xim(t)),i=1,2,…,N. The speed of particle i on generation t is Vi(t)=(vi1(t),…,vij(t),…,vim(t)). The historic optimal value of individuals is PBesti(t)=(pbesti1(t),…,pbestij(t),…,pbestim(t)).Let the global optimal value beGBest(t)=(gbest1(t),…,gbestj(t),…,gbestm(t)).As to standard particle swarm, the position and speed are updated as(10)vij(t+1)=wvij(t)+c1·r1j·(pbestij(t)-xij(t))+c2·r2j·(gbestj-xij(t)),xij(t+1)=xij(t)+vij(t+1),
where ω denotes the inertia weight [21], c1 and c2 denote the accelerating operators, andr1j and r2j are uniform distributed random numbers in [0,1].In the bare bones particle swarm optimization (PSO), (10) is replaced by (11) as the evolution equation of particle swarm algorithm:
(11)xij(t+1)=N(0.5(pbestij(t)+gbestj(t))×|pbestij(t)-gbestj(t)|).The position of particle is some random numbers which are gotten from the Gauss distribution. The distribution has the mean value of(pbestij(t)+gbestj(t))/2 and the standard deviation of |pbestij(t)-gbestj(t)|.RNA genetic algorithm is on the basis of base coding and biological molecules operation. Since in the biological molecule, every three bases compose one amino acid. In other words, the bases’ length of individuals must be divided exactly by 3. When RNA recoding and protein folding [22], to reduce calculation and to control population size, we assume that the protein folding operation only occurs on the individuals without RNA recoding. Then the most important work is to change the mutation probability [23, 24].Angeline told us that the essence of particle swarm’s position updating was one mutation operation in 1998 [25]. Traditional RNA genetic algorithm mutates as the fixed mutation probability, and the mutation is random with one direction. However HRGA can reflect the historic information of individuals and the sharing information of the population. HRGA can make every individual do directional mutation and improve search efficiency. Moreover, HRGA ensures the strong global search capability, since it does not change the selection and crossover operator.The procedure of HRGA based on bare bones particle swarm algorithm to optimize theε-SVR parameters and the dynamic correction factor is as follows.Step 1.
Get one group ofε-SVR parameters, and the dynamic correction factor randomly, code every parameter, and get the initial RNA population with N individuals, crossover probability Pc, and mutation probability Pm. Assign values for every PBesti (individual’s historic optimal solution) and  GBest (population’s global optimal solution).Step 2.
Compute its error function and get the fitness function. Comparing it with corresponding fitness value ofPBesti and GBest, then update PBesti and GBest.Step 3.
Execute the selection operation. Get current generation through copingN individuals from the initial or the last generation.Step 4.
Decide whether the value meets the RNA recoding condition or not. IfY, recode RNA, then go to Step 6. If  N, go to Step 5.Step 5.
Decide meet the protein mutual folding condition or not. IfY, execute the protein mutual folding operation. If N, execute the protein own folding operation.Step 6.
Execute the mutation operation as (11) for all the crossover individuals, on the basis of the PBesti and GBest, which have been gotten from Step 2.Step 7.
Repeat Step2 to Step 6 until the training target meets the condition. At last, we get the optimal parameters of ε-SVR and the dynamic correction factor.The flowchart of HRGA to optimize theε-SVR parameters and the dynamic correction factor is shown in Figure 1.Figure 1
The flowchart of HRGA.


### 3.1. HRGA Testing

Three classical benchmark functions shown in Table1 are used to test the property of HRGA.Table 1
Benchmark functions.
Function
Formula
Global minimum
Sphere
f
1

(
x
)
=
∑
i
=
1
n
x
i
2, xi∈[-100,100].
f
1
*


(
x
)
=
0, x*=(0,0,…,0)
Rosenbrock
f
2
=
∑
i
=
1
n
=
1
(
100
×
(
x
i
+
1
-
x
i
2
)
2
)
+
(
x
i
-
1
)
2, xi∈[-30,30].
f
2
*

(
x
)
=
0, x*=(1,1,…,1)
Griewank
f
3
=
1
4000
∑
i
=
1
n
x
i
2
-
∏
i
=
1
n
cos
(
x
i
i
)
+
1, xi∈[-600,600].
f
3
*

(
x
)
=
0, x*=(0,0,…,0)In addition, among the three functions, Sphere is unimodal function, and the other two are multimodal function.With the population sizeN=50, and other parameters determined by multiple test for each function. Each function is tested by HRGA and standard RGA in different dimensions. Each experience is carried on 100 times. Record the mean value of target function’s optimum (shown in (12)). The result is displayed in Table 2:
(12)MVO=1N∑i=1Nfi′(x).Table 2
Testing results of  HRGA and standard RGA.
Function
Hunting zone
Dimension
Iterative times
RGA
HRGA
Parameter selection
Mean value ofoptimum
Parameter selection
Mean value ofoptimum
(
-
100,100
)
2
250
P
c
=
0.9 Pm=0.1
8.3421
e
-
6
P
c
=
0.85 Pm=0.3
1.9824
e
-
125
Sphere
(
-
100,100
)
10
1000
P
c
=
0.9 Pm=0.1
0.0437
P
c
=
0.85 Pm=0.3
1.4597
e
-
67
(
-
100,100
)
20
1500
P
c
=
0.9 Pm=0.1
0.1012
P
c
=
0.85 Pm=0.3
1.5775
e
-
31
(
-
30,30
)
2
250
P
c
=
0.85 Pm=0.1
0.5311
P
c
=
0.6 Pm=0.3
0.0195
Rosenbrock
(
-
30,30
)
10
1000
P
c
=
0.85 Pm=0.1
15.0766
P
c
=
0.6 Pm=0.3
3.0922
(
-
30,30
)
20
1500
P
c
=
0.85 Pm=0.1
124.2468
P
c
=
0.6 Pm=0.3
8.0576
(
-
600,600
)
2
250
P
c
=
0.8 Pm=0.15
0.0062
P
c
=
0.5 Pm=0.4
0.0044
Griewank
(
-
600,600
)
10
1000
P
c
=
0.8 Pm=0.15
0.1468
P
c
=
0.5 Pm=0.4
0.0132
(
-
600,600
)
20
1500
P
c
=
0.8 Pm=0.15
0.0191
P
c
=
0.5 Pm=0.4
2.0109
e
-
3In this equation,MVO denotes the mean value of target function’s optimum; fi′(x) denotes the optimum of benchmark functions in every experiment.As to the experimental results, with different dimensions having the same iterative times, the mean value of optimum of HRGA is smaller than that of RGA for the three benchmark functions. The average performance of HRGA is closer to the optimum. We can increase the mutation probability appropriately and enhance the convergence speed, since the mutation operator of HRGA has directional local search.


## 3.1. HRGA Testing

Three classical benchmark functions shown in Table1 are used to test the property of HRGA.Table 1
Benchmark functions.
Function
Formula
Global minimum
Sphere
f
1

(
x
)
=
∑
i
=
1
n
x
i
2, xi∈[-100,100].
f
1
*


(
x
)
=
0, x*=(0,0,…,0)
Rosenbrock
f
2
=
∑
i
=
1
n
=
1
(
100
×
(
x
i
+
1
-
x
i
2
)
2
)
+
(
x
i
-
1
)
2, xi∈[-30,30].
f
2
*

(
x
)
=
0, x*=(1,1,…,1)
Griewank
f
3
=
1
4000
∑
i
=
1
n
x
i
2
-
∏
i
=
1
n
cos
(
x
i
i
)
+
1, xi∈[-600,600].
f
3
*

(
x
)
=
0, x*=(0,0,…,0)In addition, among the three functions, Sphere is unimodal function, and the other two are multimodal function.With the population sizeN=50, and other parameters determined by multiple test for each function. Each function is tested by HRGA and standard RGA in different dimensions. Each experience is carried on 100 times. Record the mean value of target function’s optimum (shown in (12)). The result is displayed in Table 2:
(12)MVO=1N∑i=1Nfi′(x).Table 2
Testing results of  HRGA and standard RGA.
Function
Hunting zone
Dimension
Iterative times
RGA
HRGA
Parameter selection
Mean value ofoptimum
Parameter selection
Mean value ofoptimum
(
-
100,100
)
2
250
P
c
=
0.9 Pm=0.1
8.3421
e
-
6
P
c
=
0.85 Pm=0.3
1.9824
e
-
125
Sphere
(
-
100,100
)
10
1000
P
c
=
0.9 Pm=0.1
0.0437
P
c
=
0.85 Pm=0.3
1.4597
e
-
67
(
-
100,100
)
20
1500
P
c
=
0.9 Pm=0.1
0.1012
P
c
=
0.85 Pm=0.3
1.5775
e
-
31
(
-
30,30
)
2
250
P
c
=
0.85 Pm=0.1
0.5311
P
c
=
0.6 Pm=0.3
0.0195
Rosenbrock
(
-
30,30
)
10
1000
P
c
=
0.85 Pm=0.1
15.0766
P
c
=
0.6 Pm=0.3
3.0922
(
-
30,30
)
20
1500
P
c
=
0.85 Pm=0.1
124.2468
P
c
=
0.6 Pm=0.3
8.0576
(
-
600,600
)
2
250
P
c
=
0.8 Pm=0.15
0.0062
P
c
=
0.5 Pm=0.4
0.0044
Griewank
(
-
600,600
)
10
1000
P
c
=
0.8 Pm=0.15
0.1468
P
c
=
0.5 Pm=0.4
0.0132
(
-
600,600
)
20
1500
P
c
=
0.8 Pm=0.15
0.0191
P
c
=
0.5 Pm=0.4
2.0109
e
-
3In this equation,MVO denotes the mean value of target function’s optimum; fi′(x) denotes the optimum of benchmark functions in every experiment.As to the experimental results, with different dimensions having the same iterative times, the mean value of optimum of HRGA is smaller than that of RGA for the three benchmark functions. The average performance of HRGA is closer to the optimum. We can increase the mutation probability appropriately and enhance the convergence speed, since the mutation operator of HRGA has directional local search.


## 4. Crude Oil Price Prediction Based on a Dynamic Correctingε-SVR

In this paper, we get the crude oil price from the US Energy Information Administration Web [26]. Since the oil price fluctuates violently, in order to facilitate the processing and decrease the error, we adopt the Cushing, OK WTI Spot Price FOB (dollars per barrel)monthly from January 1986 to now. We take the one hundred data from January 1986 to April 1994 as the test sample. And give the next 20-month dynamic predicting data from May 1994 to December 1995. The relative error of forecasting is shown in Table 2. The prediction effect figure of HRGA and ε-SVR with dynamic correction factor is shown in Figure 2. We use Gauss function as the kernel function of ε-SVR, which is given as follows:
(13)k(x,y)=exp(-∥x-y∥22·σ2).Figure 2
The prediction curve of HRGA-ε-SVR.Parameter setting of HRGA-ε-SVR is with population size being 100, maximum evolution generation being 150, coding length of C being 9, coding length of ε being 8, coding length of σ being 13, coding length of ε* being 8, Pc being 0.8, and Pm being 0.1.The optimization interval is set to be(14)1≤C≤10000,0.0001≤ε≤0.1,0.01≤σ≤500,0.5≤ε*≤2.When analyzing the results, we define the evaluation index:(15)Er=xi-yixi×100%,σ=1n∑i=1n(xi-yixi)2.The forecasting error analysis results are shown in Figure3. In this figure, SVM refers to ε-SVR. The BP neural network and ε-SVR are with dynamic correction factor which differs them to the traditional method. From Figure 2, we can know that the prediction result is very close to the real value. The HRGA-ε-SVR can be used to predict the crude oil price. Table 3 tells us the WTI crude oil price predicting relative errors of twenty months. Among the three methods in twenty months, the biggest absolute value of relative error of HRGA-ε-SVR is the smallest, which is 7.35%, and the smallest root-mean-square of relative error is 3.87%. As to Figure 3, the fluctuation range of HRGA-ε-SVR is smaller than those of the other two methods obviously. This means that HRGA-ε-SVR is the best one among the three methods.Table 3
Analysis results of forecasting error.
Date
BP/%
ε-SVR%
HRGA-ε-SVR /%
5-1994
3.22
3.64
3.42
6-1994
−5.1
−3.78
−4.48
7-1994
1.97
−0.57
0.61
8-1994
4.27
6.55
5.77
9-1994
2.05
4.41
3.38
10-1994
2.19
1.59
1.51
11-1994
0.69
2.3
1.87
12-1994
8.08
7.68
7.35
1-1995
−6.46
−2.79
−4.16
2-1995
7.58
2.87
4.44
3-1995
−4.23
1.61
−0.13
4-1995
−1.69
−6.46
−5.57
5-1995
1.56
2.08
2.48
6-1995
3.75
3.94
3.2
7-1995
3.7
5.19
4.83
8-1995
−1.69
−0.85
−1.75
9-1995
4.27
3.4
4.19
10-1995
4.64
5.94
5.45
11-1995
−2.58
−1.17
−2.14
12-1995
0.62
−0.92
−0.03
δ
4.09
3.96
3.87Figure 3
Errors analysis with different models.


## 5.  Conclusions

In this paper, we have presented a novel method on predicting crude oil price. This method bases on anε-support vector regression machine with dynamic correction factor correcting predicting errors. We also proposed the HRGA, with the position displacement idea of bare bones PSO changing the mutation operator, to optimize the parameters in an ε-SVR. The predicting result of crude oil price shows the validity of the proposed method. Thus, the ε-SVR model can also be applied to predict tendency in other practical areas.



---

*Source: 528678-2013-03-18.xml*