# Time Series Adaptive Online Prediction Method Combined with Modified LS-SVR and AGO

Authors: Guo Yangming; Zhang Lu; Cai Xiaobin; Ran Congbao; Zhai Zhengjun; Ma Jiezhong
Journal: Mathematical Problems in Engineering
 (2012)

---

## Abstract

Fault or health condition prediction of the complex systems has attracted more attention in recent years. The complex systems often show complex dynamic behavior and uncertainty, which makes it difficult to establish a precise physical model. Therefore, the time series of complex system is used to implement prediction in practice. Aiming at time series online prediction, we propose a new method to improve the prediction accuracy in this paper, which is based on the grey system theory and incremental learning algorithm. In this method, the accumulated generating operation (AGO) with the raw time series is taken to improve the data quality and regularity firstly; then the prediction is conducted by a modified LS-SVR model, which simplifies the calculation process with incremental learning; finally, the inverse accumulated generating operation (IAGO) is performed to get the prediction results. The results of the prediction experiments indicate preliminarily that the proposed scheme is an effective prediction approach for its good prediction precision and less computing time. The method will be useful in actual application.


---

## Body

## 1. Introduction

Prediction technique is essential to ensure the operational safety of complex systems. However, the complex systems are not easy to establish their precise physical models. In the practice, time series-based prediction methods have attracted increased attention [1–7].Compared with other reported methods, Support Vector Regression (SVR) is based on statistical theory and structural risk minimization principle [8–10], and it has a global optimum and exhibits better accuracy in nonlinear and nonstationary time series data prediction via kernel function. However, aiming at the large sample data, the quadratic programming (QP) problem becomes more complex. Thus, Least Squares Support Vector Regression (LS-SVR) was proposed by Suykens et al. [11] In LS-SVR, the inequality constrains are replaced by equality constrains, which can reduce the calculation time effectively. Thus, LS-SVR has more attention in time series forecasting [12–16].Because a great number of uncertain elements exist in practical application, online prediction scheme [17, 18] is applied to meet the actual training and prediction condition, and then achieve better prediction results. Moreover, the incompleteness, noisiness, and inconsistency existing in the raw time series sample data will reduce the prediction accuracy. In these cases, two problems should be considered simultaneously, one is how to improve prediction accuracy, the other one is how to reduce the training time and prediction time.Some researchers combined several methods to improve prediction performance [19, 20], which can fuse their advantages and avoid their drawbacks. Using this idea, we propose a scheme to fix the above problems. In the new scheme, we integrate accumulated generating operation (AGO) [21] with a modified LS-SVR-based model. It includes the following two aspects.(1)
Based on the grey system theory [21], we conduct AGO with the raw time series to improve the data quality and regularity, and finally we can obtain the results using inverse accumulated generating operation (IAGO).(2)
We set up a new adaptive online prediction model based on LS-SVR, which utilizes incremental learning algorithm to enrich information and modifies the LS-SVR model with more simple form to reduce prediction calculation time.The remainder of the paper is organized as follows. Section2 gives a brief introduction of LS-SVR; Section 3 proposes the new scheme in detail, which includes the AGO-based prediction strategy and the simple modified prediction model based on incremental learning algorithm; Section 4 shows experiments and results analysis; conclusions are given in Section 5.


## 2. A Brief Introduction of LS-SVR

Consider a training sample data set{(xk,yk)}k=1N with input data xk∈Rn and output yk∈R, where N denotes the number of training samples. The goal of LS-SVR is to obtain a function as follows:
(2.1)y(x)=wTϕ(x)+b,
where the nonlinear mapping  function ϕ(·) maps the input data into a higher dimensional feature space. It means that the method makes the nonlinear fitting problem in input feature space to be replaced by a linear fitting problem in high-dimensional feature space. w is the weight vector and b is the bias constant.According to the structure risk minimization principle,w and b can be found via a constrained convex optimization as follows:
(2.2)minJ(w,e)=12wTw+12c∑k=1Nek2,s.t.yk=wTφ(xk)+b+ek,k=1,2,…,N,
where J is the loss function, ei∈R is the slack variables, and c is a regularization parameter. Equation (2.2) can be transformed into dual form with Lagrange function as follows:
(2.3)L(w,b,e,α)=J(w,e)-∑k=1Nαk{wTφ(xk)+b+ek-yk},
where αk are the Lagrange multipliers.It is obvious that the optimal solution of (2.2) satisfies Karush-Kuhn-Tucker (KKT) conditions, then the optimal conditions are shown as follows:
(2.4)∂L∂w=w-∑i=1nαiφ(xi)=0⇒w=∑i=1nαiφ(xi),∂L∂b=-∑i=1nαi=0⇒∑i=1nαi=0,∂L∂αi=wTφ(xi)+b+ei-yi=0⇒yi=wTφ(xi)+b+ei,∂L∂ei=cei-αi=0⇒ei=1cαi.After eliminatingw and ei from (2.4), we can obtain the solution by the following linear equations:
(2.5)[01nT1nK+Ic][bα]=[0y],
where K(i,j)=k(xi,xj)=φ(xi)Tφ(xj),α=[α1,α2,…,αn]T,1n is an n-dimensional vector of all ones, I is a unite matrix, and y=[y1,y2,…,yn]T.In LS-SVR, the optimization problem is simplified to a linear equations instead of more complex quadratic programming problem in SVR. Therefore, the computational complexity is decreased significantly. Obviously, (2.5) can be factorized into a positive definite system [22]. Thus, the solutions of αk and b could be easily obtained. Then the LS-SVR model for function estimator can be expressed as follows:
(2.6)y(x)=∑k=1NαkK(x,xk)+b.


## 3. Proposed Adaptive Online Prediction Scheme for Time Series

### 3.1. AGO-Based Prediction Method for Time Series

Time series prediction models employ the historical data values for extrapolation to obtain the future values. If a time series has a regular pattern, then a value of the series should be a function of previous values. IfY is the target value that we are trying to model and predict, and Yt is the value of Y at time t, then the goal is to create a prediction model f as following form:
(3.1)Yt=f(Yt-1,Yt-2,Yt-3,…,Yt-n)+et,
where Yt-1 is the value of Y for the previous observation, Yt-2 is the value two observations ago, and et represents noise that does not follow a predictable pattern. This way, the good prediction accuracy always depends on the high quality of time series data.In 1982, a famous Chinese scholar Deng [21] proposed the grey system theory, which has already been used widely in all kinds of fields. As the core of the grey prediction theory, the accumulated generating operation (AGO) and inverse accumulated generating operation (IAGO) are the main methods which provide a manageable approach to treating disorganized evidence, that is, AGO has a main advantage that it can reduce the disturbance with the stochastic factors. AGO makes the rules hidden the raw time series to be presented fully and enhances the law of time series data.Consider a raw time seriesx(0)=(x(0)(1),x(0)(2),...,x(0)(n)), and take the AGO with x(0) as follows:
(3.2)x(1)(i)=∑r=1ix(r),i=1,2,…,n.
Then x(1)=(x(1)(1),x(1)(2),…,x(1)(n)) will be the new time series.After the new time seriesx(1) is formed, it can be used to predict the future values based on the previous and current values. The previous and the current values of the time series are used as input for the following prediction model:
(3.3){x(1)(t+1),x(1)(t+2),…,x(1)(t+h)}=F(x(1)(t),x(1)(t-1),…,x(1)(t-m+1)),
where h represents the number of ahead predictions, F is the prediction model, and m is the size of repressor. According to (3.3), we can obtain the training sample.In this paper, we use the following prediction strategy to predict the next data-point value. The strategy is that the prediction data will join into the sample data set as known data. This prediction process is called incremental algorithm. The model can be constructed with one-step prediction:(3.4)x(1)(t+1)=F(x(1)(t),x(1)(t-1),…,x(1)(t-m+1)).
The regressor of the model is defined as the vector of inputs (x(t),x(t-1),…,x(t-m+1)).After getting the prediction values sequence, we can compute the real prediction results by IAGO:(3.5)x^(0)(t+1)=x^(1)(t+1)-x^(1)(t).


### 3.2. Modified LS-SVR Model Based Incremental Learning

For adaptive online prediction, real-time is the basic requirement, that is, in the adaptive online prediction based on incremental learning algorithm, the new predicted data is constantly added, and we hope that the total computing time of training and prediction should be less than the sampling period. In this case, the most important issue is to compute Lagrange multipliersα and bias term b, which has effect on the computing time because they need recalculate when the new sample data is added in. After analyzing the research results of Yaakov et al. [23] we develop a simple prediction model based on LS-SVR to solve the problem.Suppose the training sample data are described by the input{(xi,yi)}i=1i=t,(xi∈Rd,yi∈R). In time t, give the sample data set {(x(t),y(t)}, x(t)=[x1(t),x2(t),…,xt(t)],y(t)=[y1(t),y2(t),...,yt(t)]. As time t goes on, the new sample data joins. We give a new approach to eliminate the bias term b.Letw(t)=[w(t)b/λ] and ϕi(t)=[φi(t)λ], where φi(t) is the mapping function of the ith input data at time t and λ is a constant. Then rewritten optimization problem of (2.2) as follows:
(3.6)minJ(w,e)=12w(t)Tw(t)+c2∑i=1tei2(t),s.t.yi(t)=w(t)Tϕi(t)+ei(t),i=1,2,…,t.The solution of (3.6) is also obtained after constructing the Lagrange function:
(3.7)L(w,e,a)=12w(t)Tw(t)+12c∑i=1tei2(t)-∑i=1tai(t)(w(t)Tϕi(t)+ei(t)-yi(t)),
where ai(t) is the ith Lagrange multiplier. The new optimal conditions are shown as follows
(3.8)∂L∂w(t)=w(t)-∑i=1tai(t)ϕi(t)=0⇒w(t)=∑i=1tai(t)ϕi(t),∂L∂ai(t)=w(t)Tϕi(t)+ei(t)-yi(t)=0⇒yi(t)=w(t)Tϕi(t)+ei(t),∂L∂ei(t)=cei(t)-ai(t)=0⇒ei(t)=1cai(t).
After eliminating ei(t) and w(t), the solution of (3.8) is given by the following set of linear equations:
(3.9)[φ1(t)Tφ1(t)+λ2+1cφ2(t)Tφ1(t)+λ2⋯φt(t)Tφ1(t)+λ2φ1(t)Tφ2(t)+λ2φ2(t)Tφ2(t)+λ2+1c⋯φt(t)Tφ2(t)+λ2⋮⋮⋮⋮φ1(t)Tφt(t)+λ2φ2(t)Tφt(t)+λ2⋯φt(t)Tφt(t)+λ2+1c][a1(t)a2(t)⋮at(t)]=[y1(t)y2(t)⋮yt(t)].
Let
(3.10)a(t)=[a1(t)a2(t)⋯at(t)]T,y(t)=[y1(t)y2(t)⋯yt(t)]T,kij(t)=φi(t)Tφj(t),Hij(t)=kij(t)+λ2,H(t)={Hij(t)}i,j=1t+Ic,
where I is an identity matrix with t×t. Equation (3.9) can be rewritten as follows:
(3.11)a(t)=y(t)H(t)-1.
The new regression function is presented as follows:
(3.12)yt+1(t)=∑i=1tai(t)(k(xi(t),xt+1(t))+λ2).
According to (3.11) and (3.12), when the new sample data yt+1(t) is added, the calculation only needs to compute parameter a.At timet+1, a can be calculate via (3.11), that is, a(t+1)=y(t+1)H(t+1)-1. Here,
(3.13)y(t+1)=[y(t)Tyt+1(t)]T,H(t+1)=[H(t)V(t+1)V(t+1)Tv(t+1)],V(t+1)=[φt+1(t)Tφ1(t)+λ2φt+1(t)Tφ2(t)+λ2⋮φt+1(t)Tφt(t)+λ2],v(t+1)=φt+1(t)Tφt+1(t)+λ2+1c,xt+1(t)=xt+1(t+1),yt+1(t)=yt+1(t+1),φt+1(t)=φt+1(t+1).H(t+1)-1 can be fast computed with block matrix approach [24, 25].From above description, it is obvious that the modified prediction model is a simple model, and it is maybe applied to reduce the computing time for its less number of parameters.


## 3.1. AGO-Based Prediction Method for Time Series

Time series prediction models employ the historical data values for extrapolation to obtain the future values. If a time series has a regular pattern, then a value of the series should be a function of previous values. IfY is the target value that we are trying to model and predict, and Yt is the value of Y at time t, then the goal is to create a prediction model f as following form:
(3.1)Yt=f(Yt-1,Yt-2,Yt-3,…,Yt-n)+et,
where Yt-1 is the value of Y for the previous observation, Yt-2 is the value two observations ago, and et represents noise that does not follow a predictable pattern. This way, the good prediction accuracy always depends on the high quality of time series data.In 1982, a famous Chinese scholar Deng [21] proposed the grey system theory, which has already been used widely in all kinds of fields. As the core of the grey prediction theory, the accumulated generating operation (AGO) and inverse accumulated generating operation (IAGO) are the main methods which provide a manageable approach to treating disorganized evidence, that is, AGO has a main advantage that it can reduce the disturbance with the stochastic factors. AGO makes the rules hidden the raw time series to be presented fully and enhances the law of time series data.Consider a raw time seriesx(0)=(x(0)(1),x(0)(2),...,x(0)(n)), and take the AGO with x(0) as follows:
(3.2)x(1)(i)=∑r=1ix(r),i=1,2,…,n.
Then x(1)=(x(1)(1),x(1)(2),…,x(1)(n)) will be the new time series.After the new time seriesx(1) is formed, it can be used to predict the future values based on the previous and current values. The previous and the current values of the time series are used as input for the following prediction model:
(3.3){x(1)(t+1),x(1)(t+2),…,x(1)(t+h)}=F(x(1)(t),x(1)(t-1),…,x(1)(t-m+1)),
where h represents the number of ahead predictions, F is the prediction model, and m is the size of repressor. According to (3.3), we can obtain the training sample.In this paper, we use the following prediction strategy to predict the next data-point value. The strategy is that the prediction data will join into the sample data set as known data. This prediction process is called incremental algorithm. The model can be constructed with one-step prediction:(3.4)x(1)(t+1)=F(x(1)(t),x(1)(t-1),…,x(1)(t-m+1)).
The regressor of the model is defined as the vector of inputs (x(t),x(t-1),…,x(t-m+1)).After getting the prediction values sequence, we can compute the real prediction results by IAGO:(3.5)x^(0)(t+1)=x^(1)(t+1)-x^(1)(t).


## 3.2. Modified LS-SVR Model Based Incremental Learning

For adaptive online prediction, real-time is the basic requirement, that is, in the adaptive online prediction based on incremental learning algorithm, the new predicted data is constantly added, and we hope that the total computing time of training and prediction should be less than the sampling period. In this case, the most important issue is to compute Lagrange multipliersα and bias term b, which has effect on the computing time because they need recalculate when the new sample data is added in. After analyzing the research results of Yaakov et al. [23] we develop a simple prediction model based on LS-SVR to solve the problem.Suppose the training sample data are described by the input{(xi,yi)}i=1i=t,(xi∈Rd,yi∈R). In time t, give the sample data set {(x(t),y(t)}, x(t)=[x1(t),x2(t),…,xt(t)],y(t)=[y1(t),y2(t),...,yt(t)]. As time t goes on, the new sample data joins. We give a new approach to eliminate the bias term b.Letw(t)=[w(t)b/λ] and ϕi(t)=[φi(t)λ], where φi(t) is the mapping function of the ith input data at time t and λ is a constant. Then rewritten optimization problem of (2.2) as follows:
(3.6)minJ(w,e)=12w(t)Tw(t)+c2∑i=1tei2(t),s.t.yi(t)=w(t)Tϕi(t)+ei(t),i=1,2,…,t.The solution of (3.6) is also obtained after constructing the Lagrange function:
(3.7)L(w,e,a)=12w(t)Tw(t)+12c∑i=1tei2(t)-∑i=1tai(t)(w(t)Tϕi(t)+ei(t)-yi(t)),
where ai(t) is the ith Lagrange multiplier. The new optimal conditions are shown as follows
(3.8)∂L∂w(t)=w(t)-∑i=1tai(t)ϕi(t)=0⇒w(t)=∑i=1tai(t)ϕi(t),∂L∂ai(t)=w(t)Tϕi(t)+ei(t)-yi(t)=0⇒yi(t)=w(t)Tϕi(t)+ei(t),∂L∂ei(t)=cei(t)-ai(t)=0⇒ei(t)=1cai(t).
After eliminating ei(t) and w(t), the solution of (3.8) is given by the following set of linear equations:
(3.9)[φ1(t)Tφ1(t)+λ2+1cφ2(t)Tφ1(t)+λ2⋯φt(t)Tφ1(t)+λ2φ1(t)Tφ2(t)+λ2φ2(t)Tφ2(t)+λ2+1c⋯φt(t)Tφ2(t)+λ2⋮⋮⋮⋮φ1(t)Tφt(t)+λ2φ2(t)Tφt(t)+λ2⋯φt(t)Tφt(t)+λ2+1c][a1(t)a2(t)⋮at(t)]=[y1(t)y2(t)⋮yt(t)].
Let
(3.10)a(t)=[a1(t)a2(t)⋯at(t)]T,y(t)=[y1(t)y2(t)⋯yt(t)]T,kij(t)=φi(t)Tφj(t),Hij(t)=kij(t)+λ2,H(t)={Hij(t)}i,j=1t+Ic,
where I is an identity matrix with t×t. Equation (3.9) can be rewritten as follows:
(3.11)a(t)=y(t)H(t)-1.
The new regression function is presented as follows:
(3.12)yt+1(t)=∑i=1tai(t)(k(xi(t),xt+1(t))+λ2).
According to (3.11) and (3.12), when the new sample data yt+1(t) is added, the calculation only needs to compute parameter a.At timet+1, a can be calculate via (3.11), that is, a(t+1)=y(t+1)H(t+1)-1. Here,
(3.13)y(t+1)=[y(t)Tyt+1(t)]T,H(t+1)=[H(t)V(t+1)V(t+1)Tv(t+1)],V(t+1)=[φt+1(t)Tφ1(t)+λ2φt+1(t)Tφ2(t)+λ2⋮φt+1(t)Tφt(t)+λ2],v(t+1)=φt+1(t)Tφt+1(t)+λ2+1c,xt+1(t)=xt+1(t+1),yt+1(t)=yt+1(t+1),φt+1(t)=φt+1(t+1).H(t+1)-1 can be fast computed with block matrix approach [24, 25].From above description, it is obvious that the modified prediction model is a simple model, and it is maybe applied to reduce the computing time for its less number of parameters.


## 4. Experiments and Results Analysis

We perform two simulation experiments and an application experiment to evaluate the proposed scheme. All the experiments adopt MatlabR2011b with LS-SVMlab1.8 Toolbox (The software can be downloaded fromhttp://www.esat.kuleuven.be/sista/lssvmlab) under Windows XP operating system.In this paper, we use the prediction Root Mean Squared Error (RMSE) [26] as evaluation criteria, which is defined as follows:
(4.1)RMSE=1n∑k=1n(x(k)-x'(k))2,
where n is the number of training sample data, x'(k) and x(k) are the prediction and the actual value, respectively.


### 4.1. Simulation Experiments and Results Analysis

In order to validate the performance of the proposed AGO-based method described in Section3.1 and the modified LS-SVR model-based incremental learning described in Section 3.2, we perform two simulation experiments. In the experiments, the sample time series data of variable x has 75 sample data values, which come from one complex avionics system (shown in Figure 1, omit dimension).Figure 1
Sample time series of a certain avionics system.The data points from 1 to 50 in time series are taken as 45 initial training sample data. The first sample data set consists of points 1 to 6, with the first 5 as the input sample vector and the 6th point as the output. The second sample data set consists of points 2 to 7, with the point 2 through 6 as input sample vector and 7th point as the output. This way we have 45 training data out of the first 50 data points. And then we predict the no. 51 to no. 75 time series data using the trained model. The performance is measured by the prediction mean RMSE (PrMR), training time (TrTime), and prediction time (PrTime). The experiments are repeated 100 times and the average results are obtained.In the simulation Experiment I, we compare the proposed AGO-based method with traditional LS-SVR presented in [4]. Here, Gaussian RBF kernel k(x,y)=exp(-∥x-y∥2/2σ2) is adopted as kernel function, and the parameters are jointly optimized with traditional gridding search method, where the search rang of c and σ2 is [10,3000]. The prediction results are shown in Figure 2 (ALS-SVR presents the proposed AGO-based method in the figure) and Table 1.Table 1
Prediction results of Experiment  I.
Method
TrTime/(s)
PrTime/(s)
PrMR
Traditional LS-SVR in [4]
0.0632
0.0200
2.4411
Proposed AGO-based method in Section3.1
0.0706
0.0209
2.1569Figure 2
Prediction results ofx.From Table1 and Figure 2, we can see that the proposed AGO-based method has better prediction accuracy. This is may be due to that the proposed AGO-based method can avoid the random disturbances existing in the raw time series and improve the regularity of the time series.In simulation Experiment II, we compare the two incremental learning-based methods: one is the proposed modified LS-SVR model, the other one is the traditional LS-SVR model. The other selections are the same as Experiment I. The prediction results are reported in Table2.Table 2
Prediction results of Experiment  II.
Incremental learning-based method
TrTime/(s)
PrTime/(s)
PrMR
Traditional LS-SVR model
0.1872
0.0206
2.2142
Proposed modified LS-SVR model in Section3.2
0.1186
0.0205
2.2142The results in Table2 show that the modified LS-SVR model can reduce the computing time more with the same prediction accuracy as the traditional LS-SVR model. Compared with the results of traditional LS-SVR in [4] (shown in Table 1), the incremental learning-based methods have better prediction accuracy, which is increased by 10.25%. This may be due to that the prediction information is used more fully by incremental learning algorithm.


### 4.2. Application Experiment and Results Analysis

In the application experiment, the experimental sample time series with 2000 sample data values, which come from a certain complex electronic system, shown in Figure3.Figure 3
Sample time series of a certain complex electronic system.We set the first 1000 time series data as training samples, and any continuous 11 are taken as a sample, where the data of the first 10 data compose an input sample vector and last one as the output vector, that is, in the application experiment, we have 990 training data. And then we predict the no. 1001 to no. 2000 time series data using the trained prediction model. The test is also repeated 100 times.The application experiment is executed using the proposed method presented in Section3 (called Proposed Method) and traditional incremental learning based LS-SVR method (called Traditional Method). The prediction performance is measured by training time (TrTime), prediction time (PrTime), and prediction mean RMSE (PrMR). The results are shown in Figures 4 and 5 and Table 3. In addition, we also utilize traditional gridding search method to joint optimized all the parameters, and the search rang of c and σ2 is [0.1, 200].Table 3
Prediction results of application experiment.
Method
TrTime/(s)
PrTime/(s)
PrMR
The traditional method
1.2921
0.2208
3.1941
The proposed method
0.8245
0.2289
2.0963Figure 4
Prediction results of the application experiment.Figure 5
Prediction error of the application experiment.Figures4 and 5 and Table 3 show that the proposed method has better prediction accuracy with lower computing time. This may be due to that we modify the LS-SVR model to reduce the parameters of the model, and the data quality and regularity are improved by AGO. Thus, the proposed method maybe has higher application value.


## 4.1. Simulation Experiments and Results Analysis

In order to validate the performance of the proposed AGO-based method described in Section3.1 and the modified LS-SVR model-based incremental learning described in Section 3.2, we perform two simulation experiments. In the experiments, the sample time series data of variable x has 75 sample data values, which come from one complex avionics system (shown in Figure 1, omit dimension).Figure 1
Sample time series of a certain avionics system.The data points from 1 to 50 in time series are taken as 45 initial training sample data. The first sample data set consists of points 1 to 6, with the first 5 as the input sample vector and the 6th point as the output. The second sample data set consists of points 2 to 7, with the point 2 through 6 as input sample vector and 7th point as the output. This way we have 45 training data out of the first 50 data points. And then we predict the no. 51 to no. 75 time series data using the trained model. The performance is measured by the prediction mean RMSE (PrMR), training time (TrTime), and prediction time (PrTime). The experiments are repeated 100 times and the average results are obtained.In the simulation Experiment I, we compare the proposed AGO-based method with traditional LS-SVR presented in [4]. Here, Gaussian RBF kernel k(x,y)=exp(-∥x-y∥2/2σ2) is adopted as kernel function, and the parameters are jointly optimized with traditional gridding search method, where the search rang of c and σ2 is [10,3000]. The prediction results are shown in Figure 2 (ALS-SVR presents the proposed AGO-based method in the figure) and Table 1.Table 1
Prediction results of Experiment  I.
Method
TrTime/(s)
PrTime/(s)
PrMR
Traditional LS-SVR in [4]
0.0632
0.0200
2.4411
Proposed AGO-based method in Section3.1
0.0706
0.0209
2.1569Figure 2
Prediction results ofx.From Table1 and Figure 2, we can see that the proposed AGO-based method has better prediction accuracy. This is may be due to that the proposed AGO-based method can avoid the random disturbances existing in the raw time series and improve the regularity of the time series.In simulation Experiment II, we compare the two incremental learning-based methods: one is the proposed modified LS-SVR model, the other one is the traditional LS-SVR model. The other selections are the same as Experiment I. The prediction results are reported in Table2.Table 2
Prediction results of Experiment  II.
Incremental learning-based method
TrTime/(s)
PrTime/(s)
PrMR
Traditional LS-SVR model
0.1872
0.0206
2.2142
Proposed modified LS-SVR model in Section3.2
0.1186
0.0205
2.2142The results in Table2 show that the modified LS-SVR model can reduce the computing time more with the same prediction accuracy as the traditional LS-SVR model. Compared with the results of traditional LS-SVR in [4] (shown in Table 1), the incremental learning-based methods have better prediction accuracy, which is increased by 10.25%. This may be due to that the prediction information is used more fully by incremental learning algorithm.


## 4.2. Application Experiment and Results Analysis

In the application experiment, the experimental sample time series with 2000 sample data values, which come from a certain complex electronic system, shown in Figure3.Figure 3
Sample time series of a certain complex electronic system.We set the first 1000 time series data as training samples, and any continuous 11 are taken as a sample, where the data of the first 10 data compose an input sample vector and last one as the output vector, that is, in the application experiment, we have 990 training data. And then we predict the no. 1001 to no. 2000 time series data using the trained prediction model. The test is also repeated 100 times.The application experiment is executed using the proposed method presented in Section3 (called Proposed Method) and traditional incremental learning based LS-SVR method (called Traditional Method). The prediction performance is measured by training time (TrTime), prediction time (PrTime), and prediction mean RMSE (PrMR). The results are shown in Figures 4 and 5 and Table 3. In addition, we also utilize traditional gridding search method to joint optimized all the parameters, and the search rang of c and σ2 is [0.1, 200].Table 3
Prediction results of application experiment.
Method
TrTime/(s)
PrTime/(s)
PrMR
The traditional method
1.2921
0.2208
3.1941
The proposed method
0.8245
0.2289
2.0963Figure 4
Prediction results of the application experiment.Figure 5
Prediction error of the application experiment.Figures4 and 5 and Table 3 show that the proposed method has better prediction accuracy with lower computing time. This may be due to that we modify the LS-SVR model to reduce the parameters of the model, and the data quality and regularity are improved by AGO. Thus, the proposed method maybe has higher application value.


## 5. Conclusions

In this paper, we address the issue of adaptive online prediction method based on LS-SVR model. We utilize two approaches to gain better prediction accuracy. Firstly, we make the accumulated generating operation (AGO) with raw time series to reform the quality of raw time series, which avoids the random disturbances and improves the regularity of the time series. Secondly, we modify the traditional LS-SVR model with simple form to simplify incremental learning, which helps to reduce the computing time in the process of adaptive online training and prediction.We conduct three prediction experiments to demonstrate the effectiveness of the propose method. The results show that the prediction method has better prediction performance, and it is suitable for the adaptive online prediction in practice.



---

*Source: 985930-2012-12-17.xml*