# Application of Chaos and Neural Network in Power Load Forecasting

Authors: Li Li; Liu Chong-xin
Journal: Discrete Dynamics in Nature and Society
 (2011)

---

## Abstract

This paper employs chaos theory into power load forecasting. Lyapunov exponents on chaos theory are calculated to judge whether it is a chaotic system. Delay time and embedding dimension are calculated to reconstruct the phase space and determine the structure of artificial neural network (ANN). Improved back propagation (BP) algorithm based on genetic algorithm (GA) is used to train and forecast. Finally, this paper uses the load data of Shaanxi province power grid of China to complete the short-term load forecasting. The results show that the model in this paper is more effective than classical standard BP neural network model.


---

## Body

## 1. Introduction

Chaos theory is the important component of the nonlinear science [1]. It is the random phenomena which appeared in the deterministic nonlinear dynamic system. Chaos is not a disorder but has a delicate inner structure. It reveals the order and regularity hidden behind the disordered and complex phenomena. Since the 90s, chaos theory has been well developed. Many subjects are infiltrated and promoted [2] under this tendency. So the research on chaos gets an access to a breakthrough. At the meanwhile the application about chaos theory gets a widely growing.Short-term power load forecasting is a multidimensional nonlinear system. It is easy to get the load time series in power system. But these data are nonlinear and difficult to establish a matched mathematical model to forecast the next-hour load. Recently, more and more nonlinear time series forecasting models based on chaos theory [3, 4] are applied to power short-term load forecasting. And they have achieved good prediction results. So chaos theory is employed to analyze the characters of the load time series and applied into the power system forecasting in this paper.There are many models to be adopted into power system load forecasting. They can generally be summarized as follows: time series model, regression model, expert system model, grey theory model, and fuzzy logic model. But according to chaotic characters of the load time series, ANN [5–9] is established and applied into the power system well. In view of neural network parallel processing and powerful nonlinear mapping ability, chaotic time series can be learned for unknown dynamic system and then predicted and controlled. As chaotic time series have an inner deterministic regularity which stems from nonlinearity, it represents the relevance of the time series on time delay state space. The feature makes the system have some kind of memory capacity. However it is difficult to express the regularity with the common analytical methods. So chaos theory and ANN are deserve to be studied and applied into power load forecasting.Many intrinsic deficiencies of ANN are still in existence. The structure is difficult to confirm. The blindness that initial weights are chosen results into slow convergence speed and easily falling into local minimum. However GA [10–12] has the opposite characteristics of neural network. So this paper introduces GA to overcome these deficiencies of ANN.


## 2. Chaos Theory

Power system loads are a set of time series. Chaos theory can analyze chaotic characteristics of time series and reveal the sequence itself of the objective regularities to avoid the predicted human subjectivity and improve the accuracy and credibility of load forecasting.At present, phase-space delay coordinate reconstitution method is employed to analyze chaotic characteristics of time series. Generally, the dimension is very great even infinite. In fact, phase-space delay coordinate reconstitution method can expand the given time series to three-dimensional and even higher-dimensional space, and the information which exposed sufficiently from time series can be classified and extracted.


### 2.1. Phase-Space Reconstitution

Phase-space reconstitution theory is the basis for chaotic time series forecasting. Packard and Takens proposed the technology of phase-space reconstitution for chaotic time series{x(1),x(2),…,x(n)} with the interval Δt in electric power system. The structural character of system attractors is contained in this time series. Then the time series can be marked as(2.1)X(i)=[x(i),x(i+τ),…,x(i+(m-1)τ)],i=1,2,…,M;M=n-(m-1)τ,
where M is the number of phase points of reconstructed phase-space. τ is the delay time, τ=kΔt(k=1,2,…). m is the embedding dimension, that is, the dimension of phase-space reconstitution.Takens proved that phase-space reconstitution through selecting an appropriate delay timeτ and a sufficiently large embedding dimension m has the same with the actual geometric and information properties of dynamical systems and does not depend on the specific details of the reconstitution process. Thus, these M=n-(m-1)τ phase points continuations which are reconstituted in the m-dimensional phase-space form the trajectory of the input signal data. And the trajectory describes the evolutionary trace of the system state over time, and all fall to the same chaotic attractor of phase space.


### 2.2. Lyapunov Exponents

Chaos is characterized by extreme sensitivity to movement on the initial conditions. Lyapunov exponents quantify the exponential divergence of initially close state-space trajectories and estimate the amount of chaos in a system. When the largest Lyapunov exponent [13] of the system is larger than zero (λ1>0), it indicates that there is the chaotic attractor which can be used to measure the chaotic degree.M. T. Rosenstin, J. J. Collins, and G. J. Deluca proposed an approach of small data sets. This method is more reliable, with a smaller calculation and easier to operate than others. So the largest Lyapunov exponent is computed by this approach. The process is as follows.(1)
For{x(1),x(2),…,x(n)}, compute delay time τ and mean period P by FFT.(2)
Figure out correlation dimensiond and determine the embedding dimension m through m≥2d+1.(3)
Reconstruct the phase-space based onτ and m.(4)
Find out nearest neighborYĵ of each point Yj, and limit short-term separation,
(2.2)dj(0)=min⁡Yĵ‖Yj-Yĵ‖,|j-ĵ|>P.(5)
Calculate the distance afteri discrete time step for each pair of neighbors
(2.3)dj(i)=|Yj+i-Yĵ+i|,i=1,2,…,min⁡(M-j,M-ĵ).(6)
Compute by the follow formula:(2.4)y(i)=1qΔt∑j=1qln⁡dj(i),
where q is the number of nonzero. Make regression line with the method of least squares,  and the slope of this line is the largest Lyapunov exponent λ1.


## 2.1. Phase-Space Reconstitution

Phase-space reconstitution theory is the basis for chaotic time series forecasting. Packard and Takens proposed the technology of phase-space reconstitution for chaotic time series{x(1),x(2),…,x(n)} with the interval Δt in electric power system. The structural character of system attractors is contained in this time series. Then the time series can be marked as(2.1)X(i)=[x(i),x(i+τ),…,x(i+(m-1)τ)],i=1,2,…,M;M=n-(m-1)τ,
where M is the number of phase points of reconstructed phase-space. τ is the delay time, τ=kΔt(k=1,2,…). m is the embedding dimension, that is, the dimension of phase-space reconstitution.Takens proved that phase-space reconstitution through selecting an appropriate delay timeτ and a sufficiently large embedding dimension m has the same with the actual geometric and information properties of dynamical systems and does not depend on the specific details of the reconstitution process. Thus, these M=n-(m-1)τ phase points continuations which are reconstituted in the m-dimensional phase-space form the trajectory of the input signal data. And the trajectory describes the evolutionary trace of the system state over time, and all fall to the same chaotic attractor of phase space.


## 2.2. Lyapunov Exponents

Chaos is characterized by extreme sensitivity to movement on the initial conditions. Lyapunov exponents quantify the exponential divergence of initially close state-space trajectories and estimate the amount of chaos in a system. When the largest Lyapunov exponent [13] of the system is larger than zero (λ1>0), it indicates that there is the chaotic attractor which can be used to measure the chaotic degree.M. T. Rosenstin, J. J. Collins, and G. J. Deluca proposed an approach of small data sets. This method is more reliable, with a smaller calculation and easier to operate than others. So the largest Lyapunov exponent is computed by this approach. The process is as follows.(1)
For{x(1),x(2),…,x(n)}, compute delay time τ and mean period P by FFT.(2)
Figure out correlation dimensiond and determine the embedding dimension m through m≥2d+1.(3)
Reconstruct the phase-space based onτ and m.(4)
Find out nearest neighborYĵ of each point Yj, and limit short-term separation,
(2.2)dj(0)=min⁡Yĵ‖Yj-Yĵ‖,|j-ĵ|>P.(5)
Calculate the distance afteri discrete time step for each pair of neighbors
(2.3)dj(i)=|Yj+i-Yĵ+i|,i=1,2,…,min⁡(M-j,M-ĵ).(6)
Compute by the follow formula:(2.4)y(i)=1qΔt∑j=1qln⁡dj(i),
where q is the number of nonzero. Make regression line with the method of least squares,  and the slope of this line is the largest Lyapunov exponent λ1.


## 3. Chaos Forecast Improved by ANN

Kolmogorov theory supposes ifΦ(x) is a nonconst and monotonically increasing function. M is compact subset of Rn.f(x)=f(x1,x2,…,xn) is continuous real-valued function of M. So for every ε>0, should there be N,Ci,θi(i=1,2,…,N) and ωij(i,j=1,2,…,n) to make(3.1)f̂(x1,x2,…,xn)=∑i=1NCiΦ(∑j=1nωijxj-θi)
meet(3.2)max⁡M|f̂(x1,x2,…,xn)-f(x1,x2,…,xn)|<ε.That is to say, there is a three-layer network of which hidden-layer function isΦ(x), input and output functions are linear. The network’s input-output relationship f̂(x1,x2,…,xn) can approach f(x1,x2,…,xn). Kolmogorov theory pledges the feasibility of neural network used in time series forecast on mathematics. Based on Kolmogorov and chaos theory, ANN can be designed as follows (BP is used in the paper).Chaos neural network in Figure1 contains three layer: one input layer, one hidden layer (sometime maybe more than one), one output layer X(1)=x(t),X(2)=x(t+τ),X(m)=x(t+(m-1)τ). This model has m input nodes, n hidden nodes, and one output node. Connection weights between input layer and hidden layer are marked as Wij and between hidden layer and output layer are as Tj. xi,yj,z stand for the nodes of input, hidden and output. So Figure 1 can use the following formulas to describe(3.3)x[i]=x(t+(i-1)τ),i=1,2,…,m,y[j]=f(∑i=1mWijx[i]),j=1,2,…,n,z=f(∑j=1nTjy[j]),wheref is the Sigmoid function. That is, f(x)=1/(1+e-x).Figure 1
Chaos neural network base model.In order to get the better weights and thresholds and determine the structure of BP neural network, GA can be drawn in to improve the forecasting accuracy of chaotic time series.


## 4. Improved GA Optimizes ANN

There are some deficiencies of BP neural network, such as a lower pace, being easy to local minimum, and the uncertainty structure. But GA can overcome these and improve network performance and convergence rate and optimize chaos neural network further.The methods and steps to achieve genetic algorithm and optimize chaos neural network are the following.(1) Determine the Initial Population
After analyzing the chaotic characters of the time series, we get a data set of the weights and thresholds that are unknown. These data are encoded and made as an individual. Several individuals can constitute the initial population.(2) Calculate the Fitness
Supposee is the error of the network. The fitness function is taken as like
(4.1)f=1e.
So the smaller the error is, the greater the fitness is.(3) Selecting Operation
Calculate the fitness of every individualfi. Then copy them according to the following formula pi:
(4.2)pi=fi∑fi,pi reflects the individual is copied the probability to the next generation. Use the following formula R̅i to select how many individuals could be generated:
(4.3)R̅i=fif̅i,
where f̅i is the average of fi.R̅i is rounded and marked as Ri. Ri reflects the individual is copied the times to the next generation. If Ri=0, it shows this individual is gone out and not to generate.(4) Crossover Operation
Crossover operation is used to enhance the global search ability of GA. After selecting operation, randomly two individuals are selected to match but avoid choosing the individuals of the same gene. Randomly select a cross point of each individual, and according to the probabilitypc change the gene of two cross-points to form two new individuals.(5) Mutation Operation
Mutation operation is mainly used to enhance local search ability of GA. After crossover operation, randomly select a mutation point to change the code according to the probabilitypm. If 1, change into 0. Otherwise, it is the opposite.(6) Optimization Operation
In order to overcome precocity of GA, the improvement is necessary to operate. The parent and the offspring individuals are looked as a whole to form2n individuals. Order them in the light of the fitness, and k(k=0,1,…,2n-1) is used to mark the kth individual. So the kth replication probability is determined in the way of the following formula:
(4.4)p(k)={(n-1)-kn-1,k=0,1,…,n-1,k-nn,k=n,n+1,…,2n-1.(7) Iteration Operation
For the new population, repeat the previous operation until the relative error between two iteration operations meets the accuracy requirements. Error formula is the following formula:(4.5)E(fk+1,fk)=max⁡i(fik+1)-max⁡i(fki)max⁡i(fki)<ε(k=0,1,…,i=1,2,…,n),
where E(fk+1,fk) is the relative error between two iterations; maxi(fki) and maxi(fik+1) are separately the fitness of every individual in the kth and (k+1)th iteration; ε is the given standard, and here ε=0.001. If E<ε, iteration operation will finish and the optimal solution be outputted.
On the basis of the above, GA is just like the diagram in Figure2.Figure 2
Genetic algorithm flow diagram.


## 5. Application and Analysis

There are some power data from 0:00 at 1/1/2009 to 23:00 at 3/30/2009 in Shaanxi province power grid of China to use as the sample data for short-term power load forecasting. Then the data can be established into the time series{x(t1),x(t2),…,x(tn)}. It is described in Figure 3.Figure 3
The load time series.The time series are analyzed by chaos theory. Then there are some parameters to determine. Delay timeτ=5 can be got through mutual information [14]. Embedding dimension m=5 can be calculated by Cao method [15]. Use τ and m to reconstruct the phase space. Then wolf method [16] is employed to compute the largest Lyapunov exponent λ1=0.0225. It is obvious λ1>0 that power load time series is chaotic.SinceT=1/λ1=1/0.0225=44.44h=1.85d, it complies with the requirements of short-term electric power load. The graphics of the three parameters are in Figure 4.The graphics of the three parameters.
(a)
Calculate delay time by mutual information(b)
Calculating embedding dimension by Cao method(c)
Calculating the largest Lyapunov exponent by wolf methodWhen it got the parameters, the phase space can be reconstructed in Figure5.Phase-space reconstitution.
(a)
x-y phase diagram(b)
y-z phase diagram(c)
x-z phase diagram(d)
x-y-z phase diagramFigure5 shows that power load time series are chaotic. From the phase diagrams, there is one out prominent line. It means that the power loads are affected by accidental factors. However the power load time series are in the chaotic state.After completing phase-space reconstitution, the data that come from 0:00 at 3/31/2009 to 23:00 at 3/31/2009 in Shaanxi province power grid of china are used as the testing sample. The structure of BP neural network is employed 4-9-1. Relative error and root-mean-square relative error are used as the final evaluating indicators:(5.1)E=y-xx×100%,RMSER=1n∑1n(yt-xtxt)2.The results are in Table1. It compares the method in this paper with standard BP neural network method which takes the continuous five data as the training samples without improved BP.Table 1
Comparison of the forecasting results and errors by the different methods.
Time (h)Real load (MW)This paper’s methodStandard BP network methodResults (MW)Error (%)Results (MW)Error (%)06416.696482.811.036554.612.1516111.526185.501.216329.723.5726044.526098.350.896206.502.6835998.766041.460.716031.260.5445813.065833.420.356073.574.4855945.275852.57−1.565823.41−2.0566195.156163.60−0.516075.02−1.9476863.367001.312.016956.711.3687232.717347.721.597457.663.1197781.657879.761.267949.742.16107847.127910.780.817974.201.62118000.548154.241.928151.801.89127756.467882.101.627614.51−1.83137154.437055.72−1.386967.08−2.62147340.187263.15−1.057077.44−3.58157467.577588.561.627344.41−1.65167513.927577.810.857586.820.97177856.267963.151.367955.231.26187862.167871.630.127831.54−0.39197891.197914.190.297776.06−1.46208487.148406.51−0.958617.811.54218180.108220.220.498362.522.23227476.117570.301.267682.502.76236416.696347.42−1.086632.333.36RSM////1.080//2.133Table1 shows that standard BP neural network method for RMSER is 2.133% but the method in this paper is 1.080% and most of the latter’s errors are in less than 2%. So the latter has much less errors than the former and superior to the latter.Standard BP neural network method just uses the historical load and the uncertain structure of BP neural network to forecast the next-hour load. Added the deficiencies of BP, the results could not be very accurate and the speed not so quick. But the method in this paper takes the chaotic characters of the power load time series, appropriately determining the structure of input layer and hidden layer in BP network into account. So this can make the network training well and improve the prediction accuracy further obviously.Above all, Figure6 shows the different methods, containing the results and the errors curves.The different methods forecasting comparison.
(a)
The results of the different methods(b)
The errors of the different methodsAs can be seen from the graphics in Figure6, it shows that the curve of the method in this paper can better fit real load curve than standard BP. The errors computed by the former are closer zero than by the latter. So for short-term power load forecasting, the model in this paper is more effective than classical standard BP neural network model.


## 6. Conclusion

This paper employs the chaos theory into eclectic system load forecasting. From Lyapunov exponents to phase-space reconstitution, it tells that power load time series are nonlinear and chaotic and have all the characters of chaos. The chaotic phase diagram also shows there is a chaos attractor. So chaos theory introduced into power load forecasting can describe the nonlinear dynamic behavior of the system and get the accuracy and the precision improved greatly.Meanwhile, BP neural network under being improved by GA based on chaos theory is used for forecasting to improve the accuracy and the training rate further. In the last, applied into Shaanxi province power grid short-term load forecast of China and compared with the standard BP neural network model, the method mentioned  in this paper gets more accurate results and more efficient training rate. So this way can bring a broad application prospects in power load forecasting.



---

*Source: 597634-2011-08-14.xml*