# Short-Term Load Forecasting Model Based on Quantum Elman Neural Networks

Authors: Zhisheng Zhang; Wenjie Gong
Journal: Mathematical Problems in Engineering
 (2016)

---

## Abstract

Short-term load forecasting model based on quantum Elman neural networks was constructed in this paper. The quantum computation and Elman feedback mechanism were integrated into quantum Elman neural networks. Quantum computation can effectively improve the approximation capability and the information processing ability of the neural networks. Quantum Elman neural networks have not only the feedforward connection but also the feedback connection. The feedback connection between the hidden nodes and the context nodes belongs to the state feedback in the internal system, which has formed specific dynamic memory performance. Phase space reconstruction theory is the theoretical basis of constructing the forecasting model. The training samples are formed by means ofK-nearest neighbor approach. Through the example simulation, the testing results show that the model based on quantum Elman neural networks is better than the model based on the quantum feedforward neural network, the model based on the conventional Elman neural network, and the model based on the conventional feedforward neural network. So the proposed model can effectively improve the prediction accuracy. The research in the paper makes a theoretical foundation for the practical engineering application of the short-term load forecasting model based on quantum Elman neural networks.


---

## Body

## 1. Introduction

Short-term load forecasting (STLF) is the basis for the normal and safe operation of power system. Since the introduction of competition mechanism in power market, the power company has put forward higher requirements on the accuracy and rapidity for load forecasting [1–3]. Many classical methods were applied to the study of STLF model, such as ARIMA model and ARMAX model [4, 5]. However, the ability of these classical methods to deal with the nonlinear time series is poor. With the introduction of artificial intelligence technology to the study of STLF model, it has opened up many new research methods such as neural network and expert system [6, 7]. Because of the strong nonlinear time series processing ability, neural network is widely used in the study of STLF.The process of human brain information processing may be related to the quantum phenomenon, and quantum mechanical effects may exist in the brain, which had been shown by some research results [8]. The quantum systems have the similar dynamic characteristics with the biological neural networks. The fusion of artificial neural network and quantum theory can better simulate the process of human brain information processing. Quantum neural network can improve the approximation ability and information processing efficiency of artificial neural network, which had been used for many applications [9–13].The STLF model based on quantum Elman neural networks (QENN) is constructed in this paper. QENN is composed of the quantum neurons. The quantum neuron model consists of the weighting part, the aggregating part, the activating part, and the excitation part. Different from the traditional multilayer feedforward neural network, QENN has not only the feedforward connection but also the Elman feedback connection. The Elman feedback connection between the hidden nodes and the context nodes belongs to the state feedback in the internal system, which has formed specific dynamic memory performance [14].Some studies had shown that the load time series is nonlinear and chaotic [15]. Phase space reconstruction theory (PSRT) is the important analysis method for research on chaotic time series. The training samples are formed by means of K-nearest neighbor approach (KNNA) based on PSRT.Through the actual example simulation, it is proved that the proposed model can effectively improve the prediction accuracy and has adaptability to different load time series.


## 2. Quantum Elman Neural Networks

QENN is composed of quantum neurons, and quantum neurons are the basic elements of building QENN.


### 2.1. Quantum Neuron Model

#### 2.1.1. Qubit

In the classical computation, the binary numbers “0” and “1” are used to represent the information, and they are usually called bits. In the quantum computation,0 and 1 are used to represent the two basic states of microscopic particles, and they are usually called qubits. The difference between qubits and bits is that, in addition to the state of 0 and 1, the state of qubits can be a linear combination of states, which is usually called the superposition.In quantum computation, a qubit stateϕ can be expressed as(1)
ϕ
=
α
0
+
β
1
,where 0 and 1 express the basic states of microscopic particles. α and β are the probability amplitudes, and they should meet the following relations: (2)
α
2
+
β
2
=
1
.In order to further understand the superposition of the qubits, an electron is used to illustrate. An electron can be in the ground state|
0
〉 of low energy level, and it can also be in the excitation state |
1
〉 of high energy level. According to the superposition principle, the electron can be in the state of linear combination of these two states, that is, the superposition state α
|
0
〉
+
β
|
1
〉. It is shown in Figure 1.Figure 1
Schematic diagram of superposition state of ground State|
0
〉 and excitation state |
1
〉.If the electron is in the superposition stateα
|
0
〉
+
β
|
1
〉, the electron can select the ground state |
0
〉 according to the probability α
2 and select the excitation state |
1
〉 according to the probability β
2. It is shown in Figure 2.Figure 2
Schematic diagram of the ground state and excitation state after the superposition state is collapsing.


#### 2.1.2. Quantum Neuron Model

The quantum neuron model adopted in the paper consists of the weighting part, the aggregating part, the activating part, and the excitation part. The weighting part is the simulation of the bond strength between synapses of nerve cells. The aggregating part is the simulation of the spatial and temporal integration of stimuli received by multiple dendrites. The activating part is the simulation of the interaction between the current activation values and membrane potential change of nerve cells. The excitation part is the simulation of the nonlinear characteristics of nerve cells, such as excitation, inhibition, fatigue, refractory period, and threshold. According to [16], the quantum behavior of neural cells is mainly reflected in the neurotransmitter transmission of the cells. Transmitter release from nerve terminals in the form of small packages, with multiple molecules for biological neurons, and the small package is called quantum. A small package or quantum describes the smallest unit of transmitter secretion. In general, the quantum content of a synaptic response may be different, but the number of molecules in a quantum is relatively certain; that is, the quantum size is relatively determined. Since 1990s, some research results show that the process of human brain information processing may be related to the quantum phenomenon, and quantum mechanical effects may exist in the brain, and the quantum systems have the similar dynamic characteristics with the biological neural networks. So combining artificial neural network with quantum theory can better simulate the process of human brain information processing.The quantum neuron model is shown in Figure3. Considering the actual prediction problem, the input vector X
=
(
x
1
,
x
2
,
…
,
x
n
)
T is real-valued vector, and the output y is the real number.Figure 3
Quantum neuron model.The weight and activation values are qubits.|
ϕ
〉
=
(
|
ϕ
1
〉
,
|
ϕ
2
〉
,
…
,
|
ϕ
n
〉
)
T is the weight vector, and |
ϕ
i
〉 is the weight between the input x
i and the quantum neuron. |
ϕ
〉 can be described as(3)
ϕ
=
ϕ
1
,
…
,
ϕ
n
T
=
cos
⁡
θ
1
sin
⁡
θ
1
,
…
,
cos
⁡
θ
n
sin
⁡
θ
n
T
.φ is the activation value, which can be described as(4)
φ
=
cos
⁡
ξ
,
sin
⁡
ξ
T
.The activation values are the important parameters for the activating part. The simulation of the interaction between the current activation values and membrane potential change of nerve cells is the core of the activating part. The activation values affect the accuracy of prediction. The parameters can be optimized by genetic algorithm.In Figure1, ∑ represents the aggregation operator, and F represents the activation function. f is the excitation function, which adopts theSigmoid function, and it can be described as(5)
f
x
=
1
1
+
e
-
x
.The relationship between input and output can be expressed as(6)
y
=
f
F
X
T
ϕ
,
φ
-
τ
=
f
∑
i
=
1
n
x
i
ϕ
i
∣
φ
-
τ
=
f
∑
i
=
1
n
x
i
cos
⁡
θ
i
-
ξ
-
τ
,where τ is the threshold.


### 2.2. Quantum Elman Neural Networks Model

Considering Figure4, QENN has four layers which are the input layer, the hidden layer, the context layer, and the output layer. The hidden layer and the output layer are constructed by quantum neurons.Figure 4
Quantum Elman neural networks.For QENN, each node in the hidden layer has a corresponding node connected to it in the context layer.The input nodes and the hidden nodes, the hidden nodes and the output nodes, and the context nodes and the hidden nodes are connected by adjustable weights.Compared with the feedforward network structure, QENN has the context layer. The role of the nodes in the context layer is to store the internal state of neural network. They are used to store the state of the hidden nodes at the current time, and the state will be passed to the hidden nodes at the next time. The connection between the hidden nodes and the context nodes belongs to the state feedback in the internal system, which has formed specific dynamic memory performance.QENN can not only store the information in the current input data but also store some historical information. In the training process, it can dynamically backtrack to the input at the past time related to the expected output at the current time.n, m are the number of the input nodes and the hidden nodes, respectively. The number of the context nodes is equal to the number of the hidden nodes. There is one output node. The number of iterations is denoted as k
(
k
=
1,2
,
…
,
T
). ϕ
i
j is the weight between ith input node and jth hidden node, and it can be described as(7)
ϕ
i
j
=
cos
⁡
θ
i
j
sin
⁡
θ
i
j
.ϕ
q
j
′ is the weight between qth context node and jth hidden node, and it can be described as(8)
ϕ
q
j
′
=
cos
⁡
θ
q
j
′
sin
⁡
θ
q
j
′
.ϕ
j
k
′
′ is the weight between jth hidden node and kth output node, and it can be described as(9)
ϕ
j
k
′
′
=
cos
⁡
θ
j
k
′
′
sin
⁡
θ
j
k
′
′
.x
i
(
k
) is the input value of ith external input neuron at time k.S
q
C
(
k
) is the internal state of qth context neuron at time k, which is the output of qth hidden neuron at time k
-
1, and it can be expressed as(10)
S
q
C
k
=
y
q
H
k
-
1
,where S
q
C
(
k
) is the output of qth hidden neuron at time k
-
1.y
q
C
(
k
) is the output of qth context neuron at time k, and it can be expressed as(11)
y
q
C
k
=
S
q
C
k
.S
j
H
(
k
) is the internal state of jth hidden neuron at time k, which can be expressed as(12)
S
j
H
k
=
∑
i
=
1
n
x
i
k
ϕ
i
j
∣
φ
j
+
∑
q
=
1
m
y
q
C
k
ϕ
q
j
′
∣
φ
j
-
τ
j
=
∑
i
=
1
n
x
i
k
cos
⁡
θ
i
j
-
ξ
j
+
∑
q
=
1
m
y
q
C
k
cos
⁡
θ
q
j
′
-
ξ
j
-
τ
j
,where φ
j is the activation value of jth hidden neuron, which can be described as(13)
φ
j
=
cos
⁡
ξ
j
,
sin
⁡
ξ
j
T
.y
j
H
(
k
) is the output of jth hidden neuron at time k, which can be expressed as(14)
y
j
H
k
=
f
S
j
H
k
.S
O
(
k
) is the internal state of the output neuron at time k, and it can be expressed as(15)
S
O
k
=
∑
j
=
1
m
y
j
H
k
ϕ
j
k
′
′
∣
φ
k
′
-
τ
k
′
=
∑
j
=
1
m
y
j
H
k
cos
⁡
θ
j
k
′
′
-
ξ
k
′
-
τ
k
′
,where φ
k
′ is the activation value of kth output neuron, which can be described as(16)
φ
k
′
=
cos
⁡
ξ
k
′
,
sin
⁡
ξ
k
′
T
.y
O
(
k
) is the output of the output neuron at time k, which can be expressed as(17)
y
O
k
=
f
S
O
k
.


### 2.3. Multiplicity of Convergent Solution for QENN

According to (10)–(17), y
O
(
k
) can also be expressed as(18)
y
O
k
=
f
∑
j
=
1
m
f
∑
i
=
1
n
x
i
k
cos
⁡
θ
i
j
-
ξ
j
+
∑
q
=
1
m
y
q
H
k
-
1
·
cos
⁡
θ
q
j
′
-
ξ
j
-
τ
j
cos
⁡
θ
j
k
′
′
-
ξ
k
′
-
τ
k
′
.Ifθ
∗ and ξ
∗ are the convergent solution of the sequence θ and ξ, θ
^
∗ and ξ
^
∗ which satisfy θ
^
∗
-
ξ
^
∗
=
±
(
θ
∗
-
ξ
∗
) are the convergent solution of the sequence θ and ξ.This shows that many groups ofϕ and φ are the convergent solution of the sequence ϕ and φ. It only requires the argument of |
ϕ
〉 and |
φ
〉 to satisfy a certain relation, and the specific value of the argument is not required. This ensures the multiplicity of convergent solution for QENN, which is the basis of improving the approximation capability of the neural networks and making its performance enhanced in the application to a short-term load prediction of QENN.


### 2.4. Parameters Optimization of QENN by Genetic Algorithm

Genetic algorithm is used to adjust the parameters of the QENN, which include(19)
θ
i
j
i
=
1
,
…
,
n
;
j
=
1
,
…
,
m
;
θ
q
j
′
q
=
1
,
…
,
m
;
j
=
1
,
…
,
m
;
θ
j
k
′
′
j
=
1
,
…
,
m
;
k
=
1
;
ξ
j
j
=
1
,
…
,
m
;
ξ
k
′
k
=
1
;
τ
j
j
=
1
,
…
,
m
;
τ
k
′
k
=
1
.The chromosome can be constituted by these parameters, which can be expressed asZ
=
[
θ
11
,
θ
12
,
…
,
θ
n
m
,
θ
11
′
,
θ
12
′
,
…
,
θ
m
m
′
,
θ
11
′
′
,
θ
21
′
′
,
…
,
θ
m
1
′
′
,
ζ
1
,
ξ
2
,
…
,
ξ
m
,
ξ
1
′
,
τ
1
,
τ
2
,
…
,
τ
m
,
τ
1
′
]
.Genetic algorithm uses the roulette method as selecting operator. The crossover operator adopts the nonuniform arithmetic crossover method, and the mutation operator uses the noncoherence mutation method. The fitness function mainly considers the accuracy of the output of the forecasting model; that is,f
fitness
=
1
/
f
1
(
t
). f
1
(
t
) is the mean square deviation between the desired output and the actual output of the forecasting model.The process is shown in Figure5.Figure 5
The process of parameters optimization of QENN by genetic algorithm.


#### 2.4.1. Nonuniform Arithmetic Crossover Operator

The operation object of arithmetic crossover operator is chromosomeZ, which is encoded by floating point numbers.If two chromosome individuals areZ
A
t and Z
B
t, the two new individuals generated by the crossover operation can be expressed as(20)
Z
A
t
+
1
=
α
Z
B
t
+
1
-
α
Z
A
t
,
Z
B
t
+
1
=
α
Z
A
t
+
1
-
α
Z
B
t
.For the nonuniform arithmetic crossover operator,α is the random number, and its value range is from −0.5 to 1.5.


#### 2.4.2. Noncoherence Mutation Operator

Supposing the chromosome individualZ
=
(
z
1
,
z
2
,
…
,
z
n
) is a parent solution, the component z
k is selected to mutate, and its value range is [
a
k
,
b
k
]. The mutated chromosome individual is Z
′
=
(
z
1
,
z
2
,
…
,
z
k
-
1
,
z
k
′
,
…
,
z
n
).z
k
′ can be expressed as (21)
z
k
′
=
z
k
+
Δ
t
,
b
k
-
z
k
if  rnd
2
=
0
z
k
-
Δ
t
,
z
k
-
a
k
if  rnd
2
=
1
,where t is the current evolution generation, and Δ
(
t
,
s
) can be expressed as(22)
Δ
t
,
s
=
s
×
1
-
r
1
-
t
/
T
λ
,where r is a random number and its value range is from 0 to 1. T is the maximal generation. λ is a parameter for the nonuniform arithmetic crossover operator, and its value is 2.5.


## 2.1. Quantum Neuron Model

### 2.1.1. Qubit

In the classical computation, the binary numbers “0” and “1” are used to represent the information, and they are usually called bits. In the quantum computation,0 and 1 are used to represent the two basic states of microscopic particles, and they are usually called qubits. The difference between qubits and bits is that, in addition to the state of 0 and 1, the state of qubits can be a linear combination of states, which is usually called the superposition.In quantum computation, a qubit stateϕ can be expressed as(1)
ϕ
=
α
0
+
β
1
,where 0 and 1 express the basic states of microscopic particles. α and β are the probability amplitudes, and they should meet the following relations: (2)
α
2
+
β
2
=
1
.In order to further understand the superposition of the qubits, an electron is used to illustrate. An electron can be in the ground state|
0
〉 of low energy level, and it can also be in the excitation state |
1
〉 of high energy level. According to the superposition principle, the electron can be in the state of linear combination of these two states, that is, the superposition state α
|
0
〉
+
β
|
1
〉. It is shown in Figure 1.Figure 1
Schematic diagram of superposition state of ground State|
0
〉 and excitation state |
1
〉.If the electron is in the superposition stateα
|
0
〉
+
β
|
1
〉, the electron can select the ground state |
0
〉 according to the probability α
2 and select the excitation state |
1
〉 according to the probability β
2. It is shown in Figure 2.Figure 2
Schematic diagram of the ground state and excitation state after the superposition state is collapsing.


### 2.1.2. Quantum Neuron Model

The quantum neuron model adopted in the paper consists of the weighting part, the aggregating part, the activating part, and the excitation part. The weighting part is the simulation of the bond strength between synapses of nerve cells. The aggregating part is the simulation of the spatial and temporal integration of stimuli received by multiple dendrites. The activating part is the simulation of the interaction between the current activation values and membrane potential change of nerve cells. The excitation part is the simulation of the nonlinear characteristics of nerve cells, such as excitation, inhibition, fatigue, refractory period, and threshold. According to [16], the quantum behavior of neural cells is mainly reflected in the neurotransmitter transmission of the cells. Transmitter release from nerve terminals in the form of small packages, with multiple molecules for biological neurons, and the small package is called quantum. A small package or quantum describes the smallest unit of transmitter secretion. In general, the quantum content of a synaptic response may be different, but the number of molecules in a quantum is relatively certain; that is, the quantum size is relatively determined. Since 1990s, some research results show that the process of human brain information processing may be related to the quantum phenomenon, and quantum mechanical effects may exist in the brain, and the quantum systems have the similar dynamic characteristics with the biological neural networks. So combining artificial neural network with quantum theory can better simulate the process of human brain information processing.The quantum neuron model is shown in Figure3. Considering the actual prediction problem, the input vector X
=
(
x
1
,
x
2
,
…
,
x
n
)
T is real-valued vector, and the output y is the real number.Figure 3
Quantum neuron model.The weight and activation values are qubits.|
ϕ
〉
=
(
|
ϕ
1
〉
,
|
ϕ
2
〉
,
…
,
|
ϕ
n
〉
)
T is the weight vector, and |
ϕ
i
〉 is the weight between the input x
i and the quantum neuron. |
ϕ
〉 can be described as(3)
ϕ
=
ϕ
1
,
…
,
ϕ
n
T
=
cos
⁡
θ
1
sin
⁡
θ
1
,
…
,
cos
⁡
θ
n
sin
⁡
θ
n
T
.φ is the activation value, which can be described as(4)
φ
=
cos
⁡
ξ
,
sin
⁡
ξ
T
.The activation values are the important parameters for the activating part. The simulation of the interaction between the current activation values and membrane potential change of nerve cells is the core of the activating part. The activation values affect the accuracy of prediction. The parameters can be optimized by genetic algorithm.In Figure1, ∑ represents the aggregation operator, and F represents the activation function. f is the excitation function, which adopts theSigmoid function, and it can be described as(5)
f
x
=
1
1
+
e
-
x
.The relationship between input and output can be expressed as(6)
y
=
f
F
X
T
ϕ
,
φ
-
τ
=
f
∑
i
=
1
n
x
i
ϕ
i
∣
φ
-
τ
=
f
∑
i
=
1
n
x
i
cos
⁡
θ
i
-
ξ
-
τ
,where τ is the threshold.


## 2.1.1. Qubit

In the classical computation, the binary numbers “0” and “1” are used to represent the information, and they are usually called bits. In the quantum computation,0 and 1 are used to represent the two basic states of microscopic particles, and they are usually called qubits. The difference between qubits and bits is that, in addition to the state of 0 and 1, the state of qubits can be a linear combination of states, which is usually called the superposition.In quantum computation, a qubit stateϕ can be expressed as(1)
ϕ
=
α
0
+
β
1
,where 0 and 1 express the basic states of microscopic particles. α and β are the probability amplitudes, and they should meet the following relations: (2)
α
2
+
β
2
=
1
.In order to further understand the superposition of the qubits, an electron is used to illustrate. An electron can be in the ground state|
0
〉 of low energy level, and it can also be in the excitation state |
1
〉 of high energy level. According to the superposition principle, the electron can be in the state of linear combination of these two states, that is, the superposition state α
|
0
〉
+
β
|
1
〉. It is shown in Figure 1.Figure 1
Schematic diagram of superposition state of ground State|
0
〉 and excitation state |
1
〉.If the electron is in the superposition stateα
|
0
〉
+
β
|
1
〉, the electron can select the ground state |
0
〉 according to the probability α
2 and select the excitation state |
1
〉 according to the probability β
2. It is shown in Figure 2.Figure 2
Schematic diagram of the ground state and excitation state after the superposition state is collapsing.


## 2.1.2. Quantum Neuron Model

The quantum neuron model adopted in the paper consists of the weighting part, the aggregating part, the activating part, and the excitation part. The weighting part is the simulation of the bond strength between synapses of nerve cells. The aggregating part is the simulation of the spatial and temporal integration of stimuli received by multiple dendrites. The activating part is the simulation of the interaction between the current activation values and membrane potential change of nerve cells. The excitation part is the simulation of the nonlinear characteristics of nerve cells, such as excitation, inhibition, fatigue, refractory period, and threshold. According to [16], the quantum behavior of neural cells is mainly reflected in the neurotransmitter transmission of the cells. Transmitter release from nerve terminals in the form of small packages, with multiple molecules for biological neurons, and the small package is called quantum. A small package or quantum describes the smallest unit of transmitter secretion. In general, the quantum content of a synaptic response may be different, but the number of molecules in a quantum is relatively certain; that is, the quantum size is relatively determined. Since 1990s, some research results show that the process of human brain information processing may be related to the quantum phenomenon, and quantum mechanical effects may exist in the brain, and the quantum systems have the similar dynamic characteristics with the biological neural networks. So combining artificial neural network with quantum theory can better simulate the process of human brain information processing.The quantum neuron model is shown in Figure3. Considering the actual prediction problem, the input vector X
=
(
x
1
,
x
2
,
…
,
x
n
)
T is real-valued vector, and the output y is the real number.Figure 3
Quantum neuron model.The weight and activation values are qubits.|
ϕ
〉
=
(
|
ϕ
1
〉
,
|
ϕ
2
〉
,
…
,
|
ϕ
n
〉
)
T is the weight vector, and |
ϕ
i
〉 is the weight between the input x
i and the quantum neuron. |
ϕ
〉 can be described as(3)
ϕ
=
ϕ
1
,
…
,
ϕ
n
T
=
cos
⁡
θ
1
sin
⁡
θ
1
,
…
,
cos
⁡
θ
n
sin
⁡
θ
n
T
.φ is the activation value, which can be described as(4)
φ
=
cos
⁡
ξ
,
sin
⁡
ξ
T
.The activation values are the important parameters for the activating part. The simulation of the interaction between the current activation values and membrane potential change of nerve cells is the core of the activating part. The activation values affect the accuracy of prediction. The parameters can be optimized by genetic algorithm.In Figure1, ∑ represents the aggregation operator, and F represents the activation function. f is the excitation function, which adopts theSigmoid function, and it can be described as(5)
f
x
=
1
1
+
e
-
x
.The relationship between input and output can be expressed as(6)
y
=
f
F
X
T
ϕ
,
φ
-
τ
=
f
∑
i
=
1
n
x
i
ϕ
i
∣
φ
-
τ
=
f
∑
i
=
1
n
x
i
cos
⁡
θ
i
-
ξ
-
τ
,where τ is the threshold.


## 2.2. Quantum Elman Neural Networks Model

Considering Figure4, QENN has four layers which are the input layer, the hidden layer, the context layer, and the output layer. The hidden layer and the output layer are constructed by quantum neurons.Figure 4
Quantum Elman neural networks.For QENN, each node in the hidden layer has a corresponding node connected to it in the context layer.The input nodes and the hidden nodes, the hidden nodes and the output nodes, and the context nodes and the hidden nodes are connected by adjustable weights.Compared with the feedforward network structure, QENN has the context layer. The role of the nodes in the context layer is to store the internal state of neural network. They are used to store the state of the hidden nodes at the current time, and the state will be passed to the hidden nodes at the next time. The connection between the hidden nodes and the context nodes belongs to the state feedback in the internal system, which has formed specific dynamic memory performance.QENN can not only store the information in the current input data but also store some historical information. In the training process, it can dynamically backtrack to the input at the past time related to the expected output at the current time.n, m are the number of the input nodes and the hidden nodes, respectively. The number of the context nodes is equal to the number of the hidden nodes. There is one output node. The number of iterations is denoted as k
(
k
=
1,2
,
…
,
T
). ϕ
i
j is the weight between ith input node and jth hidden node, and it can be described as(7)
ϕ
i
j
=
cos
⁡
θ
i
j
sin
⁡
θ
i
j
.ϕ
q
j
′ is the weight between qth context node and jth hidden node, and it can be described as(8)
ϕ
q
j
′
=
cos
⁡
θ
q
j
′
sin
⁡
θ
q
j
′
.ϕ
j
k
′
′ is the weight between jth hidden node and kth output node, and it can be described as(9)
ϕ
j
k
′
′
=
cos
⁡
θ
j
k
′
′
sin
⁡
θ
j
k
′
′
.x
i
(
k
) is the input value of ith external input neuron at time k.S
q
C
(
k
) is the internal state of qth context neuron at time k, which is the output of qth hidden neuron at time k
-
1, and it can be expressed as(10)
S
q
C
k
=
y
q
H
k
-
1
,where S
q
C
(
k
) is the output of qth hidden neuron at time k
-
1.y
q
C
(
k
) is the output of qth context neuron at time k, and it can be expressed as(11)
y
q
C
k
=
S
q
C
k
.S
j
H
(
k
) is the internal state of jth hidden neuron at time k, which can be expressed as(12)
S
j
H
k
=
∑
i
=
1
n
x
i
k
ϕ
i
j
∣
φ
j
+
∑
q
=
1
m
y
q
C
k
ϕ
q
j
′
∣
φ
j
-
τ
j
=
∑
i
=
1
n
x
i
k
cos
⁡
θ
i
j
-
ξ
j
+
∑
q
=
1
m
y
q
C
k
cos
⁡
θ
q
j
′
-
ξ
j
-
τ
j
,where φ
j is the activation value of jth hidden neuron, which can be described as(13)
φ
j
=
cos
⁡
ξ
j
,
sin
⁡
ξ
j
T
.y
j
H
(
k
) is the output of jth hidden neuron at time k, which can be expressed as(14)
y
j
H
k
=
f
S
j
H
k
.S
O
(
k
) is the internal state of the output neuron at time k, and it can be expressed as(15)
S
O
k
=
∑
j
=
1
m
y
j
H
k
ϕ
j
k
′
′
∣
φ
k
′
-
τ
k
′
=
∑
j
=
1
m
y
j
H
k
cos
⁡
θ
j
k
′
′
-
ξ
k
′
-
τ
k
′
,where φ
k
′ is the activation value of kth output neuron, which can be described as(16)
φ
k
′
=
cos
⁡
ξ
k
′
,
sin
⁡
ξ
k
′
T
.y
O
(
k
) is the output of the output neuron at time k, which can be expressed as(17)
y
O
k
=
f
S
O
k
.


## 2.3. Multiplicity of Convergent Solution for QENN

According to (10)–(17), y
O
(
k
) can also be expressed as(18)
y
O
k
=
f
∑
j
=
1
m
f
∑
i
=
1
n
x
i
k
cos
⁡
θ
i
j
-
ξ
j
+
∑
q
=
1
m
y
q
H
k
-
1
·
cos
⁡
θ
q
j
′
-
ξ
j
-
τ
j
cos
⁡
θ
j
k
′
′
-
ξ
k
′
-
τ
k
′
.Ifθ
∗ and ξ
∗ are the convergent solution of the sequence θ and ξ, θ
^
∗ and ξ
^
∗ which satisfy θ
^
∗
-
ξ
^
∗
=
±
(
θ
∗
-
ξ
∗
) are the convergent solution of the sequence θ and ξ.This shows that many groups ofϕ and φ are the convergent solution of the sequence ϕ and φ. It only requires the argument of |
ϕ
〉 and |
φ
〉 to satisfy a certain relation, and the specific value of the argument is not required. This ensures the multiplicity of convergent solution for QENN, which is the basis of improving the approximation capability of the neural networks and making its performance enhanced in the application to a short-term load prediction of QENN.


## 2.4. Parameters Optimization of QENN by Genetic Algorithm

Genetic algorithm is used to adjust the parameters of the QENN, which include(19)
θ
i
j
i
=
1
,
…
,
n
;
j
=
1
,
…
,
m
;
θ
q
j
′
q
=
1
,
…
,
m
;
j
=
1
,
…
,
m
;
θ
j
k
′
′
j
=
1
,
…
,
m
;
k
=
1
;
ξ
j
j
=
1
,
…
,
m
;
ξ
k
′
k
=
1
;
τ
j
j
=
1
,
…
,
m
;
τ
k
′
k
=
1
.The chromosome can be constituted by these parameters, which can be expressed asZ
=
[
θ
11
,
θ
12
,
…
,
θ
n
m
,
θ
11
′
,
θ
12
′
,
…
,
θ
m
m
′
,
θ
11
′
′
,
θ
21
′
′
,
…
,
θ
m
1
′
′
,
ζ
1
,
ξ
2
,
…
,
ξ
m
,
ξ
1
′
,
τ
1
,
τ
2
,
…
,
τ
m
,
τ
1
′
]
.Genetic algorithm uses the roulette method as selecting operator. The crossover operator adopts the nonuniform arithmetic crossover method, and the mutation operator uses the noncoherence mutation method. The fitness function mainly considers the accuracy of the output of the forecasting model; that is,f
fitness
=
1
/
f
1
(
t
). f
1
(
t
) is the mean square deviation between the desired output and the actual output of the forecasting model.The process is shown in Figure5.Figure 5
The process of parameters optimization of QENN by genetic algorithm.


### 2.4.1. Nonuniform Arithmetic Crossover Operator

The operation object of arithmetic crossover operator is chromosomeZ, which is encoded by floating point numbers.If two chromosome individuals areZ
A
t and Z
B
t, the two new individuals generated by the crossover operation can be expressed as(20)
Z
A
t
+
1
=
α
Z
B
t
+
1
-
α
Z
A
t
,
Z
B
t
+
1
=
α
Z
A
t
+
1
-
α
Z
B
t
.For the nonuniform arithmetic crossover operator,α is the random number, and its value range is from −0.5 to 1.5.


### 2.4.2. Noncoherence Mutation Operator

Supposing the chromosome individualZ
=
(
z
1
,
z
2
,
…
,
z
n
) is a parent solution, the component z
k is selected to mutate, and its value range is [
a
k
,
b
k
]. The mutated chromosome individual is Z
′
=
(
z
1
,
z
2
,
…
,
z
k
-
1
,
z
k
′
,
…
,
z
n
).z
k
′ can be expressed as (21)
z
k
′
=
z
k
+
Δ
t
,
b
k
-
z
k
if  rnd
2
=
0
z
k
-
Δ
t
,
z
k
-
a
k
if  rnd
2
=
1
,where t is the current evolution generation, and Δ
(
t
,
s
) can be expressed as(22)
Δ
t
,
s
=
s
×
1
-
r
1
-
t
/
T
λ
,where r is a random number and its value range is from 0 to 1. T is the maximal generation. λ is a parameter for the nonuniform arithmetic crossover operator, and its value is 2.5.


## 2.4.1. Nonuniform Arithmetic Crossover Operator

The operation object of arithmetic crossover operator is chromosomeZ, which is encoded by floating point numbers.If two chromosome individuals areZ
A
t and Z
B
t, the two new individuals generated by the crossover operation can be expressed as(20)
Z
A
t
+
1
=
α
Z
B
t
+
1
-
α
Z
A
t
,
Z
B
t
+
1
=
α
Z
A
t
+
1
-
α
Z
B
t
.For the nonuniform arithmetic crossover operator,α is the random number, and its value range is from −0.5 to 1.5.


## 2.4.2. Noncoherence Mutation Operator

Supposing the chromosome individualZ
=
(
z
1
,
z
2
,
…
,
z
n
) is a parent solution, the component z
k is selected to mutate, and its value range is [
a
k
,
b
k
]. The mutated chromosome individual is Z
′
=
(
z
1
,
z
2
,
…
,
z
k
-
1
,
z
k
′
,
…
,
z
n
).z
k
′ can be expressed as (21)
z
k
′
=
z
k
+
Δ
t
,
b
k
-
z
k
if  rnd
2
=
0
z
k
-
Δ
t
,
z
k
-
a
k
if  rnd
2
=
1
,where t is the current evolution generation, and Δ
(
t
,
s
) can be expressed as(22)
Δ
t
,
s
=
s
×
1
-
r
1
-
t
/
T
λ
,where r is a random number and its value range is from 0 to 1. T is the maximal generation. λ is a parameter for the nonuniform arithmetic crossover operator, and its value is 2.5.


## 3. PSRT and KNNA

### 3.1. Phase Space Reconstruction Theory

The chaotic time series is usually analyzed by PSRT. The embedding theorem of Takens and Saner et al. is the basis of the prediction theory to chaotic time series.To chaotic time seriesx
1
,
x
2
,
…
,
x
n
-
1
,
x
n
,
…, the phase space can be reconstructed by PSRT, which can be described as(23)
X
t
i
=
x
t
i
,
x
t
i
+
τ
,
…
,
x
t
i
+
m
-
1
τ
,where m is the embedding dimension and it can be acquired by the G-P method [17]. τ is the delay time, and it can be acquired by multiple-autocorrelation method [18]. In the m-dimension phase space, the evolution of the chaotic system can be described by the evolution track of phase points.According to the G-P method, the embedding dimensionm is 7. By multiple-autocorrelation method, the delay time τ is 1 for the real example simulation in this paper.PSRT provides a theoretical basis for the construction of the QENN model. For the model, the number of input nodes is decided by the embedding dimensionm according to PSRT.


### 3.2.K-Nearest Neighbor Approach

TheK-nearest neighbor approach is usually used in the forecasting research of chaotic time series [19], which is based on the Euclid distance. The similarity of two phase points is computed by the Euclid distance.In the reconstructed phase space withm-dimension, the forecasting phase point is recorded as X
r
(
t
), and X
r
i
(
t
) denotes ith neighbor phase point. d
r
i
(
t
) is the Euclid distance between the forecasting phase points and ith neighbor phase point, which is expressed as(24)
d
r
i
t
=
X
r
t
-
X
r
i
t
.According to (24), the nearest neighbor phase points set can be formed, which is the training samples set of forecasting phase points for the QENN model.


## 3.1. Phase Space Reconstruction Theory

The chaotic time series is usually analyzed by PSRT. The embedding theorem of Takens and Saner et al. is the basis of the prediction theory to chaotic time series.To chaotic time seriesx
1
,
x
2
,
…
,
x
n
-
1
,
x
n
,
…, the phase space can be reconstructed by PSRT, which can be described as(23)
X
t
i
=
x
t
i
,
x
t
i
+
τ
,
…
,
x
t
i
+
m
-
1
τ
,where m is the embedding dimension and it can be acquired by the G-P method [17]. τ is the delay time, and it can be acquired by multiple-autocorrelation method [18]. In the m-dimension phase space, the evolution of the chaotic system can be described by the evolution track of phase points.According to the G-P method, the embedding dimensionm is 7. By multiple-autocorrelation method, the delay time τ is 1 for the real example simulation in this paper.PSRT provides a theoretical basis for the construction of the QENN model. For the model, the number of input nodes is decided by the embedding dimensionm according to PSRT.


## 3.2.K-Nearest Neighbor Approach

TheK-nearest neighbor approach is usually used in the forecasting research of chaotic time series [19], which is based on the Euclid distance. The similarity of two phase points is computed by the Euclid distance.In the reconstructed phase space withm-dimension, the forecasting phase point is recorded as X
r
(
t
), and X
r
i
(
t
) denotes ith neighbor phase point. d
r
i
(
t
) is the Euclid distance between the forecasting phase points and ith neighbor phase point, which is expressed as(24)
d
r
i
t
=
X
r
t
-
X
r
i
t
.According to (24), the nearest neighbor phase points set can be formed, which is the training samples set of forecasting phase points for the QENN model.


## 4. Real Example Simulation

### 4.1. Four STLF Models for Real Example Simulation

Four STLF models are constructed, which are model-I based on the QENN, model-II based on the quantum feedforward neural network, model-III based on the conventional Elman neural network, and model-IV based on the conventional feedforward neural network.The input nodes number of four STLF models for actual simulation load system is 7, which is acquired by the G-P method. The delay timeτ is 1, which is acquired by multiple-autocorrelation method.The number of the neighbor phase points affects the prediction accuracy and calculation of the forecasting model. If the number of the neighbor phase points is too much, it will not only increase the amount of computation but also increase the number of false neighbor phase points, which has a great impact on the prediction performance. If the number of the neighbor phase points is too small, the effective information is not fully utilized, which also affects the prediction performance. In general, the relationship between the number of the neighbor phase pointsk and the embedding dimension m satisfies the following conditions: k
≥
m
+
1. Based on multiple validation tests, the number of the neighbor phase points is 10 for the actual simulation load system in this paper.For model-I and model-III, the number of the input nodes is 7, the number of the hidden nodes is 10, the number of the context nodes is 10, and the number of the output points is 1.For model-II and model-IV, the number of the input nodes is 7, the number of the hidden nodes is 10, and the number of the output points is 1.The maximum permissible error of the four STLF models isε
=
1
0
-
4. For model-I, model-II, model-III, and model-IV, genetic algorithm was used to adjusted the parameters of QENN, the quantum feedforward neural network, and the conventional Elman neural network. The fitness function mainly considers the accuracy of the output of the forecasting model. The crossover probability is 0.90, and the variation probability is 0.10.


### 4.2. Simulation Results

Four models are used to predict the actual load. The daily load forecasting errors and comparison of four models for workday and weekend are shown in Figures6 and 7, respectively.Figure 6
The daily load forecasting errors of four STLF models for workday.Figure 7
The daily load forecasting errors of four STLF models for weekend.The mean absolute percentage error and the maximum relative error of four STLF models for workday and weekend are described in Table1. The mean absolute percentage error can be calculated as(25)
E
MAPE
=
1
M
∑
i
=
1
M
Y
^
i
-
Y
i
Y
i
×
100
%
,where M is the number of the forecasting points, Y
^
i is the forecasting load, and Y
i is the actual load.Table 1
The error comparison of four STLF models for two day types.
Day type
Model-I
Model-II
Model-III
Model-IV
E
M
A
P
E
E
M
A
X
E
M
A
P
E
E
M
A
X
E
M
A
P
E
E
M
A
X
E
M
A
P
E
E
M
A
X
Workday
1.025
2.977
1.826
4.450
1.813
4.521
2.903
5.327
Weekend
1.210
3.225
2.153
4.705
2.275
4.860
3.152
5.515The maximum relative error is the maximum value of relative error of all predicted points. The relative error can be calculated as(26)
E
R
E
=
Y
^
i
-
Y
i
Y
i
×
100
%
.The error performance of the model-IV is the worst in the all models, because the conventional feedforward neural network cannot effectively characterize the chaotic dynamic behavior of the load system. The error performance of model-II is better than that of model-IV, because quantum computation can enhance information processing capability and generalization ability of quantum feedforward neural networks. The error performance of model-III is better than that of model-IV, because the conventional Elman neural network has formed specific dynamic memory performance. The forecasting performance of model-I is the best, which is the fusion of quantum computation and feedback mechanism.From Table1 it can be seen that the forecasting error is larger in weekend. In weekend, the load mode is more diverse and complex, which leads to the forecasting error is slightly larger than that of workday.In order to verify how sensitive are the simulation results onm and τ, the forecasting errors of model-I with different m are described in Table 2, and the forecasting errors of model-I with different τ are described in Table 3.Table 2
The forecasting error of model-I with differentm (τ
=
1).
m
5
6
7
8
9
E
M
A
P
E
E
M
A
X
E
M
A
P
E
E
M
A
X
E
M
A
P
E
E
M
A
X
E
M
A
P
E
E
M
A
X
E
M
A
P
E
E
M
A
X
Monday
1.629
4.512
1.466
4.011
1.025
2.977
1.358
3.965
1.540
4.250Table 3
The forecasting error of model-I with differentτ (m
=
7).
τ
1
2
3
4
E
M
A
P
E
E
M
A
X
E
M
A
P
E
E
M
A
X
E
M
A
P
E
E
M
A
X
E
M
A
P
E
E
M
A
X
Monday
1.025
2.977
1.376
3.852
1.405
4.009
1.466
4.542From Tables2 and 3, it can be seen that the forecasting error is related to m and τ. For the actual simulation load system in this paper, m
=
7 and τ
=
1 are appropriate.In order to further verify the performance of model-I, the real example are used to test in a week. The mean absolute percentage error and the maximum relative error in the week are described in Table4. It is shown that the proposed model-I has good precision performance and steadiness.Table 4
The daily load forecasting errors in a week.
Day type
Model-I
E
M
A
P
E
E
M
A
X
Monday
1.025
2.977
Tuesday
1.109
3.012
Wednesday
1.153
2.985
Thursday
1.128
2.944
Friday
1.090
2.870
Saturday
1.210
3.225
Sunday
1.227
3.412


## 4.1. Four STLF Models for Real Example Simulation

Four STLF models are constructed, which are model-I based on the QENN, model-II based on the quantum feedforward neural network, model-III based on the conventional Elman neural network, and model-IV based on the conventional feedforward neural network.The input nodes number of four STLF models for actual simulation load system is 7, which is acquired by the G-P method. The delay timeτ is 1, which is acquired by multiple-autocorrelation method.The number of the neighbor phase points affects the prediction accuracy and calculation of the forecasting model. If the number of the neighbor phase points is too much, it will not only increase the amount of computation but also increase the number of false neighbor phase points, which has a great impact on the prediction performance. If the number of the neighbor phase points is too small, the effective information is not fully utilized, which also affects the prediction performance. In general, the relationship between the number of the neighbor phase pointsk and the embedding dimension m satisfies the following conditions: k
≥
m
+
1. Based on multiple validation tests, the number of the neighbor phase points is 10 for the actual simulation load system in this paper.For model-I and model-III, the number of the input nodes is 7, the number of the hidden nodes is 10, the number of the context nodes is 10, and the number of the output points is 1.For model-II and model-IV, the number of the input nodes is 7, the number of the hidden nodes is 10, and the number of the output points is 1.The maximum permissible error of the four STLF models isε
=
1
0
-
4. For model-I, model-II, model-III, and model-IV, genetic algorithm was used to adjusted the parameters of QENN, the quantum feedforward neural network, and the conventional Elman neural network. The fitness function mainly considers the accuracy of the output of the forecasting model. The crossover probability is 0.90, and the variation probability is 0.10.


## 4.2. Simulation Results

Four models are used to predict the actual load. The daily load forecasting errors and comparison of four models for workday and weekend are shown in Figures6 and 7, respectively.Figure 6
The daily load forecasting errors of four STLF models for workday.Figure 7
The daily load forecasting errors of four STLF models for weekend.The mean absolute percentage error and the maximum relative error of four STLF models for workday and weekend are described in Table1. The mean absolute percentage error can be calculated as(25)
E
MAPE
=
1
M
∑
i
=
1
M
Y
^
i
-
Y
i
Y
i
×
100
%
,where M is the number of the forecasting points, Y
^
i is the forecasting load, and Y
i is the actual load.Table 1
The error comparison of four STLF models for two day types.
Day type
Model-I
Model-II
Model-III
Model-IV
E
M
A
P
E
E
M
A
X
E
M
A
P
E
E
M
A
X
E
M
A
P
E
E
M
A
X
E
M
A
P
E
E
M
A
X
Workday
1.025
2.977
1.826
4.450
1.813
4.521
2.903
5.327
Weekend
1.210
3.225
2.153
4.705
2.275
4.860
3.152
5.515The maximum relative error is the maximum value of relative error of all predicted points. The relative error can be calculated as(26)
E
R
E
=
Y
^
i
-
Y
i
Y
i
×
100
%
.The error performance of the model-IV is the worst in the all models, because the conventional feedforward neural network cannot effectively characterize the chaotic dynamic behavior of the load system. The error performance of model-II is better than that of model-IV, because quantum computation can enhance information processing capability and generalization ability of quantum feedforward neural networks. The error performance of model-III is better than that of model-IV, because the conventional Elman neural network has formed specific dynamic memory performance. The forecasting performance of model-I is the best, which is the fusion of quantum computation and feedback mechanism.From Table1 it can be seen that the forecasting error is larger in weekend. In weekend, the load mode is more diverse and complex, which leads to the forecasting error is slightly larger than that of workday.In order to verify how sensitive are the simulation results onm and τ, the forecasting errors of model-I with different m are described in Table 2, and the forecasting errors of model-I with different τ are described in Table 3.Table 2
The forecasting error of model-I with differentm (τ
=
1).
m
5
6
7
8
9
E
M
A
P
E
E
M
A
X
E
M
A
P
E
E
M
A
X
E
M
A
P
E
E
M
A
X
E
M
A
P
E
E
M
A
X
E
M
A
P
E
E
M
A
X
Monday
1.629
4.512
1.466
4.011
1.025
2.977
1.358
3.965
1.540
4.250Table 3
The forecasting error of model-I with differentτ (m
=
7).
τ
1
2
3
4
E
M
A
P
E
E
M
A
X
E
M
A
P
E
E
M
A
X
E
M
A
P
E
E
M
A
X
E
M
A
P
E
E
M
A
X
Monday
1.025
2.977
1.376
3.852
1.405
4.009
1.466
4.542From Tables2 and 3, it can be seen that the forecasting error is related to m and τ. For the actual simulation load system in this paper, m
=
7 and τ
=
1 are appropriate.In order to further verify the performance of model-I, the real example are used to test in a week. The mean absolute percentage error and the maximum relative error in the week are described in Table4. It is shown that the proposed model-I has good precision performance and steadiness.Table 4
The daily load forecasting errors in a week.
Day type
Model-I
E
M
A
P
E
E
M
A
X
Monday
1.025
2.977
Tuesday
1.109
3.012
Wednesday
1.153
2.985
Thursday
1.128
2.944
Friday
1.090
2.870
Saturday
1.210
3.225
Sunday
1.227
3.412


## 5. Conclusions

This paper constructs a STLF model based on QENN. The quantum computation and Elman feedback mechanism were integrated into QENN. Quantum computation can effectively improve the approximation capability and the information processing ability of the neural networks. QENN has not only the feedforward connection but also the feedback connection. Phase space reconstruction theory is the theoretical basis of constructing the forecasting model. The training samples are formed by means ofK-nearest neighbor approach. Through the example simulation, the testing results show that proposed model can effectively improve the prediction accuracy. The research in the paper makes a theoretical foundation for the practical engineering application of the STLF model based on QENN. The model based on QENN can also be used in the power prediction for new energy, such as wind power prediction and photovoltaic power prediction.



---

*Source: 7910971-2016-08-29.xml*