# Fuzzy Counter Propagation Neural Network Control for a Class of Nonlinear Dynamical Systems

Authors: Vandana Sakhre; Sanjeev Jain; Vilas S. Sapkal; Dev P. Agarwal
Journal: Computational Intelligence and Neuroscience
 (2015)

---

## Abstract

Fuzzy Counter Propagation Neural Network (FCPN) controller design is developed, for a class of nonlinear dynamical systems. In this process, the weight connecting between the instar and outstar, that is, input-hidden and hidden-output layer, respectively, is adjusted by using Fuzzy Competitive Learning (FCL). FCL paradigm adopts the principle of learning, which is used to calculate Best Matched Node (BMN) which is proposed. This strategy offers a robust control of nonlinear dynamical systems. FCPN is compared with the existing network like Dynamic Network (DN) and Back Propagation Network (BPN) on the basis of Mean Absolute Error (MAE), Mean Square Error (MSE), Best Fit Rate (BFR), and so forth. It envisages that the proposed FCPN gives better results than DN and BPN. The effectiveness of the proposed FCPN algorithms is demonstrated through simulations of four nonlinear dynamical systems and multiple input and single output (MISO) and a single input and single output (SISO) gas furnace Box-Jenkins time series data.


---

## Body

## 1. Introduction

With the rapid advance of computers, the digitization has swept in all fields of science and technology with special emphasize on modeling and identification. Most of the physical processes are continuous in nature but they are generally modeled using discrete time with the use of derivative and integral as they are easily realizable. Taking these advantages, an enormous research has been focused on discrete time methods and many techniques based on linear and nonlinear discrete systems have been developed [1, 2]. Neural network has been widely used for nonlinear dynamical systems [3–5]. Various control systems like back stepping control [6, 7], sliding mode control [8, 9], and control using soft computing [10] are continuous source of interest. The nonlinear dynamical system is a generic problem which finds its application in every field of engineering. The technique based on soft computing, fuzzy logic, and neural network has also found its application in modeling of systems in various application domains. Feedforward Neural Network (FNN) is one of the most commonly used networks for this purpose; FNN with Back Propagation algorithm is another powerful network. Neural network is most popular approach due to its capability of modeling most of the nonlinear functions approximately to any arbitrary degree of accuracy [11]. Most of the systems are designed using feedforward network with gradient descent learning but the gradient descent method encounters the problem of slow convergence. This problem of convergence can be overcome by use of some Cauchy-Newton [12] Quasi-Newtonian method and Levenberg-Marquardt algorithms [13].Optimal controllers for hybrid dynamical systems (HDS) developed using Hamilton-Jacobi-Bellman (HJB) solution method [14, 15]. Novel Lyapunov-Krasovskii functional (LKF) with triple integral time constructed for exponential synchronization of complex dynamical networks will control packet loss and additive time varying delay [16]. Control of discrete time varying system using dynamical model is difficult. To overcome this condition, new neural network approximation structure was developed to solve optimal tracking problem of nonlinear discrete time varying time system using reinforcement learning (RL) method [17]. To enhance the performance of nonlinear multivariable system, fuzzy optimal controller based Takagi-Sugeno model was developed which was used to minimize a quadratic performance index [18]. An ultimate stable training algorithm inspired by adaptive observer for black box identification of nonlinear discrete systems developed using state space recurrent neural network. The algorithm developed was using only input and output measurement in discrete time [19]. Indirect adaptive controller that uses neural network was presented for the identification and control of experimental pilot distillation column. This system is multivariable with unknown dynamics and the neural network was trained using Levenberg-Marquardt algorithm [20]. Adaptive finite time stabilization of a class of switched nonlinear systems was investigated with unknown nonlinear terms using neural network. The finite law and adaptive law were constructed using radial basis function neural network (RBFNN) to approximate the unknown packaged function. Stability analysis of RBFNN was observed to evaluate the states of the closed loop systems [21]. An adaptive fuzzy output tracking control approach was proposed in [22] for single input single output (SISO) system which is uncertain and nonlinear under arbitrary switching. An iterative learning control scheme was proposed for a class of nonlinear dynamic systems which includes holonomic systems as its subsets with linear feedback mechanism and feedforward learning strategies [23].In this proposed work we have used instar-outstar structure based CPN with Fuzzy Competitive Learning (FCL). We have designed Fuzzy Counter Propagation Network design to control some nonlinear dynamical systems. In the FCPN, CPN model is trained by FCL algorithms. The FCL learning is used for adjusting weights and update of Best Matched Node (BMN) in discrete time nonlinear dynamical system.The main contributions of this research are as follows:(1)
This paper contributes the approximation for a class of nonlinear dynamical systems using Fuzzy Competitive Learning Based Counter Propagation Network (FCPN).
(2)
FCPN is employed to optimize the Mean Absolute Error, Mean Square Error, Best Fit Rate, and so forth.
(3)
Performance criteria of proposed FCPN for nonlinear dynamical systems are effectively improved by compensating reference signal and controller signal.The paper is organized as follows. In Section 2, problem formulations for a class of nonlinear dynamical systems are given. Section 3 contains description of Feedforward Neural Network (FNN) with Back Propagation Network, Dynamic Network, and Fuzzy Learning Based Counter Propagation Network. Dynamical learning for CPN and optimal learning of dynamical system are presented in Section 4. The simulation results and comparison of different nonlinear dynamical models are presented in Section 5. Section 6 gives conclusion of the study.


## 2. Problem Formulation

Let us consider four models of discrete time nonlinear dynamical systems [24] for single input single output (SISO) and multiple input and single output (MISO) system considered in this paper and they are described by difference equations (1)–(4) and Box-Jenkins time series data [10]. Let f
:
R
n
→
R and g
:
R
m
→
R be the nonlinear continuous differentiable function of Model I–Model IV approximated by FCPN to the desired degree of accuracy. Once the system has been parameterized, the performance evaluation has been carried out by FCPN for (1) to (4).Model I:(1)yk+1=∑i=0n-1αiyk-i+guk,…,uk-m+1.Model II:(2)yk+1=fyk,…,yk-n+1+∑i=0m-1βiuk-i.Model III:(3)yk+1=fyk,…,yk-n+1+guk,…,uk-m+1.Model IV:(4)
y
k
+
1
=
f
y
k
,
…
,
y
k
-
n
+
1
;
u
k
,
…
,
u
k
-
m
+
1
,where [u
(
k
)
,
y
(
k
)] represent the input-output pair of the system at time k and their order is represented by (
n
,
m
). FCL is used to learn the system defined for (1) to (4) and the performance of the FCPN can be measured by error function given in (5)Ek=12yk-y-k2,where u
(
k
) is the input, y
k is the system output, y
-
(
k
) is neural network output, and (5) can be written for neural controller as(6)Ec=1P∑p=1Pyk-y-ck2,where y
-
c
(
k
) is neural controller output and (6) is known as minimized error function and P is total number of input patterns.


## 3. Back Propagation Network (BPN)

A neural network is one of the most popular intelligent systems which has capability to approximate calculated and target value at an arbitrary degree of accuracy. BPN is one of the most commonly used networks for this purpose. The BPN consists of input layer, hidden layer, and output layer and possesses weighted interconnections. The BPN is based on supervised learning. This method is used where error is propagated back to the hidden layer. The aim of this neural network is to train the net to achieve a balance between the net’s ability to respond (memorization) and its ability to give reasonable response to the input that is similar but not identical to the one that is used in training (generalization). For a given set of training input-output pairs, this network provides a procedure for changing weights to classify the given input patterns correctly. The weight update algorithm is based on gradient descent method [25]. The network architecture is shown in Figure 1.Figure 1
General architecture of BPN.


### 3.1. Dynamic Network

Neural network can be classified into two categories: (i) Static and (ii) Dynamic. In Static Neural Network, output can be calculated directly from input through feedforward network. But in Dynamic Network output depends on current or previous inputs, outputs, or states of network. Where the current output is function of current input and previous output, it is known as recurrent (feedback) network. Training process of Static and Dynamic Network can be differentiated by use of gradient or Jacobian which is computed. Dynamic Network contains delays and operates on a sequence of inputs. Dynamic Networks can have purely feedforward connections or they can have some recurrent connections. They can be trained using Dynamic Back Propagation Network [26].The general equation of the net inputn
m
(
t
) for mth layer is given by (7)nmt=∑l∈Lmf∑d∈DLm·lIWm·ldalt-d+∑l∈Im∑d∈DIm·lIWm·ldblt-d+bm,where b
l
(
t
) is the lth input vector at time t, I
W
m
,
l is the input weight between lth and mth layer, L
W
m
,
l is the layer weight between lth and mth layer, and b
m is the bias vector for layer m. D
L
m
,
l is the set of all delays in the Tapped Delay Line (TDL) between lth and mth layer and D
I
m
,
l is the set of all delays in the TDL between input l and mth layer, I
m is the set of indices of input vectors that connect to layer m, and L
m
f is the set of indices of layers that connect to layer m. The output of mth layer at time t is computed as(8)amt=fmnmt.Network has a TDL on the input with D
I
1,1
=
{
0
,
1
,
2
} and its output is represented as(9)
a
t
=
f
n
t
=
∑
d
I
W
d
∗
p
t
-
d
,
a
t
=
f
i
w
1,1
0
p
t
+
i
w
1,1
1
p
t
-
1
+
i
w
1,1
2
p
t
-
2
.One simple architecture with delay D
I
1,1
=
{
0
,
1
,
2
} for feedforward Dynamic Network is shown in Figure 2 known as Dynamic Adaptive Linear Neuron.Figure 2
Simple architecture of Dynamic Adaptive Linear Neuron.


### 3.2. Counter Propagation Network (CPN)

It is multilayer feedforward network based on the combination of the input, competitive, and output layers. Model of CPN is instar-outstar. It is three-layer neural network that performs input-output data mapping, that is, producing output in the response to an input vector on the basis of Fuzzy Competitive Learning. In CPN the connection between input layer and competitive layer is instar structure and connection between competitive and output layer is outstar structure. Counter Propagation Network involves two-stage training process. In first stage input vector is clustered on the basis of Euclidean distance and the performance of network is improved using linear topology. Using Euclidean distance between input and weight vector, we evaluated BMN. In phase II, the desired response is obtained by adapting the weights from competitive layer to output layer [27]. Let x
=
x
1
x
2
⋯
x
n and y
=
y
1
∗
y
2
∗
⋯
y
m
∗ be input and desired vector, respectively, let v
i
j be weight of input layer and competitive layer, where 1
≤
i
≤
n and 1
≤
j
≤
p, and let w
j
k be the weight between competitive layer and output layer, where 1
≤
k
≤
m. Euclidean distance between input vector x and weight vector v
i
j is(10)Dj=∑i=1nxi-vij2,wherej=1,2,…,p.The architecture of CPN is shown in Figure 3.Figure 3
Architecture of Counter Propagation Network.


### 3.3. Fuzzy Competitive Learning (FCL)

FCL is used for adjusting instar-outstar weights and update of BMN for discrete time nonlinear dynamical systems. Learning of CPN is divided into two phases: Fuzzy Competitive Learning phase (unsupervised) and Grossberg Learning phase (supervised) [28–31]. CPN is efficient network for mapping the input vector of size n placed in clusters of size m. Like traditional self-organizing map, the weight vector for every node is considered as a sample vector to the input pattern. The process is to determine which weight vector (say J) is more similar to the input pattern chosen as BMN. First phase is based on closeness between weight vector and input vector and second phase is weight update between competitive and output layer for desired response.Letv
i
j be weight of input node i to neuron j and let w
j
k be the weight between competitive node j and neuron k. We propose a new learning rate calculation method for use of weight update: (11)∝J=∑i=1nviJ-xi2∑j=1m∑i=1nviJ-xi2,where J denotes the index of BMN and ∝ is known as Fuzzy Competitive Learning rate.Fuzzy Competitive Learning phase is described as follows.


#### 3.3.1. Training Algorithm of FCL

Phase I (for determination of BMN). There are steps for training of CPN as follows.Step 1.
Initialize instar-outstar weights.Step 2.
Perform Steps3–8 until stopping criteria for Phase I training fail. The stopping condition may be fixed number of epochs or learning rate has reduced to negligible value.Step 3.
Perform Steps4–6 for all input training vector X.Step 4.
Set theX-input layer activation to vector X.Step 5.
Compute the Best Matched Node (BMN) (J) using Euclidean distance; find the node Z
J whose distance from the input pattern is minimum. Euclidean distance is calculated as follows:(12)Dj=∑i=1nxi-vij2,wherej=1,2,…,p.Minimum net input:(13)zinJ=∑i=1nxiviJ.Step 6.
Calculate Fuzzy Competitive Learning (14) rate for weight update using BMN.Step 7.
We update weight for unitZ
J: (14)viJnew=viJold+∝hJ;i,jxi-viJ,where i
=
1,2
,
3
,
…
,
n and h
(
·
;
·
) is neighborhood function around BMN. Consider(15)hJ;i,j=exp⁡-vij-viJ2σ-2t,wherehJ;i,j∈0,1,(16)σ-t=σ0exp⁡-tT.Step 8.
Test the stopping criteria.Phase II (to obtain desired response)Step 1.
Set activation function to(
x
,
y
) input and output layer, respectively.Step 2.
Update the BMN (J) (Step 5 from Phase I). Also update the weights into unit Z
J as(17)viJnew=viJold+αhJ;i,jxi-viJold.Step 3.
Update the weights from nodeZ
J to the output unit as(18)wJknew=wJkold+βhJ;j,kyj-wJkoldfor1≤k≤m.Step 4.
Learning rateβ is calculated as(19)βJ=∑k=1myk-wJk2∑j=1p∑k=1myk-wjk2.Step 5.
Decrease the rate of learnings.t.
β
=
β
-
ε, where ε is small positive number.Step 6.
Test the stopping condition for Phase II (i.e., fixed number of epochs or its learning rate has reduced to negligible value).


#### 3.3.2. Testing Phase (to Test FCPN)

Step 1.
Set the initial weights, that is, the weights obtained during training.Step 2.
Apply FCPN to the input vectorX.Step 3.
Find unitJ that is closest to vector X.Step 4.
Set activations of output units.Step 5.
Apply activation function aty
k, where y
k
=
∑
j
z
j
w
j
k.


## 3.1. Dynamic Network

Neural network can be classified into two categories: (i) Static and (ii) Dynamic. In Static Neural Network, output can be calculated directly from input through feedforward network. But in Dynamic Network output depends on current or previous inputs, outputs, or states of network. Where the current output is function of current input and previous output, it is known as recurrent (feedback) network. Training process of Static and Dynamic Network can be differentiated by use of gradient or Jacobian which is computed. Dynamic Network contains delays and operates on a sequence of inputs. Dynamic Networks can have purely feedforward connections or they can have some recurrent connections. They can be trained using Dynamic Back Propagation Network [26].The general equation of the net inputn
m
(
t
) for mth layer is given by (7)nmt=∑l∈Lmf∑d∈DLm·lIWm·ldalt-d+∑l∈Im∑d∈DIm·lIWm·ldblt-d+bm,where b
l
(
t
) is the lth input vector at time t, I
W
m
,
l is the input weight between lth and mth layer, L
W
m
,
l is the layer weight between lth and mth layer, and b
m is the bias vector for layer m. D
L
m
,
l is the set of all delays in the Tapped Delay Line (TDL) between lth and mth layer and D
I
m
,
l is the set of all delays in the TDL between input l and mth layer, I
m is the set of indices of input vectors that connect to layer m, and L
m
f is the set of indices of layers that connect to layer m. The output of mth layer at time t is computed as(8)amt=fmnmt.Network has a TDL on the input with D
I
1,1
=
{
0
,
1
,
2
} and its output is represented as(9)
a
t
=
f
n
t
=
∑
d
I
W
d
∗
p
t
-
d
,
a
t
=
f
i
w
1,1
0
p
t
+
i
w
1,1
1
p
t
-
1
+
i
w
1,1
2
p
t
-
2
.One simple architecture with delay D
I
1,1
=
{
0
,
1
,
2
} for feedforward Dynamic Network is shown in Figure 2 known as Dynamic Adaptive Linear Neuron.Figure 2
Simple architecture of Dynamic Adaptive Linear Neuron.


## 3.2. Counter Propagation Network (CPN)

It is multilayer feedforward network based on the combination of the input, competitive, and output layers. Model of CPN is instar-outstar. It is three-layer neural network that performs input-output data mapping, that is, producing output in the response to an input vector on the basis of Fuzzy Competitive Learning. In CPN the connection between input layer and competitive layer is instar structure and connection between competitive and output layer is outstar structure. Counter Propagation Network involves two-stage training process. In first stage input vector is clustered on the basis of Euclidean distance and the performance of network is improved using linear topology. Using Euclidean distance between input and weight vector, we evaluated BMN. In phase II, the desired response is obtained by adapting the weights from competitive layer to output layer [27]. Let x
=
x
1
x
2
⋯
x
n and y
=
y
1
∗
y
2
∗
⋯
y
m
∗ be input and desired vector, respectively, let v
i
j be weight of input layer and competitive layer, where 1
≤
i
≤
n and 1
≤
j
≤
p, and let w
j
k be the weight between competitive layer and output layer, where 1
≤
k
≤
m. Euclidean distance between input vector x and weight vector v
i
j is(10)Dj=∑i=1nxi-vij2,wherej=1,2,…,p.The architecture of CPN is shown in Figure 3.Figure 3
Architecture of Counter Propagation Network.


## 3.3. Fuzzy Competitive Learning (FCL)

FCL is used for adjusting instar-outstar weights and update of BMN for discrete time nonlinear dynamical systems. Learning of CPN is divided into two phases: Fuzzy Competitive Learning phase (unsupervised) and Grossberg Learning phase (supervised) [28–31]. CPN is efficient network for mapping the input vector of size n placed in clusters of size m. Like traditional self-organizing map, the weight vector for every node is considered as a sample vector to the input pattern. The process is to determine which weight vector (say J) is more similar to the input pattern chosen as BMN. First phase is based on closeness between weight vector and input vector and second phase is weight update between competitive and output layer for desired response.Letv
i
j be weight of input node i to neuron j and let w
j
k be the weight between competitive node j and neuron k. We propose a new learning rate calculation method for use of weight update: (11)∝J=∑i=1nviJ-xi2∑j=1m∑i=1nviJ-xi2,where J denotes the index of BMN and ∝ is known as Fuzzy Competitive Learning rate.Fuzzy Competitive Learning phase is described as follows.


### 3.3.1. Training Algorithm of FCL

Phase I (for determination of BMN). There are steps for training of CPN as follows.Step 1.
Initialize instar-outstar weights.Step 2.
Perform Steps3–8 until stopping criteria for Phase I training fail. The stopping condition may be fixed number of epochs or learning rate has reduced to negligible value.Step 3.
Perform Steps4–6 for all input training vector X.Step 4.
Set theX-input layer activation to vector X.Step 5.
Compute the Best Matched Node (BMN) (J) using Euclidean distance; find the node Z
J whose distance from the input pattern is minimum. Euclidean distance is calculated as follows:(12)Dj=∑i=1nxi-vij2,wherej=1,2,…,p.Minimum net input:(13)zinJ=∑i=1nxiviJ.Step 6.
Calculate Fuzzy Competitive Learning (14) rate for weight update using BMN.Step 7.
We update weight for unitZ
J: (14)viJnew=viJold+∝hJ;i,jxi-viJ,where i
=
1,2
,
3
,
…
,
n and h
(
·
;
·
) is neighborhood function around BMN. Consider(15)hJ;i,j=exp⁡-vij-viJ2σ-2t,wherehJ;i,j∈0,1,(16)σ-t=σ0exp⁡-tT.Step 8.
Test the stopping criteria.Phase II (to obtain desired response)Step 1.
Set activation function to(
x
,
y
) input and output layer, respectively.Step 2.
Update the BMN (J) (Step 5 from Phase I). Also update the weights into unit Z
J as(17)viJnew=viJold+αhJ;i,jxi-viJold.Step 3.
Update the weights from nodeZ
J to the output unit as(18)wJknew=wJkold+βhJ;j,kyj-wJkoldfor1≤k≤m.Step 4.
Learning rateβ is calculated as(19)βJ=∑k=1myk-wJk2∑j=1p∑k=1myk-wjk2.Step 5.
Decrease the rate of learnings.t.
β
=
β
-
ε, where ε is small positive number.Step 6.
Test the stopping condition for Phase II (i.e., fixed number of epochs or its learning rate has reduced to negligible value).


### 3.3.2. Testing Phase (to Test FCPN)

Step 1.
Set the initial weights, that is, the weights obtained during training.Step 2.
Apply FCPN to the input vectorX.Step 3.
Find unitJ that is closest to vector X.Step 4.
Set activations of output units.Step 5.
Apply activation function aty
k, where y
k
=
∑
j
z
j
w
j
k.


## 3.3.1. Training Algorithm of FCL

Phase I (for determination of BMN). There are steps for training of CPN as follows.Step 1.
Initialize instar-outstar weights.Step 2.
Perform Steps3–8 until stopping criteria for Phase I training fail. The stopping condition may be fixed number of epochs or learning rate has reduced to negligible value.Step 3.
Perform Steps4–6 for all input training vector X.Step 4.
Set theX-input layer activation to vector X.Step 5.
Compute the Best Matched Node (BMN) (J) using Euclidean distance; find the node Z
J whose distance from the input pattern is minimum. Euclidean distance is calculated as follows:(12)Dj=∑i=1nxi-vij2,wherej=1,2,…,p.Minimum net input:(13)zinJ=∑i=1nxiviJ.Step 6.
Calculate Fuzzy Competitive Learning (14) rate for weight update using BMN.Step 7.
We update weight for unitZ
J: (14)viJnew=viJold+∝hJ;i,jxi-viJ,where i
=
1,2
,
3
,
…
,
n and h
(
·
;
·
) is neighborhood function around BMN. Consider(15)hJ;i,j=exp⁡-vij-viJ2σ-2t,wherehJ;i,j∈0,1,(16)σ-t=σ0exp⁡-tT.Step 8.
Test the stopping criteria.Phase II (to obtain desired response)Step 1.
Set activation function to(
x
,
y
) input and output layer, respectively.Step 2.
Update the BMN (J) (Step 5 from Phase I). Also update the weights into unit Z
J as(17)viJnew=viJold+αhJ;i,jxi-viJold.Step 3.
Update the weights from nodeZ
J to the output unit as(18)wJknew=wJkold+βhJ;j,kyj-wJkoldfor1≤k≤m.Step 4.
Learning rateβ is calculated as(19)βJ=∑k=1myk-wJk2∑j=1p∑k=1myk-wjk2.Step 5.
Decrease the rate of learnings.t.
β
=
β
-
ε, where ε is small positive number.Step 6.
Test the stopping condition for Phase II (i.e., fixed number of epochs or its learning rate has reduced to negligible value).


## 3.3.2. Testing Phase (to Test FCPN)

Step 1.
Set the initial weights, that is, the weights obtained during training.Step 2.
Apply FCPN to the input vectorX.Step 3.
Find unitJ that is closest to vector X.Step 4.
Set activations of output units.Step 5.
Apply activation function aty
k, where y
k
=
∑
j
z
j
w
j
k.


## 4. Dynamical Learning for CPN

Learning stabilities are fundamental issues for CPN; however there are few studies on the learning issue of FNN. BPN for learning is not always successful because of its sensitivity to learning parameters. Optimal learning rate always changes during the training process. Dynamical learning of CPN is carried out using Lemmas1, 2, and 3.Assumption. Let us assume ϕ
(
x
) is a sigmoid function if it is bounded continuous and increasing function. Since input to the neural network in this model is bounded, we consider Lemmas 1 and 2 given below.Lemma 1.
Letϕ
(
x
) be a sigmoid function and let Ω be a compact set in R
n, and f
:
R
n
→
R on Ω is a continuous function and, for arbitrary ∈
>
0, ∃ integer N and real constants c
i, θ
i, and w
i
j, i
=
1
,
2
,
…
,
N, j
=
1
,
2
,
…
,
n, such that (20)f-x1,x2,…,xn=∑i=1Nci∅∑j=1nwijxi-θisatisfies(21)maxx∈Ω⁡fx1,x2,…,xn-f-x1,x2,…,xn<∈.Using Lemma 1, dynamical learning for a three-layer CPN can be formulated where the hidden layer transfer functions are ϕ
(
x
) and transfer function at output layer is linear.Let all the vectors be column vectors and superscriptd
p
k refers to specific output vectors component.LetX
=
x
1
,
x
2
,
…
,
x
p
∈
R
L
×
P, Y
=
y
1
,
y
2
,
…
,
y
p
∈
R
H
×
P, O
=
o
1
,
o
2
,
…
,
o
p
∈
R
K
×
P, D
=
d
1
,
d
2
,
…
,
d
p
∈
R
K
×
P be the input, hidden, output, and desired vector, respectively, where L, H, and K denote the number of input, hidden, and output layer neurons. Let V and W represent the input-hidden and hidden-output layer weight matrix, respectively. The objectives of the network training minimize error function J, where J is(22)J=12PK∑p=1P∑k=1Kopk-dpk2.


### 4.1. Optimal Learning of Dynamical System

Consider a three-layer CPN; the network error matrix is defined by the error between differences of desired and FCPN output at any iteration and given as(23)Et=Ot-D=WtYt-D=WtYtX-D.The objective of the network for minimization of error given in (24) is defined as follows:(24)s.t.J=12PKTrEtEtT,where T
r represent the trace of matrix. Using gradient descent method updated weight is given by(25)Wt+1=Wt-βt∂J∂Wt,Vt+1=Vt-βt∂J∂Vt,or(26)Wt+1=Wt-βtPKEtYtT,Vt+1=Vt-βtPKWtTEtXT.Using (23)–(26), we have(27)Et+1Et+1T=Wt+1Yt+1-DWt+1Yt+1-DT.To obtain minimum error for multi-layer FNN in above equation (28) after simplification, we have(28)
J
t
+
1
-
J
t
=
1
2
P
K
g
β
,
J
=
1
2
P
K
∑
p
=
1
P
∑
k
=
1
K
o
p
k
-
d
p
k
2
,
E
t
+
1
E
t
+
1
T
=
W
t
+
1
V
t
+
1
X
-
D
W
t
+
1
V
t
+
1
X
-
D
T
=
W
t
-
β
t
P
K
E
t
Y
t
T
V
t
-
β
t
P
K
W
t
T
E
t
X
T
X
-
D
W
t
-
β
t
P
K
E
t
Y
t
T
V
t
-
β
t
P
K
W
t
T
E
t
X
T
X
-
D
T
=
E
t
-
β
t
P
K
W
t
W
t
T
E
t
X
T
X
+
E
t
Y
t
T
V
t
X
+
β
t
2
P
K
2
E
t
Y
t
T
W
t
T
E
t
X
T
X
∗
E
t
-
β
t
P
K
W
t
W
t
T
E
t
X
T
X
+
E
t
Y
t
T
V
t
X
+
β
t
2
P
K
2
E
t
Y
t
T
W
t
T
E
t
X
T
X
T
=
E
t
E
t
T
-
β
t
P
K
E
t
W
t
W
t
T
E
t
X
T
X
T
+
E
t
E
t
Y
t
T
V
t
X
T
+
W
t
W
t
T
E
t
X
T
X
E
t
T
+
E
t
Y
t
T
V
t
X
E
t
T
+
β
t
2
P
K
2
E
t
E
t
Y
t
T
W
t
T
E
t
X
T
X
T
+
E
t
Y
t
T
W
t
T
E
t
X
T
X
E
t
T
+
E
t
Y
t
T
V
t
X
W
t
W
t
T
E
t
X
T
X
T
+
W
t
W
t
T
E
t
X
T
X
E
t
Y
t
T
V
t
X
T
+
E
t
Y
t
T
V
t
X
E
t
Y
t
T
V
t
X
T
+
W
t
W
t
T
E
t
X
T
X
W
t
W
t
T
E
t
X
T
X
T
-
β
t
3
P
K
3
W
t
W
t
T
E
t
X
T
X
E
t
Y
t
T
W
t
T
E
t
X
T
X
+
E
t
Y
t
T
V
t
X
E
t
Y
t
T
W
t
T
E
t
X
T
X
T
+
E
t
Y
t
T
W
t
T
E
t
X
T
X
W
t
W
f
T
E
t
X
T
X
T
+
E
t
Y
t
T
W
t
T
E
t
X
T
X
E
t
Y
t
T
V
t
X
T
+
β
t
4
P
K
4
E
t
Y
t
T
W
t
T
E
t
X
T
X
E
t
Y
t
T
W
t
T
E
t
X
T
X
T
.For simplification, omit subscript t, we have; that is, (29)Jt+1-Jt=12PKAβ4+Bβ4+Cβ4+Mβwhere(30)
A
=
1
P
K
4
T
r
E
Y
T
W
E
X
T
X
E
Y
T
W
T
E
X
T
X
T
,
B
=
1
P
K
3
T
r
W
W
T
E
X
T
X
E
Y
T
W
T
E
X
T
X
T
+
E
Y
T
V
X
E
Y
T
W
T
E
X
T
X
T
+
E
Y
T
W
T
E
X
T
X
W
W
T
E
X
T
X
T
+
E
Y
T
W
T
E
X
T
X
E
Y
T
V
X
,
C
=
1
P
K
2
T
r
E
Y
T
W
T
E
X
T
X
T
+
E
Y
T
W
E
X
T
X
E
T
+
E
Y
T
V
X
W
W
T
E
X
T
X
T
+
W
W
T
E
X
T
X
+
E
Y
T
V
X
T
+
E
Y
T
V
X
E
Y
T
V
X
T
+
W
W
T
E
X
T
X
W
W
T
E
X
T
X
T
,
M
=
1
P
K
T
r
E
W
W
T
E
X
T
X
T
+
E
E
Y
T
V
X
T
+
W
W
T
E
X
T
X
+
E
Y
T
V
X
E
T
,where (31)gβ=Aβ4+Bβ3+Cβ2+Mβ,(32)∂g∂β=4Aβ3+aβ2+bβ+c,where a
=
3
B
/
4
A, b
=
2
C
/
4
A, and c
=
M
/
4
A.Lemma 2.
For solution of general real cubic equation, one uses the following lemma:(33)fx=x3+ax2+bx+c,LetD=-27c2+18cab+a2b2-4a3b3-4b3,where D is discriminant of f
(
x
).
Then we have the following:(1)
IfD
<
0, f
(
x
) has one real root.
(2)
IfD
≥
0, f
(
x
) has three real roots:
(a)
D
>
0; f
(
x
) has three different real roots,
(b)
D
=
0, 6
b
-
2
a
2
#
0; f
(
x
) has one single root and one multiple root,
(c)
D
=
0, 66
-
2
a
2
=
0; f
(
x
) has one root of three multiplicities.Lemma 3.
For given polynomialg
(
β
) given (31) if optimum β
=
β
i
∣
g
β
i
=
min
⁡
g
β
1
,
g
β
2
,
g
β
3
,
i
∈
1
,
2
,
3, where β
i is real root of∂
g
/
∂
β, the optimum (β) is the optimal learning rate and this learning process is stable.Proof.
To find stable learning range ofβ, consider that Lyapunov function is(34)Vt=Jt2,ΔVt=Jt+12-Jt2ifΔVt<0,and then dynamical system is guaranteed to be stable if Δ
V
t
<
0; that is, J
t
+
1
-
J
t
=
1
/
2
P
K
A
β
4
+
B
β
4
+
C
β
4
+
M
β
<
0 (29), where β is learning rate, since in the training process input matrix remains the same during the whole training process. To find the range of β, which satisfy g
β
=
A
β
4
+
B
β
3
+
C
β
2
+
M
β
<
0. Since ∂
g
/
∂
β, where has at least one real root (Lemma 2) and one of them must give optimum g
β. Obviously minimum value of g
β gives the largest reduction in J
t at each step of learning process. Equation (31) shows that g
β has two or four real roots, one including β
=
0 (Lemma 2), such that minimum value of β shows largest reduction in error at two successive times and minimum value is obtained by differentiating (31) with respect to β we have from (32): (35)∂g∂β=4Aβ3+aβ2+bβ+C,where a
=
3
B
/
4
A, b
=
2
C
/
4
A, and c
=
M
/
4
A.
Solving∂
g
/
∂
β
=
0 gives β which minimizes error in (5).


## 4.1. Optimal Learning of Dynamical System

Consider a three-layer CPN; the network error matrix is defined by the error between differences of desired and FCPN output at any iteration and given as(23)Et=Ot-D=WtYt-D=WtYtX-D.The objective of the network for minimization of error given in (24) is defined as follows:(24)s.t.J=12PKTrEtEtT,where T
r represent the trace of matrix. Using gradient descent method updated weight is given by(25)Wt+1=Wt-βt∂J∂Wt,Vt+1=Vt-βt∂J∂Vt,or(26)Wt+1=Wt-βtPKEtYtT,Vt+1=Vt-βtPKWtTEtXT.Using (23)–(26), we have(27)Et+1Et+1T=Wt+1Yt+1-DWt+1Yt+1-DT.To obtain minimum error for multi-layer FNN in above equation (28) after simplification, we have(28)
J
t
+
1
-
J
t
=
1
2
P
K
g
β
,
J
=
1
2
P
K
∑
p
=
1
P
∑
k
=
1
K
o
p
k
-
d
p
k
2
,
E
t
+
1
E
t
+
1
T
=
W
t
+
1
V
t
+
1
X
-
D
W
t
+
1
V
t
+
1
X
-
D
T
=
W
t
-
β
t
P
K
E
t
Y
t
T
V
t
-
β
t
P
K
W
t
T
E
t
X
T
X
-
D
W
t
-
β
t
P
K
E
t
Y
t
T
V
t
-
β
t
P
K
W
t
T
E
t
X
T
X
-
D
T
=
E
t
-
β
t
P
K
W
t
W
t
T
E
t
X
T
X
+
E
t
Y
t
T
V
t
X
+
β
t
2
P
K
2
E
t
Y
t
T
W
t
T
E
t
X
T
X
∗
E
t
-
β
t
P
K
W
t
W
t
T
E
t
X
T
X
+
E
t
Y
t
T
V
t
X
+
β
t
2
P
K
2
E
t
Y
t
T
W
t
T
E
t
X
T
X
T
=
E
t
E
t
T
-
β
t
P
K
E
t
W
t
W
t
T
E
t
X
T
X
T
+
E
t
E
t
Y
t
T
V
t
X
T
+
W
t
W
t
T
E
t
X
T
X
E
t
T
+
E
t
Y
t
T
V
t
X
E
t
T
+
β
t
2
P
K
2
E
t
E
t
Y
t
T
W
t
T
E
t
X
T
X
T
+
E
t
Y
t
T
W
t
T
E
t
X
T
X
E
t
T
+
E
t
Y
t
T
V
t
X
W
t
W
t
T
E
t
X
T
X
T
+
W
t
W
t
T
E
t
X
T
X
E
t
Y
t
T
V
t
X
T
+
E
t
Y
t
T
V
t
X
E
t
Y
t
T
V
t
X
T
+
W
t
W
t
T
E
t
X
T
X
W
t
W
t
T
E
t
X
T
X
T
-
β
t
3
P
K
3
W
t
W
t
T
E
t
X
T
X
E
t
Y
t
T
W
t
T
E
t
X
T
X
+
E
t
Y
t
T
V
t
X
E
t
Y
t
T
W
t
T
E
t
X
T
X
T
+
E
t
Y
t
T
W
t
T
E
t
X
T
X
W
t
W
f
T
E
t
X
T
X
T
+
E
t
Y
t
T
W
t
T
E
t
X
T
X
E
t
Y
t
T
V
t
X
T
+
β
t
4
P
K
4
E
t
Y
t
T
W
t
T
E
t
X
T
X
E
t
Y
t
T
W
t
T
E
t
X
T
X
T
.For simplification, omit subscript t, we have; that is, (29)Jt+1-Jt=12PKAβ4+Bβ4+Cβ4+Mβwhere(30)
A
=
1
P
K
4
T
r
E
Y
T
W
E
X
T
X
E
Y
T
W
T
E
X
T
X
T
,
B
=
1
P
K
3
T
r
W
W
T
E
X
T
X
E
Y
T
W
T
E
X
T
X
T
+
E
Y
T
V
X
E
Y
T
W
T
E
X
T
X
T
+
E
Y
T
W
T
E
X
T
X
W
W
T
E
X
T
X
T
+
E
Y
T
W
T
E
X
T
X
E
Y
T
V
X
,
C
=
1
P
K
2
T
r
E
Y
T
W
T
E
X
T
X
T
+
E
Y
T
W
E
X
T
X
E
T
+
E
Y
T
V
X
W
W
T
E
X
T
X
T
+
W
W
T
E
X
T
X
+
E
Y
T
V
X
T
+
E
Y
T
V
X
E
Y
T
V
X
T
+
W
W
T
E
X
T
X
W
W
T
E
X
T
X
T
,
M
=
1
P
K
T
r
E
W
W
T
E
X
T
X
T
+
E
E
Y
T
V
X
T
+
W
W
T
E
X
T
X
+
E
Y
T
V
X
E
T
,where (31)gβ=Aβ4+Bβ3+Cβ2+Mβ,(32)∂g∂β=4Aβ3+aβ2+bβ+c,where a
=
3
B
/
4
A, b
=
2
C
/
4
A, and c
=
M
/
4
A.Lemma 2.
For solution of general real cubic equation, one uses the following lemma:(33)fx=x3+ax2+bx+c,LetD=-27c2+18cab+a2b2-4a3b3-4b3,where D is discriminant of f
(
x
).
Then we have the following:(1)
IfD
<
0, f
(
x
) has one real root.
(2)
IfD
≥
0, f
(
x
) has three real roots:
(a)
D
>
0; f
(
x
) has three different real roots,
(b)
D
=
0, 6
b
-
2
a
2
#
0; f
(
x
) has one single root and one multiple root,
(c)
D
=
0, 66
-
2
a
2
=
0; f
(
x
) has one root of three multiplicities.Lemma 3.
For given polynomialg
(
β
) given (31) if optimum β
=
β
i
∣
g
β
i
=
min
⁡
g
β
1
,
g
β
2
,
g
β
3
,
i
∈
1
,
2
,
3, where β
i is real root of∂
g
/
∂
β, the optimum (β) is the optimal learning rate and this learning process is stable.Proof.
To find stable learning range ofβ, consider that Lyapunov function is(34)Vt=Jt2,ΔVt=Jt+12-Jt2ifΔVt<0,and then dynamical system is guaranteed to be stable if Δ
V
t
<
0; that is, J
t
+
1
-
J
t
=
1
/
2
P
K
A
β
4
+
B
β
4
+
C
β
4
+
M
β
<
0 (29), where β is learning rate, since in the training process input matrix remains the same during the whole training process. To find the range of β, which satisfy g
β
=
A
β
4
+
B
β
3
+
C
β
2
+
M
β
<
0. Since ∂
g
/
∂
β, where has at least one real root (Lemma 2) and one of them must give optimum g
β. Obviously minimum value of g
β gives the largest reduction in J
t at each step of learning process. Equation (31) shows that g
β has two or four real roots, one including β
=
0 (Lemma 2), such that minimum value of β shows largest reduction in error at two successive times and minimum value is obtained by differentiating (31) with respect to β we have from (32): (35)∂g∂β=4Aβ3+aβ2+bβ+C,where a
=
3
B
/
4
A, b
=
2
C
/
4
A, and c
=
M
/
4
A.
Solving∂
g
/
∂
β
=
0 gives β which minimizes error in (5).


## 5. Simulation Results

To demonstrate the effectiveness and merit of proposed FCPN, simulation results are presented and discussed. Four different nonlinear dynamical systems and one general benchmark problem known as Box-Jenkins model with time series data are considered.


### 5.1. Performance Criteria

For accessing the performance criteria of FCPN and comparing with Dynamic and Back Propagation Network, we have evaluated various errors as given below. In case of four dynamical models, it is recommended to access criterion such as subset of the following.GivenN pair of data points (
y
k
,
y
-
k
), where y
k is output of system and y
-
k is output of controller, the Maximum Absolute Error (MAE) is(36)JMAE=Jmax=max1≤k≤N⁡yk-y-k,the Sum of Squared Errors (SSE) is(37)JSSE=∑k=1Nyk-y-k2,the Mean Squared Error (MSE) is(38)JMSE=1N∑k=1Nyk-y-k2,and/or the Root Mean Squared Error (RMSE) is(39)JRMSE=JMSE.The measure Variance Accounting Factor (VAF) is(40)JVAF=1-Varyk-y-kVaryk100%.A related measure is the Normalized Mean Squared Error (NMSE):(41)JNMSE=∑k=1Nyk-y-k2∑k=1Nyk-y-2,and the Best Fit Rate (BFR) is(42)JBFR=1-∑k=1Nyk-y-k2∑k=1Nyk-y^2100%.Example 1.
The nonlinear dynamical system [24] is governed by the following difference equation:(43)yk+1=0.3yk+0.6yk-1+fuk.The nonlinear function in the system in this case is f
(
u
)
=
u
3
+
0.3
u
2
-
0.4
u, where y and u are uniformly distributed in [
-
2
,
2
]. The objective of Example 1 is to control the system to track reference output given as 250 sample data points. The model has two inputs y
k and u
k and single output y
k
+
1 and the system identification was initially performed with the system input being uniformly distributed over [
-
2
,
2
]. The FCPN controller uses two inputs y
k and u
k to produce output y
¯
k
+
1. FCPN model governed by the difference equation y
¯
k
+
1
=
0.3
y
k
+
0.6
y
k
-
1
+
N
[
u
k
] was used. Figures 4(a) and 4(b) show the error e
k
+
1
=
y
k
+
1
-
y
¯
k
+
1 and outputs of the dynamical system and the FCPN. As can be seen from the figure, the identification error is small even when the input is changed to a sum of two sinusoids u
(
k
)
=
sin
⁡
(
2
π
k
/
250
)
+
sin
⁡
(
2
π
k
/
25
) at k
=
250.Figure 4
(a) Mean Square Error of system (Example1) using FCPN. (b) Performance of the controller using FCPN algorithm for Example 1.
(a)
(b)Table1 includes various calculated errors of different NN models for Example 1. It can be seen from the table that various errors calculated for FCPN are minimum as compared to the DN and BPN.Table 1
Calculated various errors for Example1.
NN model
Different error calculation
MAE
SSE
MSE
RMSE
NMSE
FCPN
1.3364
0.1143
4.5709
e
-
004
0.0214
7.1627
e
-
004
Dynamic
5.8231
3.1918
0.0128
0.1130
0.0028
BPN
38.0666
48.1017
0.4810
0.6936
0.1035Example 2.
The nonlinear dynamical system [24] is governed by the following difference equation:(44)yk+1=fypk,ypk-1+uk,where(45)fypk,ypk-1=ypkypk-1ypk+2.51+yp2k+yp2k-1.The objective of (44) is to control the system to track reference output given 100 sample data points. The model has two inputs y
k and u
k and single output y
k
+
1 and the system identification was initially performed with the system input being uniformly distributed over [
-
2
,
2
]. Training and testing samples contain 100 sample data points. The FCPN controller uses two inputs y
k and u
k to output y
¯
k
+
1. FCPN network discussed earlier is used to identify the system from input-output data and is described by the equation(46)y¯k+1=Nyk,yk-1+uk.For FCPN, the identification process involves the adjustment of the weights of N using FCL. FCPN identifier needs some prior information concerning the input-output behavior of the system before identification can be undertaken. FCL is used to adjust the weights of the neural network so that the error e
k
+
1
=
y
k
+
1
-
y
¯
(
k
+
1
) is minimized as shown in Figure 5(a). The behavior of the FCPN model for (16) is shown in Figure 5(b). The input u
(
k
) was assumed to be a random signal uniformly distributed in the interval [
-
2
,
2
]. The weights in the neural network were adjusted.Figure 5
(a) Mean Square Error of system (Example2) using FCPN. (b) Performance of the controller using FCPN algorithm for Example 2.
(a)
(b)Table2 includes various calculated errors of different NN models for Example 2. It can be seen from the table that various errors calculated for FCPN are minimum as compared to the DN and BPN.Table 2
Calculated various errors for Example2.
NN models
Different error calculation
MAE
SSE
MSE
RMSE
NMSE
FCPN
0.1956
0.0032
3.2328
e
-
005
0.0057
2.7164
e
-
005
Dynamic
9.5152
2.9856
0.0299
0.1728
0.0308
BP
10.1494
4.5163
0.0452
0.2125
0.0323Example 3.
The nonlinear dynamical system [23] is governed by the following difference equation: (47)yk+1=yk1+yk2+u3k,which corresponds to f
y
k
=
y
(
k
)
/
1
+
y
(
k
)
2 and g
u
k
=
u
3
(
k
). FCPN network equation (47) is used to identify the system from input-output data. Weights in the neural networks were adjusted for every instant of discrete time steps.
The objective of (47) is to control the system to track reference output given 100 sample data points. The model has two inputs y
k and u
k and single output y
k
+
1 and the system identification was initially performed with the system input being uniformly distributed over [
-
2
,
2
]. Training and testing samples contain 250 sample data points. The FCPN controller uses two inputs y
k and u
k to output y
¯
k
+
1. The input data is generated using uniform distribution in interval [
-
2
,
2
]. The function f
¯ obtained by FCPN is used to adjust the weights of the neural network so that the error e
k
+
1
=
y
k
+
1
-
y
¯
(
k
+
1
) is minimized as shown in Figure 6(a). In Figure 6(b), the desired outputs of the system as well as the FCPN model are shown and are seen to be indistinguishable.Figure 6
(a) Mean Square Error of system (Example3) using FCPN. (b) Performance of the controller using FCPN algorithm for Example 3.
(a)
(b)Table3 includes various calculated errors of different NN models for Example 3. It can be seen from the table that various errors calculated for FCPN are minimum as compared to the DN and BPN.Table 3
Calculated various errors and BFR for Example3.
NN model
MAE
SSE
MSE
RMSE
NMSE
FCPN
0.3519
0.0032
3.1709
e
-
005
0.0056
5.6371
e
-
006
Dynamic
5.5652
3.6158
0.0362
0.1902
0.0223
BP
42.6524
46.9937
0.4699
0.6855
0.2892Example 4.
The nonlinear dynamical system [24] is governed by the difference equation (48). Generalized form of this equation is Model IV. In this example, the system is assumed to be of the form(48)ypk+1=fypk,ypk-1,ypk-2,uk,uk-1,where the unknown function f has the form(49)fx1,x2,x3,x4,x5=x1x2x3x5x3-1+x41+x32+x22.In the identification model, which is approximated by FCPN, Figure 7(b) shows the output of the system and the model when the identification procedure carried random input signal uniformly distributed in the interval [
-
1
,
1
]. The performance of the model is studied and the error e
k
+
1
=
N
[
y
p
k
+
1
] is minimized as shown in Figure 7(a). In Figure 7(b), the outputs of the system as well as output of FCPN model are shown. Input to the system and the identified model is given by u
k
=
sin
⁡
(
2
π
k
/
250
) for k
≤
500 and u
k
=
0.8
sin
⁡
(
2
π
k
/
250
)
+
0.2
sin
⁡
(
2
π
k
/
25
) for k
>
500.Figure 7
(a) Mean Square Error of system (Example4) using FCPN. (b) Performance of the controller using FCPN algorithm for Example 4.
(a)
(b)Table4 includes various calculated errors of different NN models for Example 4. It can be seen from the table that various errors calculated for FCPN are minimum as compared to the DN and BPN.Table 4
Calculated various errors and BFR for Example4.
NN models
MAE
SSE
MSE
RMSE
NMSE
FCPN
30.4348
2.1931
0.0027
0.0524
0.0186
Dynamic
32.8952
6.4059
0.0080
0.0895
0.0276
BP
53.1988
13.4174
0.0168
0.1295
0.0844Example 5.
In this example we have used Box-Jenkins time series [32], of 296 pairs of data measured from a gas furnace system with single input u
(
t
) being gas flow rate and single output y
(
t
) being CO2 concentration in outlet gas. Training samples and testing samples contained 196 and 100 data points, respectively. The FCPN uses the two inputs, the current state y
(
k
) and the desired state y
d
(
k
), to produce an output which is y
¯
(
k
) [33].
Figure8(a) shows the mean squared control errors of FCPN methods. Figure 8(b) shows the performance of the controller with FCPN algorithm. Result shows that FCPN algorithm enable us to appropriate the approximation using fuzzy learning as given in equation (11) of Box Jenkins time series data, based on calculation of BMN.Figure 8
(a) Mean Square Error of system (Example5) using FCPN. (b) Performance of the controller using FCPN algorithm for Example 5.
(a)
(b)Table5 shows BFR (%) various NN models for all Examples 1–5, respectively. It can be observed from Table 5 that the Best Fit Rate found for FCPN is maximum as compared to DN and BPN which shows better performance of the FCPN network for nonlinear system.Table 5
BFR (%) values of NN models for various examples.
NN models
BFR (%)
Example1
Example2
Example3
Example4
FCPN
97.32
99.48
99.76
86.35
DN
94.69
79.32
85.08
83.39
BPN
67.83
79.28
46.2234
70.94


## 5.1. Performance Criteria

For accessing the performance criteria of FCPN and comparing with Dynamic and Back Propagation Network, we have evaluated various errors as given below. In case of four dynamical models, it is recommended to access criterion such as subset of the following.GivenN pair of data points (
y
k
,
y
-
k
), where y
k is output of system and y
-
k is output of controller, the Maximum Absolute Error (MAE) is(36)JMAE=Jmax=max1≤k≤N⁡yk-y-k,the Sum of Squared Errors (SSE) is(37)JSSE=∑k=1Nyk-y-k2,the Mean Squared Error (MSE) is(38)JMSE=1N∑k=1Nyk-y-k2,and/or the Root Mean Squared Error (RMSE) is(39)JRMSE=JMSE.The measure Variance Accounting Factor (VAF) is(40)JVAF=1-Varyk-y-kVaryk100%.A related measure is the Normalized Mean Squared Error (NMSE):(41)JNMSE=∑k=1Nyk-y-k2∑k=1Nyk-y-2,and the Best Fit Rate (BFR) is(42)JBFR=1-∑k=1Nyk-y-k2∑k=1Nyk-y^2100%.Example 1.
The nonlinear dynamical system [24] is governed by the following difference equation:(43)yk+1=0.3yk+0.6yk-1+fuk.The nonlinear function in the system in this case is f
(
u
)
=
u
3
+
0.3
u
2
-
0.4
u, where y and u are uniformly distributed in [
-
2
,
2
]. The objective of Example 1 is to control the system to track reference output given as 250 sample data points. The model has two inputs y
k and u
k and single output y
k
+
1 and the system identification was initially performed with the system input being uniformly distributed over [
-
2
,
2
]. The FCPN controller uses two inputs y
k and u
k to produce output y
¯
k
+
1. FCPN model governed by the difference equation y
¯
k
+
1
=
0.3
y
k
+
0.6
y
k
-
1
+
N
[
u
k
] was used. Figures 4(a) and 4(b) show the error e
k
+
1
=
y
k
+
1
-
y
¯
k
+
1 and outputs of the dynamical system and the FCPN. As can be seen from the figure, the identification error is small even when the input is changed to a sum of two sinusoids u
(
k
)
=
sin
⁡
(
2
π
k
/
250
)
+
sin
⁡
(
2
π
k
/
25
) at k
=
250.Figure 4
(a) Mean Square Error of system (Example1) using FCPN. (b) Performance of the controller using FCPN algorithm for Example 1.
(a)
(b)Table1 includes various calculated errors of different NN models for Example 1. It can be seen from the table that various errors calculated for FCPN are minimum as compared to the DN and BPN.Table 1
Calculated various errors for Example1.
NN model
Different error calculation
MAE
SSE
MSE
RMSE
NMSE
FCPN
1.3364
0.1143
4.5709
e
-
004
0.0214
7.1627
e
-
004
Dynamic
5.8231
3.1918
0.0128
0.1130
0.0028
BPN
38.0666
48.1017
0.4810
0.6936
0.1035Example 2.
The nonlinear dynamical system [24] is governed by the following difference equation:(44)yk+1=fypk,ypk-1+uk,where(45)fypk,ypk-1=ypkypk-1ypk+2.51+yp2k+yp2k-1.The objective of (44) is to control the system to track reference output given 100 sample data points. The model has two inputs y
k and u
k and single output y
k
+
1 and the system identification was initially performed with the system input being uniformly distributed over [
-
2
,
2
]. Training and testing samples contain 100 sample data points. The FCPN controller uses two inputs y
k and u
k to output y
¯
k
+
1. FCPN network discussed earlier is used to identify the system from input-output data and is described by the equation(46)y¯k+1=Nyk,yk-1+uk.For FCPN, the identification process involves the adjustment of the weights of N using FCL. FCPN identifier needs some prior information concerning the input-output behavior of the system before identification can be undertaken. FCL is used to adjust the weights of the neural network so that the error e
k
+
1
=
y
k
+
1
-
y
¯
(
k
+
1
) is minimized as shown in Figure 5(a). The behavior of the FCPN model for (16) is shown in Figure 5(b). The input u
(
k
) was assumed to be a random signal uniformly distributed in the interval [
-
2
,
2
]. The weights in the neural network were adjusted.Figure 5
(a) Mean Square Error of system (Example2) using FCPN. (b) Performance of the controller using FCPN algorithm for Example 2.
(a)
(b)Table2 includes various calculated errors of different NN models for Example 2. It can be seen from the table that various errors calculated for FCPN are minimum as compared to the DN and BPN.Table 2
Calculated various errors for Example2.
NN models
Different error calculation
MAE
SSE
MSE
RMSE
NMSE
FCPN
0.1956
0.0032
3.2328
e
-
005
0.0057
2.7164
e
-
005
Dynamic
9.5152
2.9856
0.0299
0.1728
0.0308
BP
10.1494
4.5163
0.0452
0.2125
0.0323Example 3.
The nonlinear dynamical system [23] is governed by the following difference equation: (47)yk+1=yk1+yk2+u3k,which corresponds to f
y
k
=
y
(
k
)
/
1
+
y
(
k
)
2 and g
u
k
=
u
3
(
k
). FCPN network equation (47) is used to identify the system from input-output data. Weights in the neural networks were adjusted for every instant of discrete time steps.
The objective of (47) is to control the system to track reference output given 100 sample data points. The model has two inputs y
k and u
k and single output y
k
+
1 and the system identification was initially performed with the system input being uniformly distributed over [
-
2
,
2
]. Training and testing samples contain 250 sample data points. The FCPN controller uses two inputs y
k and u
k to output y
¯
k
+
1. The input data is generated using uniform distribution in interval [
-
2
,
2
]. The function f
¯ obtained by FCPN is used to adjust the weights of the neural network so that the error e
k
+
1
=
y
k
+
1
-
y
¯
(
k
+
1
) is minimized as shown in Figure 6(a). In Figure 6(b), the desired outputs of the system as well as the FCPN model are shown and are seen to be indistinguishable.Figure 6
(a) Mean Square Error of system (Example3) using FCPN. (b) Performance of the controller using FCPN algorithm for Example 3.
(a)
(b)Table3 includes various calculated errors of different NN models for Example 3. It can be seen from the table that various errors calculated for FCPN are minimum as compared to the DN and BPN.Table 3
Calculated various errors and BFR for Example3.
NN model
MAE
SSE
MSE
RMSE
NMSE
FCPN
0.3519
0.0032
3.1709
e
-
005
0.0056
5.6371
e
-
006
Dynamic
5.5652
3.6158
0.0362
0.1902
0.0223
BP
42.6524
46.9937
0.4699
0.6855
0.2892Example 4.
The nonlinear dynamical system [24] is governed by the difference equation (48). Generalized form of this equation is Model IV. In this example, the system is assumed to be of the form(48)ypk+1=fypk,ypk-1,ypk-2,uk,uk-1,where the unknown function f has the form(49)fx1,x2,x3,x4,x5=x1x2x3x5x3-1+x41+x32+x22.In the identification model, which is approximated by FCPN, Figure 7(b) shows the output of the system and the model when the identification procedure carried random input signal uniformly distributed in the interval [
-
1
,
1
]. The performance of the model is studied and the error e
k
+
1
=
N
[
y
p
k
+
1
] is minimized as shown in Figure 7(a). In Figure 7(b), the outputs of the system as well as output of FCPN model are shown. Input to the system and the identified model is given by u
k
=
sin
⁡
(
2
π
k
/
250
) for k
≤
500 and u
k
=
0.8
sin
⁡
(
2
π
k
/
250
)
+
0.2
sin
⁡
(
2
π
k
/
25
) for k
>
500.Figure 7
(a) Mean Square Error of system (Example4) using FCPN. (b) Performance of the controller using FCPN algorithm for Example 4.
(a)
(b)Table4 includes various calculated errors of different NN models for Example 4. It can be seen from the table that various errors calculated for FCPN are minimum as compared to the DN and BPN.Table 4
Calculated various errors and BFR for Example4.
NN models
MAE
SSE
MSE
RMSE
NMSE
FCPN
30.4348
2.1931
0.0027
0.0524
0.0186
Dynamic
32.8952
6.4059
0.0080
0.0895
0.0276
BP
53.1988
13.4174
0.0168
0.1295
0.0844Example 5.
In this example we have used Box-Jenkins time series [32], of 296 pairs of data measured from a gas furnace system with single input u
(
t
) being gas flow rate and single output y
(
t
) being CO2 concentration in outlet gas. Training samples and testing samples contained 196 and 100 data points, respectively. The FCPN uses the two inputs, the current state y
(
k
) and the desired state y
d
(
k
), to produce an output which is y
¯
(
k
) [33].
Figure8(a) shows the mean squared control errors of FCPN methods. Figure 8(b) shows the performance of the controller with FCPN algorithm. Result shows that FCPN algorithm enable us to appropriate the approximation using fuzzy learning as given in equation (11) of Box Jenkins time series data, based on calculation of BMN.Figure 8
(a) Mean Square Error of system (Example5) using FCPN. (b) Performance of the controller using FCPN algorithm for Example 5.
(a)
(b)Table5 shows BFR (%) various NN models for all Examples 1–5, respectively. It can be observed from Table 5 that the Best Fit Rate found for FCPN is maximum as compared to DN and BPN which shows better performance of the FCPN network for nonlinear system.Table 5
BFR (%) values of NN models for various examples.
NN models
BFR (%)
Example1
Example2
Example3
Example4
FCPN
97.32
99.48
99.76
86.35
DN
94.69
79.32
85.08
83.39
BPN
67.83
79.28
46.2234
70.94


## 6. Conclusions

FCPN is a neural network control method which was developed and presented. It is based on the concept of combining FCL algorithm and CPN. The key ideas explored are the use of the FCL algorithm for training the weights of the instar-outstar. The performances of the FCPN controller based training are tested using four nonlinear dynamical systems and on time series (Box-Jenkins) data and compared with the Dynamic Network and standard Back Propagation algorithm. The comparative performances of the FCPN algorithm and Dynamic Network, such as the number of iterations and performance functions like MAE, MSE, SSE, NMSE, BFR, and so forth, of FCPN and Dynamic Network error are summarized in Tables1–4. It can be seen that the FCPN algorithm gives minimum errors as compared to the Dynamic Network and standard Back Propagation Network for all four models of nonlinear dynamical systems and Box-Jenkins time series data. Results obtained from FCPN were compared for various errors and it is clearly shown that FCPN works much better for the control of nonlinear dynamical systems.



---

*Source: 719620-2015-08-20.xml*