# Controller Design Based on Echo State Network with Delay Output for Nonlinear System

Authors: Xianshuang Yao; Siyuan Fan; Bo Zhao; Shengxian Cao
Journal: Complexity
 (2020)

---

## Abstract

For the nonlinear systems with delay output, the control performance of the system is affected by the previous output of the system, such as crawling robots of photovoltaic panels. In this paper, an improved controller design method based on echo state network with delay output (DO-ESN) is proposed for designing the controller of a class of nonlinear system. According to the internal characteristics of DO-ESN, the DO-ESN can match the system characteristics of nonlinear systems with delay output, such that the proposed controller can quickly meet the control performance of the nonlinear system. In order to ensure the stability of the controller, a sufficient condition is given for the echo state property of DO-ESN. Finally, a simulation example is used to illustrate the effectiveness of the proposed method.


---

## Body

## 1. Introduction

For the nonlinear system, the proportional-integral-derivative (PID) method is a very effective control method, which has been widely applied in many fields, for example, load frequency control [1], automatic voltage regulator [2], chemical engineering [3], vehicle dynamic [4], hydrogen energy [5], and others [6]. However, the setting problem of PID controller parameters is always the focus in the research of academic and industry community. In recent years, based on neural network, many different controller design methods have been presented, for example, BPNN [7], RBFNN [8, 9], single neuron adaptive neural networks (SNANN) [10], adaptive neural networks [11], and artificial intelligence algorithm [12]. For these design methods based on artificial neural networks, some appropriate PID parameters can be obtained to meet the control accuracy requirement in some industrial applications.However, there are still some problems for these design methods, for example, through using BPNN, the obtained PID controller parameters are easily trapped in local minima. Because of the randomness of the center vector, it is difficult to obtain the appropriate PID controller parameters by using RBFNN in a short time. Through analyzing these design methods based on artificial neural networks, one can summarize these drawbacks including training weights, local minima, and slow convergence. Therefore, an improved neural network should be introduced into the controller design method to solve the above shortcomings.Echo state network (ESN) [13, 14] uses a dynamical reservoir with many random connected neurons to replace the hidden layer of recurrent neural network (RNN) [15–20]. Compared with the traditional RNN, the advantages of ESN are reflected in the weight selection and weight learning of network, i.e., only the output weight needs to be learned. Therefore, ESN not only has the network structure of traditional RNN but also has the characteristics of deep learning, such that ESN can be applied in many fields, for example, time-series prediction [21–24], filtering or control [25–28], dynamic pattern recognition [29–31], optimization [32], system identification [31, 33, 34], and big data application [35, 36]. Thus, comparing with the existing controller design methods based on neural network, ESN can avoid lots of adjusting parameters and the limitation of calculation. Because the reservoir of ESN has the memory ability, the previous calculation results can be used for the next parameter calculation. In addition, ESN can also deepen the calculation of input information to improve the convergence speed.Combining with the advantages of ESN, some improved methods have been applied in many control fields. For example, in [26], a PID adaptive controller based on RLSESN is proposed for realizing high-accurate tracking of the rehabilitation robotic hand. In [27], ESNBIMC is proposed for the pneumatic muscle system. The ESNBIMC is regarded as a replacement of the conventional PID controller which is used to adjust the input of the controlled plant. In [28], through using the approximation capability of ESN, the ESN is regarded as a controller which is applied for nonlinear dynamical systems. However, for a class of nonlinear systems with delay output, the current moment output of the system is affected by some previous moment output of the system, such that these existing control methods based on echo state network cannot well match the characteristics of the system. Thus, for the nonlinear systems with delay output, how to build a suitable echo state network to meet the characteristics of the system is an interesting problem.Thus, in this paper, an improved echo state network with delay output (DO-ESN) is proposed. Based on the DO-ESN, an improved controller design method is given for a class of nonlinear systems with delay output, such that the output of the DO-ESN controller can quickly meet the control performance of the nonlinear system. In order to ensure the stability of the controller, the echo state property of DO-ESN should be first guaranteed, and thus, a sufficient condition is given for the echo state property of DO-ESN.The main work of this paper is given as follows:(i)
An improved controller design method based on DO-ESN is given for a class of nonlinear systems with delay output, such that the control performance of this nonlinear system can be quickly satisfied(ii)
A sufficient condition is given for the echo state property of DO-ESN, and the proposed controller can be stably applied to nonlinear systems with delay output


## 2. Echo State Network with Delay Output

ESN is a class of recurrent neural networks, whose structure is shown in Figure1. From Figure 1, the ESN comprises a K-dimensional input layer, a N-dimensional reservoir, and a L-dimensional output layer. Let u=un, x=xn, and y=yn denote the external input vector, the reservoir state, and the output vector, respectively. Win∈ℝN×K denotes the input weight matrix, W∈ℝN×N denotes the reservoir weight matrix, Wfb∈ℝN×L denotes the output feedback weight matrix and Wout∈ℝL×K+N denotes the output weight matrix. For the standard ESN, the reservoir state updated equation is given as follows [14]:Figure 1
Structure of ESN.(1)xn+1=fWinun+1+Wxn+Wfbyn,yn=gWoutxn;un,wheref⋅ and g⋅ denote the activation function of reservoir state and network output, respectively.Consider the characteristics of the nonlinear system with delay output, the previous moment output of ESN can be introduced into the reservoir state equation, and thus an improved echo state network with delay output (DO-ESN) is proposed. The reservoir state updated equation of DO-ESN is given as follows:(2)xn+1=axn+fWinun+1+Wxn+Wfb1yn+Wfb2yn−τ,(3)yn=Woutxn;un−αyn−τ,where τ denote the index of delay output.Remark 1.
For the DO-ESN, the current reservoir state and the current output are affected by the previous output. The current reservoir state can use the previous system characteristic, and the current output can use the previous output characteristic. Therefore, the DO-ESN can better match the characteristics of the nonlinear system with delayed output.


## 3. Controller Design Method Based on DO-ESN

### 3.1. Controller Based on DO-ESN

Considering a class of nonlinear system with delay output,(4)yn=fsyn−1,…,yn−τ1,ucn,…,ucn−τ2,where τ1 and τ2 denote the delay of yn and ucn, respectively. fs denotes the nonlinear polynomial.According to the learning ability and approximation ability of DO-ESN in control and optimization, the DO-ESN can be embedded into the controller to control the nonlinear system. The design controller method based on the DO-ESN is illustrated in Figure2.Figure 2
The controller design based on DO-ESN.From Figure2, esn denotes the system output error, ysn denotes the actual system output, and rn denotes the desired system output.


### 3.2. Stability Analysis of Controller Based on DO-ESN

In order to guarantee the stability and convergence of the controller model, the echo state property of DO-ESN should be ensured. Thus, in the following, a sufficient condition will be provided to guarantee the echo state property of DO-ESN.Because theWoutn can be written as follows:(5)Woutn=WxoutWuout.The output of DO-ESN is modified as follows:(6)yn=Woutxn;un−αyn−τ=Wxoutxn+Wuoutun−αyn−τ.Thus, the reservoir state is modified as follows:(7)xn+1=axn+fWinun+1+Wxn+Wfb1Wxoutxn+Wuoutun−αyn−τ+Wfb2yn−τ=axn+fWinun+1+Wxn+Wfb1Wxoutxn+Wfb1Wuoutun+Wfb2−αWfb1yn−τ=axn+fWinun+1+W∗xn++Wu∗un+Wfb∗yn−τ,where Wfb∗=Wfb2−αWfb1, W∗=W+Wfb1Wxout, and Wu∗=Wfb1Wuout.Theorem 1.
For a DO-ESN model (equations (2) and (3)), if the following conditions are satisfied(1)
f is a Lipschitz continuous function, and the Lipschitz constant is less than 1(2)
Wfb1≠0 and Wfb2≠0(3)
n=mτ(4)
a+δmax∗+δfbmax∗1−α+α2−α3+⋯+−1mαm−1<1 (where δmax∗ and δfbmax∗ are the maximal singular value of W∗ and Wfb∗Wxout, respectively); then, the DO-ESN model has the echo state property in each iterationProof.
Forxn+1 and x′n+1, we have(8)xn+1−x′n+1=axn+fWinun+1+W∗xn+Wu∗un+Wfb∗yn−τ−ax′n−fWinun+1+W∗x′n+Wu∗un+Wfb∗y′n−τ≤axn−ax′n+fWinun+1+W∗xn+Wu∗un+Wfb∗yn−τ−fWinun+1+W∗x′n+Wu∗un+Wfb∗y′n−τ≤axn−x′n+Winun+1+W∗xn+Wu∗un+Wfb∗yn−τ−Winun+1−W∗x′n−Wu∗un−Wfb∗y′n−τ≤axn−x′n+W∗xn−W∗x′n+Wfb∗yn−τ−Wfb∗y′n−τ≤axn−x′n+W∗xn−x′n+Wfb∗yn−τ−y′n−τ.
For the termyn−τ−y′n−τ of equation (8), we have(9)yn−τ−y′n−τ=Wxoutxn−τ+Wuoutun−τ−αyn−2τ−Wxoutx′n−τ−Wuoutun−τ+αy′n−2τ≤Wxoutxn−τ−x′n−τ−αyn−2τ−y′n−2τ.
Similarly, we also have(10)yn−2τ−y′n−2τ=Wxoutxn−2τ+Wuoutun−2τ−αyn−3τ−Wxoutx′n−2τ−Wuoutun−2τ+αy′n−3τ≤Wxoutxn−2τ−x′n−2τ−αyn−3τ−y′n−3τ,(11)⋮yn−m−1τ−y′n−m−1τ=Wxoutxnm−1τ+Wuoutunm−1τ−αyn−mτ−Wxoutx′nm−1τ−Wuoutunm−1τ+αy′n−mτ≤Wxoutxnm−1τ−x′nm−1τ−αy0−y′0.
Substituting equations (10) and (11) into equation (9), we have(12)yn−τ−y′n−τ≤Wxoutxn−τ−x′n−τ−αWxoutxn−2τ−x′n−2τ+α2Wxoutxn−3τ−x′n−3τ−α3Wxoutxn−4τ−x′n−4τ+⋯+−1mαm−1Wxoutxn−mτ−x′n−mτ.
Substituting equation (12) into equation (8), we have(13)xn+1−x′n+1≤axn−x′n+W∗xn−x′n+Wfb∗Wxoutxn−τ−x′n−τ−αWxoutxn−2τ−x′n−2τ+α2Wxoutxn−3τ−x′n−3τ−α3Wxoutxn−4τ−x′n−4τ+⋯+αm−1Wxoutxn−mτ−x′n−mτ≤axn−x′n+W∗xn−x′nWfb∗Wxoutxn−τ−x′n−τ−αxn−2τ−x′n−2τ+α2xn−3τ−x′n−3τ−α3xn−4τ−x′n−4τ+⋯+−1mαm−1xn−mτ−x′n−mτ.
If(14)a+δmax∗+δfbmax∗1−α+α2−α3+⋯+−1mαm−1<1.
Then, the DO-ESN has the echo state property in each iteration.Remark 2.
For the proposed controller design method based on DO-ESN, the stability of the controller should be guaranteed, and then, a sufficient condition need be given for the echo state property of DO-ESN. Thus, in this section, the proposed controller design method can be stably applied to nonlinear systems with delay output.


### 3.3. Learning the Output Weights of DO-ESN

The purpose of designing the controller is to minimize the system output error. The expression of the system output erroresn is given as follows: (15)esn=rn−ysn.For the output weight learning of DO-ESN, the partial derivative∂esn/∂Woutn need be computed. ∂esn/∂Woutn is formulated as follows: (16)∂esn∂Woutn=∂esn∂ysn∗∂ysn∂Wout.Then, the output weightsWoutn+1 is updated as follows:(17)Woutn+1=Woutn+β∂esn∂Woutn,where β denotes the learning rate.Remark 3.
In this paper, we usually give a definite value for the output weight learning rate. Since the learning rate is also affected by the reservoir parameters of DO-ESN, for different nonlinear systems, the learning rate can be given according to the changes of the reservoir state in the process of training network.


## 3.1. Controller Based on DO-ESN

Considering a class of nonlinear system with delay output,(4)yn=fsyn−1,…,yn−τ1,ucn,…,ucn−τ2,where τ1 and τ2 denote the delay of yn and ucn, respectively. fs denotes the nonlinear polynomial.According to the learning ability and approximation ability of DO-ESN in control and optimization, the DO-ESN can be embedded into the controller to control the nonlinear system. The design controller method based on the DO-ESN is illustrated in Figure2.Figure 2
The controller design based on DO-ESN.From Figure2, esn denotes the system output error, ysn denotes the actual system output, and rn denotes the desired system output.


## 3.2. Stability Analysis of Controller Based on DO-ESN

In order to guarantee the stability and convergence of the controller model, the echo state property of DO-ESN should be ensured. Thus, in the following, a sufficient condition will be provided to guarantee the echo state property of DO-ESN.Because theWoutn can be written as follows:(5)Woutn=WxoutWuout.The output of DO-ESN is modified as follows:(6)yn=Woutxn;un−αyn−τ=Wxoutxn+Wuoutun−αyn−τ.Thus, the reservoir state is modified as follows:(7)xn+1=axn+fWinun+1+Wxn+Wfb1Wxoutxn+Wuoutun−αyn−τ+Wfb2yn−τ=axn+fWinun+1+Wxn+Wfb1Wxoutxn+Wfb1Wuoutun+Wfb2−αWfb1yn−τ=axn+fWinun+1+W∗xn++Wu∗un+Wfb∗yn−τ,where Wfb∗=Wfb2−αWfb1, W∗=W+Wfb1Wxout, and Wu∗=Wfb1Wuout.Theorem 1.
For a DO-ESN model (equations (2) and (3)), if the following conditions are satisfied(1)
f is a Lipschitz continuous function, and the Lipschitz constant is less than 1(2)
Wfb1≠0 and Wfb2≠0(3)
n=mτ(4)
a+δmax∗+δfbmax∗1−α+α2−α3+⋯+−1mαm−1<1 (where δmax∗ and δfbmax∗ are the maximal singular value of W∗ and Wfb∗Wxout, respectively); then, the DO-ESN model has the echo state property in each iterationProof.
Forxn+1 and x′n+1, we have(8)xn+1−x′n+1=axn+fWinun+1+W∗xn+Wu∗un+Wfb∗yn−τ−ax′n−fWinun+1+W∗x′n+Wu∗un+Wfb∗y′n−τ≤axn−ax′n+fWinun+1+W∗xn+Wu∗un+Wfb∗yn−τ−fWinun+1+W∗x′n+Wu∗un+Wfb∗y′n−τ≤axn−x′n+Winun+1+W∗xn+Wu∗un+Wfb∗yn−τ−Winun+1−W∗x′n−Wu∗un−Wfb∗y′n−τ≤axn−x′n+W∗xn−W∗x′n+Wfb∗yn−τ−Wfb∗y′n−τ≤axn−x′n+W∗xn−x′n+Wfb∗yn−τ−y′n−τ.
For the termyn−τ−y′n−τ of equation (8), we have(9)yn−τ−y′n−τ=Wxoutxn−τ+Wuoutun−τ−αyn−2τ−Wxoutx′n−τ−Wuoutun−τ+αy′n−2τ≤Wxoutxn−τ−x′n−τ−αyn−2τ−y′n−2τ.
Similarly, we also have(10)yn−2τ−y′n−2τ=Wxoutxn−2τ+Wuoutun−2τ−αyn−3τ−Wxoutx′n−2τ−Wuoutun−2τ+αy′n−3τ≤Wxoutxn−2τ−x′n−2τ−αyn−3τ−y′n−3τ,(11)⋮yn−m−1τ−y′n−m−1τ=Wxoutxnm−1τ+Wuoutunm−1τ−αyn−mτ−Wxoutx′nm−1τ−Wuoutunm−1τ+αy′n−mτ≤Wxoutxnm−1τ−x′nm−1τ−αy0−y′0.
Substituting equations (10) and (11) into equation (9), we have(12)yn−τ−y′n−τ≤Wxoutxn−τ−x′n−τ−αWxoutxn−2τ−x′n−2τ+α2Wxoutxn−3τ−x′n−3τ−α3Wxoutxn−4τ−x′n−4τ+⋯+−1mαm−1Wxoutxn−mτ−x′n−mτ.
Substituting equation (12) into equation (8), we have(13)xn+1−x′n+1≤axn−x′n+W∗xn−x′n+Wfb∗Wxoutxn−τ−x′n−τ−αWxoutxn−2τ−x′n−2τ+α2Wxoutxn−3τ−x′n−3τ−α3Wxoutxn−4τ−x′n−4τ+⋯+αm−1Wxoutxn−mτ−x′n−mτ≤axn−x′n+W∗xn−x′nWfb∗Wxoutxn−τ−x′n−τ−αxn−2τ−x′n−2τ+α2xn−3τ−x′n−3τ−α3xn−4τ−x′n−4τ+⋯+−1mαm−1xn−mτ−x′n−mτ.
If(14)a+δmax∗+δfbmax∗1−α+α2−α3+⋯+−1mαm−1<1.
Then, the DO-ESN has the echo state property in each iteration.Remark 2.
For the proposed controller design method based on DO-ESN, the stability of the controller should be guaranteed, and then, a sufficient condition need be given for the echo state property of DO-ESN. Thus, in this section, the proposed controller design method can be stably applied to nonlinear systems with delay output.


## 3.3. Learning the Output Weights of DO-ESN

The purpose of designing the controller is to minimize the system output error. The expression of the system output erroresn is given as follows: (15)esn=rn−ysn.For the output weight learning of DO-ESN, the partial derivative∂esn/∂Woutn need be computed. ∂esn/∂Woutn is formulated as follows: (16)∂esn∂Woutn=∂esn∂ysn∗∂ysn∂Wout.Then, the output weightsWoutn+1 is updated as follows:(17)Woutn+1=Woutn+β∂esn∂Woutn,where β denotes the learning rate.Remark 3.
In this paper, we usually give a definite value for the output weight learning rate. Since the learning rate is also affected by the reservoir parameters of DO-ESN, for different nonlinear systems, the learning rate can be given according to the changes of the reservoir state in the process of training network.


## 4. Simulation Examples

In this section, a simulation example is selected to show the performance of the proposed control method. For different output error accuracy (5%, 2%, 0.1%, or E-10), several neural networks methods (e.g., BPNN [7], RBFNN [9], and ESN [15]) are selected to illustrate the performance of the proposed design method.Considering the following nonlinear system with time delay,(18)yn=−0.1yn−1+ucn−11+yn−12.The desired outputrn of system is(19)rn=0.5signsin2ntsπ,where ts=0.001.According to the characteristic of the nonlinear system, the reservoir size of DO-ESN is given asN=25. Through equation (17) with β=0.0002, the final reservoir parameters of DO-ESN are obtained as follows:(20)ax=0.000231,af=0.995,ρ=0.623,sin=0.324,α=0.821.The simulation results of these methods (BPNN [7], RBFNN [9], ESN [15], and DO-ESN) are shown in Table 1. The curves of system output and system output error are shown in Figures 3 and 4, respectively.Table 1
Comparison of error precision and iterations for BPNN, RBFNN, ESN, and DO-ESN.
IterationsHiddenError precisionSize0.0250.015E-4E-10BPNN [7]20145212462–RBFNN [9]15105123245–ESN [15]256074125359DO-ESN25223456172Figure 3
The output response curve of the control system.Figure 4
The output error curve of the control system.From Table1, we can see that when the output value of the system reaches the expected fluctuation range (5%–2%), the DO-ESN needs less iteration step. For example, when the error precision is 0.025(5%), the iteration step of DO-ESN is 22, and the iteration step of those methods [7, 9, 15, 21] are 60, 145, 40, and 105, respectively. When the error precision is 0.01 (2%), the iteration of DO-ESN is 34, and the iteration of other methods [7, 9, 15, 21] is 74, 212, 62, and 123, respectively. When the error precision is 0.0005(0.1%), the iteration of DO-ESN is 56, and the iteration of other methods [7, 9, 15, 21] is 125, 462, 96, and 245, respectively. From Figures 3 and 4, we can see that, for the same error precision, the iteration step of DO-ESN is far less than that of those methods in [7, 9, 15].


## 5. Discussion

In this paper, an improved controller design method based on DO-ESN is proposed for a class of nonlinear system with delay output. According to the internal characteristics of DO-ESN, the DO-ESN can match the system characteristics of nonlinear systems with delay output, such that the DO-ESN controller can quickly meet the control performance of the nonlinear system. Meanwhile, a sufficient condition for the stability of the DO-ESN controller is given, such that the controller can be stably applied to the given controlled plant. Finally, the simulation result shows the effectiveness of the proposed method.



---

*Source: 8643029-2020-09-27.xml*