# Adaptive Control Using Neural Networks and Approximate Models for Nonlinear Dynamic Systems

Authors: Khadija El Hamidi; Mostafa Mjahed; Abdeljalil El Kari; Hassan Ayad
Journal: Modelling and Simulation in Engineering
 (2020)

---

## Abstract

In this research, a comparative study of two recurrent neural networks, nonlinear autoregressive with exogenous input (NARX) neural network and nonlinear autoregressive moving average (NARMA-L2), and a feedforward neural network (FFNN) is performed for their ability to provide adaptive control of nonlinear systems. Three dynamical nonlinear systems of different complexity are considered. The aim of this work is to make the output of the plant follow the desired reference trajectory. The problem becomes more challenging when the dynamics of the plants are assumed to be unknown, and to tackle this problem, a multilayer neural network-based approximate model is set up which will work in parallel to the plant and the control scheme. The network parameters are updated using the dynamic backpropagation (BP) algorithm.


---

## Body

## 1. Introduction

Linear control methods are based on the existence of an analytical model of the system. However, most physical systems have nonlinearity, and their mathematical model is unknown or partially known and variable in time. So, conventional methods suffer some limitation in terms of stabilization and performance [1, 2]. With the considerable development of artificial intelligence, which is found to be more suitable in handling such complex processes, the artificial neural network (ANN) is one of the powerful tools that has been recognized for tracking highly nonlinear dynamic and complex systems [2–5].In the field of control engineering for a class of nonlinear discrete-time systems, a neural network has recently emerged as adjustable approximators capable of reproducing the complex behavior of nonlinear systems. Artificial neural networks have been effectively used as tracking controllers for unknown linear and nonlinear dynamic plants [6, 7]. ANNs have been employed in various fields, like time series prediction, system identification and control, and function approximation [8]. It has been shown that ANNS can efficiently approximate dynamics without requiring detailed knowledge of the plant [8, 9]. Another advantage of ANNs is their possibility of learning, which can reduce the human effort during the design of the controllers and allows discovering more effective control structures than those already known.There are different types of ANNs proposed in the literature: feedforward neural networks (FFNNs), Kohonen self-organizing network, radial basis function (RBF), and recurrent neural network [10–12]. In fact, two classes of neural networks are the most popular in practical applications and have received considerable attention in the area of artificial neural networks in recent years: multilayer feedforward neural networks and recurrent networks. Multilayer feedforward neural networks have an ability to model any nonlinear function and have proven extremely successful in pattern recognition tasks [13, 14] while recurrent networks have been used in associative memories as well as for the solution of optimization problems, and it constitutes also a powerful computational tool for sequence modeling and prediction. Both of these networks have common characteristics like parallelism in their operation, generalization, and learning capability, which make them suitable candidates for the control of nonlinear systems [2]; there are compelling reasons to view them in a unified fashion.Various structures of neural networks have been proposed in the literature. In [2], the authors have applied a diagonal recurrent neural network (DRNN) as a controller for both single-input single-output (SISO) and multiple-input multiple-output (MIMO) plants. The responses of plants obtained with DRNN are compared with those obtained when a multilayer feedforward neural network is used as a controller. In [15], recurrent neural networks have been used for providing speed control to the nonlinear motor-drive system using a model-following control scheme. In [4], an adaptive controller for a class of unknown nonlinear discrete-time systems based on a multi-input fuzzy rules emulated network (MIFREN) is introduced; the neural network is assigned to identify the unknown system under control. The authors in [3] designed a nonlinear autoregressive moving average (NARMA-L2) controller, which is based on an adaptive neurofuzzy inference system. The NARMA-L2 controller for a single-link manipulator has also been used in [16, 17]. A Lyapunov function-based neural network tracking (LNT) strategy for SISO discrete-time nonlinear dynamic systems is proposed. The proposed scheme uses two Lyapunov function neural networks operating as the controller and estimator. In [18], both the feedforward and recurrent neural network approaches are proposed, tested, and compared.The main contribution of this paper is to propose a strong nonlinear adaptive controller of unknown nonlinear dynamical systems based on the approximate models. Another objective is to present a prescriptive method for the dynamic adjustment of the parameters based on backpropagation. The big advantage of the proposed control system is that it does not require previous knowledge of the model. Our ultimate goal is to determine the control input using only the values of the input and output.In our research study, the strong learning capability of the dynamic neural network in identification and control is combined with the functionality of the approximate model controller structure to propose a novel online NN-based controller for nonlinear single-input single-output (SISO) dynamical systems. The main issues in the field of ANN-based control are the choice of neural network structure to be used as a controller. In our paper, an attempt has been made to compare the three types of neural networks: two recurrent neural networks, nonlinear autoregressive with exogenous input (NARX) neural network and nonlinear autoregressive and moving average (NARMA-L2), and a feedforward neural network (FFNN). The control system configuration employed in our paper consists of an ANN-based controller in cascade with the plant, and the training is performed online.This study differentiates from the studies in the literature in terms of mathematically obtaining nonlinear SISO controller parameters. Thus, the main novelty of this paper is that the parameters of the nonlinear SISO controller can be identified as neural network expressions for nonlinear SISO systems. All neural network simulations are performed in a Matlab environment. The results indicate that the online NARMA-L2, NARX, and FFNN controllers attain good modeling and control performances.The remainder of the paper is organized as follows: Section2 includes a brief introduction about NARX, NARMA-L2, and FFNN models and their mathematical formulation. Section 3 includes the discussion on the adaptive control of nonlinear systems. Section 4 contains the simulation study. In Section 5, the conclusion of the paper is given.


## 2. Neural Network Approximate Models and Their Mathematical Formulation

Generally, there are many different neural networks (NN) of nonlinear models. In this research study, the recurrent and feedforward neural networks are presented. Since a lot of literature exists on NARX, NARMA-L2, and FFNN, in this section, a brief introduction regarding their mathematical formulation is given. Figures1–3 show the structures of FFNN, NARX, and NARMA-L2 models, respectively.Figure 1
A feedforward neural network structure.Figure 2
Nonlinear autoregressive with exogenous input (NARX) model.Figure 3
Nonlinear autoregressive and moving average.


### 2.1. Feedforward Neural Network Model

In a feedforward neural network, the information moves only in one direction, from the input layer to the output layer, but not vice versa as presented in Figure1 [19]. The input vector of FFNN at any kth time instant is denoted by Xk=x1k,x2k,⋯xpk. Thus, there are p numbers of inputs. The weighted sums of inputs of hidden neurons are denoted by the vector Sk where Sk=S1k,S2k,⋯Sqk. The output of hidden neurons is denoted by a vector ak=a1k,a2k,⋯aqk. Further, a tangent hyperbolic function is used as an activation function for hidden neurons and a linear activation function for output neuron. The input weight vector WIk shows the connection weight between the external applied inputs and the neurons of a hidden layer. The adjustable output weight vector, WOk=W1Ok,W2Ok,⋯WqOk, represents the hidden to the output layer connections, and yFFNNk+1 denotes the FFNN output. The mathematical model of FFNN is given by
(1)Sjk=∑i=1pWijIxik+bjj=1⋯q,ajk=fSjk,yFFNNk+1=∑j=1qajkW1jok+bj.


### 2.2. NARX Model

The NARX model represents a generic recurrent neural network having one step ahead output,yNARXk+1, depending upon the present and past values of the input which are called the exogenous inputs, namely, uk,uk−1,⋯uk−du+1, as well as on its delayed values of the output, that is, yNARXk,yNARXk−1,⋯yNARXk−dy+1 [10]. In the NARX neural network model, the internal architecture that performs this approximation is the multilayer perceptron (MLP). The dynamic behavior of the NARX model may be written in the following form:
(2)yNARXk+1=g∑h=1NWhof∑i=1duWihIuk−j+1+∑j=1dyWjhIyNARXk−j+1+b.Note that in Figure2, we have assumed that the two delay line memories du and dy are both equal to the order of the plant or different with du≤dy, where f and g are a hyperbolic tangent and a linear activation function characterizing the hidden and output layers of the MLP, respectively, and N denotes the number of hidden layers.


### 2.3. NARMA-L2 Model

The nonlinear autoregressive moving average (NARMA) (Figure3) is one of the most certified representations of general discrete-time nonlinear systems. This model representation is used in the form of the past, the current, and the future system parameters, as shown in [3]
(3)yk+d=Φyk,yk−1,⋯yk−n+1,uk,uk−1,⋯uk−n+1,where yk and uk are the input and output of the system, respectively, the relative degree d represents the delay of the system from control effort u to the output y, and Φ· is a nonlinear function. This model is not convenient for finding a control signal uk. To overcome this problem, an efficient method is proposed by Narendra and Mukhopadhyay by introducing approximation models. Two classes of the NARMA model have been proposed, namely, NARMA-L1 and NARMA-L2. It was found that the second class involving two subapproximation functions is more efficient and adequate in the identification and adaptation of control contexts [20].
(4)yNARMA−L2k+d=f0yk,yk−1,⋯yk−n+1,uk−1,⋯uk−n+1+g0yk,yk−1,⋯yk−n+1,uk−1,⋯uk−n+1uk,(5)yNARMA−L2k+d=N1yk,yk−1,⋯yk−n+1,uk−1,⋯uk−n+1+N2yk,yk−1,⋯yk−n+1,uk−1,⋯uk−n+1uk.The NARMA-L2 is obtained by(6)yNARMA−L2k+d=g∑h=1h1Wh1of∑i=1duWih1Iuk−j+1+∑j=1dyWjh1Iypk−j+1+b1+g∑h=1h2Wh2of∑i=1duWih2Iuk−j+1+∑j=1dyWjh2Iypk−j+1+b2,where f and g are, respectively, the activation functions of the hidden layers and the linear layers of the networks N1 and N2.Equations (5) to (7) represent single-input single-output (SISO) systems. As it can be seen from equation (4), NARMA-L2 consists of two nonlinear functions, f0 and g0, which can be approximated using two subnetworks (N1 and N2) as presented in equation (5). To design the neurocontroller, the number of delayed plant inputs and outputs is chosen based on a structural model. The size of the hidden layer is chosen such that the network can accurately approximate the nonlinearity of the system. There are two steps to construct the NARMA-L2 controller: identification and control.The two subfunctions,N1 and N2, are used in the identification phase as well as to compute a signal as follows:
(7)uk=yrk+d−N1yk,⋯yk−n+1,uk−1,⋯,uk−n+1N2yk,⋯,yk−n+1,uk−1,⋯,uk−n+1,where yrk+d is the reference signal to follow. This controller can be implemented using the model of the NARMA-L2 system previously identified. If the system is precisely approximated, the output of the system will be equal to the output of the reference model.


## 2.1. Feedforward Neural Network Model

In a feedforward neural network, the information moves only in one direction, from the input layer to the output layer, but not vice versa as presented in Figure1 [19]. The input vector of FFNN at any kth time instant is denoted by Xk=x1k,x2k,⋯xpk. Thus, there are p numbers of inputs. The weighted sums of inputs of hidden neurons are denoted by the vector Sk where Sk=S1k,S2k,⋯Sqk. The output of hidden neurons is denoted by a vector ak=a1k,a2k,⋯aqk. Further, a tangent hyperbolic function is used as an activation function for hidden neurons and a linear activation function for output neuron. The input weight vector WIk shows the connection weight between the external applied inputs and the neurons of a hidden layer. The adjustable output weight vector, WOk=W1Ok,W2Ok,⋯WqOk, represents the hidden to the output layer connections, and yFFNNk+1 denotes the FFNN output. The mathematical model of FFNN is given by
(1)Sjk=∑i=1pWijIxik+bjj=1⋯q,ajk=fSjk,yFFNNk+1=∑j=1qajkW1jok+bj.


## 2.2. NARX Model

The NARX model represents a generic recurrent neural network having one step ahead output,yNARXk+1, depending upon the present and past values of the input which are called the exogenous inputs, namely, uk,uk−1,⋯uk−du+1, as well as on its delayed values of the output, that is, yNARXk,yNARXk−1,⋯yNARXk−dy+1 [10]. In the NARX neural network model, the internal architecture that performs this approximation is the multilayer perceptron (MLP). The dynamic behavior of the NARX model may be written in the following form:
(2)yNARXk+1=g∑h=1NWhof∑i=1duWihIuk−j+1+∑j=1dyWjhIyNARXk−j+1+b.Note that in Figure2, we have assumed that the two delay line memories du and dy are both equal to the order of the plant or different with du≤dy, where f and g are a hyperbolic tangent and a linear activation function characterizing the hidden and output layers of the MLP, respectively, and N denotes the number of hidden layers.


## 2.3. NARMA-L2 Model

The nonlinear autoregressive moving average (NARMA) (Figure3) is one of the most certified representations of general discrete-time nonlinear systems. This model representation is used in the form of the past, the current, and the future system parameters, as shown in [3]
(3)yk+d=Φyk,yk−1,⋯yk−n+1,uk,uk−1,⋯uk−n+1,where yk and uk are the input and output of the system, respectively, the relative degree d represents the delay of the system from control effort u to the output y, and Φ· is a nonlinear function. This model is not convenient for finding a control signal uk. To overcome this problem, an efficient method is proposed by Narendra and Mukhopadhyay by introducing approximation models. Two classes of the NARMA model have been proposed, namely, NARMA-L1 and NARMA-L2. It was found that the second class involving two subapproximation functions is more efficient and adequate in the identification and adaptation of control contexts [20].
(4)yNARMA−L2k+d=f0yk,yk−1,⋯yk−n+1,uk−1,⋯uk−n+1+g0yk,yk−1,⋯yk−n+1,uk−1,⋯uk−n+1uk,(5)yNARMA−L2k+d=N1yk,yk−1,⋯yk−n+1,uk−1,⋯uk−n+1+N2yk,yk−1,⋯yk−n+1,uk−1,⋯uk−n+1uk.The NARMA-L2 is obtained by(6)yNARMA−L2k+d=g∑h=1h1Wh1of∑i=1duWih1Iuk−j+1+∑j=1dyWjh1Iypk−j+1+b1+g∑h=1h2Wh2of∑i=1duWih2Iuk−j+1+∑j=1dyWjh2Iypk−j+1+b2,where f and g are, respectively, the activation functions of the hidden layers and the linear layers of the networks N1 and N2.Equations (5) to (7) represent single-input single-output (SISO) systems. As it can be seen from equation (4), NARMA-L2 consists of two nonlinear functions, f0 and g0, which can be approximated using two subnetworks (N1 and N2) as presented in equation (5). To design the neurocontroller, the number of delayed plant inputs and outputs is chosen based on a structural model. The size of the hidden layer is chosen such that the network can accurately approximate the nonlinearity of the system. There are two steps to construct the NARMA-L2 controller: identification and control.The two subfunctions,N1 and N2, are used in the identification phase as well as to compute a signal as follows:
(7)uk=yrk+d−N1yk,⋯yk−n+1,uk−1,⋯,uk−n+1N2yk,⋯,yk−n+1,uk−1,⋯,uk−n+1,where yrk+d is the reference signal to follow. This controller can be implemented using the model of the NARMA-L2 system previously identified. If the system is precisely approximated, the output of the system will be equal to the output of the reference model.


## 3. Controller Design

The control configuration based on the NN approximate model is shown in Figure4. In this control scheme, a NN model is utilized to approximate the nonlinear function of the unmodeled dynamics. The system model is considered unknown. The use of neural networks is justified by this absence of a model. In this case, three approaches predominate. In the two approaches based on the FFNN and NARX models, a single network is used for system control. The third approach based on the NARMA-L2 model uses two networks to deduce the controller adaptation law.Figure 4
Adaptive control using neural network approximate models.


### 3.1. Learning Algorithm

The process of training consists in modifying the weights in an organized way using an appropriate algorithm. During the training process, a specified number of inputs and their desired output are introduced in the network. Then, the weights are tuned so that the neural network produces an output close to the target values. The fundamental training algorithm for multilayer networks is the backpropagation (BP) algorithm [20]. This algorithm is of iterative type, and it is based on the minimization of a sum-squared error (MSE) utilizing the optimization gradient descent method. The MSE is used as the cost function which is a function of error, which is defined as follows:
(8)Ek=12∑k=1Nyrk−yk2,where yrk and yk are the desired output of the network and the actual response of the network on the given input pattern uk, respectively, and N is the dimension of the training set. The modification of the weights of the jth neuron in the μth layer is performed according to the formula:
(9)Wijμk+1=Wijμk−η∂E∂Wijμ,biμk+1=biμk−η∂E∂biμ,where
(10)∂E∂Wijμ=∂E∂Sijμ×∂Sijμ∂Wijμ,∂E∂biμ=∂E∂Sijμ×∂Sijμ∂biμ,Sijμ=∑j=1μ−1Wijμajμ−1+biμ.Therefore,aμ=fμSijμ, ∂Siμ/∂Wijμ=ajμ−1, and ∂Siμ/∂biμ=1.If we define∂E/∂Sijμ=δiμ, then, ∂E/∂Wijμ=δiμajμ−1.Thus, each element inWμk and bμk can be updated as
(11)Wijμk+1=Wijμk−ηδiμajμ−1,biμk+1=biμk−ηδiμ.The errorδ is defined as follows:
(12)δjμk=−f′Sjμkek,forμ=MOL,f′Sjμk∑p=1μ+1δpμ+1kWpjμ+1,forμ=1,⋯,M−1HL,δjμk=−ek,forμ=MOL,1−aj2kδμ+1kWijμ+1,forμ=1,⋯,M−1HL,where fμx=tanhx=ex−e−x/ex+e−x and fμ+1x=x.


## 3.1. Learning Algorithm

The process of training consists in modifying the weights in an organized way using an appropriate algorithm. During the training process, a specified number of inputs and their desired output are introduced in the network. Then, the weights are tuned so that the neural network produces an output close to the target values. The fundamental training algorithm for multilayer networks is the backpropagation (BP) algorithm [20]. This algorithm is of iterative type, and it is based on the minimization of a sum-squared error (MSE) utilizing the optimization gradient descent method. The MSE is used as the cost function which is a function of error, which is defined as follows:
(8)Ek=12∑k=1Nyrk−yk2,where yrk and yk are the desired output of the network and the actual response of the network on the given input pattern uk, respectively, and N is the dimension of the training set. The modification of the weights of the jth neuron in the μth layer is performed according to the formula:
(9)Wijμk+1=Wijμk−η∂E∂Wijμ,biμk+1=biμk−η∂E∂biμ,where
(10)∂E∂Wijμ=∂E∂Sijμ×∂Sijμ∂Wijμ,∂E∂biμ=∂E∂Sijμ×∂Sijμ∂biμ,Sijμ=∑j=1μ−1Wijμajμ−1+biμ.Therefore,aμ=fμSijμ, ∂Siμ/∂Wijμ=ajμ−1, and ∂Siμ/∂biμ=1.If we define∂E/∂Sijμ=δiμ, then, ∂E/∂Wijμ=δiμajμ−1.Thus, each element inWμk and bμk can be updated as
(11)Wijμk+1=Wijμk−ηδiμajμ−1,biμk+1=biμk−ηδiμ.The errorδ is defined as follows:
(12)δjμk=−f′Sjμkek,forμ=MOL,f′Sjμk∑p=1μ+1δpμ+1kWpjμ+1,forμ=1,⋯,M−1HL,δjμk=−ek,forμ=MOL,1−aj2kδμ+1kWijμ+1,forμ=1,⋯,M−1HL,where fμx=tanhx=ex−e−x/ex+e−x and fμ+1x=x.


## 4. Results and Discussion

Simulation studies have been carried out using the Matlab software environment to verify the performance of the proposed methods. Three different nonlinear dynamical systems are presented to evaluate the performance of the NN controllers. Two hidden layers are used withH1NN=20 and H2NN=10 (thus, 20 neurons are present in the first and 10 neurons in the second hidden layers).


### 4.1. Example 1

Consider the nonlinear discrete time which is described by the third-order difference equation [19]:
(13)yk+1=5ypkypk−11+yp2k+yp2k−1+yp2k−2+uk+0.8uk−1.The system can be reformulated as(14)yk+1=hyk,yk−1,yk−2+uk+0.8uk−1.A reference model is described by the third-order difference equation:(15)yrk+1=0.32yrk+0.64yrk−1−0.5yrk−2+rk,where rk is a bounded reference input. The function h is considered to be unknown; it can be estimated using neural network Nh. The control input to the plant at any instant k is computed using Nh− in place of h.Ifyk+1=yrk+1, then,
(16)uk=−Nhyk,yk−1,yk−2−0.8uk−1+0.32yrk+0.64yrk−1−0.5yrk−2+rk,whereas for NARMA-L2 the control signal is generated based on equation (7):
(17)uk=yrk+d−f0yk,yk−1,yk−2,uk−1g0yk,yk−1,yk−2,uk−1.N1 and N2 are used to learn f0· and g0−; then,
(18)uk=yrk+d−N1yk,yk−1,yk−2,uk−1N2yk,yk−1,yk−2,uk−1.The simulation reported in Figures5–7 indicates stable and efficient online control. The improvements in the responses, when the neural networks in the approximate model are used to generate the control input to the plant, are evident from the figure. The outputs of the controlled plant and the reference model are shown and indicate that the output error is almost zero. All three neural networks FFNN-, NARX-, and NARMA-L2-based controllers provided satisfactory results.Figure 5
Example 1: (a) response for no control action (withr=u=sin2πk/25+sin2πk/10), (b) outputs of the reference model and the plant with control action, (c) FFNN controller output, and (d) control error.
(a)(b)(c)(d)Figure 6
Example 1: (a) response of the reference model, (b) outputs of the reference model and the plant with control action, (c) NARX controller output, and (d) control error.
(a)(b)(c)(d)Figure 7
Example 1: (a) response of the reference model, (b) outputs of the reference model and the plant with control action, (c) NARMA-L2 controller output, and (d) control error.
(a)(b)(c)(d)The performance index of the different controllers with various neural network models is given in Table1. We can notice that all the designed controllers give better performance. Nevertheless, the performance of the FFNN and NARX indicated better attainment regarding the performance indices (MSE).Table 1
Comparison of the performance index of the proposed controller and other controllers (example 1).
ControllerMSENbr. of iterationsLearning rateRBFN [9]0.01220000.005DRNN [2]0.025320000.0154FFNN1.6366×10−311000.1NARX5.1093×10−191000.1NARMA-L20.251000.1


### 4.2. Example 2

Consider the nonlinear discrete-time system [1, 17]:
(19)yk+1=sinyk+uk5+cosykuk,where yk stands for the system output and uk is the control input at time index k. The output yk+1 is required to track the desired trajectory yrk+1 given in
(20)yrk=sin2πk10+sin2πk25×rk.The order of the plant isn=1, so the inputs for the FFNN controller are yrkandyk and the inputs for the NARMA-L2 controller are ykanduk, and NARX models are y∗k,yk,anduk.The control results for different epochs and learning rates along with control error are shown in Figure8. As we can see from the results, the network is unstable using n=m=1 in the delayed input/output (not enough information).Figure 8
Plant output and reference model response forn=m=1.The model of discrete-time plants introduced here can be also described by the following nonlinear difference equations:(21)yk+1=hyk,yk−1,⋯,yk−n+1+∑i=0m−1βuk−i.The same proposed scheme in Figure4 is considered. To approximate the unknown h, an FNN-based identification model is used:
(22)y∗k+1=Nhyk,yk−1+uk.The control action is required to getyrk=yk.In this example, performance comparisons of the BP-NNs are carried out with the three approximate models, FFNN, NARX, and NARMA-L2. The results are shown in Figures9–11. Also, notice that after sufficient online training of the plant, it has also converged to the desired response. The result during the training is presented in the figures. It can be seen from these figures that the control error (ec) reduces quickly to zero in the case of the NARX and NARMA-L2 controllers as compared to the control error obtained with FFNN. This suggests the strong learning ability of recurrent NN models over FFNN in this example.Figure 9
Example 2: (a) response of an ANN-NARX identification model, (b) response of the plant under the NARX controller, (c) NARX controller output, and (d) control error with the NARX model (usingn=3 and m=2).
(a)(b)(c)(d)Figure 10
Example 2: (a) response of the reference model, (b) response of the plant under the NARMA-L2 controller, (c) NARMA-L2 controller output, and (d) control error with the NARMA-L2 model (usingn=3 and m=2).
(a)(b)(c)(d)Figure 11
Example 2: (a) response of an ANN-FFNN identification model, (b) response of the plant under the FFNN controller, (c) FFNN controller output, and (d) control error with the FFNN model (usingn=3 and m=2).
(a)(b)(c)(d)The numerical parameters used in the second system are presented in Table2, as well as the mean square error (MSE) obtained with different controllers.Table 2
Comparison of the performance index of the proposed controller (example 2).
ControllerMSENbr. of iterationsLearning rateFFRN0.06981000.01NARX1.0027×10−133000.01NARMA-L22.99×10−61000.01


### 4.3. Example 3

The second-order differential equation which describes the dynamics of a single-link robotic manipulator is given as(23)ml2d2ytdt2+Ddytdt+mglcosyt=ut.The angular position of the robotic manipulator arm is represented byyt. The other parameters include link length which is denoted by l. Also, the term Ddyt/dt quantifies the viscous friction torque, and g=9.8m/s2 is the acceleration due to gravity. For simplicity, the values of m=l=D=1 are used in this paper. Further, ut is the control torque exerted on the robotic arm. The corresponding state space representation of the above differential equation is given by
(24)ẏy¨=01−gcosyl−Dml2yẏ+01ml2u.The difference equation of the one-link robotic manipulator with a sampling period is given by(25)yk+1=hyk,yk−1+T2uk−1,where the nonlinear function h is
(26)hyk,yk−1=2−Tyk+yk−1T−1−9.8T2cosyk−1.The desired reference model dynamics is given by the following difference equation:(27)yrk+1=yrk2−1.5T+yrk−1−1−2.5T2+1.5T+T2rk−1,where rk=sinkT is the externally applied input.Before control action, the sampling periodT should be chosen with care. It is taken to be T=0.4. In this part, the control is to make the robotic link output follow the reference trajectory signal. The response of the plant without control is shown in Figure 12. Clearly, the plant’s output is not following the reference input. So, the control scheme is set up and the training is continued for 100 epochs, and each epoch contains 100 training samples.Figure 12
Reference model and open-loop response of the plant without a controller (example 3).In the simulation studies,N1 and N2 were multilayer neural networks chosen with three-layer networks with p inputs, 20 neurons in the first hidden layer, 10 neurons in the second hidden layer, and one output neuron.From equation (25), the control input uk can be computed from acknowledging yk+1 and its past values as
(28)uk=−fyk,yk−1+yk+1×1T2,uk=−fyk,yk−1+yk2−1.5T+yk−1−1−2.5T2+1.5T+T2rk−1×1T2,rk=sin2πk25+sin2πk10.The mean square error (MSE) obtained with different controllers for example 3 is presented in Table3. From the table, it can be seen that the MSE obtained with FFNN, NARX, and NARMA-L2 is minimum as compared to the MSE obtained with the RBFN- and DRNN-based controllers.Table 3
Comparison of the performance index of the proposed controller and other controllers (example 3).
ControllerMSENbr. of iterationsLearning rateRBFN [9]0.142720000.027DRNN [2]0.02888000.0354FFNN9.644×10−3210000.05NARX2.1077×10−315000.1NARMA-L21.802×10−311000.01In this study, all the three methods provide simultaneous structure and parameter learning within the approximate models. The learning rate,η, is taken to be 0.01. From Figures 13–15, it can be seen that the response of the robotic link is successfully controlled.Figure 13
Example 3: (a) response of the reference model, (b) response of the plant under the adaptive controller, (c) FFNN controller output, and (d) control error (usingn=2 and m=2) (online learning).
(a)(b)(c)(d)Figure 14
Example 3: (a) response of the reference model, (b) response of the plant under the adaptive controller, (c) NARX controller output, and (d) control error (usingn=2 and m=2).
(a)(b)(c)(d)Figure 15
Example 3: (a) response of the reference model, (b) response of the plant under the adaptive NARMA-L2 controller, (c) controller output, (d) control error (usingn=2 and m=2).
(a)(b)(c)(d)


## 4.1. Example 1

Consider the nonlinear discrete time which is described by the third-order difference equation [19]:
(13)yk+1=5ypkypk−11+yp2k+yp2k−1+yp2k−2+uk+0.8uk−1.The system can be reformulated as(14)yk+1=hyk,yk−1,yk−2+uk+0.8uk−1.A reference model is described by the third-order difference equation:(15)yrk+1=0.32yrk+0.64yrk−1−0.5yrk−2+rk,where rk is a bounded reference input. The function h is considered to be unknown; it can be estimated using neural network Nh. The control input to the plant at any instant k is computed using Nh− in place of h.Ifyk+1=yrk+1, then,
(16)uk=−Nhyk,yk−1,yk−2−0.8uk−1+0.32yrk+0.64yrk−1−0.5yrk−2+rk,whereas for NARMA-L2 the control signal is generated based on equation (7):
(17)uk=yrk+d−f0yk,yk−1,yk−2,uk−1g0yk,yk−1,yk−2,uk−1.N1 and N2 are used to learn f0· and g0−; then,
(18)uk=yrk+d−N1yk,yk−1,yk−2,uk−1N2yk,yk−1,yk−2,uk−1.The simulation reported in Figures5–7 indicates stable and efficient online control. The improvements in the responses, when the neural networks in the approximate model are used to generate the control input to the plant, are evident from the figure. The outputs of the controlled plant and the reference model are shown and indicate that the output error is almost zero. All three neural networks FFNN-, NARX-, and NARMA-L2-based controllers provided satisfactory results.Figure 5
Example 1: (a) response for no control action (withr=u=sin2πk/25+sin2πk/10), (b) outputs of the reference model and the plant with control action, (c) FFNN controller output, and (d) control error.
(a)(b)(c)(d)Figure 6
Example 1: (a) response of the reference model, (b) outputs of the reference model and the plant with control action, (c) NARX controller output, and (d) control error.
(a)(b)(c)(d)Figure 7
Example 1: (a) response of the reference model, (b) outputs of the reference model and the plant with control action, (c) NARMA-L2 controller output, and (d) control error.
(a)(b)(c)(d)The performance index of the different controllers with various neural network models is given in Table1. We can notice that all the designed controllers give better performance. Nevertheless, the performance of the FFNN and NARX indicated better attainment regarding the performance indices (MSE).Table 1
Comparison of the performance index of the proposed controller and other controllers (example 1).
ControllerMSENbr. of iterationsLearning rateRBFN [9]0.01220000.005DRNN [2]0.025320000.0154FFNN1.6366×10−311000.1NARX5.1093×10−191000.1NARMA-L20.251000.1


## 4.2. Example 2

Consider the nonlinear discrete-time system [1, 17]:
(19)yk+1=sinyk+uk5+cosykuk,where yk stands for the system output and uk is the control input at time index k. The output yk+1 is required to track the desired trajectory yrk+1 given in
(20)yrk=sin2πk10+sin2πk25×rk.The order of the plant isn=1, so the inputs for the FFNN controller are yrkandyk and the inputs for the NARMA-L2 controller are ykanduk, and NARX models are y∗k,yk,anduk.The control results for different epochs and learning rates along with control error are shown in Figure8. As we can see from the results, the network is unstable using n=m=1 in the delayed input/output (not enough information).Figure 8
Plant output and reference model response forn=m=1.The model of discrete-time plants introduced here can be also described by the following nonlinear difference equations:(21)yk+1=hyk,yk−1,⋯,yk−n+1+∑i=0m−1βuk−i.The same proposed scheme in Figure4 is considered. To approximate the unknown h, an FNN-based identification model is used:
(22)y∗k+1=Nhyk,yk−1+uk.The control action is required to getyrk=yk.In this example, performance comparisons of the BP-NNs are carried out with the three approximate models, FFNN, NARX, and NARMA-L2. The results are shown in Figures9–11. Also, notice that after sufficient online training of the plant, it has also converged to the desired response. The result during the training is presented in the figures. It can be seen from these figures that the control error (ec) reduces quickly to zero in the case of the NARX and NARMA-L2 controllers as compared to the control error obtained with FFNN. This suggests the strong learning ability of recurrent NN models over FFNN in this example.Figure 9
Example 2: (a) response of an ANN-NARX identification model, (b) response of the plant under the NARX controller, (c) NARX controller output, and (d) control error with the NARX model (usingn=3 and m=2).
(a)(b)(c)(d)Figure 10
Example 2: (a) response of the reference model, (b) response of the plant under the NARMA-L2 controller, (c) NARMA-L2 controller output, and (d) control error with the NARMA-L2 model (usingn=3 and m=2).
(a)(b)(c)(d)Figure 11
Example 2: (a) response of an ANN-FFNN identification model, (b) response of the plant under the FFNN controller, (c) FFNN controller output, and (d) control error with the FFNN model (usingn=3 and m=2).
(a)(b)(c)(d)The numerical parameters used in the second system are presented in Table2, as well as the mean square error (MSE) obtained with different controllers.Table 2
Comparison of the performance index of the proposed controller (example 2).
ControllerMSENbr. of iterationsLearning rateFFRN0.06981000.01NARX1.0027×10−133000.01NARMA-L22.99×10−61000.01


## 4.3. Example 3

The second-order differential equation which describes the dynamics of a single-link robotic manipulator is given as(23)ml2d2ytdt2+Ddytdt+mglcosyt=ut.The angular position of the robotic manipulator arm is represented byyt. The other parameters include link length which is denoted by l. Also, the term Ddyt/dt quantifies the viscous friction torque, and g=9.8m/s2 is the acceleration due to gravity. For simplicity, the values of m=l=D=1 are used in this paper. Further, ut is the control torque exerted on the robotic arm. The corresponding state space representation of the above differential equation is given by
(24)ẏy¨=01−gcosyl−Dml2yẏ+01ml2u.The difference equation of the one-link robotic manipulator with a sampling period is given by(25)yk+1=hyk,yk−1+T2uk−1,where the nonlinear function h is
(26)hyk,yk−1=2−Tyk+yk−1T−1−9.8T2cosyk−1.The desired reference model dynamics is given by the following difference equation:(27)yrk+1=yrk2−1.5T+yrk−1−1−2.5T2+1.5T+T2rk−1,where rk=sinkT is the externally applied input.Before control action, the sampling periodT should be chosen with care. It is taken to be T=0.4. In this part, the control is to make the robotic link output follow the reference trajectory signal. The response of the plant without control is shown in Figure 12. Clearly, the plant’s output is not following the reference input. So, the control scheme is set up and the training is continued for 100 epochs, and each epoch contains 100 training samples.Figure 12
Reference model and open-loop response of the plant without a controller (example 3).In the simulation studies,N1 and N2 were multilayer neural networks chosen with three-layer networks with p inputs, 20 neurons in the first hidden layer, 10 neurons in the second hidden layer, and one output neuron.From equation (25), the control input uk can be computed from acknowledging yk+1 and its past values as
(28)uk=−fyk,yk−1+yk+1×1T2,uk=−fyk,yk−1+yk2−1.5T+yk−1−1−2.5T2+1.5T+T2rk−1×1T2,rk=sin2πk25+sin2πk10.The mean square error (MSE) obtained with different controllers for example 3 is presented in Table3. From the table, it can be seen that the MSE obtained with FFNN, NARX, and NARMA-L2 is minimum as compared to the MSE obtained with the RBFN- and DRNN-based controllers.Table 3
Comparison of the performance index of the proposed controller and other controllers (example 3).
ControllerMSENbr. of iterationsLearning rateRBFN [9]0.142720000.027DRNN [2]0.02888000.0354FFNN9.644×10−3210000.05NARX2.1077×10−315000.1NARMA-L21.802×10−311000.01In this study, all the three methods provide simultaneous structure and parameter learning within the approximate models. The learning rate,η, is taken to be 0.01. From Figures 13–15, it can be seen that the response of the robotic link is successfully controlled.Figure 13
Example 3: (a) response of the reference model, (b) response of the plant under the adaptive controller, (c) FFNN controller output, and (d) control error (usingn=2 and m=2) (online learning).
(a)(b)(c)(d)Figure 14
Example 3: (a) response of the reference model, (b) response of the plant under the adaptive controller, (c) NARX controller output, and (d) control error (usingn=2 and m=2).
(a)(b)(c)(d)Figure 15
Example 3: (a) response of the reference model, (b) response of the plant under the adaptive NARMA-L2 controller, (c) controller output, (d) control error (usingn=2 and m=2).
(a)(b)(c)(d)


## 5. Conclusions

This paper describes the application of NN models as a controller. Both feedforward and two recurrent NARX and NARMA-L2 predictors are proposed, tested, and compared. The method for the adjustment of parameters in generalized neural networks is treated, and the concept of dynamic back propagation is introduced. The training of all the three neural networks is done online. Simulation and comparative studies demonstrate the superior performance of the proposed approaches. It can be concluded that the ANNs can be successfully utilized for controlling different nonlinear dynamic systems. In addition, future works will explore the adaptive neural network control for nonlinear MIMO systems under disturbances using the hybrid learning method.



---

*Source: 8642915-2020-08-26.xml*