# Higher-Order Iterative Learning Control with Optimal Control Gains Based on Evolutionary Algorithm for Nonlinear System

Authors: Yun-Shan Wei; Xiaofen Yang; Wenli Shang; Ying-Yu Chen
Journal: Complexity
 (2021)

---

## Abstract

For the nonlinear discrete-time system, higher-order iterative learning control (HOILC) with optimal control gains based on evolutionary algorithm (EA) is developed in this paper. Since the updating actions are constituted by the tracking information from several previous iterations, the suitably designed HOILC schemes with appropriate control gains usually achieve fast convergence speed. To optimize the control gains in HOILC approach, EA is introduced. The encoding strategy, population initialization, and fitness function in EA are designed according to the HOILC characteristics. With the global optimization of EA, the optimal control gains of HOILC are selected adaptively so that the number of convergence iteration is reduced in ILC process. It is shown in simulation that the sum absolute error, total square error, and maximum absolute error of tracking in the proposed HOILC based on EA are convergent faster than those in conventional HOILC.


---

## Body

## 1. Introduction

In real applications such as robot manipulator systems [1–5] and flexible systems [6–8], there are many unmanned autonomous systems in complex environments. The exact mathematical model is hard to construct. For these systems, iterative learning control (ILC) is proposed. It is an effective intelligent control approach applied in dynamical systems that perform repetitive tasks to track a specific trajectory in a certain time interval. By using the control input and tracking information of previous iterations, the control input signal can be gradually updated from iteration to iteration such that the tracking performance can be improved. Less previous knowledge about the controlled systems makes ILC popular in theoretical fields [9–14] as well as applicable fields [15–19].First-order ILC, which generates the control input from tracking information at last iteration, is widely applied to dynamical systems for perfect tracking in a finite time interval [20–26]. However, only the tracking information of last iteration is utilized to update the current control input in first-order ILC, and thus it is difficult to obtain a satisfactory convergence speed. To achieve faster convergence speed, higher-order ILC (HOILC) adopting the tracking information of many previous iterations to generate the current control input signal was proposed [27–31]. Since the updating actions are constituted by the tracking information from several previous iterations, the tracking performance of suitably designed HOILC is better than that of first-order ILC. Specifically, the appropriate control gains can accelerate the convergence process of HOILC. Thus, how to select optimal control gains is a significant issue in HOILC designs.Motivated by the above observation, in this paper, the evolutionary algorithm (EA) originating from biological evolutionism is adopted to choose the optimal control gains in HOILC scheme adaptively. EA is a heuristic optimizing algorithm which simulates the reproduction, selection, crossover, and mutation in biological evolution process. It has been widely introduced to deal with various optimal issues [32–34]. In this paper, the encoding strategy, population initialization, and fitness function of EA are designed according to the HOILC characteristics such that the generations in EA are reduced. Then, the designed EA is involved in HOILC to optimize the control gains. After that, the optimal control gains and the control inputs are generated simultaneously. Comparing with traditional HOILC, the number of convergence iteration is reduced in the proposed EA scheme based HOILC (EA-HOILC). The EA with global optimization is introduced to optimize the control gains of HOILC in this paper.The rest of paper is organized as follows. The problem formulation is given in Section2. The HOILC with its convergence analysis is provided in Section 3. Section 4 presents the designed EA-HOILC scheme with optimal control gains. In Section 5, an example is provided to illustrate the effectiveness of the proposed EA-HOILC. Section 6 concludes this paper.


## 2. Problem Formulation

Consider the following nonlinear discrete-time system which performs repetitive operation:(1)xji+1=fxji,i+Biuji,yji=Cixji,where j∈0,1,2,… and i∈0,1,…,M represent the iteration index and the time point, respectively. xji∈Rw, uji∈R, and yji∈R denote the state, control input, and output of system (1), respectively. Bi∈Rw, Ci∈R1×w, and f⋅,⋅∈Rw. ydi=Cixdi for i∈0,1,…,M is the reference output, where xdi is the corresponding reference state. eji=ydi−yji is the ILC tracking error at j th iteration for i∈0,1,…,M. The following assumptions are required for the technical analysis. ⋅ represents the required norm in this paper.Assumption 1.
For allj, the initial state satisfies(2)xj0=xd0.
As the identical initial condition considered in Assumption1 cannot be satisfied, the techniques proposed in [29–31] can be introduced to deal with the vibration of initial state.Assumption 2.
The nonlinear functionf⋅,⋅ in system (1) is assumed to be differentiable to i and to be globally Lipschitz in the first variable, that is, ∀x⌢,x⌣∈Rw,(3)fx⌢,i−fx⌣,i≤cfx⌢−x⌣,where cf>0 is the Lipschitz constant.Assumption 3.
The numberCi+1Bi≠0.Remark 1.
It is noted that Assumption3 implies that the relative degree of system (1) is one. For the nonlinear discrete-time systems with higher relative degree, the ILC law can be modified according to the order of system relative degree as discussed in [31].
Suppose that the reference outputydi is realizable, there exists a unique control input udi∈R such that(4)xdi+1=fxdi,i+Biudi,ydi=Cixdi.
The objective of this paper is to develop an EA-HOILC method, which generates the control input from the tracking information of several previous iterations. The control gains are optimized by EA to reduce the number of convergence iteration. For HOILC convergence analysis, the following lemma is adopted.Lemma 1 (see [31]).
Letgj be a real sequence defined as(5)gj≤ω1gj−1+ω2gj−2+⋯+ωNgj−N+dj,for j≥N+1, where dj is a specific real sequence. If ω1,ω2,…,ωN are nonnegative numbers satisfying(6)ω=∑v=1Nωv<1,then limj⟶+∞dj=0 implies that limj⟶+∞gj=0.


## 3. HOILC Design and Convergence Analysis

In this section, for nonlinear discrete-time system (1) under Assumptions 1–3, the following HOILC law is designed for i∈0,1,…,M and j≥N−1:(7)uj+1i=∑v=1NWvuj−v+1i+∑v=1NLvej−v+1i+1,where N≥1 is the order of HOILC law (7), and Wv∈R and Lv∈R (Lv≤sL) for v=1,2,…,N are the control gains.Remark 2.
In the existing HOILC schemes [30, 31], the initial control inputs u0i,u1i,…,uN−1i are normally set as zero vectors. In this paper, since the control inputs can be obtained by EA along with the optimal control gains, we can set the initial control inputs u0i,u1i,…,uN−1i same as the generated control inputs. It means that the initial control inputs are optimized by EA, which can also accelerate the convergence speed.Theorem 1.
For nonlinear discrete-time system (1) under Assumptions 1–3, the HOILC law (7) is applied. If the control gains Wv and Lv (Lv≤sL) for v=1,2,…,N are selected to make(8)∑v=1NWv=1,(9)Wv−LvCi+1Bi≤γv,(10)∑v=1Nγv<1,then limj⟶+∞eji=0 for i∈0,1,…,M+1.Proof.
Letδuji=udi−uji and δxji=xdi−xji. Subtracting both sides of (7) with udi and considering (1), (4), and (8), we obtain(11)δuj+1i=∑v=1NWvδuj−v+1i−∑v=1NLvej−v+1i+1,=∑v=1NWvδuj−v+1i−∑v=1NLvCi+1δxj−v+1i+1,=∑v=1NWvδuj−v+1i−∑v=1NLvCi+1fxdi,i−fxj−v+1i,i+Biδuj−v+1i,=∑v=1NWv−LvCi+1Biδuj−v+1i−∑v=1NLvCi+1fxdi,i−fxj−v+1i,i.
Then, noting convergence condition (9) and Assumption 2, we can further deduce that(12)δuj+1i≤∑v=1Nγvδuj−v+1i+∑v=1NsLsCcfδxj−v+1i,where Ci≤sC and Lv≤sL for v=1,2,…,N.
On the other hand, it follows from (1) and (4) that(13)δxji=fxdi−1,i−1−fxji−1,i−1+Bi−1δuji−1.
Taking norm on both sides of (13) and considering Assumptions 1-2, it yields(14)δxji≤cfδxji−1+sBδuji−1≤cf2δxji−2+cfsBδuji−2+sBδuji−1≤⋯⋯≤cfiδxj0+∑k=0i−1cfi−k−1sBδujk=∑k=0i−1cfi−k−1sBδujk,where Bi≤sB. Substituting (14) into (12),(15)δuj+1i≤∑v=1Nγvδuj−v+1i+∑v=1N∑k=0i−1sLsCsBcfi−kδuj−v+1k.
Asi=0, considering (2) of Assumption 1, it is derived from (12) that(16)δuj+10≤∑v=1Nγvδuj−v+10.
Applying Lemma1 to (16) with convergence condition (10), we have(17)limj⟶+∞δuj0=0.
Asi=1, from (16), there is(18)δuj+11≤∑v=1Nγvδuj−v+11+∑v=1NsLsCsBcfδuj−v+10.
Applying Lemma1 to (18) with convergence condition (10) and considering (17), we obtain(19)limj⟶+∞δuj1=0.
Assume that fori=1,2,⋯,n−1, there is(20)limj⟶+∞δuji=0.
Asi=n, it follows from (15) that (21)δuj+1n≤∑v=1Nγvδuj−v+1n+∑v=1N∑k=0n−1sLsCsBcfn−kδuj−v+1k.
Applying Lemma1 to (21) with convergence condition (10) and considering (17) and (20), we can derive(22)limj⟶+∞δujn=0.
Finally, based on the mathematical induction, the following result can be deduced:(23)limj⟶+∞δuji=0,i∈0,1,…,M.
Noting (2) in Assumption 1, then it can be obtained from (15) and (24) that(24)limj⟶+∞δxji=0,i∈0,1,…,M+1.
Furthermore, fori∈0,1,…,M+1, it follows from (1) and (4) that(25)limj⟶+∞eji≤limj⟶+∞sCδxji=0.
Then, we havelimj⟶+∞eji=0 for i∈0,1,…,M+1. The proof is completed.


## 4. EA-HOILC Scheme with Optimal Control Gains

Theorem1 provides the asymptotic convergence of the proposed HOILC. It is well known that the control gains can affect the convergence performance significantly. In this section, the control gains of the HOILC developed in Section 3 are optimized by EA to reduce the number of convergence iteration.EA is an intelligent optimization algorithm which simulates the process of biological evolution to gain the optimal solution. The main idea of EA-HOILC is presented as follows.


### 4.1. Encoding Strategy

In this paper, the control gains of HOILC are real numbers, so it is appropriate to choose the real encoding strategy. The control gains to be optimized in HOILC law (7) are Wv and Lv (v=1,2,…,N). Since the convergence condition (8) holds, it is easily obtained that WN=1−∑v=1N−1Wv. As a result, we can assume the variable vector in EA to be λ∈R1×2N−1, and the encoding strategy is represented as(26)λ=W1W2⋯WN−1L1L2⋯LN.


### 4.2. Population Initialization and Individual Evaluation

Based on the convergence conditions (8)–(10), the value range of control gains Wv and Lv for v=1,2,…,N could be determined. Thus, the initial population can be produced according to the convergent conditions. Let P be the population size, without loss of generality, assume that P is even. The variable vector of h th individual in the population is represented as λh=W1,hW2,h⋯WN−1,hL1,hL2,h⋯LN,h∈R1×2N−1 which are initialized to λhini∈R1×2N−1 for h=1,2,…,P. On the other hand, for variable λh, let the system output for h th individual at i th time point be Yhi,λh. To evaluate the individual superiority, the following fitness function fithλh∈R of h th individual is established:(27)fithλh=M¯−Ehλh,where M¯ is a constant large enough and Ehλh is the sum of absolute value of tracking error represented as(28)Ehλh=∑i=0Mydi−Yhi,λh.From the fitness function (27) and the initial variable λhini, the initial fitness value of h th individual fithini is obtained. Then, we have the following initial fitness vector fitini∈RP of population:(29)fitini=fit1inifit2ini⋯fitPiniT.Hence, the initial populationpopini∈RP×2N is constructed as(30)popini=λinifitini,where the initial variable of population is(31)λini=λ1iniλ2ini⋯λPiniT.From (31), the α th (1≤α≤2N−1) variable of the β th (1≤β≤P) individual is represented as the β th row of α th column. The last column of popini shown in (30) is the fitness value of the corresponding initial variable in population.


### 4.3. Selection Strategy

The individuals are selected into next generation by roulette strategy and elitism strategy. The individual with bigger fitness value is selected at higher probability by the roulette strategy. However, one shortcoming of the roulette strategy is that the best individual in old population might be missed. So, we adopt elitism strategy to ensure that the best individual of last generation can be retained. Due to these two strategies, the number of convergence generation of EA can be reduced.


### 4.4. Crossover Operator

The crossover probability0<pc<1 depends whether an individual needs to cross. For h′ th individual, where h′=1,3,5,…,P−1, a random number is produced between 0 and 1, which is represented as rh′. If rh′>pc, the crossover operation occurs. Otherwise, the crossover operation does not occur. Due to the real encoding strategy, arithmetical crossover operator is adopted. Assume the variable vectors of h′ th and h′+1 th parent individuals to be par_λh′ and par_λh′+1, respectively, which are selected to cross. After crossover, they generate two new individuals, of which the variable vectors are represented as chi_λh′ and chi_λh′+1. For h′=1,3,5,…,P−1, the crossover operation is expressed as(32)chi_λh′=ηh′⋅par_λh′+1−ηh′⋅par_λh′+1,chi_λh′+1=1−ηh′⋅par_λh′+ηh′⋅par_λh′+1,where ηh′∈0,1 is the crossover weighting for the h′ th and h′+1 th parent individuals.


### 4.5. Mutation Operator

In this paper, we adopt the stochastic mutation strategy. Let0<pm<1 be the mutation probability. For h th individual, a number is produced randomly between 0 and 1 represented as rh, h=1,2,…,P. If rh>pm, the mutation operation occurs. Otherwise, the mutation operation does not occur. Let par_λh be the variable vector of h th individual which is selected to mutate. After mutation, a new variable vector chi_λh is produced. The mutation operator is defined as(33)chi_λh=par_λh+mh,h=1,2,…,P,where mh∈R1×2N−1 is the mutation weighting for h th individual.


### 4.6. Terminative Conditions

The terminative conditions can be determined by the fitness value or by the tracking error. In simulation, the number of generation in EA is set to be 100. Finally, we can obtain the optimal control gainsWvbest (v=1,2,…,N−1) and Lvbest (v=1,2,…,N) from the best individual produced by EA. According to the convergence condition (8), the last control gain WNbest is derived by WNbest=1−∑v=1N−1Wvbest.


### 4.7. Overview of the Proposed EA-HOILC

The flowchart of the proposed EA-HOILC is depicted in Figure1. First of all, according to the control gains characteristics and convergence conditions (9) and (10), the initial variable λhini (h=1,2,…,P) is obtained. Then, we apply the traditional HOILC at i∈0,1,…,M with initial control gains in each λhini, where convergence condition (8) is considered. By using the tracking error Ehλhini with λhini as shown in (29) produced by HOILC, the corresponding fitness fithini (h=1,2,…,P) is derived from (27). Combining λhini and fithini for h=1,2,…,P, the initial population popini as shown in (31) with (30) and (32) is produced. Secondly, selection, crossover, and mutation are performed by the selection strategy, crossover operator, and mutation operator, respectively. After that, a new input upi, i∈0,1,⋯,M, is obtained along with the optimal control gains. Set the initial control inputs of the EA-HOILC u0i=u1i=⋯=uN−1i=upi, i=0,1,…,M. Then, the HOILC with optimal control gains process begins.Figure 1
The flowchart of the proposed EA-HOILC.


## 4.1. Encoding Strategy

In this paper, the control gains of HOILC are real numbers, so it is appropriate to choose the real encoding strategy. The control gains to be optimized in HOILC law (7) are Wv and Lv (v=1,2,…,N). Since the convergence condition (8) holds, it is easily obtained that WN=1−∑v=1N−1Wv. As a result, we can assume the variable vector in EA to be λ∈R1×2N−1, and the encoding strategy is represented as(26)λ=W1W2⋯WN−1L1L2⋯LN.


## 4.2. Population Initialization and Individual Evaluation

Based on the convergence conditions (8)–(10), the value range of control gains Wv and Lv for v=1,2,…,N could be determined. Thus, the initial population can be produced according to the convergent conditions. Let P be the population size, without loss of generality, assume that P is even. The variable vector of h th individual in the population is represented as λh=W1,hW2,h⋯WN−1,hL1,hL2,h⋯LN,h∈R1×2N−1 which are initialized to λhini∈R1×2N−1 for h=1,2,…,P. On the other hand, for variable λh, let the system output for h th individual at i th time point be Yhi,λh. To evaluate the individual superiority, the following fitness function fithλh∈R of h th individual is established:(27)fithλh=M¯−Ehλh,where M¯ is a constant large enough and Ehλh is the sum of absolute value of tracking error represented as(28)Ehλh=∑i=0Mydi−Yhi,λh.From the fitness function (27) and the initial variable λhini, the initial fitness value of h th individual fithini is obtained. Then, we have the following initial fitness vector fitini∈RP of population:(29)fitini=fit1inifit2ini⋯fitPiniT.Hence, the initial populationpopini∈RP×2N is constructed as(30)popini=λinifitini,where the initial variable of population is(31)λini=λ1iniλ2ini⋯λPiniT.From (31), the α th (1≤α≤2N−1) variable of the β th (1≤β≤P) individual is represented as the β th row of α th column. The last column of popini shown in (30) is the fitness value of the corresponding initial variable in population.


## 4.3. Selection Strategy

The individuals are selected into next generation by roulette strategy and elitism strategy. The individual with bigger fitness value is selected at higher probability by the roulette strategy. However, one shortcoming of the roulette strategy is that the best individual in old population might be missed. So, we adopt elitism strategy to ensure that the best individual of last generation can be retained. Due to these two strategies, the number of convergence generation of EA can be reduced.


## 4.4. Crossover Operator

The crossover probability0<pc<1 depends whether an individual needs to cross. For h′ th individual, where h′=1,3,5,…,P−1, a random number is produced between 0 and 1, which is represented as rh′. If rh′>pc, the crossover operation occurs. Otherwise, the crossover operation does not occur. Due to the real encoding strategy, arithmetical crossover operator is adopted. Assume the variable vectors of h′ th and h′+1 th parent individuals to be par_λh′ and par_λh′+1, respectively, which are selected to cross. After crossover, they generate two new individuals, of which the variable vectors are represented as chi_λh′ and chi_λh′+1. For h′=1,3,5,…,P−1, the crossover operation is expressed as(32)chi_λh′=ηh′⋅par_λh′+1−ηh′⋅par_λh′+1,chi_λh′+1=1−ηh′⋅par_λh′+ηh′⋅par_λh′+1,where ηh′∈0,1 is the crossover weighting for the h′ th and h′+1 th parent individuals.


## 4.5. Mutation Operator

In this paper, we adopt the stochastic mutation strategy. Let0<pm<1 be the mutation probability. For h th individual, a number is produced randomly between 0 and 1 represented as rh, h=1,2,…,P. If rh>pm, the mutation operation occurs. Otherwise, the mutation operation does not occur. Let par_λh be the variable vector of h th individual which is selected to mutate. After mutation, a new variable vector chi_λh is produced. The mutation operator is defined as(33)chi_λh=par_λh+mh,h=1,2,…,P,where mh∈R1×2N−1 is the mutation weighting for h th individual.


## 4.6. Terminative Conditions

The terminative conditions can be determined by the fitness value or by the tracking error. In simulation, the number of generation in EA is set to be 100. Finally, we can obtain the optimal control gainsWvbest (v=1,2,…,N−1) and Lvbest (v=1,2,…,N) from the best individual produced by EA. According to the convergence condition (8), the last control gain WNbest is derived by WNbest=1−∑v=1N−1Wvbest.


## 4.7. Overview of the Proposed EA-HOILC

The flowchart of the proposed EA-HOILC is depicted in Figure1. First of all, according to the control gains characteristics and convergence conditions (9) and (10), the initial variable λhini (h=1,2,…,P) is obtained. Then, we apply the traditional HOILC at i∈0,1,…,M with initial control gains in each λhini, where convergence condition (8) is considered. By using the tracking error Ehλhini with λhini as shown in (29) produced by HOILC, the corresponding fitness fithini (h=1,2,…,P) is derived from (27). Combining λhini and fithini for h=1,2,…,P, the initial population popini as shown in (31) with (30) and (32) is produced. Secondly, selection, crossover, and mutation are performed by the selection strategy, crossover operator, and mutation operator, respectively. After that, a new input upi, i∈0,1,⋯,M, is obtained along with the optimal control gains. Set the initial control inputs of the EA-HOILC u0i=u1i=⋯=uN−1i=upi, i=0,1,…,M. Then, the HOILC with optimal control gains process begins.Figure 1
The flowchart of the proposed EA-HOILC.


## 5. Simulation

To verify the effectiveness of the proposed EA-HOILC, a two-link robotic fish is employed. The system dynamic of the two-link robotic fish is described as follows [22]:(34)MfV˙jt=−ξVj2t+Fjt,where Mf=0.4kg is the mass of robotic fish, ξ=165.7056 kg/m is the water resistance coefficient, Vjt denotes the velocity, and Fjt is the forward thrust produced by the tail motion. Let the velocity Vji⋅TS and the forward thrust Fji⋅TS be the system state xji and the control input uji, respectively, where TS=0.1 is the sampling time. We can discretize system (34) by using V˙ji⋅TS≈Vji+1⋅TS−Vji⋅TS/TS. Thus, the discrete-time system is(35)xji+1=xji+Afxj2i+Bfuji,yji=xji,where Af=−ξ⋅TS/Mf and Bf=TS/Mf.The reference output trajectory is represented as(36)ydi=0.0161+sin2πi/M−π/2,i∈0,1,…,M+1,with M=99. For the proposed HOILC algorithm, set the order N=2. The control gains W1, L1, and L2 are selected by EA. Another control gain W2=1−W1 is obtained from convergence condition (8). The crossover probability pc=0.7, and the mutation probability pm=0.3. To evaluate the tracking performance, three tracking indexes on sum absolute error SEk, total square error TEk, and maximum absolute error MEk are defined as follows:(37)SEk=∑i=0100eji,TEk=∑i=0100eji2,MEk=supi∈0,1,…,100eji.In the simulation, the EA-HOILC is run 10 times, and the optimized control gains are shown in Table1.Table 1
Optimized control gains of EA-HOILC in different times.
TimesW1W2L1L210.74740.25260.49320.457020.78270.21730.47240.456930.73130.26870.49240.406840.79720.20280.40670.485650.71610.28390.42090.417760.72430.27570.44560.474470.74650.25350.44240.467880.75020.24980.42090.468290.75430.24570.49540.4156100.74740.25260.49320.4570Average0.78270.21730.47240.4569Figure2 exhibits the system output performance at iterations j=15, j=22, and j=39 by using EA-HOILC with the average values of optimal control gains in 10 times. To compare the convergence speed between EA-HOILC and conventional HOILC with different parameters, the control gains in HOILC proposed in [30] with 2-order are, respectively, chosen as following two cases. Case 1: W1=0.7, W2=0.3, L1=0.3, and L2=0.2 and Case 2: W1=0.6, W2=0.4, L1=0.2, and L2=0.1. The corresponding sum absolute error SEk, total square error TEk, and maximum absolute error MEk of tracking are shown in Figure 3. From Figure 3, one can observe that the case with lager control gains in W1, L1, and L2 can achieve faster convergent speed in conventional HOILC. Moreover, it is clearly revealed that the proposed EA-HOILC can make the convergence iterations less than the conventional HOILC with the same order.Figure 2
Output performance of EA-HOILC with the average values of optimized control gains at iterationsj=15, j=22, and j=39.Figure 3
The tracking indexes (a)SEk, (b) TEk, and (c) MEk at different iterations by using EA-HOILC and conventional HOILC with two cases.


## 6. Conclusions

In this paper, an HOILC law utilizing the tracking information of several previous iterations is proposed for the nonlinear discrete-time system. The convergence is rigorously analyzed based on the mathematical induction. In order to improve the convergence performance of the developed HOILC, the EA with global optimization is introduced to optimize the control gains. With the optimal control gains, the proposed EA-HOILC can achieve faster convergence speed. In simulation, it is shown that the sum absolute error, total square error, and maximum absolute error of tracking in EA-HOILC are convergent faster than those in the conventional HOILC with same order. However, it is worth noting that because EA is adopted to select the control gains, the offline computing time of EA-HOILC is longer. It is very suitable for the cases in which fewer iterations are required only. For instance, to destroy a target with bombs, the proposed method can reduce the number of bombs at the cost of computing time. Future research will extend the EA-HOILC developed in this paper to the dynamical systems with uncertainties in real application [35–37].



---

*Source: 4281006-2021-12-30.xml*