# Design of Iterative Learning Control Method with Global Convergence Property for Nonlinear Systems

Authors: Guang-Wei Xu; Cheng Shao; Yu Han
Journal: Journal of Control Science and Engineering
 (2014)

---

## Abstract

We address an iterative learning control (ILC) method for overcoming initial value problem caused by local convergence methods. Introducing a feedback recursive form of tracking errors into iterative learning law, this algorithm can avoid a crude linear approximation to nonlinear plants to reach global convergence property. The algorithm’s structure is entirely illustrated. Under assumptions, it is guaranteed that tracking errors of the closed-loop system converge to zero. Besides, we discuss the roles of parameters in iterative learning law for algorithm realization, and a nonlinear case study is presented to demonstrate the effectiveness
and tracking performance of the proposed algorithm.


---

## Body

## 1. Introduction

Iterative learning control (ILC) is a methodology for reducing errors from trial to trial for systems that operate repetitively. The objective of iterative learning control is to overcome the imperfect knowledge of system structure to improve tracking performance as few trials as possible. Since ILC issue is originally proposed by Arimoto et al. [1], applications of ILC can be widely found in industrial robot manipulator, chemical batch process, some medical equipment, manufacturing, and so forth [2–7].The research of iterative learning control has been focusing on nonlinear systems. ILC algorithms based on optimization theory are effective methods to improve tracking performance of nonlinear systems. Xu and Tan [8] proposed Newton-type ILC scheme for nonlinear systems, and Du et al. [9] also provided Newton-type ILC scheme for known or identified continuous differential and monotonic system. Besides, Lin et al. [10] presented ILC algorithm based on Newton method for discrete nonlinear systems, and algorithm was implemented by decomposing nonlinear ILC problem into the sequence of linear time-varying ILC problems. Xu et al. [11] developed rank-one update to derive recurrent formula for approximating inverse matrix of Jacobian. Benefiting from usage of optimization theory, useful insights on performance improvement are presented in these articles. However, due to local or semilocal convergence properties, the difficult and important problem of finding an initial value close enough to the desired input so that error e
k converges to zero remains. In fact, due to the unknown desired input in control process, it is rigorously hard to satisfy additional conditions of initial value. Therefore, convergence performance of methods mentioned above could have a greatly reduced quality.This paper develops an iterative learning control method with global convergence property for nonlinear systems. We disclose the relationship between optimization and general ILC nonlinear scheme. Based on this relationship, iterative learning control issue can be reasonably transformed into seeking root for an equation. Defining a new learning gain related with two contiguous iterations, we address iterative learning law with recursive form of errors, and the proposed algorithm avoids a crude linear approximation to nonlinear systems to overcome initial value issue. Giving algorithm’s structure entirely, we theoretically guarantee global convergence property of this algorithm. Afterwards, the importance of learning gain and parameter is discussed. A nonlinear case study is given to demonstrate the effectiveness and tracking performance of the proposed algorithm.The remainder of this paper is organized as follows. In Section2, a focused nonlinear ILC problem is considered. In Section 3, an iterative leaning law with recursive form of errors is proposed. Afterwards, algorithm’s structure is given entirely. In Section 4, under our assumptions, theoretical analysis is derived to ensure global convergence. We discuss the roles of parameter and learning gain for algorithm realization. In Section 5, simulation example is presented to demonstrate the effectiveness of the proposed algorithm. Finally, conclusions are given in Section 6.


## 2. Problem Statement

The class of stable continuous-time nonlinear systems is considered:(1)
x
˙
k
t
=
g
x
k
t
,
u
k
t
,
t
,
x
0
=
x
0
,
y
k
t
=
h
x
k
t
,
u
k
t
,
t
,
where u
k
(
t
)
∈
U
⊂
R is control input and y
k
(
t
)
∈
Y
⊂
R is system output at the kth trial with t
∈
[
0
,
T
]. Besides, x
k
(
t
)
∈
D
⊂
R
n denotes system state. In addition, a reference signal y
d
(
t
) is given. Suppose T
s is sample period and time interval [
0
,
T
] is partitioned in N equal division; that is, T
=
(
N
-
1
)
T
s. Therefore, u
k is redefined in the vector form
(2)
u
k
=
u
k
0
,
u
k
T
s
,
…
,
u
k
N
-
1
T
s
T
,
as well as system state and output, respectively,
(3)
x
k
=
x
k
0
,
x
k
T
s
,
…
,
x
k
N
-
1
T
s
T
,
y
k
=
y
k
0
,
y
k
T
s
,
…
,
y
k
N
-
1
T
s
T
.
In sampled system, corresponding to y
k, y
d also need to be set piecewise, or rather
(4)
y
d
=
y
d
0
,
y
d
T
s
,
…
,
y
d
N
-
1
T
s
T
.The tracking error at thekth trial is defined as e
k
(
t
)
=
y
d
(
t
)
-
y
k
(
t
), and then we could rewrite it in the vector form
(5)
e
k
=
e
k
0
,
e
k
T
s
,
…
,
e
k
N
-
1
T
s
T
=
y
d
0
-
y
k
0
,
y
d
T
s
-
y
k
T
s
,
…
,
N
-
1
T
s
W
W
y
d
N
-
1
T
s
-
y
k
N
-
1
T
s
T
.The control inputu
k
+
1
(
t
) at the next trial can be constructed by using effectiveness of previous inputs and errors, such as the typical ILC scheme u
k
+
1
=
u
k
+
L
e
k with learning gain L. Under iterative learning law, algorithm can reach convergence in the sense that
(6)
lim
⁡
k
→
∞
e
k
=
0
,
lim
⁡
k
→
∞
u
d
-
u
k
=
0
,
where u
d is the desired reference input and · denotes the Euclidean norm; that is, ·
2
=
〈
·
,
·
〉. The inner product in both input space and output space is defined such that 〈
x
,
y
〉
=
x
T
y.Then, sampled nonlinear system (1) in ILC context can be stated as a mapping
(7)
y
k
=
f
u
k
,
f
:
R
N
⟶
R
N
,
where u
k and y
k, respectively, denote the piecewise input and the corresponding sampled system output. In mathematics context, the problem is described that, for a given desired output y
d, it is to find the desired input control u
d to satisfy functional equation
(8)
y
d
-
f
u
d
=
0
.
Let F
(
u
k
)
:
=
y
d
-
f
(
u
k
), and then iterative learning control scheme can be transformed into seeking root for functional equation
(9)
F
u
k
=
0
,
where F
(
·
) is a nonlinear operator, and it is obviously shown that e
k
=
F
(
u
k
).


## 3. Design of Iterative Learning Control Method

Aiming at studying the effective global scheme for nonlinear ILC issue, we solve nonlinear equation (9) by the following iterative learning law with memoryless iterative process:
(10)
u
k
+
1
t
=
u
k
t
+
λ
k
d
k
t
,
k
=
1,2
,
…
,
where λ
k
>
0 is the iteration-varying learning gain, served as the role of a dynamical controller. The learning gain λ
k satisfies
(11)
F
u
k
+
λ
k
d
k
,
d
k
≤
σ
F
u
k
,
d
k
,
(12)
e
k
-
e
k
+
1
>
δ
λ
k
e
k
,
d
k
,
with 0
<
δ
<
σ
<
1. Constants, δ and σ, play the role of constraining choice range of learning gain to avoid its value fluctuation. It is noted that δ and σ do not need to adjust values for every iteration.Besides, we defined
k by the following recursive form:
(13)
d
k
t
=
e
k
t
for
k
=
1
;
e
k
t
+
α
k
d
k
-
1
t
for
k
≥
2
,
where α
k is a scalar with our new formula
(14)
α
k
=
e
k
2
d
k
-
1
,
e
k
-
1
-
e
k
.Forα
k, applying (13) and (14), we can obtain that
(15)
d
k
,
e
k
=
e
k
2
+
α
k
d
k
-
1
,
e
k
=
e
k
2
+
α
k
d
k
-
1
,
e
k
-
e
k
-
1
+
α
k
d
k
-
1
,
e
k
-
1
=
α
k
d
k
-
1
,
e
k
-
1
.
Therefore, formula (14) of scalar α
k can be rewritten as
(16)
α
k
=
d
k
,
e
k
d
k
-
1
,
e
k
-
1
.
It shows that α
k is a link between the current iteration and the last iteration information. Value of α
k describes the ratio of two contiguous 〈
d
k
,
e
k
〉. This formula is very important for subsequent convergence analysis. Now we can define the improved algorithm for nonlinear systems as follows.Algorithm 1 (a new ILC method).Step 1.
Input any initialu
0, and compute e
1, d
1
:
=
e
1, k
:
=
1.Step 2.
Compute aλ
k satisfying (11) and (12).Step 3.
Letu
k
+
1
=
u
k
+
λ
k
d
k. If e
k
+
1
=
0, then stop.Step 4.
Chooseα
k satisfied (14) and generate d
k
+
1 by (13),setk
:
=
k
+
1, and go to Step 2.Remark 2.
If tracking errore
k achieves zero, α
k will be singular, and then it is noted that the algorithm can be convergent and an improved ILC scheme can be reached.


## 4. Convergence of ILC Method

In this section, we turn our attention to establishing a convergence theorem for this ILC algorithm.


### 4.1. Convergence Analysis

The following conditions for nonlinear system are assumed.Assumption 3.
(i) The level set of control inputsu
k
(
t
), denoted by S
0
=
{
u
∣
F
(
u
k
(
t
)
)
≤
F
(
u
0
(
t
)
)
}, is bounded.
(ii) System functionf is continuously differentiable and satisfies Lipschitz continuous condition; that is, there exists a constant L
>
0 such that
(17)
f
u
1
-
f
u
2
≤
L
u
1
-
u
2
,
for all u
1
,
u
2
∈
S
0.
In fact, these assumptions are reasonable for actual control process. Imply that the error generated by algorithm is bounded; that is, there is a constantb such that
(18)
e
k
t
≤
b
.
In addition, Lipschitz continuous condition implies that error e
k
:
=
F
(
u
k
) is also Lipschitz continuous.
Under Assumption3, we present a useful lemma.Lemma 4.
Suppose that Assumption3 holds, and consider iterative learning law (10) where learning gain λ
k satisfies conditions (11) and (12). Then,
(19)
∑
k
≥
1
e
k
,
d
k
2
d
k
2
<
∞
.Proof.
From (11) we have
(20)
e
k
-
e
k
+
1
,
d
k
≥
1
-
σ
e
k
,
d
k
.
Applying Lipschitz continuous condition (17) and iterative learning law (10), the following estimate is given:
(21)
e
k
-
e
k
+
1
,
d
k
≤
L
λ
k
d
k
2
.
Then it is immediately obtained that
(22)
λ
k
≥
1
-
σ
L
·
e
k
,
d
k
d
k
2
,
which with (12) shows that
(23)
e
k
-
e
k
+
1
≥
δ
1
-
σ
L
·
e
k
,
d
k
d
k
2
.
Summing the above inequalities with k
=
1,2
…, and using the bound (18), we can obtain that (19) is proved.Note that it is necessary and important to discuss the convergence of infinite series including〈
e
k
,
d
k
〉
2
/
d
k
2, because recursive formula (13) can be also regarded as the sum of errors e
k when iteration k approaches infinity. And it ensures that the errors can be convergent to zero. Above all, convergence theorem is provided.Theorem 5.
Consider system (1) with learning control law (10), and suppose that u
0
(
t
) is an initial input for which Assumption 3 holds. Then errors e
k generated by Algorithm 1 converge to zero in the sense that
(24)
lim
⁡
k
→
∞
e
k
=
0
.Proof.
From (13) for k
≥
2, we can obtain
(25)
d
k
-
e
k
=
α
k
d
k
-
1
.
Squaring both sides of the above equation, we have
(26)
d
k
2
=
α
k
2
d
k
-
1
2
+
2
e
k
,
d
k
-
e
k
2
.
Then, divide both sides by 〈
e
k
,
d
k
〉
2 and apply (16) as follows:
(27)
d
k
2
e
k
,
d
k
2
=
d
k
-
1
2
e
k
-
1
,
d
k
-
1
2
+
2
e
k
,
d
k
-
e
k
2
e
k
,
d
k
2
=
d
k
-
1
2
e
k
-
1
,
d
k
-
1
2
-
e
k
e
k
,
d
k
-
1
e
k
2
+
1
e
k
2
≤
d
k
-
1
2
e
k
-
1
,
d
k
-
1
2
+
1
e
k
2
.
Due to d
1
2
/
〈
e
1
,
d
1
〉
2
=
1
/
e
1
2, it is shown from the above inequality that
(28)
d
k
2
e
k
,
d
k
2
≤
∑
i
=
1
k
1
e
i
2
for all k. The algorithm does not converge after finite many iterations; that is, there exists some ɛ
>
0 such that e
k
≥
ɛ
>
0 for all k. This implies, by (28), that
(29)
d
k
2
e
k
,
d
k
2
≤
k
ɛ
2
,
which implies that
(30)
∑
k
≥
1
e
k
,
d
k
2
d
k
2
=
∞
.
It is concluded by noting that (30) and (19) in Lemma 4 are contradictory.The proposed ILC method could enjoy the advantage of global convergence and solve the problem of the initial value close enough to the unknown desired input.


### 4.2. Discussion

It is shown in the previous subsection that this ILC method could reach convergence under Assumption3. Now it is worth noting the role of α
k. From condition (11), we can obtain that
(31)
ρ
k
≔
e
k
,
d
k
-
1
e
k
-
1
,
d
k
-
1
≤
σ
.
Applying formula (14) and (16), we have
(32)
e
k
,
d
k
=
α
k
e
k
-
1
,
d
k
-
1
=
1
1
-
ρ
k
e
k
2
,
due to 0
<
σ
<
1, and the above equation can be rewritten as
(33)
e
k
2
≤
e
k
,
d
k
.
From (32), for k
-
1, the above inequality can establish the following relationship:
(34)
e
k
2
≤
α
k
e
k
-
1
,
d
k
-
1
≤
α
k
1
-
ρ
k
-
1
e
k
-
1
2
≤
α
k
1
-
σ
e
k
-
1
2
.This indicates thatα
k and σ could influence the convergence speed of our algorithm. As is known in the previous section, α
k is a key for constructing iteration learning law. α
k in our definition could reflect the sufficient descent property if α
k
<
1
-
σ holds.Subsequently, the choice of learning gain is considered.λ
k could be achieved by satisfying (11) and (12), which are a group of good guide for identifying a value λ
k. It is noted that constants δ
,
σ
∈
(
0,1
) actually play the role of bound on 〈
e
k
,
d
k
〉
/
e
k
2. And σ can be considered as a factor impacting on the rate of convergence by inequality (11). If σ is too small, error e
k
+
1 at the next trial could decline so fast that it is difficult to find a proper learning gain λ
k. Therefore, a large value between 0 and 1 for σ could remain more space for the choice of λ
k.


## 4.1. Convergence Analysis

The following conditions for nonlinear system are assumed.Assumption 3.
(i) The level set of control inputsu
k
(
t
), denoted by S
0
=
{
u
∣
F
(
u
k
(
t
)
)
≤
F
(
u
0
(
t
)
)
}, is bounded.
(ii) System functionf is continuously differentiable and satisfies Lipschitz continuous condition; that is, there exists a constant L
>
0 such that
(17)
f
u
1
-
f
u
2
≤
L
u
1
-
u
2
,
for all u
1
,
u
2
∈
S
0.
In fact, these assumptions are reasonable for actual control process. Imply that the error generated by algorithm is bounded; that is, there is a constantb such that
(18)
e
k
t
≤
b
.
In addition, Lipschitz continuous condition implies that error e
k
:
=
F
(
u
k
) is also Lipschitz continuous.
Under Assumption3, we present a useful lemma.Lemma 4.
Suppose that Assumption3 holds, and consider iterative learning law (10) where learning gain λ
k satisfies conditions (11) and (12). Then,
(19)
∑
k
≥
1
e
k
,
d
k
2
d
k
2
<
∞
.Proof.
From (11) we have
(20)
e
k
-
e
k
+
1
,
d
k
≥
1
-
σ
e
k
,
d
k
.
Applying Lipschitz continuous condition (17) and iterative learning law (10), the following estimate is given:
(21)
e
k
-
e
k
+
1
,
d
k
≤
L
λ
k
d
k
2
.
Then it is immediately obtained that
(22)
λ
k
≥
1
-
σ
L
·
e
k
,
d
k
d
k
2
,
which with (12) shows that
(23)
e
k
-
e
k
+
1
≥
δ
1
-
σ
L
·
e
k
,
d
k
d
k
2
.
Summing the above inequalities with k
=
1,2
…, and using the bound (18), we can obtain that (19) is proved.Note that it is necessary and important to discuss the convergence of infinite series including〈
e
k
,
d
k
〉
2
/
d
k
2, because recursive formula (13) can be also regarded as the sum of errors e
k when iteration k approaches infinity. And it ensures that the errors can be convergent to zero. Above all, convergence theorem is provided.Theorem 5.
Consider system (1) with learning control law (10), and suppose that u
0
(
t
) is an initial input for which Assumption 3 holds. Then errors e
k generated by Algorithm 1 converge to zero in the sense that
(24)
lim
⁡
k
→
∞
e
k
=
0
.Proof.
From (13) for k
≥
2, we can obtain
(25)
d
k
-
e
k
=
α
k
d
k
-
1
.
Squaring both sides of the above equation, we have
(26)
d
k
2
=
α
k
2
d
k
-
1
2
+
2
e
k
,
d
k
-
e
k
2
.
Then, divide both sides by 〈
e
k
,
d
k
〉
2 and apply (16) as follows:
(27)
d
k
2
e
k
,
d
k
2
=
d
k
-
1
2
e
k
-
1
,
d
k
-
1
2
+
2
e
k
,
d
k
-
e
k
2
e
k
,
d
k
2
=
d
k
-
1
2
e
k
-
1
,
d
k
-
1
2
-
e
k
e
k
,
d
k
-
1
e
k
2
+
1
e
k
2
≤
d
k
-
1
2
e
k
-
1
,
d
k
-
1
2
+
1
e
k
2
.
Due to d
1
2
/
〈
e
1
,
d
1
〉
2
=
1
/
e
1
2, it is shown from the above inequality that
(28)
d
k
2
e
k
,
d
k
2
≤
∑
i
=
1
k
1
e
i
2
for all k. The algorithm does not converge after finite many iterations; that is, there exists some ɛ
>
0 such that e
k
≥
ɛ
>
0 for all k. This implies, by (28), that
(29)
d
k
2
e
k
,
d
k
2
≤
k
ɛ
2
,
which implies that
(30)
∑
k
≥
1
e
k
,
d
k
2
d
k
2
=
∞
.
It is concluded by noting that (30) and (19) in Lemma 4 are contradictory.The proposed ILC method could enjoy the advantage of global convergence and solve the problem of the initial value close enough to the unknown desired input.


## 4.2. Discussion

It is shown in the previous subsection that this ILC method could reach convergence under Assumption3. Now it is worth noting the role of α
k. From condition (11), we can obtain that
(31)
ρ
k
≔
e
k
,
d
k
-
1
e
k
-
1
,
d
k
-
1
≤
σ
.
Applying formula (14) and (16), we have
(32)
e
k
,
d
k
=
α
k
e
k
-
1
,
d
k
-
1
=
1
1
-
ρ
k
e
k
2
,
due to 0
<
σ
<
1, and the above equation can be rewritten as
(33)
e
k
2
≤
e
k
,
d
k
.
From (32), for k
-
1, the above inequality can establish the following relationship:
(34)
e
k
2
≤
α
k
e
k
-
1
,
d
k
-
1
≤
α
k
1
-
ρ
k
-
1
e
k
-
1
2
≤
α
k
1
-
σ
e
k
-
1
2
.This indicates thatα
k and σ could influence the convergence speed of our algorithm. As is known in the previous section, α
k is a key for constructing iteration learning law. α
k in our definition could reflect the sufficient descent property if α
k
<
1
-
σ holds.Subsequently, the choice of learning gain is considered.λ
k could be achieved by satisfying (11) and (12), which are a group of good guide for identifying a value λ
k. It is noted that constants δ
,
σ
∈
(
0,1
) actually play the role of bound on 〈
e
k
,
d
k
〉
/
e
k
2. And σ can be considered as a factor impacting on the rate of convergence by inequality (11). If σ is too small, error e
k
+
1 at the next trial could decline so fast that it is difficult to find a proper learning gain λ
k. Therefore, a large value between 0 and 1 for σ could remain more space for the choice of λ
k.


## 5. Simulation Example

In this section, simulation example is presented to demonstrate the effectiveness of the proposed ILC method. The following system is the same as the first example in [10]. State space model of the objective plant is
(35)
x
1
t
+
1
=
x
2
t
,
x
2
t
+
1
=
-
0.5
+
ϵ
x
1
t
+
1.5
+
ϵ
cos
⁡
x
1
t
x
2
t
+
2
+
ϵ
sin
x
1
t
u
t
,
y
t
=
x
1
t
,
where ϵ
∈
(
0,1
) is a coefficient related to the above plant’s nonlinearity. Larger value of ϵ denotes more serious nonlinearity of the plant.The reference output signal is(36)
y
d
t
=
sin
0.02
π
t
,
t
∈
0,100
.Set the initial condition of statex as [
x
1
(
0
)
,
x
2
(
0
)
]
T
=
[
y
d
(
0
)
,
y
d
(
1
)
]
T. The proposed ILC method is then applied to the above nonlinear ILC problem. For theoretical analysis on global convergence property, now we examine that the simulation results correspond with the previous analysis.The performance comparison of the proposed ILC method and Newton method based ILC [10] on magnitude of nonlinearity ϵ
=
0.3 is shown in Figure 1. The initial input value is set as u
0
=
0. Due to numerical noise generated by normal errors implicit in simulation, algorithm ends when errors e
k converge to around 1
0
-
15. Figure 1 shows simulation results of the proposed ILC method by solid line and Newton method based ILC denoted by dash-dotted line at the beginning of initial value u
0
=
0. In the light of Figure 1, Newton method based ILC with local convergence reaches convergence in 12 iterations, while it shows that tracking errors of the proposed algorithm drop to 1
0
-
15 until 15 iterations. Once the control input becomes close enough to the desired one, Newton method based ILC converges to the tolerance range in a much faster speed.Figure 1
Simulation results of both the proposed ILC method and Newton method based ILC at the initial valueu
0
=
0 with ϵ
=
0.3.However, due to the unknown desired input, it is hard to get an ideal initial value close enough to the desired input without any additional requirement. As usual, the performances of algorithms vary with different initial inputs. If initial value of input is far from the desired one, a very large tracking error may be generated, and even divergence could happen. Therefore, an ILC method with global convergence property becomes a feasible alternative scheme for overcoming this problem.To clarify the importance of initial control input, take the caseu
0
=
0.7 with ϵ
=
0.4, when the system’s nonlinearity is severe. As shown in Figure 2, the proposed ILC method is convergent until the 18th trial, while Newton method based ILC cannot converge due to the initial value that is far from the desired input. Then, it is worth receiving robustness for different initial values at the cost of convergence speed.Figure 2
Convergence of the proposed ILC method at the initial valueu
0
=
0.7 with ϵ
=
0.4.The above results indicate that the proposed ILC method can converge even in the presence of different initial values. Although convergence speed of the proposed ILC method slows down with the system’s nonlinearity getting severe, it overcomes the limitation of divergence for local convergence algorithms when initial values are far from the unknown desired input.


## 6. Conclusions

This paper has proposed an iterative learning control method to avoid initial value issue caused by local convergence algorithms. The chief idea behind this algorithm is to construct iterative learning law with feedback recursive formula of tracking errors. Aiming to satisfy the global convergence performance, a proper learning gain is the key part in this formula, and the definition of learning gain is given. The problem of initial value close to unknown desired input is removed by this improvement in theoretical circumstance. Afterwards, under our assumptions, convergence proof of this algorithm is derived, and the proposed algorithm ensures that tracking errors of closed-loop system uniformly converge to zero. Besides, we discuss the role of learning gain for algorithm realization. Finally, a nonlinear case study is given to demonstrate the effectiveness of this ILC algorithm. In the future, interesting questions could include the possibility of uncertainty for plant and robustness issue.



---

*Source: 351568-2014-10-02.xml*