# On Iterative Learning Control for Remote Control Systems with Packet Losses

Authors: Chunping Liu; Rong Xiong; Jianxin Xu; Jun Wu
Journal: Journal of Applied Mathematics
 (2013)

---

## Abstract

The problem of iterative learning control (ILC) is considered
for a class of time-varying systems with random packet dropouts. It is assumed that an ILC scheme is implemented via a remote control system
and that packet dropout occurs during the packet transmission between the ILC controller and the actuator of
remote plant. The packet dropout is viewed as a binary switching sequence which is subject to the Bernoulli distribution.
In order to eliminate the effect of packet dropouts on the convergence property of output error,
the hold-input scheme is adopted  to compensate the packet dropout at the actuator.
It is shown that the hold-input scheme with average ILC can achieve asymptotical convergence
along the iteration axis for discrete time-varying linear system. Numerical examples are provided to show the effectiveness of the proposed method.


---

## Body

## 1. Introduction

Iterative learning control (ILC) is an attractive technique when dealing with systems that execute the same task repeatedly over a finite time interval [1]. This technique has been the center of interest of many researchers over the two decades [2–5] and covered a wide scope of research issues such as model uncertainty [6–8], disturbance uncertainty and stochastic noise [9], the initial condition and desired trajectory uncertainty [10–12], continuous-time nonlinear system control [13], and parameter interval uncertainty [14].On the other hand, the remote control systems have been the focus of several research studies over the last few years [15–21]. In the remote control systems, one feature is that the control loops are closed through a real-time communication channel which transmits signals from the sensors to the controller and from the controller to the actuators [17]. The remote control systems eliminate unnecessary wiring reducing the complexity and overall cost in designing and implementing the control systems. However, the introduction of communication networks makes the analysis and control design more complicated than classical feedback loops. Data packet dropout can randomly occur due to node failure or network congestion and impose one of the most important issues in remote control systems. In [18, 19], the authors are concerned with the stability problem for remote control systems with the packet dropout. In the work [20, 21], decentralized stabilization of remote control systems with nonlinear perturbations is studied.Besides the stability issue, trajectory tracking is a challenging issue for remote control systems. Fortunately, for periodic systems, iterative learning control offers a systematic design that can improve the tracking performance by iterations in a fixed time interval. ILC is in principle a feedforward technique; thus it can send the controller signals obtained from previous trials. It is still an open research area in ILC which is implemented  via a remote systems setting, except for certain pioneer works [22–29]. In [22, 23], the authors designed an optimal ILC controller for a class of linear systems with random packet dropouts. Bu et al. [26] studied the stability of first and high order ILC with data dropout when the plant is subject to measurement signal dropout. In [24, 25], the authors investigated the implementation of ILC in a remote control systems environment and specifically focused on compensation when both random data dropouts and delays occur at the communication network between the sensors and the controller. In [27], a sampled-data ILC approach was proposed for a class of nonlinear remote control systems to analyze the effect of packet loss. In [28], the author considered the problem of ILC for a class of nonlinear systems with control signal dropouts and measurement signal dropouts, but the convergence analysis needs controller and actuator to know the received signal whether lost or not. Huang and Fang [29] discussed the wireless remote iterative learning control system with random data dropouts.In this paper, we proposed an ILC for a time-varying system with random packet dropouts. As depicted in previous studies [22–29], there are two different kinds of packet dropouts in remote ILC systems: control input signal dropouts and output measurement signal dropouts. For the sake of convenience, we only consider the control signal dropouts in this paper, but the results can be extended to the measurement signal dropouts. The packet dropouts would be described as a binary sequence which is subject to a Bernoulli distribution taking the value of one or zero with certain probability. The ILC law adopts an iteration-average operator and a revised learning gain that takes into consideration the probabilities of data-dropout factors. As a result, the ensemble average of the output tracking errors can be made to converge along the iteration axis. In this paper, we consider a class of discrete time linear plants with output matrix C and input matrix B; our results refer only to CB of full-column rank.The paper is organized as follows. Section2 formulates the system problem. Section 3 formulates the hold-input scheme with average ILC algorithm and proves the convergence property of ILC for linear varying discrete-time plants. Section 4 presents numerical examples, and Section 5 draws the conclusions.


## 2. Problem Formulation

Consider the ILC system with network communication depicted in Figure1. The discrete time linear plant with actuators and sensors is described by
(1)xi(t+1)=A(t)xi(t)+B(ui(t)+wi(t))yi(t)=Cxi(t)+vi(t),t∈{0,1,2,…,T},
where i∈ℤ+ denotes the iteration index; T∈ℤ+ is a given finite time; xi(t)∈ℝn, ui(t)∈ℝq, and yi(t)∈ℝm are state, control, and output, respectively; A(t)∈ℝn×n is unknown matrix, while B∈ℝn×q and C∈ℝm×n are known; wi(t)∈ℝq and vi(t)∈ℝm are random noises with ℰ[wi(t)]=0 and ℰ[vi(t)]=0; for all i∈ℤ+, the initial state xi(0) is a random variable of ℰ[xi(0)]=x0 with a fixed point x0∈ℝn. Assume that CB has full-column rank. The discrete time controller consists of a ILC algorithm and a memory. The controller and the actuators are connected via a communication network through which the controller transmits data to the actuators, while the controller is directly connected to the sensors. The plant and the controller are assumed to be time driven and synchronized.Figure 1
The schematic diagram of the networked control system.At eacht∈{0,…,T} of the ith iteration stage, the controller output u^i(t) is computed, the controller transmits u^i(t) to the actuators through the network. The transmission may succeed or fail. For a successful transmission, it is assumed that the transmission delay through the network is negligible. With the negligible delay, the actuators can employ ui(t)=u^i(t), when u^i(t) is transmitted successfully. Of course, when the transmission fails, the actuators receive no u^i(t) and have to employ ui(t)=ui(t-1) (this paper prescribes ui(-1)=0). Overall, the scheme of actuators is
(2)ui(t)=γi(t)u^i(t)+(1-γi(t))ui(t-1),
where
(3)γi(t)={1,ifthe transmission ofu^i(t)succeeds,0,if the transmission ofu^i(t)fails.
Specially, this paper assumes that, for all i∈ℤ+, for all t∈{0,…,T}, γi(t) is a random variable of ℰ[γi(t)]=γ with a constant γ∈(0,1) as well as that γi(t1) and γj(t2) are independent either when i≠j or when t1≠t2. In addition, TCP-like protocol is assumed, in which there is an acknowledgment for a successful transmission, and hence the controller has indicators of whether the current controller output is received or not by the actuators.Assumption 1.
Given an output reference trajectoryyd(t), which is realizable; that is, there exists a unique desired control input ud(t)∈ℝq such that
(4)xd(t+1)=A(t)xd(t)+Bud(t)yd(t)=Cxd(t),xd(0)=x0.The purpose of this paper is to design an iterative learning control law for the above plant with network communication such thatyi(t) tracks yd(t) as closely as possible when i is large enough.


## 3. ILC Algorithms and Convergence Analysis

Denoteei(t)≜yd(t)-yi(t). The control law is a D-type ILC with average operator that employs updating mechanism:
(5)u^i+1(t)=1i∑j=1i‍uj(t)+i+1iL∑j=1i‍ej(t+1)=𝒜[ui(t)]+(i+1)L𝒜[ei(t+1)],
where the gain matrix L∈ℝq×m. From (2) and (5), the hold-input scheme with average ILC is expressed as
(6)ui+1(t)=γi+1(t)u^i+1(t)+(1-γi+1(t))ui+1(t-1)=γi+1(t)(𝒜[ui(t)]+(i+1)L𝒜[ei(t+1)])+(1-γi+1(t))ui+1(t-1).
Define the input and state errors
(7)Δui+1(t)≜ud(t)-ui+1(t),Δxi+1(t)≜xd(t)-xi+1(t).
And subtracting ud(t) from both sides of (6) yields
(8)Δui+1(t)=γi+1(t)(𝒜[Δui(t)]-(i+1)L𝒜[ei(t+1)])+(1-γi+1(t))(Δui+1(t-1)+δ(t)),
where (this paper prescribes ud(-1)=0 and hence δ(0)=ud(0)) δ(t)≜ud(t)-ud(t-1). Noticing that γi+1(t) is independent of 𝒜[Δui(t)],𝒜[ei(t+1)] and Δui+1(t-1) and taking expectation on both sides of (8), we have
(9)ℰ[Δui+1(t)]=γ(ℰ[𝒜[Δui(t)]]-(i+1)Lℰ[𝒜[ei(t+1)]])+(1-γ)(ℰ[Δui+1(t-1)]+δ(t)).Expanding expression (9) from ℰ[Δui+1(t-1)] to ℰ[Δui+1(0)], we have
(10)ℰ[Δui+1(t-1)]=γ(ℰ[𝒜[Δui(t-1)]]-(i+1)Lℰ[𝒜[ei(t)]])+(1-γ)(ℰ[Δui+1(t-2)]+δ(t-1)),ℰ[Δui+1(t-2)]=γ(ℰ[𝒜[Δui(t-2)]]-(i+1)Lℰ[𝒜[ei(t-1)]])+(1-γ)(ℰ[Δui+1(t-3)]+δ(t-2))⋮ℰ[Δui+1(0)]=γ(ℰ[𝒜[Δui(0)]]-(i+1)Lℰ[𝒜[ei(1)]])+(1-γ)(ℰ[Δui+1(-1)]+δ(0))=γ(ℰ[𝒜[Δui(0)]]-(i+1)Lℰ[𝒜[ei(1)]])+(1-γ)δ(0).
The above expression can be arranged later below (this paper prescribes ∑k=k1k2=0 when k2<k1)
(11)ℰ[Δui+1(t-1)]=∑k=0t-1‍γ(1-γ)kℰ[𝒜[Δui(t-k-1)]]-(i+1)L∑k=0t-1‍γ(1-γ)kℰ[𝒜[ei(t-k)]]+∑k=0t-1‍(1-γ)k+1δ(t-k-1).From (1) and (4), we have
(12)Δxi(t+1)=A(t)Δxi(t)+B(Δui(t)-wi(t))ei(t)=CΔxi(t)-vi(t).Taking expectation on both sides of (12) and expanding expression from ℰ[Δxi(t+1)] to ℰ[Δxi(1)], we obtain
(13)ℰ[Δxi(t+1)]=A(t)ℰ[Δxi(t)]+Bℰ[Δui(t)],ℰ[Δxi(t)]=A(t-1)ℰ[Δxi(t-1)]+Bℰ[Δui(t-1)],⋮ℰ[Δxi(1)]=A(0)ℰ[Δxi(0)]+Bℰ[Δui(0)].
The above expression can be arranged later (this paper prescribes ∏τ=τ1τ2=I when τ2<τ1)
(14)ℰ[Δxi(t+1)]=∑τ=0t‍∏ν=τ+1t‍A(ν)Bℰ[Δui(τ)]ℰ[𝒜[ei(t+1)]]=Cℰ[𝒜[Δxi(t+1)]]=C∑τ=0t‍∏ν=τ+1t‍A(ν)Bℰ[𝒜[Δui(τ)]].For anya>1 and any λ>1, denote
(15)∥ℰ[𝒜[Δui]]∥(λ,a)≜maxt∈{0,1,…,T}a-λt∥ℰ[𝒜[Δui(t)]]∥2,(16)∥ℰ[𝒜[ei]]∥(λ,a)≜maxt∈{0,1,…,T}a-λt∥ℰ[𝒜[ei(t)]]∥2.Lemma 2.
For alla>1, for all λ>1, and for all i∈ℤ+,
(17)γmaxt∈{0,…,T}a-λt∑k=0t‍(1-γ)k∑τ=0t-k-1‍at-k-τ∥ℰ[𝒜[Δui(τ)]]∥2≤∥ℰ[𝒜[Δui]]∥(λ,a)aλ-1-1.Proof.
From (17), we have
(18)γmaxt∈{0,…,T}a-λt∑k=0t‍(1-γ)k×∑τ=0t-k-1‍at-k-τ∥ℰ[𝒜[Δui(τ)]]∥2≤γmaxt∈{0,…,T}∑k=0t‍(1-γ)k×∑τ=0t-k-1‍a-λτ∥ℰ[𝒜[Δui(τ)]]∥2a-(λ-1)ta(λ-1)τ≤γmaxt∈{0,…,T}∑k=0t‍(1-γ)k∥ℰ[𝒜[Δui]]∥(λ,a)×∑τ=0t-k-1‍a-(λ-1)ta(λ-1)τ≤∥ℰ[𝒜[Δui]]∥(λ,a)maxt∈{0,…,T}γ×∑k=0t‍(1-γ)ka-(λ-1)k-a-(λ-1)taλ-1-1≤∥ℰ[𝒜[Δui]]∥(λ,a)aλ-1-1maxt∈{0,…,T}γ1-(1-γ)t+11-(1-γ)≤∥ℰ[𝒜[Δui]]∥(λ,a)aλ-1-1.Theorem 3.
For the system with network communication described in Section2 and the iterative learning controller (5), suppose
(19)ρ≜∥I-γLCB∥<1.
Then for all A0,…,AT∈ℝn×n, for all ϵ>0, there exist a>1 and λ>1 such that
(20)limi→∞∥ℰ[ei]∥(λ,a)<ϵ.Proof.
From definition of average operator, note the relation(21)𝒜[Δui+1(t)]=1i+1(Δui+1(t)+∑j=1i‍Δuj(t))=1i+1(Δui+1(t)+i𝒜[Δui(t)]).
Applying the ensemble operatorℰ[·] to both sides of (21) and substituting the relationship (8), we can obtain
(22)ℰ[𝒜[Δui+1(t)]]=1i+1ℰ[Δui+1(t)]+ii+1ℰ[𝒜[Δui(t)]]=γi+1(ℰ[𝒜[Δui(t)]]-(i+1)Lℰ[𝒜[ei(t+1)]])+1-γi+1(ℰ[Δui+1(t-1)]+δ(t))+ii+1ℰ[𝒜[Δui(t)]].
Substituting (11) into (22) leads to the following relationship:
(23)ℰ[𝒜[Δui+1(t)]]=i+γi+1ℰ[𝒜[Δui(t)]]-γLℰ[𝒜[ei(t+1)]]+γi+1∑k=0t-1‍(1-γ)k+1ℰ[𝒜[Δui(t-k-1)]]-γL∑k=0t-1‍(1-γ)k+1ℰ[𝒜[ei(t-k)]]+1-γi+1(∑k=0t-1‍(1-γ)k+1δ(t-k-1)+δ(t))=i+γi+1ℰ[𝒜[Δui(t)]]+γi+1×∑k=0t-1‍(1-γ)k+1ℰ[𝒜[Δui(t-k-1)]]-γL∑k=0t‍(1-γ)kℰ[𝒜[ei(t-k+1)]]+1i+1∑k=0t‍(1-γ)k+1δ(t-k).
Now let us handle the third term on the right hand side of (23); we will express ℰ[𝒜[ei(t-k+1)]] with ℰ[𝒜[ui]]. Substituting the state error dynamics (14) into (23) leads to the following relationship:
(24)ℰ[𝒜[Δui+1(t)]]=i+γi+1ℰ[𝒜[Δui(t)]]+γi+1×∑k=0t-1‍(1-γ)k+1ℰ[𝒜[Δui(t-k-1)]]-γLC∑k=0t‍(1-γ)k×∑τ=0t-k‍∏ν=τ+1t-k‍A(ν)Bℰ[𝒜[Δui(τ)]]+μi(t)=i+γi+1ℰ[𝒜[Δui(t)]]+γi+1×∑k=1t‍(1-γ)kℰ[𝒜[Δui(t-k)]]-γLC∑k=0t‍(1-γ)k×∑τ=0t-k-1‍∏ν=τ+1t-k‍A(ν)Bℰ[𝒜[Δui(τ)]]-γLC∑k=0t‍(1-γ)kBℰ[𝒜[Δui(t-k)]]+μi(t),
where μi(t)≜(1/(i+1))∑k=0t(1-γ)k+1δ(t-k).
Next, combining analogous terms on the right hand of (22), we obtain
(25)ℰ[𝒜[Δui+1(t)]]=i+γi+1ℰ[𝒜[Δui(t)]]+γi+1×∑k=1t‍(1-γ)kℰ[𝒜[Δui(t-k)]]-γLC∑k=0t‍(1-γ)k×∑τ=0t-k-1‍∏ν=τ+1t-k‍A(ν)Bℰ[𝒜[Δui(τ)]]-γLC∑k=1t‍(1-γ)kBℰ[𝒜[Δui(t-k)]]-γLCBℰ[𝒜[Δui(t)]]+μi(t)=(i+γi+1I-γLCB)ℰ[𝒜[Δui(t)]]+(γi+1I-γLCB)∑k=1t‍(1-γ)kℰ[𝒜[Δui(t-k)]]-γLC∑k=0t‍(1-γ)k×∑τ=0t-k-1‍∏ν=τ+1t-k‍A(ν)Bℰ[𝒜[Δui(τ)]]+μi(t).
The relationship (25) can be rewritten as follows:
(26)ℰ[𝒜[Δui+1(t)]]=(i+γi+1I-γLCB)ℰ[𝒜[Δui(t)]]+(γi+1I-γLCB)∑τ=0t-1‍(1-γ)t-τℰ[𝒜[Δui(τ)]]-γLC∑k=0t‍(1-γ)k×∑τ=0t-k-1‍∏ν=τ+1t-k‍A(ν)Bℰ[𝒜[Δui(τ)]]+μi(t).
To simplify expression of ((i+γ)/(i+1))I-γLCB, (γ/(i+1))I-γLCB, and γLC∑k=0t(1-γ)k, we choose a>1 and λ>1 such that
(27)a>∥A(ν)∥∀ν∈{0,…,T},(28)γ∥LCB∥+γ+∥LC∥∥B∥aλ-1-1<1-ρ4,(29)∥C∥∥B∥1aλ-1-12α1-ρ<ϵ,
where
(30)α≜maxt∈{0,…,T}a-λt2∑k=0t‍(1-γ)k+1∥δ(t-k)∥2≥maxt∈{0,…,T}a-λt∥μi(t)∥2.
Takingλ-norm on both sides of (26), we obtain
(31)∥ℰ[𝒜[Δui+1]]∥(λ,a)≤∥I-γLCB-1-γi+1I∥∥ℰ[𝒜[Δui]]∥(λ,a)+maxt∈{0,…,T}a-λt∥μi(t)∥2+∥γi+1I-γLCB∥maxt∈{0,…,T}a-λt×∑τ=0t-1‍(1-γ)t-τ∥ℰ[𝒜[Δui(τ)]]∥2+γ∥LC∥∥B∥maxt∈{0,…,T}a-λt×∑k=0t‍(1-γ)k∑τ=0t-k-1‍at-k-τ∥ℰ[𝒜[Δui(τ)]]∥2.
Using Lemma2, it can be proved that, for all, a>1, for all λ>1, and for all i∈ℤ+,
(32)maxt∈{0,…,T}a-λt∑τ=0t-1‍(1-γ)t-τ∥ℰ[𝒜[Δui(τ)]]∥2≤∥ℰ[𝒜[Δui]]∥(λ,a)aλ-1-1.
Combining Lemma2, (31) and (32) yields
(33)∥ℰ[𝒜[Δui+1]]∥(λ,a)≤(ρ+1-γi+1+γ∥LCB∥+γ+∥LC∥∥B∥aλ-1-1)×∥ℰ[𝒜[Δui(t)]]∥(λ,a)+α.
There exists M∈ℤ+ such that (1-γ)/(i+1)<(1-ρ)/4 when i≥M. Now for i>M, (28) and (33) imply that
(34)∥ℰ[𝒜[Δui+1]]∥(λ,a)≤1+ρ2∥ℰ[𝒜[Δui]]∥(λ,a)+α,∥ℰ[𝒜[Δui+1]]∥(λ,a)≤(1+ρ2)i+1-M∥ℰ[𝒜[ΔuM]]∥(λ,a)+∑j=0i-M‍(1+ρ2)jα≤(1+ρ2)i+1-M∥ℰ[𝒜[ΔuM]]∥(λ,a)+1-((1+ρ)/2)i+1-M1-((1+ρ)/2)α.
Consequently, we obtain
(35)limi→∞∥ℰ[𝒜[Δui+1]]∥(λ,a)≤2α1-ρ.
According to the relationship (14) between the input error and output error, we have
(36)ℰ[𝒜[ei(t)]]=C∑τ=0t-1‍∏ν=τ+1t-1‍A(ν)Bℰ[𝒜[Δui(τ)]].
Similar to the proof of Lemma2, one can prove that
(37)maxt∈{0,…,T}a-λt∥ℰ[𝒜[ei(t)]]∥2≤∥C∥∥B∥maxt∈{0,…,T}a-λt∑τ=0t-1‍at-1-τ∥ℰ[𝒜[Δui]]∥2≤∥C∥∥B∥∥ℰ[𝒜[Δui]]∥(λ,a)aλ-1-1.
Finally, from (29), (35), and (37), we can obtain
(38)limi→∞∥ℰ[𝒜[ei+1]]∥(λ,a)≤∥C∥∥B∥2α1-ρ1aλ-1-1<ϵ
because lambda can be chosen arbitrarily large in (38).
This completes the proof.Remark 4.
In this paper, we consider D-type iterative learning control with average operator, and the result obtained can be extended to P-type iterative learning control with average operator.


## 4. Numerical Examples

In this simulation test, let us consider system (1) and matrices given by
(39)A(t)=[01-1-10-3t-2-10-3t],B=[01],C=[11].
The random noises wi(t) and vi(t) have uniform distribution on the intervals [-0.05,0.05] and [-0.01,0.01], respectively. In this control problem, the desired output trajectory for t∈{0,1,…,50} is given to be yd(t)=sin(2πt/50), and the initial states x1i(0) and x2i(0) have uniform distribution on the intervals [-0.01,0.01] and [-0.02,0.02], respectively. The fixed time interval T is 50. The control profile of the first iteration is u1(t)=0. Random packet dropout in controller-actuator channel is subject to Bernoulli distribution of expected value γ (1 means transmission success while 0 means transmission failure).For expected valueγ=0.95, we compare our algorithm with the other 2 algorithms.Algorithm 1 (classic
I
L
C
). The control signal is constructed as
(40)ui+1(t)=γi+1(t)(ui(t)+Lei(t+1))+(1-γi+1(t))ui+1(t-1)
with L=0.2 satisfying ∥1-γLCB∥=0.81<1.Algorithm 2 (zero-input scheme with average ILC). The control signal is constructed as
(41)ui+1(t)=γi+1(t)(𝒜[ui(t)]+(i+1)L𝒜[ei(t+1)])
with L=0.2.Algorithm 3. Now, we consider the proposed algorithm. From (2) and (5), the control signal ui+1(t) is constructed as
(42)ui+1(t)=γi+1(t)u^i+1(t)+(1-γi+1(t))ui+1(t-1)=γi+1(t)(𝒜[ui(t)]+(i+1)L𝒜[ei(t+1)])+(1-γi+1(t))ui+1(t-1),
where the learning gain L=0.2 and expected value γ=0.95.As shown in Figure2, the tracking error profiles for the proposed algorithm are much lower than the other two algorithms with 5% packet dropout. In Figure 3, the mathematical expectation of the tracking error versus iterations is shown, and the proposed hold-input scheme with average ILC achieves the convergent performance.Figure 2
The max tracking error versus iteration with 5% packet dropout.Figure 3
The mathematical expectation of the tracking error versus iteration with 5% packet dropout.


## 5. Conclusion

In this work we address a remote control system problem with random packet dropout in controller-actuator channel. The hold-input scheme with average ILC is applied to handle this remote control problem with repeated tracking tasks. Through analysis we illustrate the desired convergence property of the hold-input scheme with average ILC. In our future work, we will also explore the extension to more generic stochastic process such as Markov chain.



---

*Source: 245372-2013-11-20.xml*