# Iterative Learning Control for Remote Control Systems with Communication Delay and Data Dropout

Authors: Chunping Liu; Jianxin Xu; Jun Wu
Journal: Mathematical Problems in Engineering
 (2012)

---

## Abstract

Iterative learning control (ILC) is applied to remote control systems in which communication channels from the plant to the controller are subject to random data dropout and communication delay. Through analysis, it is shown that ILC can achieve asymptotical convergence along the iteration axis, as far as the probabilities of the data dropout and communication delay are knowna priori. Owing to the essence of feedforward-based control ILC can perform trajectory-tracking tasks while both the data-dropout and the one-step delay phenomena are taken into consideration. Theoretical analysis and simulations validate the effectiveness of the ILC algorithm for network-based control tasks.


---

## Body

## 1. Introduction

Iterative leaning control (ILC) is a control method that achieves perfect trajectory tracking when the system operates repeatedly. ILC has made significant progresses over the past two decades [1–3] and covered a wide scope of research issues such as continuous-time nonlinear system control [4], discrete-time nonlinear system [5], the initial reset problem [6, 7], stochastic process control [8], state delays [9], and data dropout [10].On the other hand, the research on networked control systems has attracted much attention [11, 12] over the past few years. In network control, two frequently encountered issues are data dropout and communication delays, which are causes of poor performance of remote control systems. A central research area in remote control systems is to evaluate and compensate data dropout and time-delay factors [13–16]. Since data dropout and delay are random and time varying by nature, the existing control methods for deterministic data dropout and communication delays cannot be directly applied. Significant research efforts have been made on the control problems for networked systems with random data dropout and communication delays that are modeled in various ways in terms of the probability and characteristics of sources and destinations, for instance [10, 17].It is in general still an open research area in ILC when remote control systems problems are concerned, except for certain pioneer works that address linear systems associated with either random data dropout [10, 18] or random communication delays [17, 19–21]. This paper investigates the implementation of ILC in a remote control systems setting, specifically focusing on compensation when both random data dropout and delays occur at the communication channels between the plant output and the controller.Since ILC is in principle a feedforward technique, it is possible to send the controller signal before the task is executed. This would not be possible for feedback-based control systems. Hence, the data dropout can be circumvented to certain extent by using network protocols that assure the delivery of data packets. Likewise, the large delay due to large data package can also be avoided when the package is used for repeated task executions, namely, in future executions. ILC task is carried out in a finite-time interval, hence the time-domain stability is not a concern. Thus, unlike most network control works that focus on the stability issue, ILC can be applied to address trajectory-tracking tasks and the learning convergence is achieved in the iteration domain.On the other hand, the use of data in the feedforward fashion would require the temporal analysis and management of data packages as well as resending the missing data package, which may not be available in certain remote control systems tasks. In this work, we adopt an ILC scheme that uses pastcontrol signals, as well as the error signals that are perturbed by the data dropout and communication delay. The ILC law adopts classical D-type algorithm and a revised learning gain that takes into consideration the probabilities of both data-dropout and communication-delay factors. As a result, the output tracking errors can be made to converge along the iteration axis. The ILC scheme can be applied to linear discrete-time plants with trajectory-tracking tasks.The paper is organized as below. Section2 formulates the remote control systems problem. Sections 3 and 4 prove the convergence property of ILC for linear discrete-time plants. Section 5 presents a numerical examples.Throughout the paper, the following notations are used. Letℰ[·] be the expected value of a random variable, 𝒫[·] the probability of an event, ∥·∥2 the Euclidean norm of a vector, and ∥·∥ the maximal singular value of a matrix. Let z(t) is a discrete time signal with t∈{0,1,…,T}. For any a>1 and any λ>1, define
(1.1)‖z‖(λ,a)≜sup⁡t∈[0,T]a-λt‖z(t)‖2,
where [0,T]={0,1,…,T}.


## 2. Problem Formulation

Consider a deterministic discrete-time linear time-invariant dynamics system:(2.1)xi(t+1)=Axi(t)+Bui(t),yi(t)=Cxi(t),
where “i” and “t” denote the iteration index and discrete time, respectively. xi(t)∈ℝn, ui(t)∈ℝp, and yi(t)∈ℝr for all t∈[0,T] are system states, inputs, and outputs, respectively, at the ith iteration. A, B, and C are constant matrices with appropriate dimensions.The schematic diagram of the remote control systems under consideration is shown in Figure1.Figure 1
The schematic diagram of the remote control system.It should be noted that the open-loop system from the ILC input to the plant output is deterministic. The randomness occurs during the data transmission from the plant output to the ILC input. There are two approaches in analyzing the closed-loop system. The first approach is to treat the entire closed-loop system as a random or stochastic process. In such circumstances, the topology of the overall system keeps changing and the control process is either a Markovian jump process or a switching process. Another approach, which is adopted in this work, is to retain the essentially deterministic structure of the original open-loop system, meanwhile model the random data dropout and communication delay into two random factors with known probability distributions. As a consequence, the signals used in ILC,ỹi(t) are the modulated plant output with the two random factors.When the control process is deterministic, an effective ILC law for the linear system (2.1) is
(2.2)ui+1(t)=ui(t)+Lei(t+1),
where ui+1(t) and ui(t) are control inputs at the (i+1)th and ith iterations, namely, the present trial and the previous trial, respectively. ei(t+1)=yd(t+1)-yi(t+1) is the output tracking error at the time (t+1)th time instance of the ith iteration. L is a learning gain matrix.Remark 2.1.
Note that in the ILC law (2.2), the control signal of the present iteration, ui+1(t), consists of both the pastcontrol input, ui(t) and the past error with one-step temporal advance, ei(t+1). The current-cycle feedback errors, such as ei+1(t), are not used. Since ILC does not require the current-cycle feedback nor the temporal stability, it is an effective control method for remote control systems problems with random data dropout and communication delay.To facilitate the ILC design and convergence analysis, data dropout and one-step communication delay are formulated. First formulate the data-dropout problem. Denoteγ(t) a stochastic variable with Bernoulli distribution taking binary values 0 and 1, where γ(t)=0 denotes an occurrence of data dropout and γ(t)=1 denotes a normal data communication. The probabilities of γ(t) are
(2.3)P[γ(t)=1]=E[γ(t)]=γ¯,P[γ(t)=0]=1-E[γ(t)]=1-γ¯,
where γ¯>0 is a known constant. Here, we assume that γ(t) is a stationery stochastic process, thus the data dropout rate is independent of the time t. In subsequent derivations, we treat γ as time invariant.When the data dropout occurs in multiple communication channels, we can similarly defineℰ[γj]=γ¯j>0 for the jth communication channel. Thus, denote
(2.4)Γ=diag⁡(γj)=[γ10⋯00γ2⋯0⋮⋮⋱⋮00⋯γr],
the corresponding mathematical expectation is
(2.5)E[Γ]=Γ¯,
where Γ¯>0 is known a  priori.Due to the data dropout, the plant output received by the controller at the(i+1)th iteration is
(2.6)Γi+1yi+1(t).
Generally speaking, the occurrences of data dropouts at two iterations are uncorrelated, thus independent. On the other hand, ILC law at the current iteration, the (i+1)th iteration, uses only signals of the previous iteration, namely, ith iteration, as shown in (2.2). Thus yi+1(t) with the control input ui+1(t) contains data dropouts upto the ith iteration. Therefore, Γi+1 and yi+1(t) are independent, that is,
(2.7)E[Γi+1yi+1(t)]=E[Γi+1]E[yi+1(t)]=Γ¯i+1E[yi+1(t)].
Without the loss of generality, we assume ℰ[Γi]=Γ¯, namely, the data dropout rate is invariant at different iterations.Next formulate the one-step communication delay problem. Denotew(t) is a random delay factor with Bernoulli distribution, which takes binary values 0 and 1 that indicate, respectively, the presence and absence of an one-step communication delay. Here we assume that w(t) is a stationery stochastic process, thus the occurrence of the communication delay is independent of the time t. In subsequent derivations we treat w as time invariant. With multiple communication channels, we define matrix W=diag(wj), where wj denotes the occurrence of communication delay at the jth communication channel. Denote ℰ[w]=w¯ and ℰ[W]=W¯. The plant output received by ILC with possible communication delay is formulated by
(2.8)yio(t)=Wiyi(t)+[1-Wi]yi(t-1),
where Wi is the communication delay at the ith iteration. Without the loss of generality, we assume ℰ[Wi]=W¯, namely, the probability of the communication delay is invariant at different iterations. Analogous to data dropout, assume that communication delay at any two iterations are independent, then Wi+1 and Wi are independent, so are Wi+1 and yi+1(t) because yi+1(t) contains communication delays upto the ith iteration through the ILC law (2.2).It is worthwhile noting that stochastic variablesγ and w are not completely independent. A delayed or nondelayed communication occurs only when γ=1, that is, no data dropout. Hence, we should have the condition probability for data transmission without delay
(2.9)Prob[γ=1,w=1]=P[γ=1]P[w=1∣γ=1]=γ¯w¯,
and the condition probability for data transmission with one-step delay(2.10)P[γ=1,w=0]=P[γ=1]P[w=0∣γ=1]=P[γ=1](1-P[w=1∣γ=1])=γ¯(1-w¯).
As a consequence, we have(2.11)E[γw]=γ¯w¯.
The relationship between data drop out and communication delay, (2.11), can be extended to multiple channels at the ith iteration
(2.12)E[ΓiWi]=Γ¯W¯.At theith iteration, the output signals perturbed by data dropout and one-step communication delay can be expressed as
(2.13)ỹi(t)=Γiyio(t)=Γi[Wiyi(t)+(I-Wi)yi(t-1)],
where I is a unity matrix of appropriate dimensions. The mathematical expectation of ỹi(t) can be derived using the independence property between Γi, Wi, and yi, as well as the relationship  (2.12)
(2.14)E[ỹi(t)]=E{Γi[Wyi(t)+(I-W)yi(t-1)]}=Γ¯{W¯E[yi(t)]+(I-W¯)E[yi(t-1)]}.The objective of control design is to seek an appropriate ILC law that can take into consideration data dropout and communication delay concurrently. The following ILC law is adopted(2.15)ui+1(t)=ui(t)+Lẽi(t+1),
where
(2.16)ẽi(t+1)=Γi[yd(t+1)-ỹi(t+1)]=Γi[yd(t+1)-Wiyi(t+1)-(I-Wi)yi(t)]=Γi[Wiei(t+1)+(I-Wi)ei(t)+(I-Wi)δ(t)],
where δ(t)≜yd(t+1)-yd(t).


## 3. Convergence Analysis for Left-Invertible Systemsr≥p

In this section, we derive the convergence property of the ILC (2.15) in the presence of data dropout and communication delays.In ILC, the learning convergence can be derived in terms of either the output tracking error,ei(t), or the input tracking error, Δui(t). In this section, we prove the learning convergence property of Δui(t).Assumption 3.1.
For a given output reference trajectoryyd(t), which is realizable, there exists a unique desired control inputud(t)∈ℝpsuch that(3.1)xd(t+1)=Axd(t)+Bud(t),yd(t)=Cxd(t),
where ud(t) is uniformly bounded for all t∈[0,T]. It is assumed that for alli∈ℤ+,xi(0) is a random variable with ℰ[xi(0)]=x0=xd(0).Define the input and state errors(3.2)Δui+1≜ud(t)-ui+1(t),Δxi+1≜xd(t)-xi+1(t),
then from (2.1) and (3.1), we have
(3.3)Δxi(t+1)=AΔxi(t)+BΔui(t),ei(t)=CΔxi(t).From (2.15), using the relationship (2.12), we have(3.4)E[ẽi+1(t+1)]=E[Γi[yd(t+1)-Wiyi(t+1)-(I-Wi)yi(t)]]=E[Γi[Wiei(t+1)+(I-Wi)ei(t)+(I-Wi)(yd(t+1)-yd(t))]]=Γ¯{W¯E[ei(t+1)]+(I-W¯)E[ei(t)]+(I-W¯)E[δ(t)]}.Theorem 3.2.
Suppose that the update law (2.15) is applied to the networked control system and satisfied the Assumption 3.1. If there exist ρ satisfying
(3.5)‖Ip-LΓ¯W¯CB‖≤ρ<1,
then the input error along the iteration axis, ℰ[Δui(t)], converges to a bound that is proportional to the factor δ(t).Proof.
First, subtractingud(t) from both sides of the ILC law (2.15) yields
(3.6)Δui+1(t)=Δui(t)-Lẽi(t+1).
Applying the ensemble operator ℰ[·] to both sides of (3.6) and substituting the relationship (3.4) with ei(t)=CΔxi(t), we obtain
(3.7)E[Δui+1(t)]=E[Δui(t)]-LΓ¯W¯CE[Δxi(t+1)]-LΓ¯(I-W¯)CE[Δxi(t)]-LΓ¯(I-W¯)δ(t).
Substituting the state error dynamics (3.3) into (3.7) leads to the following relationship:
(3.8)E[Δui+1(t)]=(Ip-LΓ¯W¯CB)E[Δui(t)]-L[Γ¯W¯CA+Γ¯(I-W¯)C]E[Δxi(t)]-LΓ¯(I-W¯)δ(t).
Define ρ≜∥Ip-LΓ¯W¯CB∥.
Now let us handle the second term on the right hand side of (3.8), which is related to Δxi(t). Applying the ensemble operation to the following relationship:
(3.9)Δxi(t)=At[xd(0)-xi(0)]+∑k=0t-1At-1-kBΔui(k),
Substituting the relation (3.9) into (3.8), taking the norm ∥·∥2 on both sides, the following relationship is derived:(3.10)‖E[Δui+1(t)]‖2≤ρ‖E[Δui(t)]‖2+‖LΓ¯W¯CA+LΓ¯(I-W¯)C‖at‖xd(0)-E[xi(0)]‖2+‖LΓ¯W¯CA+LΓ¯(I-W¯)C‖‖B‖∑k=0t-1at-1-k‖E[Δui(k)]‖2+‖LΓ¯(I-W¯)‖‖δ(t)‖2,
where a≥∥A∥ and in this work we choose a>1 if ∥A∥≤1.
In order to handle the exponential term withat in (3.11), we introduce the λ norm. From Assumption 3.1, multiplying both sides of (3.10) by a-λt and taking the supermum over [0,T] yield(3.11)sup⁡t∈[0,T]a-λt‖E[Δui+1(t)]‖2≤ρsup⁡t∈[0,T]a-λt‖E[Δui(t)]‖2+β1sup⁡t∈[0,T]a-λt∑k=0t-1at-1-k‖E[Δui(k)]‖2+β2sup⁡t∈[0,T]a-λt‖δ(t)‖2,
where β1=∥LΓ¯W¯CA+LΓ¯(I-W¯)C∥∥B∥, β2=∥LΓ¯(I-W¯)∥, and ηi=∥xd(0)-ℰ[xi(0)]∥ that is independent of t. Since ∥δ(t)∥ is bounded, so is
(3.12)μ=sup⁡t∈[0,T]a-λt‖δ(t)‖2.
Substituting the properties of LemmaA.1 into (3.11) yields
(3.13)‖E[Δui+1]‖(λ,a)≤(ρ+β11-a-(λ-1)Taλ-a)‖E[Δui]‖(λ,a)+β2μ.
Since0≤ρ<1, it is possible to choose λ sufficiently large such that
(3.14)ρ1=ρ+β11-a-(λ-1)Taλ-a<1.
Therefore we can rewrite (3.13) as(3.15)‖E[Δui+1]‖(λ,a)≤ρ1‖E[Δui]‖(λ,a)+β2μ,
which implies
(3.16)lim⁡i→∞‖E[Δui]‖(λ,a)≤β2μ1-ρ1.Note thatμ is proportional to δ(t), namely, the maximum difference between yd(t+1)-yd(t) in t∈[0,T], which is bounded and small when the reference trajectory is smooth or the sampling interval is sufficiently small. When the probability associated with the data communication delay, W¯, is known a  priori, we can further revise the reference trajectory to an augmented one, such that the resulting δ(t)=0.Corollary 3.3.
Revising the original referenceyd(t) into an augmented one  yda(t)=W¯yd(t)+(I-W¯)yd(t-1), then  δ(t)=0 and the ILC (2.15) ensures a zero-tracking error.Proof.
Note thatℰ[yio(t)]=W¯ℰ[yi(t)]+(I-W¯)ℰ[yi(t-1)]. Suppose that yi(t)=yd(t), then the delay-perturbed output should be W¯ℰ[yd(t)]+(I-W¯)ℰ[yd(t-1)]. In other words, the augmented reference trajectory for yio(t+1) should be yda(t)=W¯yd(t)+(I-W¯)yd(t-1). As a result, yio(t)=yda(t) implies yi(t)=yd(t). Now replacing yd(t+1) in (2.16) with yda(t+1), we can derive
(3.17)ẽj(t+1)=Γj[yda(t+1)-yjo(t+1)]=Γj[Wjyd(t+1)-Wjyj(t+1)+(I-Wj)yd(t-1)-(I-Wj)yj(t)]=ΓjWjej(t+1)+Γj(I-Wj)ej(t).
Comparing the above expression with (2.16), we conclude that δ(t)=0, subsequently μ=0, which implies a zero-tracking error according to (3.16).


## 4. Convergence Analysis for Right-Invertible Systemsr≤p

In this section, we prove the learning convergence property ofei(t).Assumption 4.1.
(Ir-CBLΓ¯W¯)-1always exists.Theorem 4.2.
Suppose that the update law (2.15) is applied to the networked control system and satisfied the  Assumption 4.1. If(4.1)ρ′≜‖Ir-CBLΓ¯W¯‖<1,
then the tracking error along the iteration axis, ℰ[ei(t)], converges to a bound that is proportional to the factor δ(t).Proof.
First note the relationship:(4.2)ei+1(t+1)=yd(t+1)-yi+1(t+1)=ei(t+1)+yi(t+1)-yi+1(t+1)=ei(t+1)+CA[xi(t)-xi+1(t)]+CB[ui(t)-ui+1(t)],(4.3)xi(t)=Atxi(0)+∑k=0t-1At-1-kBui(k).
Substituting ILC law (2.15), (2.16), and (4.3) into (4.2) yields
(4.4)ei+1(t+1)=ei(t+1)+CAt+1[xi(0)-xi+1(0)]-C∑k=0t-1At-kBLẽi(k+1)-CBLẽi(t+1)=ei(t+1)+CAt+1[xi(0)-xi+1(0)]-C∑k=0t-1At-kBLΓi[Wiei(k+1)+(I-Wi)ei(k)+(I-Wi)δ(k)]-CBLΓi[Wiei(t+1)+(I-Wi)ei(t)+(I-Wi)δ(t)]=(Ir-CBLΓiWi)ei(t+1)+CAt+1[xi(0)-xi+1(0)]-C∑k=0tAt-kBLΓiWiei(k+1)-C∑k=0t-1At-k-1BLΓi(I-Wi)ei(k)-C∑k=0tAt-kBLΓi(I-Wi)δ(k).Assumption 4.3.
Assumeℰ[xi+1(0)]-ℰ[xi(0)]=0.Applying the ensemble operator ℰ[·] to both sides of (4.4) and substituting the relationship (4.2), we obtain
(4.5)E[ei+1(t+1)]=(Ir-CBLΓ¯W¯)E[ei(t+1)]-C∑k=0t-1At-k-1BLΓ¯W¯E[ei(k+1)]-C∑k=0tAt-kBLΓ¯(I-W¯)E[ei(k)]-C∑k=0tAt-kBLΓ¯(I-W¯)E[δ(t)].
Taking the norm ∥·∥2 on both sides of (4.5), the following relationship is derived
(4.6)‖E[ei+1(t+1)]‖2≤‖Ir-CBLΓ¯W¯‖‖E[ei(t+1)]‖2+‖C‖∑k=0t-1at-k-1‖BL‖Γ¯W¯‖E[ei(k+1)]‖2+‖C‖∑k=0tat-k‖BL‖Γ¯(I-W¯)‖E[ei(k)]‖2+‖C‖∑k=0tat-k‖BL‖Γ¯(I-W¯)‖E[δ(t)]‖2,
where a≥∥A∥ and in this work we choose a>1 if ∥A∥≤1.
In order to handle the exponential term withat in (4.5), we introduce the λ norm. Multiplying both sides of (4.5) by a-λt and taking the supermum over [0,T] yield(4.7)sup⁡t∈[0,T]a-λt‖E[ei+1(t)]‖2≤sup⁡t1∈[0,T]‖I-CBLΓ¯W¯‖sup⁡t∈[0,T]a-λt‖E[ei(t)]‖2+‖C‖sup⁡t∈[0,T]a-λt∑k=1tat-k‖BL‖Γ¯W¯‖E[ei(k)]‖2+‖C‖sup⁡t∈[0,T]a-λt∑k=0t-1at-k-1‖BL‖Γ¯(I-W¯)‖E[ei(k)]‖2+‖C‖sup⁡t∈[0,T]a-λt∑k=0tat-k‖BL‖Γ¯(I-W¯)‖E[δ(t)]‖2.
Substituting the properties of Lemma A.2 into (4.7) yields
(4.8)‖E[ei+1]‖(λ,a)≤(ρ′+β41-a-(λ-1)Ta(λ-1)-1)‖E[ei]‖(λ,a)+β5μ,
where β3≜∥C∥∥BL∥Γ¯ and β4≜∥C∥∥BL∥Γ¯(I-W¯)((1-a-(λ-1)T)/(a(λ-1)-1)).
Since0≤ρ′<1, it is possible to choose λ sufficiently large such that(4.9)ρ2=ρ′+β31-a-(λ-1)Ta(λ-1)-1<1.
Therefore, we can rewrite (4.8) as
(4.10)‖E[ei+1]‖(λ,a)≤ρ2‖E[ei]‖(λ,a)+β4μ,
which implies
(4.11)lim⁡i→∞‖E[ei]‖(λ,a)≤β4μ1-ρ2.


## 5. Numerical Examples

Consider the following linear discrete-time system:(5.1)xi(t+1)=[0.50-0.251.000.150.30-0.50-0.750.25-0.25]xi(t)+[001]ui(t),yi(t)=[001.0]xi(t),
with the initial condition xi(0)=0. The desired trajectory is yd(t)=sin(2πt/50). The tracking period is {0,1,…,49}. The control profile of the first iteration is u0(t)=0 for t=0,1,…,49. Two sets of probabilities for the data dropout rate and communication delay are considered, which are γ¯=0.9, w¯=0.9, γ¯=0.6, and w¯=0.6, respectively. The learning gain is L=0.5, which yields ∥I-Lγ¯w¯CB∥=0.595<1, and ∥I-Lγ¯w¯CB∥=0.82<1 with respect to the two sets of probabilities. The tracking performance of two ILC algorithms is given in Figure 2, where Max Error denotes the maximum absolute error of each iteration.The tracking error profiles for the discrete-time linear system with data dropout and one-step communication delay. (a) is learning results with the data dropout rateγ¯=0.9 and communication delay rate w¯=0.9. (b) is learning results with the data dropout rate γ¯=0.6 and communication delay rate w¯=0.6.
(a)(b)


## 6. Conclusion

In this work, we address a class of networked control system problems with random data dropout and communication delay. D-type ILC is applied to handle this remote control systems problem with repeated tracking tasks. Through analysis, we illustrate the desired convergence property of the ILC. Although we focus on one-step communication delay in this work, the results could be extended to multiple delays, which is one of our ongoing research topics. In our future work, we will also explore the extension to more generic nonlinear dynamic processes.



---

*Source: 705474-2012-03-22.xml*