# Formation Tracking via Iterative Learning Control for Multiagent Systems with Diverse Communication Time-Delays

Authors: Jialu Zhang; Yong Fang; Chenlong Li; Wenbo Zhu
Journal: Mathematical Problems in Engineering
 (2019)

---

## Abstract

In this paper, we consider the formation tracking problem for multiagent systems with diverse communication time-delays by using iterative learning control (ILC) method based on the frequency domain analysis. A first-order ILC law for multiagent systems with diverse communication time-delays is first proposed and its convergence conditions are given by the general Nyquist stability criterion and Gershgorin’s disk theorem. Then, in order for the system to track accurately, a second-order ILC law is presented. The conditions for system tracking with zero error are established. Numerical simulations show that the proposed ILC laws for multiagent systems with diverse communication time-delays are able to achieve effectively formation tracking. And the convergence speed remains the same as the learning control algorithm without communication delay.


---

## Body

## 1. Introduction

Multiagent systems can make it more efficient to complete some parallel and complicated tasks than a single agent and more and more scholars have begun to pay attention to multiagent systems cooperation [1–3]. As well known, ILC has been widely utilized for dynamic systems that operate repetitively on the finite time interval, because of its low complexity and high precision. Therefore, the multiagent systems that can operate repetitively using ILC method are a wise choice to achieve flexible task. In 2009, Ahn et al. firstly used ILC to implement circular formation for multiagent systems [4]. Afterward, ILC for multiagent systems was widely used in practical applications. For example, in [5], multiple satellites cooperatively kept the trajectory around the earth; in [6], multiple robots formation moved in a shape; in [7], multiple high-speed trains cooperatively controlled driving speed; in [8, 9], multiple UAVs flew for trajectory tracking. And all the above-mentioned papers were based on ILC method.However, note that all the above-mentioned results were based on an ideal communication environment, in which data transmitted between multiple agents will not be mistaken. Due to the limited bandwidth of communication channels, noise interference, and signal fading, multiagents inevitably have communication time-delays during signal transmission [10]. Since communication time-delays extensively exit, the performance of systems may be greatly reduced. For example, in [11–14], multiagent systems with communication time-delays have attracted great attention. Since ILC is introduced into multiagent systems, more and more scholars have paid attention to ILC for multiagent systems with communication time-delays. An ILC tracking strategy was proposed for nonlinear multiagent systems with communication time-delays in [15]. Using the 2D analysis, [16] obtained convergence conditions in terms of linear matrix inequality and addressed both exponential convergence and monotonic convergence problems associated with the ILC process corresponding to the proposed consensus learning protocol. In [17], a distributed ILC algorithm was proposed for multiagent systems subject to 2D switching topologies and varying communication time-delays. In [18], an adaptive iterative learning control method, which includes a P-type feedback term and an iterative learning term along the iteration axis, was designed for the consensus problem of homogeneous multiagent systems with state time-delays. In [19], an ILC scheme for multiagent systems with one-step random time-delay was proposed. All the above-mentioned papers have proposed suitable learning law and convergence conditions to guarantee convergence of system errors. However, the bounds of tracking errors were subject to the accuracy of the estimated delay in [15]. The works [16, 18] discussed consensus problem of multiagent systems not formation tracking. And [19] showed that the convergence speed got slower as the increase of the time-delays rate. Therefore, time-delays compensation method for formation tracking of multiagent systems with diverse time-delays is a more challenging issue.Motivated by the above observations, this paper proposes an ILC time-delays compensation method for multiagent systems. And we can keep convergence speed as multiagent systems without time-delays. A frequency domain ILC model is established for multiagent systems with linear SISO discrete time dynamics. Since ILC is a two-dimensional system with evolution along two independent axes, frequency domain and iteration domain, we discuss the convergence of arbitrary frequency in iteration dimension by exchange variable frequency and parameter iteration. Then we investigate eigenvalues of the characteristic equation to analyze the condition of convergence by taking z-transformation in iteration domain. Because the characteristic equations are in matrix form, it is difficult to analyze their eigenvalues. To this end, we use the generalized Nyquist criteria and Gershgorin disc theorem to analyze the range of eigenvalues and obtain the conditions for system convergence. The results show that the convergence condition of the system has nothing to do with the communication time-delays. The system errors cannot converge to zero but a certain value. In order to compensate for the influence of time-delays, this paper proposes a method using second-order ILC to make the system error converge to zero. The same frequency domain analysis method as first-order ILC was used to analyze the convergence conditions of the second-order ILC for multiagent systems with communication time-delays. Simulations demonstrate that second-order ILC can effectively compensate for the effects of communication delay.The remainder of the paper is organized as follows. In Section2, an ILC model is established for multiagent systems with linear SISO discrete time dynamics based on frequency domain. Moreover, we proposed the control objective which is formation tracking for multiagent systems. The conditions of system convergence are obtained in Section 3 when using first-order ILC and second-order ILC, respectively. In Section 4, numerical simulations show the effectiveness of the proposed protocol.Notations.1 and 0 denote the column vectors of appropriate dimensions whose elements are all ones and all zeros, respectively; I denotes an identity matrix with the required dimensions.


## 2. Problem Formulation

### 2.1. System

Consider the multiagent systems consisting ofn agents labeled from 1 through n, whose communication topology graph is denoted by G. And the dynamic of each agent is a linear SISO discrete time system:(1)xi,kt+1=Axi,kt+Bui,ktyi,kt=Cxi,ktwhere i=1,2,…,n represents the ith agent, t=1,2,…,T is the discrete time index, k=1,2… is iteration index, xi,kt∈R is the ith agent state of the kth iteration in time t, ui,kt∈R is the control input signal, and yi,kt∈R is the output. Multiple agents have the same dynamics A, B, and C. Without loss of generality, let system (1) relative degree be 1; that is, CB≠0. And the dynamics for each agent is stable; that is, A<1. Taking the z-transformation of system (1), we get(2)Yi,kz=BCz-AUi,kz+Cz-Axi,k0where z=ejω,ω∈-∞,∞ with j=-1,ω is frequency, and xi,k(0) is the ith agent initial state of the kth iteration; let Gz=BCz-A-1,γi,kz=Cz-A-1xi,k(0), then (2) can be rewritten as(3)Yi,kz=GzUi,kz+γi,kzAssumption 1.
The initial resetting condition for all agents and all the iteration satisfied desired input. Then,xi,k0 can be abbreviated as xi0.Remark 2.
Assumption1 is common in ILC for multiagent systems such as [16, 19]. If the condition is not met, we can think of it as a robust problem against initial shifts. In our future work, the ILC with initial-state learning of formation tracking control will be one of our objectives to consider.


### 2.2. Control Objective

For system (1), the objective to realize in this paper is that multiagent systems can move in the desired formation tracking reference trajectory as the increase of iteration. That is, for i=1,2,…,n, t=1,2,…,T, we have limk→∞⁡yi,kt⁡=yidt, and yidt is desired output of the ith agent and denoted as(4)yidt=rt+ditwhere rt, t=1,2,…,T is desired reference trajectory of multiagent systems. In practice, not all agents can obtain rt, but a portion of agents can. To this end, we denote the reference-accessibility matrix Φ=diag{φ1,φ2,⋯φn}, which is a diagonal, nonnegative, real matrix. If the ith agent has direct information about rt, then φi>0; otherwise, φi=0. di(t) is the desired output deviation from desired reference trajectory of the ith agent. If multiple agents have the same desired output deviation, then the agents will have the same output. So dit≠djt,i≠j. dijt=dit-djt represents the desired relative formation between the ith agent and the jth agent. It is worth pointing out that each agent can obtain its own output yi,kt and its own desired output deviation di(t). For convenience, we denote(5)δi,kt=dit-yi,ktWe denote the output error of the ith agent in the kth iteration:(6)ei,kt=rt+dit-yi,ktTaking z-transformation for (6), we get(7)Ei,kz=Rz+Diz-Yi,kzWhen it is satisfied limk→∞⁡Ei,kz=0, that is, limk→∞⁡Yi,kz=Yidz, multiagent systems realize formation tracking control.


### 2.3. Preliminaries in Graph Theory

LetG= (V,E,A) be directed graph, where V=v1,…,vn is the set of vertices, E⊆V×V is the set of edge, and A=[aij]aij≥0 is the weighted adjacency matrix of the graph G. If there is an edge from the ith agent to the jth agent, that is, (vi,vj)∈E, then aji>0, representing the fact that information is transmitted from ith agent to jth agent. Otherwise, aji=0. Moreover, we assume that aii=0. The index set of neighbors of node vi is Ni={j:(vi,vj)∈E}. The Laplacian matrix L=lij∈Rn×n of graph G is denoted as L=D-A, where D≜ diag{d1,…,dn} with di=∑j=1naij. A path in the directed graph G is a finite sequence vi1,vi2,…,vij, and vil,vil+1∈E,l=1,2,⋯j-1. If there is a vertex that can be connected to all other vertices through paths, then G is said to have a spanning tree, and this vertex is called the root vertex.Assumption 3.
The graph of multiagent system (1) is a directed graph. And its graph is connected; that is, each agent has a path to the other agents so that they can exchange information.Lemma 4.
Vector1 is a right eigenvector of the Laplacian L with eigenvalue 0; i.e.,(8)L1=01Lemma 5 (see [20]).
If an irreducible matrixM=ai,j∈Cn×n,n≥1 is weakly generalized diagonally dominant and at least one of the rows is strictly diagonally dominant, A is nonsingular.Remark 6.
Assumption3 implies L+Φ is a matrix that satisfies the conditions ofLemma5. In L+Φ, the magnitude of the diagonal entry is φi+∑jϵNiai,j,i=1,2,…,n, and the sum of the magnitudes of all nondiagonal entries is ∑jϵNiai,j,i=1,2,…,n. Since φi+∑jϵNiai,j≥∑jϵNiai,j,i=1,2,…,n and at least one of the rows is satisfied φi+∑jϵNiai,j>∑jϵNiai,j, matrix L+Φ is nonsingular.


### 2.4. Communication Time-Delays

When multiagent systems cooperate through the network, there are time-delays in transmission due to the limited bandwidth of communication channels, noise interference, and signal fading. Therefore, we introduce communication time-delays into multiagent systems to research more close to the practical systems. A block diagram of multiagent systems with communication time-delays is illustrated in Figure2 if the communication graph of multiagent systems is shown in Figure 1. When the ith agent can obtain δj,kt from the jth agent, we denote δ˙ji,kt as the signal which the ith agent received from the jth agent:(9)δ˙ji,kt=δj,kt-τi,jwhere τi,j is communication time-delay when the jth agent sends signal to the ith agent. τi,j∈0,1,2⋯,τmax,i≠j,τmax<T.Figure 1
Communication graph of the multiagent system.Figure 2
The structure of multiagent systems with communication time-delays.


## 2.1. System

Consider the multiagent systems consisting ofn agents labeled from 1 through n, whose communication topology graph is denoted by G. And the dynamic of each agent is a linear SISO discrete time system:(1)xi,kt+1=Axi,kt+Bui,ktyi,kt=Cxi,ktwhere i=1,2,…,n represents the ith agent, t=1,2,…,T is the discrete time index, k=1,2… is iteration index, xi,kt∈R is the ith agent state of the kth iteration in time t, ui,kt∈R is the control input signal, and yi,kt∈R is the output. Multiple agents have the same dynamics A, B, and C. Without loss of generality, let system (1) relative degree be 1; that is, CB≠0. And the dynamics for each agent is stable; that is, A<1. Taking the z-transformation of system (1), we get(2)Yi,kz=BCz-AUi,kz+Cz-Axi,k0where z=ejω,ω∈-∞,∞ with j=-1,ω is frequency, and xi,k(0) is the ith agent initial state of the kth iteration; let Gz=BCz-A-1,γi,kz=Cz-A-1xi,k(0), then (2) can be rewritten as(3)Yi,kz=GzUi,kz+γi,kzAssumption 1.
The initial resetting condition for all agents and all the iteration satisfied desired input. Then,xi,k0 can be abbreviated as xi0.Remark 2.
Assumption1 is common in ILC for multiagent systems such as [16, 19]. If the condition is not met, we can think of it as a robust problem against initial shifts. In our future work, the ILC with initial-state learning of formation tracking control will be one of our objectives to consider.


## 2.2. Control Objective

For system (1), the objective to realize in this paper is that multiagent systems can move in the desired formation tracking reference trajectory as the increase of iteration. That is, for i=1,2,…,n, t=1,2,…,T, we have limk→∞⁡yi,kt⁡=yidt, and yidt is desired output of the ith agent and denoted as(4)yidt=rt+ditwhere rt, t=1,2,…,T is desired reference trajectory of multiagent systems. In practice, not all agents can obtain rt, but a portion of agents can. To this end, we denote the reference-accessibility matrix Φ=diag{φ1,φ2,⋯φn}, which is a diagonal, nonnegative, real matrix. If the ith agent has direct information about rt, then φi>0; otherwise, φi=0. di(t) is the desired output deviation from desired reference trajectory of the ith agent. If multiple agents have the same desired output deviation, then the agents will have the same output. So dit≠djt,i≠j. dijt=dit-djt represents the desired relative formation between the ith agent and the jth agent. It is worth pointing out that each agent can obtain its own output yi,kt and its own desired output deviation di(t). For convenience, we denote(5)δi,kt=dit-yi,ktWe denote the output error of the ith agent in the kth iteration:(6)ei,kt=rt+dit-yi,ktTaking z-transformation for (6), we get(7)Ei,kz=Rz+Diz-Yi,kzWhen it is satisfied limk→∞⁡Ei,kz=0, that is, limk→∞⁡Yi,kz=Yidz, multiagent systems realize formation tracking control.


## 2.3. Preliminaries in Graph Theory

LetG= (V,E,A) be directed graph, where V=v1,…,vn is the set of vertices, E⊆V×V is the set of edge, and A=[aij]aij≥0 is the weighted adjacency matrix of the graph G. If there is an edge from the ith agent to the jth agent, that is, (vi,vj)∈E, then aji>0, representing the fact that information is transmitted from ith agent to jth agent. Otherwise, aji=0. Moreover, we assume that aii=0. The index set of neighbors of node vi is Ni={j:(vi,vj)∈E}. The Laplacian matrix L=lij∈Rn×n of graph G is denoted as L=D-A, where D≜ diag{d1,…,dn} with di=∑j=1naij. A path in the directed graph G is a finite sequence vi1,vi2,…,vij, and vil,vil+1∈E,l=1,2,⋯j-1. If there is a vertex that can be connected to all other vertices through paths, then G is said to have a spanning tree, and this vertex is called the root vertex.Assumption 3.
The graph of multiagent system (1) is a directed graph. And its graph is connected; that is, each agent has a path to the other agents so that they can exchange information.Lemma 4.
Vector1 is a right eigenvector of the Laplacian L with eigenvalue 0; i.e.,(8)L1=01Lemma 5 (see [20]).
If an irreducible matrixM=ai,j∈Cn×n,n≥1 is weakly generalized diagonally dominant and at least one of the rows is strictly diagonally dominant, A is nonsingular.Remark 6.
Assumption3 implies L+Φ is a matrix that satisfies the conditions ofLemma5. In L+Φ, the magnitude of the diagonal entry is φi+∑jϵNiai,j,i=1,2,…,n, and the sum of the magnitudes of all nondiagonal entries is ∑jϵNiai,j,i=1,2,…,n. Since φi+∑jϵNiai,j≥∑jϵNiai,j,i=1,2,…,n and at least one of the rows is satisfied φi+∑jϵNiai,j>∑jϵNiai,j, matrix L+Φ is nonsingular.


## 2.4. Communication Time-Delays

When multiagent systems cooperate through the network, there are time-delays in transmission due to the limited bandwidth of communication channels, noise interference, and signal fading. Therefore, we introduce communication time-delays into multiagent systems to research more close to the practical systems. A block diagram of multiagent systems with communication time-delays is illustrated in Figure2 if the communication graph of multiagent systems is shown in Figure 1. When the ith agent can obtain δj,kt from the jth agent, we denote δ˙ji,kt as the signal which the ith agent received from the jth agent:(9)δ˙ji,kt=δj,kt-τi,jwhere τi,j is communication time-delay when the jth agent sends signal to the ith agent. τi,j∈0,1,2⋯,τmax,i≠j,τmax<T.Figure 1
Communication graph of the multiagent system.Figure 2
The structure of multiagent systems with communication time-delays.


## 3. Compensation to ILC for Multiagent System with Communication Time-Delays

In order to make the multiagent systems more close to the practical engineering applications, we analyze the ILC for multiagent systems with communication time-delays first. The result shows the system error of first-order ILC with communication time-delays cannot converge to zero. Second, the method to compensate communication time-delay is given. And the convergence analysis is obtained in Section3.2.


### 3.1. First-Order ILC for Multiagent Systems with Communication Time-Delays

For multiagent systems with communication time-delays, a first-order ILC law is proposed as(10)ui,k+1t=ui,kt+Γ∑jϵNiai,jδi,kt+1-δj,kt+1-τi,j+φiei,kt+1where Γ∈R and Γ≠0 is a gain to be determined. This ILC law is multiple agents joint learning law. The control input of the ith agent needs previous iterative control input, self-error, and adjacent agent information. In (10), τi,j is communication time-delay mentioned above.Taking the z-transformation of (10), we get(11)Ui,k+1z=Ui,kz+zΓ∑jϵNiai,jΔi,kz-z-τi,jΔj,kz+φiEi,kzBy (7), (11) andAssumption1, we have that(12)Ei,k+1z-Ei,kz=GzUi,kz-Ui,k+1z=-zΓGz∑jϵNiai,jΔi,kz-z-τi,jΔj,kz+φiEi,kzLet Ekz=E1,kz,E2,kz,…,En,kzT; then (12) can be rewritten as(13)Ek+1z-Ekz=-zΓGzLz+ΦEkz-Rz1-zΓGzΦRz1Further, we can show that(14)Ek+1z=I-zΓGzLz+ΦEkz+zΓGzRzLz1where Lz is(15)Lz=-ai,jz-τi,j,jϵNi∑jϵNiai,j,i=j0,othersTheorem 7.
Consider multiagent system (1); let Assumptions 1 and 3 hold and the learning law (10) be applied. Then the system error of multiagent systems can converge, given that any one of the following conditions is satisfied for all i=1,2,…,n:(16)11>A>0∧Pi,1>0∧Pi,2>0∧Pi,3>0∧Pi,4>0∧Pi,5>02-1<A<0∧Pi,1>0∧Pi,2<0∧Pi,3>0∧Pi,4<0∧Pi,5>0where(17)Pi,1=2-QiPi,2=-161-A2+41-A3-AQi+A-2Qi2Pi,3=21-A2-1-AQiPi,4=-41-A2+21-A3-AQi+A-2Qi2Pi,5=41-A2-41-AQi+SiQi=ΓBCKi+φiSi=ΓBC22Kiφi+φi2Ki=∑j∈Niai,jProof.
As we know, ILC is employed for dealing with the repeated tracking control. For this kind of machines, let them learn as people by using previous errors correct this control input. Thus, by repeating the learning process, the tracking performance can be achieved with high accuracy. Since ILC is a two-dimensional system with evolution along two independent axes, time domain and iteration domain, the scholars Fang et al. proposed an ILC 2D model [21, 22]. This paper considers a two-dimensional system of frequency domain and iteration domain after taking z-transformation. Considering Ekz as two-dimensional function Ek,z, we discuss the convergence of arbitrary frequency in iteration dimension by exchange variable frequency and parameter iteration. Thus, we can exchange variable z and parameter k. And Ekz can be rewritten as Ezk; that is, Ezk=E1,zk,E2,zk,…,En,zkT. Then, (14) can be rewritten as(18)Ezk+1=I-zΓGzLz+ΦEzk+zΓGzRzLz1Taking the z-transformation of (18), we get(19)zEzz=I-zΓGzLz+ΦEzz+zz-1zΓGzRzLz1Further, we get z-1I+zΓG(z)Lz+ΩEzz-zz-1-1zΓG(z)RzLz1=0. The characteristic equation is(20)detz-1I+zΓGzLz+Φ=0
Whenz=1, detz-1I+zΓG(z)Lz+Φ=detΓG(z)Lz+Φ. FromRemark6, we can get that Lz+Ω is nonsingular; that is, detLz+Φ≠0. Since ΓG(z)≠0, z=1 is not the root of characteristic equation.
Whenz≠1, both sides of (20) are divided by z-1:(21)detI+zΓGzLz+Φz-1=0
Next, we need to prove that all roots of (21) have modulus less than unity. Let Fz=zΓG(z)Lz+Φz-1-1. Based on the general Nyquist stability criterion [23], the roots of (21) have modulus less than unity, if the eigenloci λFω,β of(22)Fω,β=ejβΓGejβLejβ+Φejω-1do not enclose the point -1,j0 for β∈[-π,π), ω∈[-π,π), and ω≠0. By Gershgorin’s disk theorem, we have λFω,β∈⋃i∈nFi for all β∈[-π,π), ω∈[-π,π), and ω≠0, where(23)Fi=ς:ς∈C,ς-ejβΓBCKi+φiejω-1ejβ-A≤∑j∈NiKiΓBCejω-1ejβ-ASo the eigenloci λFω,β do not enclose the point -1,j0 for β∈[-π,π), ω∈[-π,π), and ω≠0 when the point -a,j0 with a≥1 is not in the disc Fi for all i=1,2,…,n, β∈-π,π, ω∈-π,π, and ω≠0. That is, a+ejβΓBCKi+φi/ejω-1ejβ-A>KiΓBC/ejω-1ejβ-A for all i=1,2,…,n, β∈[-π,π), ω∈[-π,π) and ω≠0, when a≥1. We denote fia as(24)fia=aejω-1ejβ-A+ejβΓBCKi+φi2-KiΓBC2Note that a+ejβΓBCKi+φi/ejω-1ejβ-A2-KiΓBC/ejω-1ejβ-A2=1/ejω-1ejβ-A2fia for all a≥1, β∈[-π,π), ω∈[-π,π) and ω≠0. So, a+ejβΓBCKi+φi/ejω-1ejβ-A2-KiΓBC/ejω-1ejβ-A2>0 as long as fia>0 for all a≥1, β∈[-π,π), ω∈[-π,π), and ω≠0. Further, we can show that(25)fia=21-cos⁡ω1+A2-2Acos⁡βa2+2aΓBCKi+φicos⁡ω-1-Acos⁡ω-β+Acos⁡β+ΓBC22Kiφi+φi2Using the conditions ofTheorem7, we can prove fia>0 for all a≥1, β∈[-π,π), ω∈[-π,π), and ω≠0 in Appendix. Then eigenloci of λFω,β for all β∈[-π,π), ω∈[-π,π), and ω≠0 do not enclose the point -1,j0. Thus the roots of (21) have modulus less than unity. That is, the system achieves a formation tracking asymptotically.
In addition, let the final value ofEzk be Ez∞. Multiply z-1 on both sides of z-1I+zΓG(z)Lz+ΦEzz=z/z-1zΓG(z)RzLz1 and take the limit:(26)limz→1z-1I+zΓGzLz+Φz-1Ezz=limz→1zzΓGzRzLz1By final value theorem, we get(27)zΓGzLz+ΦEz∞=zΓGzRzLz1Since rankLz+Φ=n, Lz+Φ is invertible. And zΓG(z)≠0 is satisfied; thus Ez∞=RzLz+Φ-1Lz1. This completes the proof.When there is not communication time-delay in multiagent systems, we can get system error converging to zero combining equation (27) andLemma4. Otherwise, system error cannot converge to zero and converges to a certain value about desired reference trajectory, graph, and time-delays as shown in (27).


### 3.2. Second-Order ILC for Multiagent Systems with Time-Delays

In this subsection, we analyze convergence condition of second-order ILC for multiagent systems. Using previous two errors to correct the control input can compensate the influence of communication time-delays. Therefore, we propose the second-order ILC law:(28)ui,k+1t=ui,kt+Γ∑jϵNiai,jδi,kt+1-δj,kt+1-τi,j+ϕiei,kt+1+Λ∑jϵNiai,jδi,k-1t+1-δj,k-1t+1-τi,j+ϕiei,k-1t+1where Γ is first-order learning gain, Λ is second-order learning gain, Γ,Λ∈R, and Γ≠0,Λ≠0.Theorem 8.
Consider multiagent system (1); let Assumptions 1 and 3 hold and the learning law (28) be applied. Then the formation tracking objective limk→∞⁡Yi,kz=Yidz is achieved, given that Λ=-Γ, Qi>0, and any one of the following conditions are satisfied for all i=1,2,…,n:(29)11>A>0∧Pi,6>0∧Pi,7>02-1<A<0∧Pi,7>0∧Pi,8<031>A>0∧Pi,9>0∧Pi,10>0where(30)Pi,6=Qi-21+APi,7=1+A2+21+AQi+SPi,8=Qi-1-APi,9=1-A-QiPi,10=1-A2-21-AQi+SiQi=ΓBCKi+φiSi=ΓBC22Kiφi+φi2Ki=∑j∈Niai,jProof.
Taking the z-transformation of (28), we get(31)Ui,k+1z=Ui,kz+zΓ∑jϵNiai,jΔi,kz-z-τi,jΔj,kz+ϕiEi,kz+zΛ∑jϵNiai,jΔi,k-1z-z-τi,jΔj,k-1z+ϕiEi,k-1zBy (7), (31), andAssumption1, we have that(32)Ei,k+1z-Ei,kz=-zΓGz∑jϵNiai,jΔi,kz-z-τi,jΔj,kz+ϕiEi,kz-zΛGz∑jϵNiai,jΔi,k-1z-z-τi,jΔj,k-1z+ϕiEi,k-1zLet Ekz=E1,kz,E2,kz,…,En,kzT; (32) can be rewritten as(33)Ek+1z-Ekz=-zΓGzLz+ΦEkz-Rz1-zΓGzΦRz1-zΛGzLz+ΦEk-1z-Rz1-zΛGzΦRz1=-zΓGzLz+ΦEkz-zΛGzLz+ΦEk-1z+zGzRzΛ+ΓLz1Exchanging variable frequency z and parameter iteration k, Ekz can be rewritten as Ezk in (33):(34)Ezk+1-Ezk=-zΓGzLz+ΦEzk-zΛGzLz+ΦEzk-1+zGzRzΛ+ΓLz1Taking z-transformation of (34), we get(35)zEzz-Ezz=-zΓGzLz+ΦEzz-z-1zΛGzLz+ΦEzz+zz-1zGzRzΛ+ΓLz1FromTheorem8, Λ=-Γ. Further, we have that(36)z-1I+zz-1ΓGzLz+ΦEzz=0The characteristic equation of system is(37)detz-1I+zz-1ΓGzLz+Φ=0Let pz=z-1I+zz-1ΓG(z)Lz+Φ. Since p1=0 and detI+zΓG(z)Lz+Φ≠0, z=1 is the only root of (37).
Whenz≠1, both sides of (37) are divided by z-1:(38)detI+zΓGzLz+Φz=0
Next, we need to prove that all roots of (38) have modulus less than unity. Similar to the proof ofTheorem7, the conditions ofTheorem8 are easily verified.
Taking the limit of (36) yields(39)limz→1⁡z-1I+zz-1ΓGzLz+ΦEzz=0By final value theorem, we get(40)I+zΓGzLz+ΦEz∞=0Since detI+zΓG(z)Lz+Φ≠0 as proved above, rankI+zΓG(z)Lz+Φ=n. Based on Sylvester inequality, we get rankI+zΓG(z)Lz+Φ+rankEz∞≤n. Therefore, rankEz∞=0; that is Ez∞=0.Theorem8 is thus proved.The result ofTheorem7 shows that the error of multiagent system cannot converge to zero considering diverse communication time-delays. InTheorem8, using second-order ILC and the fact that the second-order learning gain is opposite of the first-order learning gain, the error of multiagent system can converge to zero considering diverse communication time-delays. In addition, the convergence speed can be kept as convergence speed of multiagent systems without time-delays.


## 3.1. First-Order ILC for Multiagent Systems with Communication Time-Delays

For multiagent systems with communication time-delays, a first-order ILC law is proposed as(10)ui,k+1t=ui,kt+Γ∑jϵNiai,jδi,kt+1-δj,kt+1-τi,j+φiei,kt+1where Γ∈R and Γ≠0 is a gain to be determined. This ILC law is multiple agents joint learning law. The control input of the ith agent needs previous iterative control input, self-error, and adjacent agent information. In (10), τi,j is communication time-delay mentioned above.Taking the z-transformation of (10), we get(11)Ui,k+1z=Ui,kz+zΓ∑jϵNiai,jΔi,kz-z-τi,jΔj,kz+φiEi,kzBy (7), (11) andAssumption1, we have that(12)Ei,k+1z-Ei,kz=GzUi,kz-Ui,k+1z=-zΓGz∑jϵNiai,jΔi,kz-z-τi,jΔj,kz+φiEi,kzLet Ekz=E1,kz,E2,kz,…,En,kzT; then (12) can be rewritten as(13)Ek+1z-Ekz=-zΓGzLz+ΦEkz-Rz1-zΓGzΦRz1Further, we can show that(14)Ek+1z=I-zΓGzLz+ΦEkz+zΓGzRzLz1where Lz is(15)Lz=-ai,jz-τi,j,jϵNi∑jϵNiai,j,i=j0,othersTheorem 7.
Consider multiagent system (1); let Assumptions 1 and 3 hold and the learning law (10) be applied. Then the system error of multiagent systems can converge, given that any one of the following conditions is satisfied for all i=1,2,…,n:(16)11>A>0∧Pi,1>0∧Pi,2>0∧Pi,3>0∧Pi,4>0∧Pi,5>02-1<A<0∧Pi,1>0∧Pi,2<0∧Pi,3>0∧Pi,4<0∧Pi,5>0where(17)Pi,1=2-QiPi,2=-161-A2+41-A3-AQi+A-2Qi2Pi,3=21-A2-1-AQiPi,4=-41-A2+21-A3-AQi+A-2Qi2Pi,5=41-A2-41-AQi+SiQi=ΓBCKi+φiSi=ΓBC22Kiφi+φi2Ki=∑j∈Niai,jProof.
As we know, ILC is employed for dealing with the repeated tracking control. For this kind of machines, let them learn as people by using previous errors correct this control input. Thus, by repeating the learning process, the tracking performance can be achieved with high accuracy. Since ILC is a two-dimensional system with evolution along two independent axes, time domain and iteration domain, the scholars Fang et al. proposed an ILC 2D model [21, 22]. This paper considers a two-dimensional system of frequency domain and iteration domain after taking z-transformation. Considering Ekz as two-dimensional function Ek,z, we discuss the convergence of arbitrary frequency in iteration dimension by exchange variable frequency and parameter iteration. Thus, we can exchange variable z and parameter k. And Ekz can be rewritten as Ezk; that is, Ezk=E1,zk,E2,zk,…,En,zkT. Then, (14) can be rewritten as(18)Ezk+1=I-zΓGzLz+ΦEzk+zΓGzRzLz1Taking the z-transformation of (18), we get(19)zEzz=I-zΓGzLz+ΦEzz+zz-1zΓGzRzLz1Further, we get z-1I+zΓG(z)Lz+ΩEzz-zz-1-1zΓG(z)RzLz1=0. The characteristic equation is(20)detz-1I+zΓGzLz+Φ=0
Whenz=1, detz-1I+zΓG(z)Lz+Φ=detΓG(z)Lz+Φ. FromRemark6, we can get that Lz+Ω is nonsingular; that is, detLz+Φ≠0. Since ΓG(z)≠0, z=1 is not the root of characteristic equation.
Whenz≠1, both sides of (20) are divided by z-1:(21)detI+zΓGzLz+Φz-1=0
Next, we need to prove that all roots of (21) have modulus less than unity. Let Fz=zΓG(z)Lz+Φz-1-1. Based on the general Nyquist stability criterion [23], the roots of (21) have modulus less than unity, if the eigenloci λFω,β of(22)Fω,β=ejβΓGejβLejβ+Φejω-1do not enclose the point -1,j0 for β∈[-π,π), ω∈[-π,π), and ω≠0. By Gershgorin’s disk theorem, we have λFω,β∈⋃i∈nFi for all β∈[-π,π), ω∈[-π,π), and ω≠0, where(23)Fi=ς:ς∈C,ς-ejβΓBCKi+φiejω-1ejβ-A≤∑j∈NiKiΓBCejω-1ejβ-ASo the eigenloci λFω,β do not enclose the point -1,j0 for β∈[-π,π), ω∈[-π,π), and ω≠0 when the point -a,j0 with a≥1 is not in the disc Fi for all i=1,2,…,n, β∈-π,π, ω∈-π,π, and ω≠0. That is, a+ejβΓBCKi+φi/ejω-1ejβ-A>KiΓBC/ejω-1ejβ-A for all i=1,2,…,n, β∈[-π,π), ω∈[-π,π) and ω≠0, when a≥1. We denote fia as(24)fia=aejω-1ejβ-A+ejβΓBCKi+φi2-KiΓBC2Note that a+ejβΓBCKi+φi/ejω-1ejβ-A2-KiΓBC/ejω-1ejβ-A2=1/ejω-1ejβ-A2fia for all a≥1, β∈[-π,π), ω∈[-π,π) and ω≠0. So, a+ejβΓBCKi+φi/ejω-1ejβ-A2-KiΓBC/ejω-1ejβ-A2>0 as long as fia>0 for all a≥1, β∈[-π,π), ω∈[-π,π), and ω≠0. Further, we can show that(25)fia=21-cos⁡ω1+A2-2Acos⁡βa2+2aΓBCKi+φicos⁡ω-1-Acos⁡ω-β+Acos⁡β+ΓBC22Kiφi+φi2Using the conditions ofTheorem7, we can prove fia>0 for all a≥1, β∈[-π,π), ω∈[-π,π), and ω≠0 in Appendix. Then eigenloci of λFω,β for all β∈[-π,π), ω∈[-π,π), and ω≠0 do not enclose the point -1,j0. Thus the roots of (21) have modulus less than unity. That is, the system achieves a formation tracking asymptotically.
In addition, let the final value ofEzk be Ez∞. Multiply z-1 on both sides of z-1I+zΓG(z)Lz+ΦEzz=z/z-1zΓG(z)RzLz1 and take the limit:(26)limz→1z-1I+zΓGzLz+Φz-1Ezz=limz→1zzΓGzRzLz1By final value theorem, we get(27)zΓGzLz+ΦEz∞=zΓGzRzLz1Since rankLz+Φ=n, Lz+Φ is invertible. And zΓG(z)≠0 is satisfied; thus Ez∞=RzLz+Φ-1Lz1. This completes the proof.When there is not communication time-delay in multiagent systems, we can get system error converging to zero combining equation (27) andLemma4. Otherwise, system error cannot converge to zero and converges to a certain value about desired reference trajectory, graph, and time-delays as shown in (27).


## 3.2. Second-Order ILC for Multiagent Systems with Time-Delays

In this subsection, we analyze convergence condition of second-order ILC for multiagent systems. Using previous two errors to correct the control input can compensate the influence of communication time-delays. Therefore, we propose the second-order ILC law:(28)ui,k+1t=ui,kt+Γ∑jϵNiai,jδi,kt+1-δj,kt+1-τi,j+ϕiei,kt+1+Λ∑jϵNiai,jδi,k-1t+1-δj,k-1t+1-τi,j+ϕiei,k-1t+1where Γ is first-order learning gain, Λ is second-order learning gain, Γ,Λ∈R, and Γ≠0,Λ≠0.Theorem 8.
Consider multiagent system (1); let Assumptions 1 and 3 hold and the learning law (28) be applied. Then the formation tracking objective limk→∞⁡Yi,kz=Yidz is achieved, given that Λ=-Γ, Qi>0, and any one of the following conditions are satisfied for all i=1,2,…,n:(29)11>A>0∧Pi,6>0∧Pi,7>02-1<A<0∧Pi,7>0∧Pi,8<031>A>0∧Pi,9>0∧Pi,10>0where(30)Pi,6=Qi-21+APi,7=1+A2+21+AQi+SPi,8=Qi-1-APi,9=1-A-QiPi,10=1-A2-21-AQi+SiQi=ΓBCKi+φiSi=ΓBC22Kiφi+φi2Ki=∑j∈Niai,jProof.
Taking the z-transformation of (28), we get(31)Ui,k+1z=Ui,kz+zΓ∑jϵNiai,jΔi,kz-z-τi,jΔj,kz+ϕiEi,kz+zΛ∑jϵNiai,jΔi,k-1z-z-τi,jΔj,k-1z+ϕiEi,k-1zBy (7), (31), andAssumption1, we have that(32)Ei,k+1z-Ei,kz=-zΓGz∑jϵNiai,jΔi,kz-z-τi,jΔj,kz+ϕiEi,kz-zΛGz∑jϵNiai,jΔi,k-1z-z-τi,jΔj,k-1z+ϕiEi,k-1zLet Ekz=E1,kz,E2,kz,…,En,kzT; (32) can be rewritten as(33)Ek+1z-Ekz=-zΓGzLz+ΦEkz-Rz1-zΓGzΦRz1-zΛGzLz+ΦEk-1z-Rz1-zΛGzΦRz1=-zΓGzLz+ΦEkz-zΛGzLz+ΦEk-1z+zGzRzΛ+ΓLz1Exchanging variable frequency z and parameter iteration k, Ekz can be rewritten as Ezk in (33):(34)Ezk+1-Ezk=-zΓGzLz+ΦEzk-zΛGzLz+ΦEzk-1+zGzRzΛ+ΓLz1Taking z-transformation of (34), we get(35)zEzz-Ezz=-zΓGzLz+ΦEzz-z-1zΛGzLz+ΦEzz+zz-1zGzRzΛ+ΓLz1FromTheorem8, Λ=-Γ. Further, we have that(36)z-1I+zz-1ΓGzLz+ΦEzz=0The characteristic equation of system is(37)detz-1I+zz-1ΓGzLz+Φ=0Let pz=z-1I+zz-1ΓG(z)Lz+Φ. Since p1=0 and detI+zΓG(z)Lz+Φ≠0, z=1 is the only root of (37).
Whenz≠1, both sides of (37) are divided by z-1:(38)detI+zΓGzLz+Φz=0
Next, we need to prove that all roots of (38) have modulus less than unity. Similar to the proof ofTheorem7, the conditions ofTheorem8 are easily verified.
Taking the limit of (36) yields(39)limz→1⁡z-1I+zz-1ΓGzLz+ΦEzz=0By final value theorem, we get(40)I+zΓGzLz+ΦEz∞=0Since detI+zΓG(z)Lz+Φ≠0 as proved above, rankI+zΓG(z)Lz+Φ=n. Based on Sylvester inequality, we get rankI+zΓG(z)Lz+Φ+rankEz∞≤n. Therefore, rankEz∞=0; that is Ez∞=0.Theorem8 is thus proved.The result ofTheorem7 shows that the error of multiagent system cannot converge to zero considering diverse communication time-delays. InTheorem8, using second-order ILC and the fact that the second-order learning gain is opposite of the first-order learning gain, the error of multiagent system can converge to zero considering diverse communication time-delays. In addition, the convergence speed can be kept as convergence speed of multiagent systems without time-delays.


## 4. Simulation

In this section, numerical simulations are presented to illustrate the first-order and second-order ILC for multiagent systems. Consider a multiagent system consisting of 4 agents in the directed graph as shown in Figure3. It can be seen that the graph is connected graph. In addition, only the 1st agent can obtain the desired reference trajectory.Figure 3
Directed graph of multiagent system.The Laplacian matrixL of graph G and the reference-accessibility matrix Φ are(41)L=1.50-0.5-1-0.41-0.60-0.300.5-0.2-0.7-0.100.8,Φ=1000000000000000From L and Φ, we get maxi=1,2,3,4⁡2Ki+ϕi=4. Let desired reference trajectory be rt=150+80×sin⁡t. The desired output deviation of each agent is chosen as(42)d1t=200-td2t=80d3t=-80d4t=-200+tSet the initial state of 4 agents as 875, 575, 175, and -125. It satisfiesAssumption1 that the initial resetting conditions for all agents and all the iterations meet desired input. When dynamics of system is Case 1, A=0.5,B=-0.8,C=-0.7, we set Γ=0.6. It can be seen that above learning gains satisfy condition 1 ofTheorem7. When dynamics of system is Case 2, A=-0.5,B=0.8,C=0.7, we set Γ=0.6. It can be seen that above learning gains satisfy condition 2 ofTheorem7. In Case 1 or Case 2, when t=1,2,…,100 and iteration is 200, the trajectory of the multiagent system without communication time-delays is shown as Figure 4. From this figure, it can be seen that the desired formation is well achieved to track desired reference trajectory.Figure 4
Trajectory of multiagent system without communication time-delays at iteration 200.In order to measure the formation accuracy quantitatively, the disagreement among all agents on their output errors is defined aserrork=1/nT∑i=1n∑t=1Tei,kt. When limk→∞⁡errork=0, multiagent system achieves formation tacking. Based on this, Figure 5 shows formation performance of the system at the first 200 iterations in Cases 1 and 2. From Figure 5, when iteration is around 120, the multiagent systems error in Case 1 can converge to zero and when iteration is around 150, the multiagent systems error in Case 2 can converge to zero.Figure 5
Convergence of formation tracking errors for multiagent system without communication time-delays.When there are communication time-delays in the multiagent system, we set 3 time-delays graphs of the multiagent system as shown in Figure6.Figure 6
The time-delays graphs of multiagent systems. (a) Time-delays graph 1. (b) Time-delays graph 2. (c) Time-delays graph 3.
(a) (b) (c)From Figure7, the system errors cannot converge to zero but a certain value as the increase of the iteration when there are time-delays. In addition, the bigger the communication time-delays are, the bigger the bounds of tracking errors are. Multiagent system cannot achieve formation tracking and the trajectory of multiagent system is as shown in Figure 8 when time-delays graph 1 holds.Figure 7
Convergence of formation tracking errors for multiagent system with communication time-delays. (a) Case 1. (b) Case 2.
(a) (b)Figure 8
Trajectory of multiagent system with communication time-delays graph 1 at iteration 200.ByTheorem8, we use second-order ILC and set Λ=-Γ when the time-delays graphs of multiagent systems are as shown in Figure 6. It can be seen that Case 1, A=0.5,B=-0.8,C=-0.7,Λ=-0.6, satisfies conditions 1 and 3 ofTheorem8 and Case 2, A=-0.5,B=0.8,C=0.7,Λ=-0.6, satisfies condition 2 ofTheorem8. The system errors are as shown in Figure 9. From Figure 9, when iteration is around 120 in Case 1, all the multiagent systems errors with communication time-delays and without communication time-delays can converge to zero. When iteration is around 150 in Case 2, all the multiagent systems errors with communication time-delays and without communication time-delays can converge to zero. Therefore, second-order ILC formula can keep the same learning error convergence speed, no matter what time-delays are. In Case 1 or Case 2, multiagent systems can achieve formation tracking and the trajectory of multiagent systems is shown as Figure 10.Figure 9
Convergence of formation tracking errors via second-order ILC for multiagent system with communication time-delays and via first-order ILC for multiagent system without communication time-delays. (a) Case 1. (b) Case 2.
(a) (b)Figure 10
Trajectory of multiagent systems with communication time-delays via second-order ILC at iteration 200.


## 5. Conclusion

When using first-order ILC to implement the formation tracking for multiagent system with communication time-delays, the system error cannot converge to zero but a certain value. Using second-order ILC and making second-order learning gain be opposite of first-order learning gain can compensate influence of communication time-delays. Based on the frequency domain ILC model for multiagent systems, we analyze the convergence condition by considering the convergence of arbitrary frequency in iteration dimension. We use the generalized Nyquist criteria and Gershgorin disc theorem to analyze the range of eigenvalues of the system characteristic equation and obtain the conditions for system convergence. The second-order ILC proposed in this paper for compensating for the influence of time-delays not only can make the system error converge to zero, but also can ensure that the convergence speed is the same as the convergence speed for system without time-delays. The simulation results also verify the effectiveness of the conclusion.



---

*Source: 8164297-2019-01-20.xml*