# Event-Triggered Adaptive Dynamic Programming Consensus Tracking Control for Discrete-Time Multiagent Systems

Authors: Yuyang Zhao; Xiaolin Dai; Dawei Gong; Xinzhi Lv; Yang Liu
Journal: Complexity
 (2022)

---

## Abstract

This paper proposes a novel adaptive dynamic programming (ADP) approach to address the optimal consensus control problem for discrete-time multiagent systems (MASs). Compared with the traditional optimal control algorithms for MASs, the proposed algorithm is designed on the basis of the event-triggered scheme which can save the communication and computation resources. First, the consensus tracking problem is transferred into the input-state stable (ISS) problem. Based on this, the event-triggered condition for each agent is designed and the event-triggered ADP is presented. Second, neural networks are introduced to simplify the application of the proposed algorithm. Third, the stability analysis of the MASs under the event-triggered conditions is provided and the estimate errors of the neural networks’ weights are also proved to be ultimately uniformly bounded. Finally, the simulation results demonstrate the effectiveness of the event-triggered ADP consensus control method.


---

## Body

## 1. Introduction

Because of the wide applications in the control field [1–6], the consensus control of MASs gained more and more attentions. In recent years, quite a few methods have been reported to solve the consensus control problem of MASs, such as adaptive control [7, 8] and sliding mode control [9, 10]. It is worth mentioning that the previous methods focus on the stability of the MASs. However, the optimal characteristic is also worth considering in the consensus control problem. Optimal consensus control problem aims to find the optimal control policies which guarantee the stability of MASs and minimize the energy cost. As one of the core methods to achieve the optimal control policies, ADP approaches address the issue abovementioned by approximating the solutions of Hamilton–Jacobi–Bellman (HJB) equation [11–13].Till now, ADP approaches have been applied in the optimal consensus control of MASs [14–20]. In [14], an optimal coordination control algorithm has been designed to address the consensus problem of the multiagent differential games through fuzzy ADP. The optimal output heterogeneous MASs was considered in [15]. Based on this work, Gao et al. [16] considered the dynamic uncertainties factor in the cooperative output regulation problems. Zhang et al. [17, 18] considered the optimal consensus tracking control for discrete-time/continuous-time MASs. In order to address the optimal consensus problem for unknown MASs with input delay, the authors proposed a data-driven disturbed adaptive controller based on ADP technique in [19]. In [20], the problem of data-based optimal consensus control was studied for MASs with multiple time delays. All the above results are based on the assumption that the communication and computing resources are big enough to transmit system data and update the control policy in every time step. However, it is difficult to be satisfied in practice.Event-triggered control (ETC) is a well-recognized technology to address the above issue [21–24]. Different from the time-triggered control, whether the systems sample the signals or not only depends on the event-triggered condition. If it is satisfied at some time instants, then the data will be transmitted and the control policy will be updated. Therefore, compared with the control algorithms based on time-triggered scheme, the event-trigger control algorithms can efficiently save the computation resources [25]. In the past years, ETC is introduced to solve the optimal control problem under the limited computing resources [26–29]. In [26], an ETC method based on ADP is developed for continuous-time MASs. The authors considered the unknown internal states factor in the event-triggered optimal control for continuous-time MASs in [27]. The multiplayer zero-sum differential games are considered in [28] and an optimal consensus tracking control based on event-triggered is designed to solve this problem. In [29], an event-triggered optimal control algorithm is designed for unmatched uncertain nonlinear continuous-time systems. In [30], to save the limited network resources, an event-triggered mechanism was introduced to address the consensus problem of linear discrete-time MASs. The authors considered the event-triggered consensus problem of discrete-time multiagent networks in [31]. It is worthy to say, all the results in [26–29] studied the event-triggered optimal control for continuous-time MASs, but there were few works [30, 31] which consider the discrete-time MASs.Motivated by the above discussions, an event-triggered ADP control algorithm is designed to address the optimal consensus tracking problem for discrete-time MASs. The major contributions of this paper are emphasized as follows:(1)
Comparing with the existing event-triggered ADP consensus control methods [27–29], we design the adaptive ET condition for every agent in the MASs. Then, the agent samples the data and communicates with the neighbors only when its event-triggered condition is satisfied. That means the agents in the MASs may not communicate with their neighbors or update their control policies at the same time instant, and then, the communicate resources are saved.(2)
In this paper, we give the stability analysis for the MASs under the event-triggered condition. It shows all agents in the discrete-time MASs will achieve consensus under the ET condition. And, we also prove the weight estimate errors for the critic neural networks (NNs) and actor NNs are uniformly ultimately bounded during the learning process.The rest of this paper is organized as follows. In Section2, the discrete-time MASs are considered and the consensus problem is formed. The event-triggered conditions for each agent in the system are introduced and the stability analysis is given in Section 3. Then, NN-based event-triggered ADP algorithm is introduced in Section 4, and the simulation results of this algorithm are given in Section 5. Finally, the conclusions are shown in Section 6.


## 2. Problem Formation

Consider the discrete-time MASs:(1)xik+1=Axik+Biuik,where xik∈Rn×1anduik∈Rmi×1 denote the state and the coordination control of agent i,i∈1,2,…,N, respectively. A∈Rn×nandBi∈Rn×mi are the constant matrices.The leader’s dynamics function is defined as(2)x0k+1=Ax0k,where x0k∈Rn denotes the state of the leader.The local neighbor consensus tracking errorξi is defined as(3)ξik=∑j∈Niαijxik−xjk+βixik−x0k.where αij denotes the adjacency elements, aij>0 if agent i can communicate with agent j, otherwise, αij=0, and βi denotes the pinning gain, βi>0, if agent i can communicate with the leader, otherwise, βi=0. We assume that there is at least one agent who can get the information from the leader.Under the event-triggered scheme, the discrete-time MASs transmit the systems’ data only when the event is triggered. Here, we define that the event is triggered at the discrete-time instants’ sequenceki,1,ki,2,…,ki,p−1,ki,p, for i=1,2,…,Nandp=1,2,…,∞. At the pth event-triggered instant of agent i, the consensus errors of agent i denote as ξ^iki,p=∑j∈Niαijxiki,p−xjkj,q+βixiki,p−x0ki,p.The event-triggered error is defined as(4)δk=ξ^iki,p−ξik,which means the difference between the consensus tracking errors at the pth event-triggered instant and the current local neighbor consensus tracking errors.Then, the consensus problem of the discrete-time MASs is to find the distributed feedback control law,uik=χξ^iki,p, which becomes a continuous signal through a zero-order hold (ZOH) device when ki,p≤k<ki,p+1.Then, the local cost function is defined as(5)Jiξik,uik,ujk=∑l=k∞ρt−kUiξit,uit,ujt=Uiξik,uik,ujk+ρJiξik+1,uik+1,ujk+1,where(i)
Uiξik,uik,ujk: the utility function, for agent i,(6)Uiξik,uik,ujk=ξiΤkQiiξik+uiΤkRiiuik+∑j∈NiujΤkSijujk.(ii)
ujk: the control of the neighbors of agent i.(iii)
Qii, Rii, and Sij: positive symmetric weighting matrices.(iv)
p: the discount factor, 0<ρ≤1.According to Bellman’s principle, the optimal local cost functionJi∗ξik,uik,ujk can be defined as(7)Ji∗ξik,uik,ujk=minuiξiki,pUiξik,uik,ujk+ρJi∗ξik+1,uik+1,ujk+1,which is also called discrete-time HJB equations.The optimal disturbed control lawui∗ξiki,p is defined as(8)ui∗ξiki,p=argminuiξiki,pUiξik,uik,ujk+ρJi∗ξik+1,uik+1,ujk+1.


## 3. Stability Analysis

Assumption 1 (see [32]).
There exist positive constantsL, L1, ϕ, and ψ, a C1 function V: Rn⟶R≥0, and class κ∞ functions γ1 and γ2, such that(9)A≤L,B≤L,χξik+δ(k≤Lξ^iki,p,(10)γ1x≤Vxk≤γ2x∀x∈ℝn,(11)VAxik+Biχξik+δk−Vxik≤−ϕVxik+ψδk.
If (10) and (11) are satisfied, function V is called an ISS-Lyapunov function for the discrete-time MAS.
Let us consider a situation thatk∈ki,p,ki,p+1, which means that the ET condition is satisfied at the sampling instant ki,p. In this situation, it is obvious that δi,pk+1=ξ^iki,p−ξik+1. Then, we have(12)δk+1≤ξ^iki,p+ξik+1.
Substituting (1) and (2) into (3), we have(13)ξik+1=∑j∈NiαijAxik−xjk+Biuik−Bjujk+βiA(xik−x0k+Biuik=Aξik+∑j∈NiαijBiuik−Bjujk+βiBiuik).
Then, we can have(14)ξik+1≤Aξik+∑j∈NiαijBiuik−Bjujk+βiBiuik≤Aξik+∑j∈NiαijBiuik−Bjujk+βiBiuik≤Aξik+∑j∈NiαijBiuik+Bjujk+βiBiuik.
Substituting (9) into (14), we have(15)ξik+1≤Aξik+∑j∈NiαijBiuik+Bjujk+βiLuik≤Lξik+L∑j∈Niαijujk+uik+βiLuik≤Lξik+L∑j∈Niαijujk+Lξ^iki,p+βiL2ξ^iki,p.
Therefore,(16)δk≤ξ^iki,p+ξik−1≤ξ^iki,p+Lξik−1+L∑j∈Niαijujk+Lξ^iki,p+βiL2ξ^iki,p=ξ^iki,p+βiL2ξ^iki,p+L∑j∈Niαijujk+Lξ^iki,p+Lξik−1≤1+βiL2ξ^iki,p+L∑j∈Niαijujk+Lξ^iki,p+LLξik−2+L∑j∈Niαijujk+Lξ^iki,p+βiL2ξ^iki,p⋯≤1+βiL2+βiL3+⋯+βiLk−ki,p+1ξ^iki,p+1+L+⋯+Lk−ki,p−1L∑j∈Niαijujk+Lξ^iki,p+Lk−ki,pξ^iki,p.
Then, we can rewrite the ET condition as(17)δi,pk≤δiT=1+Lk−ki,pξ^ki,p+βiL21−Lk−ki,p1−Lξ^ki,p+L1−Lk−ki,p1−L∑j∈Niαijujk+Lξ^iki,p,for every k∈ki,p,ki,p+1.
To better illustrate the control process, a flowchart has been displayed in Figure1. The transmitted data and control policies are updated at ki,p instant, and the event-triggered error is reset to zero. Once the event-triggered condition is satisfied, the current instant becomes the next triggering instant ki,p+1, and the system data are transmitted. Otherwise, keep the transmitted data and control policies unchanged.
Then, we will prove the discrete-time MAS is stable under our event-triggered conditions.Figure 1
The flowchart of the event-triggered consensus control algorithm.Theorem 1.
If a discrete-time MASs which is under assumption 1 and satisfies the function,(18)Vxk≤Vxki,p+1=−φϕV(x(ki,p)ki,p+1−ki,p+ψVxki,p,for every k∈ki,p,ki,p+1, where φ∈0,1, then the system is asymptotically stable.Proof.
According to (9) and (11), we obtain(19)VAxik+Biχξik+δk≤1−ϕVxik+ψδk.
Then, applying (18) into (19), we have(20)VAxik+Biχξik+δk≤1−ϕVxik+ψδk≤1−ϕVxik+ψ1+Lk−ki,p+βiL21−Lk−ki,p1−Lξ^iki,p+L1−Lk−ki,p1−L∑j∈Niαijujk+Lξ^iki,p.
Solving (20), we can obtain(21)VAxik+Biχξik+δk≤1−ϕk−ki,pVxiki,p+ψ1−1−ϕk−ki,pϕ1+Lk−ki,p+βiL21−Lk−ki,p1−Lξ^iki,p+L1−Lk−ki,p1−L∑j∈Niαijujk+Lξ^iki,p.
We define a function as(22)Fxk=−φϕV((xki,pk−ki,p+ψVxki,p,∀k∈ki,p,ki,p+1.
According to (18), we have(23)Vxk≤Fxk,for every k∈ki,p,ki,p+1.
From (22), we obtain(24)ΔF=Fxk+1−Fxk=−φϕVxki,p.
Applying (9) into (24), we have(25)ΔF≤−φϕγ1xki,p∀k∈ki,p,ki,p+1.
Since (23) and (25) hold, the stability of the discrete-time MAS is proved.Remark 1.
We give the event-triggered condition for each agent in the discrete-time MASs. Moreover, the stability of the systems is also proved in this paper.


## 4. Event-Trggered Controller Design

In this section, considering the good fitting characteristics of the neural networks (NN) [33, 34], the actor-critic neural network structure is introduced to approximate the local cost function Jiξik,uik,ujk and the distributed feedback control law uix. The actor-critic NNs are defined as(26)ℱ^ω,z,w=ωΤΨwΤz=ωΤΨZ,where z denotes the input data, Ψ· denotes the activation functions, and w and ω denote the weight matrices of the NNs.


### 4.1. Formulation of the Critic Networks

The critic NN approximates the local cost functionJiξik,uik,ujk in this paper as follows:(27)V^ik=ω^ciΤΨciw^ciΤzcik,where zcik denotes the input vector of the critic NN which is constituted by ξik, uik, and uNik, Ψci· denotes the activation function of the critic NN, and w^ci and ω^ci are the weight matrices for the critic NN.We define the difference between the current cost value and the estimate value as the error function of the critic NN as follows:(28)εcik=−Uiξik,uik,ujk+ρω^ciTΨciw^ciΤzcik+1+ω^ciTΨciw^ciTzcik.Then, the loss function for the critic NN is given as(29)Eci=12εciΤkεcik.Our objective is to minimize the loss function during the critic NN training.The weights for the critic NN are updated according to the gradient-based rule, which is given as follows:(30)ω^cik+1=ω^cik−Kci∂Ecik∂εcik∂εcik∂ω^cik=ω^cik−KciεciTk−ρψciw^ciTzcik+1+ψciw^ciTzcik,where Kci denotes the learning rate.


### 4.2. Formulation of the Actor Networks

The actor NN approximates the disturbed control lawuik, which can be formulated as(31)u^ik=ω^aiTΨaiw^aiTzaik,where zaik is the input vector of the actor NN, ψai· is the activation function for the actor NN, and ω^ai and w^ai are the weight matrices for the actor NN.We define the difference between the current local cost valueV^ik and the target cost value Pik as the error function, which is given as(32)εai=V^ik−Pik.In this paper, the target cost value is defined as 0.Then, the loss function for the actor NN is given as(33)Eai=12εaiTkεaik.Our objective is to minimize the loss function during the actor NN training.The weights for the actor NN is updated according to the gradient-based rule, which is given as follows:(34)ω^aik+1=ω^aik−Kai∂Eaik∂εaik∂εaik∂V^ik∂V^ik∂u^ik∂u^ik∂ω^aik=ω^aik−KaiεaiTkω^ciΩkCiΨaiw^aiTzaik,where Ωk=∂Ψciw^Tcizci/∂zci, Ci=∂zci/∂ui, and Kai is the learning rate for the actor NN.The procedure of the NN-based event-triggered optimal consensus control algorithm for discrete-time MASs is shown in Algorithm 1.Algorithm 1: NN-based event-triggered optimal consensus control algorithm for discrete-time MASs.
Initialization:Give the computation precisionτ and the initial state xi0 for agent i;Give the initial statex00 for the leader;Select the learning rateKai and Kci;Give the positive matricesQii, Rii, and Sij;Initialize the event-triggered error conditionδiT0=0;Select the positive constantL;Iteration:Let the iteration indexk=0;repeat:Calculate the tracking errorξ^ik and the event-triggered error δik;IFδik≥δiTk:Event-triggered errorδik=0;Event-triggered indexki,p=k;Compute the control lawu^ik;Compute the local cost functionV^ik;Compute the next statex^ik+1 of agent i and the next state x^0k+1 of the leader agent;Calculate the next tracking errorξik;Compute the control lawu^ik+1;Compute the local cost functionV^ik+1;Update the weights matrix of the critic NN;Update the weights matrix of the actor NN;ELSE:The control lawu^ik=u^ik−1;Compute the control lawu^ik;Compute the next statexik+1 of agent i and the next sate x0k+1 of the leader agentaccording to the model NN;k=k+1 ;Untilω^cik+1−ω^cik≤τ;EndTheorem 2.
Consider a discrete-time MAS. The weights of critic NN and actor NN are updated following (30) and (34), respectively, under condition (17). The state xi, the critic NN weight estimation error, ω˜ci=ωci−ωci∗, and the action weight estimation error, ω˜ai=ωai−ωai∗, in the close loop system are UUB.Proof.
Case 1: the ET condition is satisfied at iteration indexk. The Lyapunov function for agent i can be defined as follows:(35)ℒik=ℒi,1k+ℒi,2k+ℒi,3k,whereLi,1k=xiTkxik, Li,2k=1/Kcitrω˜ciTkω˜cik, and Li,3k=1/Kai·trω˜aiTkω˜aik. The difference between ℒi,1k+1 and ℒi,1k can be given as(36)Δℒi,1=xiTk+1xik+1−xiTkxik,=xik+12−xik2,=−xik2+Aixik+Biωai∗Ψwaiξi2.The difference betweenℒi,2k+1 and ℒi,2k can be given as(37)Δℒi,2=1/Kcitrω˜ciTk+1ω˜cik+1−1/Kcitrω˜ciTkω˜cik=1/Kcitrω˜ciTk+1ω˜cik+1−ω˜ciTkω˜cik.According to the update function for the weight matrix of critic NN (30), we have(38)ω˜cik+1=ω˜cik−KciεciTk−ρψciw^ciTzcik+1+ψciw^ciTzcik=ω˜cik−Kci−ρψciw^ciTzcik+1+ψciw^ciTzcik×−Uik+ρω^ciTΨciw^ciTzcik+1+ω^ciTΨciw^ciTzcik=ω˜cik−Kciηk−Uik+ω^ciTηk=I−KciηkηTkω˜cik+KciUikηk,whereηk=−ρΨciw^ciTzcik+1+Ψciw^ciTzcik.Substituting (38) into 37 we have(39)Δℒi,2=1/KcitrI−KciηkηTkω˜cik+KciUikηkT×I−KciηkηTkω˜cik+KciUikηk−ω˜ciTkω˜cik≤1/KciKciηkηTkF2ω˜cik2+KciUikηk2=ηkηTkF2ω˜cik2+Uikηk2=ηk4ω˜cik2+Uikηk2.The difference betweenℒi,3k+1 and ℒi,3k can be given as(40)Δℒi,3=1/Kai2trω˜aiTk+1ω˜aik+1−1/Kaitrω˜aiTkω˜aik,=1/Kai2trω˜aiTk+1ω˜aik+1−ω˜aiTkω˜aik.According to the update function for the weight matrix of critic NN (34), we have(41)ω˜aik+1=ω˜aik−KaiεaiTkω^ciΩkCiΨaiw^aiTzaik=ω˜aik−KaiV^ikω^ciΩkCiΨaiw^aiTzaik.Substituting (41) into (40), we have(42)Δℒi,3=1Kaitrω˜aik−KaiV^ikω^ciΩkCiΨaiw^aiTzaikT×ω˜aik−KaiV^ikω^ciΩkCiΨaiw^aiTzaik−ω˜aiTkω˜aik=1Kaitr−ω˜aiTkKaiV^ikω^ciΩkCiΨaiw^aiTzaik−KaiV^ikΨaiTw^aiTzaikCiTΩTkω^Tciω˜aik+Kai2V^i2kΨaiTw^aiTzaikCiTΩTkω^Tciω^ciΩkCiΨaiw^aiTzaik=tr−2V^ikΨaiTw^aiTzaikCiTΩTkω^Tciω˜aik+KaiV^i2kΨaiTw^aiTzaikCiTΩTkω^Tciω^ciΩkCiΨaiw^aiTzaik=−2V^iktrω˜aiTkω^ciΩkCiΨaiw^aiTzaik+KaiV^i2kω^ciΩkCiΨaiw^aiTzaikF2.Combining (36), (39), and (42), the difference between Δℒk and Δℒk+1 is given as(43)Δℒ≤−xik2−2V^iktrω˜aiTkω^ciΩkCiΨaiw^aiTzaik+Aixik+Biωai∗Ψwaiξi2+ηk4+Uikηk2+KaiV^i2kω^ciΩkCiΨaiw^aiTzaikF2≤−xik2−2V^iktrω˜aiTkω^ciΩkCiΨaiw^aiTzaik+Dmi,whereDmi=supAixik+Biωai∗Ψwaiξi2+ηk4+Uikηk2+KaiV^i2kω^ciΩkCiΨaiw^aiTzaikF2.If one of the conditionsxk≥Dmi or V^iktrω˜aiTkω^ciΩkCiΨaiw^aiTzaik≥Dmi holds, the difference is Δℒ<<0. This means the states of the system and the error of the weight matrices for critic NN and actor NN are UUB.Case 2: if the ET condition is not satisfied at iteration instantk, consider the Lyapunov function (35) in case 1.The difference betweenℒi,1k+1 and ℒi,1k can be given as(44)Δℒi,1=xiTk+1xik+1−xiTkxik=xik+12−xik2≤−xik2+Lxk+L2ξk.The weight matrices for the critic NN and actor NN are not updated when the ET condition is not satisfied, so the differences areΔℒi,2=0 and Δℒi,3=0.CombiningΔℒi,1, Δℒi,2, and Δℒi,3, the difference between Δℒk and Δℒk+1 is given as(45)Δℒ=Δℒi,1+Δℒi,2+Δℒi,3≤−xik2+Lxk+L2ξk.If the conditionxik≥Lxk+L2ξk holds, the difference is Δℒ<0. This means when the ET condition is not satisfied at the time index k, the states of the system and the error of the weights matrices for the critic NN and actor NN are UUB.


## 4.1. Formulation of the Critic Networks

The critic NN approximates the local cost functionJiξik,uik,ujk in this paper as follows:(27)V^ik=ω^ciΤΨciw^ciΤzcik,where zcik denotes the input vector of the critic NN which is constituted by ξik, uik, and uNik, Ψci· denotes the activation function of the critic NN, and w^ci and ω^ci are the weight matrices for the critic NN.We define the difference between the current cost value and the estimate value as the error function of the critic NN as follows:(28)εcik=−Uiξik,uik,ujk+ρω^ciTΨciw^ciΤzcik+1+ω^ciTΨciw^ciTzcik.Then, the loss function for the critic NN is given as(29)Eci=12εciΤkεcik.Our objective is to minimize the loss function during the critic NN training.The weights for the critic NN are updated according to the gradient-based rule, which is given as follows:(30)ω^cik+1=ω^cik−Kci∂Ecik∂εcik∂εcik∂ω^cik=ω^cik−KciεciTk−ρψciw^ciTzcik+1+ψciw^ciTzcik,where Kci denotes the learning rate.


## 4.2. Formulation of the Actor Networks

The actor NN approximates the disturbed control lawuik, which can be formulated as(31)u^ik=ω^aiTΨaiw^aiTzaik,where zaik is the input vector of the actor NN, ψai· is the activation function for the actor NN, and ω^ai and w^ai are the weight matrices for the actor NN.We define the difference between the current local cost valueV^ik and the target cost value Pik as the error function, which is given as(32)εai=V^ik−Pik.In this paper, the target cost value is defined as 0.Then, the loss function for the actor NN is given as(33)Eai=12εaiTkεaik.Our objective is to minimize the loss function during the actor NN training.The weights for the actor NN is updated according to the gradient-based rule, which is given as follows:(34)ω^aik+1=ω^aik−Kai∂Eaik∂εaik∂εaik∂V^ik∂V^ik∂u^ik∂u^ik∂ω^aik=ω^aik−KaiεaiTkω^ciΩkCiΨaiw^aiTzaik,where Ωk=∂Ψciw^Tcizci/∂zci, Ci=∂zci/∂ui, and Kai is the learning rate for the actor NN.The procedure of the NN-based event-triggered optimal consensus control algorithm for discrete-time MASs is shown in Algorithm 1.Algorithm 1: NN-based event-triggered optimal consensus control algorithm for discrete-time MASs.
Initialization:Give the computation precisionτ and the initial state xi0 for agent i;Give the initial statex00 for the leader;Select the learning rateKai and Kci;Give the positive matricesQii, Rii, and Sij;Initialize the event-triggered error conditionδiT0=0;Select the positive constantL;Iteration:Let the iteration indexk=0;repeat:Calculate the tracking errorξ^ik and the event-triggered error δik;IFδik≥δiTk:Event-triggered errorδik=0;Event-triggered indexki,p=k;Compute the control lawu^ik;Compute the local cost functionV^ik;Compute the next statex^ik+1 of agent i and the next state x^0k+1 of the leader agent;Calculate the next tracking errorξik;Compute the control lawu^ik+1;Compute the local cost functionV^ik+1;Update the weights matrix of the critic NN;Update the weights matrix of the actor NN;ELSE:The control lawu^ik=u^ik−1;Compute the control lawu^ik;Compute the next statexik+1 of agent i and the next sate x0k+1 of the leader agentaccording to the model NN;k=k+1 ;Untilω^cik+1−ω^cik≤τ;EndTheorem 2.
Consider a discrete-time MAS. The weights of critic NN and actor NN are updated following (30) and (34), respectively, under condition (17). The state xi, the critic NN weight estimation error, ω˜ci=ωci−ωci∗, and the action weight estimation error, ω˜ai=ωai−ωai∗, in the close loop system are UUB.Proof.
Case 1: the ET condition is satisfied at iteration indexk. The Lyapunov function for agent i can be defined as follows:(35)ℒik=ℒi,1k+ℒi,2k+ℒi,3k,whereLi,1k=xiTkxik, Li,2k=1/Kcitrω˜ciTkω˜cik, and Li,3k=1/Kai·trω˜aiTkω˜aik. The difference between ℒi,1k+1 and ℒi,1k can be given as(36)Δℒi,1=xiTk+1xik+1−xiTkxik,=xik+12−xik2,=−xik2+Aixik+Biωai∗Ψwaiξi2.The difference betweenℒi,2k+1 and ℒi,2k can be given as(37)Δℒi,2=1/Kcitrω˜ciTk+1ω˜cik+1−1/Kcitrω˜ciTkω˜cik=1/Kcitrω˜ciTk+1ω˜cik+1−ω˜ciTkω˜cik.According to the update function for the weight matrix of critic NN (30), we have(38)ω˜cik+1=ω˜cik−KciεciTk−ρψciw^ciTzcik+1+ψciw^ciTzcik=ω˜cik−Kci−ρψciw^ciTzcik+1+ψciw^ciTzcik×−Uik+ρω^ciTΨciw^ciTzcik+1+ω^ciTΨciw^ciTzcik=ω˜cik−Kciηk−Uik+ω^ciTηk=I−KciηkηTkω˜cik+KciUikηk,whereηk=−ρΨciw^ciTzcik+1+Ψciw^ciTzcik.Substituting (38) into 37 we have(39)Δℒi,2=1/KcitrI−KciηkηTkω˜cik+KciUikηkT×I−KciηkηTkω˜cik+KciUikηk−ω˜ciTkω˜cik≤1/KciKciηkηTkF2ω˜cik2+KciUikηk2=ηkηTkF2ω˜cik2+Uikηk2=ηk4ω˜cik2+Uikηk2.The difference betweenℒi,3k+1 and ℒi,3k can be given as(40)Δℒi,3=1/Kai2trω˜aiTk+1ω˜aik+1−1/Kaitrω˜aiTkω˜aik,=1/Kai2trω˜aiTk+1ω˜aik+1−ω˜aiTkω˜aik.According to the update function for the weight matrix of critic NN (34), we have(41)ω˜aik+1=ω˜aik−KaiεaiTkω^ciΩkCiΨaiw^aiTzaik=ω˜aik−KaiV^ikω^ciΩkCiΨaiw^aiTzaik.Substituting (41) into (40), we have(42)Δℒi,3=1Kaitrω˜aik−KaiV^ikω^ciΩkCiΨaiw^aiTzaikT×ω˜aik−KaiV^ikω^ciΩkCiΨaiw^aiTzaik−ω˜aiTkω˜aik=1Kaitr−ω˜aiTkKaiV^ikω^ciΩkCiΨaiw^aiTzaik−KaiV^ikΨaiTw^aiTzaikCiTΩTkω^Tciω˜aik+Kai2V^i2kΨaiTw^aiTzaikCiTΩTkω^Tciω^ciΩkCiΨaiw^aiTzaik=tr−2V^ikΨaiTw^aiTzaikCiTΩTkω^Tciω˜aik+KaiV^i2kΨaiTw^aiTzaikCiTΩTkω^Tciω^ciΩkCiΨaiw^aiTzaik=−2V^iktrω˜aiTkω^ciΩkCiΨaiw^aiTzaik+KaiV^i2kω^ciΩkCiΨaiw^aiTzaikF2.Combining (36), (39), and (42), the difference between Δℒk and Δℒk+1 is given as(43)Δℒ≤−xik2−2V^iktrω˜aiTkω^ciΩkCiΨaiw^aiTzaik+Aixik+Biωai∗Ψwaiξi2+ηk4+Uikηk2+KaiV^i2kω^ciΩkCiΨaiw^aiTzaikF2≤−xik2−2V^iktrω˜aiTkω^ciΩkCiΨaiw^aiTzaik+Dmi,whereDmi=supAixik+Biωai∗Ψwaiξi2+ηk4+Uikηk2+KaiV^i2kω^ciΩkCiΨaiw^aiTzaikF2.If one of the conditionsxk≥Dmi or V^iktrω˜aiTkω^ciΩkCiΨaiw^aiTzaik≥Dmi holds, the difference is Δℒ<<0. This means the states of the system and the error of the weight matrices for critic NN and actor NN are UUB.Case 2: if the ET condition is not satisfied at iteration instantk, consider the Lyapunov function (35) in case 1.The difference betweenℒi,1k+1 and ℒi,1k can be given as(44)Δℒi,1=xiTk+1xik+1−xiTkxik=xik+12−xik2≤−xik2+Lxk+L2ξk.The weight matrices for the critic NN and actor NN are not updated when the ET condition is not satisfied, so the differences areΔℒi,2=0 and Δℒi,3=0.CombiningΔℒi,1, Δℒi,2, and Δℒi,3, the difference between Δℒk and Δℒk+1 is given as(45)Δℒ=Δℒi,1+Δℒi,2+Δℒi,3≤−xik2+Lxk+L2ξk.If the conditionxik≥Lxk+L2ξk holds, the difference is Δℒ<0. This means when the ET condition is not satisfied at the time index k, the states of the system and the error of the weights matrices for the critic NN and actor NN are UUB.


## 5. Simulation Analysis

To test the effectiveness of the proposed algorithm, we apply the proposed algorithm in a numerical example. Consider a discrete-time leader-follower MAS consisting of 4 agents with a network topology, as shown in Figure2. In the topology, agent 0 denotes the leader and the followers are labeled as agent 1 to agent 4. The adjacency elements α21, α31, and α42 are set to 1. The other adjacency elements are set to 0. In this numerical example, only agent 1 can communicate with the leader, which means β1=1 and β2=β3=β4=0. The weight matrices of the utility function are selected as Q11=Q11=Q11=Q11=I2×2, R11=R22=R33=R44=1, and S12=S13=S14=S23=S24=S32=S34=S41=S43=0.Figure 2
The network topology for the discrete-time MAS.The dynamics matrix for the leader are set toA=0.99500.0798−0.07980.9950. The dynamics matrices for the followers are set to A=0.99500.0798−0.07980.9950, B1=0.20470.0898, B2=0.21470.2895, B3=0.20970.1897, and B4=0.20000.1000.The parameters for the critic NN and the actor NN are set toρ=0.9, and Kc1=Kc2=Kc3=0.01, Kc4=0.001, and Ka1=Ka2=Ka3=Ka4=0.01. Ψc1k1,p=ξ^112k1,p,ξ^122k1,p,u^1k1,p, Ψc2k2,p=ξ^212k2,p,ξ^222k2,p,u^21k2,p,u^22k2,p, Ψc3k3,p=ξ^312k3,p,ξ^322k3,p,u^21k3,p,u^23k3,p, and Ψc4k4,p=ξ^412k4,p,ξ^422k4,p,u^2k4,p,u^4k4,p are the activation functions of the critic NNs. The activation functions of the actor NNs are set to Ψaiki,p=ξ^iki,p. x00=0.6311,0.0899T, x10=0.9954,0.3321T, x20=0.2973,0.0620T, x30=0.2982,0.0464T, and x40=0.5054,0.7614T are chosen as the initial states for the leader and the follower agents in the system. We set L=0.1.The tracking path for every agent in the discrete-time MAS is shown in Figure3. From Figure 3, we can observe that all the agents in the system can reach the same state as the leader, and then, they achieve synchronization. The driving errors for the agents in the system are shown in Figure 4. All the agents’ driving error all are not updated at every instant k, that is to say, all the agents are driven when the ET condition is satisfied. Figure 5 shows the comparisons of event-triggered errors and thresholds for every agent in the system. In Figure 5, we can observe that the event-triggered errors are always smaller than the thresholds during the tracking process, and we only sample the data when the event-triggered errors are bigger or equal to the thresholds, so we sample the less data and save computing resources using our algorithm. Figure 6 shows the comparisons of the required number of transmitting data under the time-triggered and event-triggered ADP algorithm for every agent in the system. We can observe the required number of the event-triggered algorithm is much less than the required number of the time-triggered algorithm.Figure 3
The tracking path for the system under the ET condition.Figure 4
The driving error for every agent in the system under the ET condition. (a) Driving error for agent1. (b) Driving error for agent2. (c) Driving error for agent3. (d) Driving error for agent4.
(a)(b)(c)(d)Figure 5
Comparisons of event-triggered errors and thresholds for every agent in the system. (a) Comparisons of event-triggered errors and thresholds for agent1. (b) Comparisons of event-triggered errors and thresholds for agent2. (c) Comparisons of event-triggered errors and thresholds for agent3. (d) Comparisons of event-triggered errors and thresholds for agent4.
(a)(b)(c)(d)Figure 6
Comparisons of required number of transmitting data under the time-triggered and event-triggered ADP for every agent in the system. (a) The required number of transmitting data under the time-triggered ADP and event-triggered ADP for agent1. (b) The required number of transmitting data under the time-triggered ADP and event-triggered ADP for agent2. (c) The required number of transmitting data under the time-triggered ADP and event-triggered ADP for agent1. (d) The required number of transmitting data under the time-triggered ADP and event-triggered ADP for agent1.
(a)(b)(c)(d)


## 6. Conclusion

An event-triggered optimal consensus tracking control algorithm based on the ADP structure is proposed in this paper. To save the communication and computation resources, we introduce the event-triggered scheme to the optimal consensus tracking control algorithm. The neural networks technology is introduced to simplify the application of the proposed algorithm. It is proved the discrete-time MASs are stable with the proposed algorithm and the estimate errors of the weights for NNs are UUB. The simulation results illustrate the efficiency of the proposed method.



---

*Source: 6028054-2022-03-01.xml*