$Event-Triggered Control for Human-in-the-Loop Multi-Agent Systems under DoS Attacks$

Authors: yuanyuan  xu
china  chengdu
kai  liu
hongjing  liang
tieshan  li
qidong  liu
ximing  yang
Venue: IEEE_org_ICIST_2024_Conference
Published: 2024-08-01 11:11:42

Abstract: This paper introduces an event-triggered secure control scheme for human-in-the-loop multi-agent systems in the context of DoS attacks. The integration of human intelligence and decision-making significantly enhances system security, as a human provides command signals to a non-autonomous leader agent. To determine unknown states, an adaptive neural state observer utilizes neural networks to approximate nonlinear functions, while a relative threshold-based event-triggered control strategy is introduced to optimize communication resource usage. At the same time, a predictor is developed to monitor potential compromises in the edges of the multi-agent network to counteract attacks. Using Lyapunov analysis, it is shown that the proposed secure control protocol is capable of maintaining bounded closed-loop signals despite the occurrence of attacks. Finally, the effectiveness of the proposed scheme is validated by the simulation results.

Main paper: 
 i. introduction: 
 I N many earlier studies on coordination control [1]- [5], it is commonly assumed that every agent, including the leader, operates independently within a multi-agent system (MAS). This setup takes advantage of advancements in artificial intelligence, which allows for decreased human involvement. Although it offers a clear benefit, the idea of complete autonomy seems somewhat idealistic when considering the complexities of decision-making in emergencies. This can lead to significant consequences, as evidenced by the documented fatalities associated with Tesla's self-driving cars and the crashes involving Boeing 737 aircraft [6] . Given the potential for emergencies, it is standard practice to incorporate a human operator to support the autonomous system in performing tasks. Consequently, research on human-in-the-loop (HiTL) control has been carried out, as detailed in [7] - [9] .Generally, network attacks are broadly classified as deception attacks [10] and denial of service (DoS) attacks [11] . In contrast to the injection of malicious information, loss of communication can isolate an agent, thus impeding the effective functioning of multi-agent systems. Taking this into account, experts in control and computational intelligence have made efforts to combat DoS attacks, resulting in successful outcomes. The authors in [12] established sufficient conditions for multi-agent systems under DoS attacks with limited energy. Subsequently, this result was further extended in [13] , which also addressed attacks on multiple transmission channels. Similarly, in [14] , the authors proposed a recovery mechanism designed to restore communication in network systems during interruptions. Beyond mitigating network attacks, the effective use of communication resources significantly impacts system performance. Event-triggered control is crucial for reducing unnecessary information updates and deserves greater focus. In an event-triggered framework, to avoid unnecessary transmissions, the actuator interacts with the device only when specific trigger conditions are satisfied. In [15] , Tabuada et al. first introduced an event-triggered control scheme for analyzing nonlinear systems. Following this, a fixed-threshold strategy with a constant upper limit for the trigger condition was implemented in [16] to design a control scheme for linear systems. Although this strategy is simple and easy to implement, it may lead to reduced system performance. To address this issue, In [17] , Xing et al. advanced the approach from a fixed-threshold to a relative-threshold method, allowing the trigger threshold to be dynamically adjusted according to the magnitude of the trigger error. This approach provides a better balance between system performance and communication resource consumption. Drawing from the previous research, this paper investigates secure control mechanisms within HiTL multiagent systems using an event-triggered approach, where the control inputs of leaders vary dynamically over time due to human intervention. Unlike existing control schemes [18] - [22] , this paper explores a more comprehensive approach, taking into account that inter-agent communication within multiagent networks may be compromised by malicious attacks. To solve the issue of unavailable state variables, a method for state reconstruction is implemented, alongside an approach that reduces communication frequency and minimizes triggering events by using a threshold-based event-trigger mechanism.A graph is specified by P = (S, W, A), where S = {S 1 , . . . , S N +M }, with S 1 = {1, . . . N} and S N +M = {N + 1, . . . N + M }. The set W = {(i, m) ∈ S × S} represents the edges, while A = [a im ] ∈ R (N +M )×(N +M ) denotes the adjacency matrix. These elements are associated with the sets of nodes, followers, leaders, edges, and the matrix of adjacency in that sequence. Define i as the collection of nodes adjacent to node i, given by i = {m|(i, m) ∈ W }. The entries in the adjacency matrix A can be characterized by a im > 0 if (m, i) ∈ W ; otherwise, a im = 0. This implies that (m, i) ∈ W , then agent m obtains data from agent i. The scenario where self-connections are absent, indicated by (i, i) / ∈ W , is analyzed. The adjacency matrix element a im is defined as 1 if (i, i) ∈ W , and 0 if(i, i) / ∈ W . The Laplacian matrix L is given by L = D -A ∈ R (N +M )×(N +M ) , where D = diag {d 1 , . . . d N } ∈ R (N +M )×(N +M ) represents the in- degree matrix, with d i = N +M m=1,m =i a im .In a MAS with N followers and M leaders, the behavior for each ith follower is described byκi,m = κ i,m+1 + g i,m (κ i,m ) κi,n = u i + g i,n (κ i,n ) y i = κ i,1 , (1) i = 1, 2, . . . , N, m = 1, 2, . . . , n -1where the state vectors are given by κi,m =[κ i,1 , κ i,2 , . . . , κ i,m ]T ∈ R m , and y i ∈ R denotes the system outputs. The nonlinear functions g i,m (•) and g i,n (•) denote not explicitly known and smooth.The behavior of the leader is described by κ0,j = -κ 0,j + u 0 ȳ0,j = κ 0,jwhere the states of the leader are represented by κ 0,j , while ȳ0,j denotes the system output, j = 1, 2. The control input u 0 is an unknown bounded parameter.The subsequent lemma will be utilized in the design of the adaptive controller.Lemma 1: ( [23] ) For any variable π ∈ R, the following inequality holds0 ≤ |π| - π 2 √ π 2 + o 2 ≤ owhere o > 0 is a constant. To identify potential issues, a detector h (i, m, t) has been created to monitor the status of the edge (i, m) ∈ W . When agent i obtains data from agent m at time t, h (i, m, t) is set to 0; otherwise, it is set to 1. This mechanism helps agent verify whether the communication link to agent m is compromised.Therefore, the adjacency matrix Ā = [ā im ], which includes the monitoring information, is expressed asEQUATIONTo reduce the consumption of communication resources, an adaptive relative threshold-based event-triggered control method [17] was developed.The following outlines the devised triggering mechanism:u i (t) = ϑ i (t i,χ ) , ∀t ∈ [t i,χ , t i,χ+1 ) , χ ∈ N * (3) t i,χ+1 = inf {t > t i,χ ||α i (t)| ≥ ζ i |u i | + μ i } (4)where ϑ i (t) represents the controller to be designed subsequently. The decision to update the input u i (t) is determined by the triggering error α i (t) = ϑ i (t)u i (t). The constants ζ i and μ i , which are greater than 0.From [24] , within the time interval[t i,χ , t i,χ+1 ), the expres- sion ϑ i (t) = (1 + r i,1 (t) ζ i ) u i (t) + r i,2 (t) μ i holds, where |r i,1 (t)| ≤ 1 and |r i,2 (t)| ≤ 1. Consequently, the input u i (t) can be expressed as: u i (t) = ϑi(t) 1+ri,1(t)ζi - ri,2(t)μi1+ri,1(t)ζi . Radial basis function neural networks (RBFNNs) are frequently used for modeling nonlinear functions due to their strong ability to approximate functions effectively.The following assumption is essential for designing the neural state observer.Assumption 1: ( [25]) Assuming function g (•) meets the global Lipschitz condition, specific constants i for i = 1, 2, ..., n can be identified to satisfyEQUATIONwhere κi = [κ i,1 , . . . , κi,n ] T represents the estimation of κ i .Given the system's unobserved states, it is crucial to design an adaptive state estimation approach as outlined below.κi,m = κi,m+1 + i,m (y i -κi,1 )EQUATIONwhere κi = [κ i,1 , . . . , κi,n ] T represents the estimated values of κ i .Reformulate the the dynamics described in (1)EQUATION)whereEQUATIONΘ i = ⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ 0 1 0 • • • 0 0 0 1 • • • 0 . . . . . . . . . . . . . . . 0 0 0 • • • 1 0 0 0 • • • 0 ⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦ M i = [ i,1 , i,2 , • • • , i,n ] T , K T i = [1, 0, • • • , 0] U i,m = [0, • • • , 1, • • • , 0] T , Ūi = [0, • • • , 0, 1]T The observer outlined in (6) undergoes reformulation toκi = Θi κi + n m=1 U i,m (g i,m (κ i ) + g i,m ) + Ūi u i + M i y i (8)Let e i = κi -κi represent the estimation error. The differential of this error is given byėi = Θi e i + n m=1 U i,m βT i,m ω i,m (κ i ) + ε i,m + g i,mwhere e i = [e i,1 , . . . , e i,n ] T . |ε i,m | ≤ δ i,m . The vector M i is selected, and the matrix Θi is strictly Hurwitz. Consequently, there exists a matrix Ξi = ΞT i > 0, and the matrix Ψi = ΨT i > 0 satisfies ΘT i Ψi + Ψi Θi = -Ξi The candidate Lyapunov function V e is given byV 0 = e Ti Ψi e i The differential of the Lyapunov function V 0 can be represented byEQUATIONBy evaluatingEQUATIONEQUATIONSubstituting ( 11), ( 12) into (10) , it can be determined thatV0 ≤ -λ min Ξi -3n -2 -Ψi 2 n m=1 2 i,m e T i e i + Ψi 2 n m=1 βT i,m βi,m + δ 2 i,mThe coordinate transformations can be specified in the following way:EQUATIONwhere the dynamics errors are denoted by z i,m (for m = 2, . . . , n), while θ i,m denotes the filter's output.The outputs y jr , ẏj,r of multiple leaders are bounded in their trajectories.The following expression represents the error compensation signal:EQUATION)wherem = 1, • • • , n. The constants c i,1 , c i,2 , . . . , c i,m > 0. ρ i,m (0) set to 0 for m from 1 to n. i,1 =N +M m=1 āim . Details regarding the virtual controllers τ i,1 , τ i,2 , . . . , τ i,m will be presented in later sections.The formulation for the error compensation tracking signal is:EQUATIONDesign the virtual controller τ i,1 , along with the adaptive laws βi,1 and θi,1EQUATIONwhere β i,1 and ϑ i,1 are the weight vectors. ω i,1 and Υ i are the basis function vectors. The constants c i,1 , l i,1 , ς i,1 , ci,1 , ςi,1 > 0. Design the virtual controller τ i,m , along with the adaptive law βi,m , m = 2, . . . n -1.EQUATIONwhere β i,m is the weight vector. ω i,m is the basis function vectors. The constants c i,m , l i,m , i,m , ς i,m > 0. Design the virtual controller τ i,n , along with the adaptive laws βi,n and φiEQUATIONwhere the constants c i,n , a i , l i,n , ς i,n , ρ i , ςi,n > 0.Design the adaptive controller in the following manner:EQUATIONwith γi , i > 0.Theorem 1: Consider the multi-agent systems (1) subject to DoS attacks, considering Assumptions 1 and 2. The virtual controllers ( 15), ( 18), (20) , the event-triggered adaptive controller (23) , the observer (6), and the adaptive laws ( 16), ( 17), ( 19), (21) , and (22) provides assurance that, despite the occurrence of DoS attacks, all closed-loop signals will stay within bounded limits.Proof. Establish the total Lyapunov function asEQUATION)Utilizing the aforementioned analysis, the results in ( 24) can be expressed asEQUATION)whereH = min λ min Ξi -3n -5 2 -Ψi 2 n m=1 2 i,m λ min Ψi , 2c i,1 , 2 (c i,2 -1) , • • • , 2 (c i,n -1) , 2l i,s ς i,s 2a i,s -Ψi 2 - 1 4 , ςi,1 , ςi,n Y = N i=1 n m=1 δ 2 i,m + N i=1 n m=1 ς i,s 2l i,s β T i,s β i,s + N i=1i 2o i + 1 2 N i=1 η 2 i,1 + ι 2 i,1 + a 2 i + h2 i,n + N i=1 ςi,1 2r i,1 ψ T i,1 ψ i,1 + N i=1 ρ i 2σ i ςi,n ϕ 2 iFrom [26] , it follows that θ i,m+1τ i,m ≤ ¯ i . As a result, θ i,m will be constrained, ensuring that all signals stay bounded even when DoS attacks occur. Theorem 1 has been fully proved.From [24] , it can be concluded thatd dt |α i (t)| = d dt (α i (t) × α i (t)) 1 2 = sign (α i (t)) αi (t) ≤ θ (t)where θ (t) ≤ σ, σ > 0.Based on (3), ( 4) and α i (t), it can be determined that |α i (t)| ≥ ζ i |u i | + μ i . There exists a time interval t * such that t i,χ+1t i,χ ≥ t * . Consequently, the lower bound for the interval between executions t * satisfies t * ≥ ζi|ui|+μi σ , thus successfully avoiding Zeno behavior.In this section, simulation examples will be provided to validate the theoretical results.Investigate the dynamics related to the ith agent in the following manner:κi,1 = κ i,2 + g i,1 κi,2 = u i + g i,2 i = 1, 2, 3whereg 1,1 = cos (κ 1,1 ) , g 1,2 = cos (κ 1,1 κ 1,2 ) g 2,1 = sin(0.5κ 2,1 ) , g 2,2 = sin (κ 2,1 κ 2,2 ) g 3,1 = sin(0.2κ 3,1 ) , g 3,2 = sin (κ 3,1 κ 3,2 )the state vectors and control input are given by κi = [κ i,1 , κ i,2 ]T and u i , respectively.The dynamics of the leader can be described by the following equation: κ0,j = -κ 0,j + u 0 ȳ0,j = κ 0,j Values Initial states Values κ 1,1 (0) where κ 0,j represents the states of the leader, and ȳ0,j denotes the system output, j = 1, 2. For containment control purposes, the outputs are defined as y j,1 = ȳ0,1 + 0.5 and y j,2 = ȳ0,2 -0.5. Additionally, the control input u 0 is an unknown bounded variable. The initial conditions are specified in Table I . Moreover, the control input of the leader, represented as u 0 (t) and provided by a human operator, is not available to all followers. The design of the control input u 0 (t) is outlined as follows:EQUATIONu 0 (t) = ⎧ ⎨ ⎩ sin 2 (t) , 0 ≤ t ≤ 15 sin (t) cos (t) , 15 < t ≤ 30 cos 2 (t) , t > 30Fig. 1 illustrates the trajectories of followers κ i,1 for i = 1, 2, 3 and leaders y j,r for r = 1, 2 during the DoS attacks. The shaded flesh area indicates the period of the DoS attacks. With the implementation of the proposed secure control scheme, the system returns to normal operation after a brief period. Fig. 2 depicts the timing and intervals of the event-triggered communication strategy.This paper has presented a secure control scheme for HiTL MASs utilizing an event-triggered mechanism to address DoS attacks. By integrating human intelligence and decision-making, the system's security has been significantly improved as human operators send command signals to a nonautonomous leader agent. To manage limited communication resources, a relative threshold event-triggered control strategy has been adopted, which effectively reduces data transmissions. Simultaneously, the system's vulnerability to malicious attacks in the network, which could compromise control tasks, is acknowledged. Therefore, the effects of DoS attacks are considered. 
 
 Appendix:  



Reviews: review: title: title: This paper can be accepted.
main_review: main_review: This paper investigates an event-triggered secure control scheme for human-in-the-loop multi-agent systems in the context of DoS attacks. 
However in the reviewer's opinion, there are somecomments in the paper which should be addressed by the authors:
1. In addition to the contribution of the proposed work, it is suggested to discuss its main limitations/shortcomings.
2. Please add a structural flowchart of the paper's framework to enhance the readability of the paper.
3.Muhc more simulation results are needed to validate the effectiveness of the proposed method in the paper..
4.The English grammar and format of this paper could be further polished and checked carefully.
title: title: Please the author double-check this paper.
main_review: main_review: 1. Please clarify the role of human-in-the-loop control when the system is under attack.
2. There is an incorrect description in Part A of Section II.
3. The authors mentioned human-in-the-loop control in the abstract and Section I, but the designed human-in-the-loop control cannot be found in Section II.
4. The authors have not modeled DoS attacks in this paper and have not specified which term of the proposed controller is designed to counteract them.
5. The authors should explicitly outline the motivation and contributions of this paper in comparison to existing work in the field。
6. Many symbols have not been defined.
title: title: Comments to paper 52
main_review: main_review: This paper introduces an event-triggered secure control scheme for human-in-the-loop multi-agent systems in the context of DoS attacks. The effectiveness of the proposed scheme is validated by the simulation results. The presented results are impressive. The paper can be accepted for publication after some necessary revisions. The following issues are suggested for improving the paper:
1. The controller has many parameters. Please give a remark to illustrate the effect of the controller
parameters.
2. Some future works are also suggested in the conclusion section.
3. Some typos and grammar errors should be modified in this paper. Please carefully double check the manuscript.