SUGISAKAMASANORI

Journal of Robotics, Networking and Artificial Life
Vol. 9(4); March (2023), pp. 374–378
ISSN (Online):2352-6386; ISSN (Print):2405-9021
https://alife-robotics.org/jrnal.html

Jichao Zhao1, Fengzhi Dai1, Yunzhong Song2
1

College of Electronic Information and Automation, Tianjin University of Science and Technology, 300222, China;
School of Electrical Engineering and Automation, Henan Polytechnic University, 454003, China

2

Received 22 November 2021
Accepted 23 October 2022

Multi-agent system
UAV
Optimal control
Distributed formation control

In this paper, the distributed optimization problem for multi-agent system (MAS) formation
control of unmanned aerial vehicles (UAVs) is suggested here. The situation that the internalstates of a single UAV can be made fully available was aimed at, the internal optimal control lawof a single UAV is designed using the optimal control theory. To cope with the obstacle that eachagent in the system can only communicate with its neighbor agents, the distributed formation
control law of the system is introduced based on the communication topology of the system, andfurther the stability of the system is analyzed in helps of graph theory. The validity of thesuggested scheme is verified by both the numerical simulation and UAV platform.
© 2022 The Author. Published by Sugisaka Masanori at ALife Robotics Corporation Ltd.
This is an open access article distributed under the CC BY-NC 4.0 license(http://creativecommons.org/licenses/by-nc/4.0/).

1. Introduction

system topology. And this result is applied to the formation
control of multiple unmanned aerial vehicles (UAVs),
which is shown in Fig.1.
The main contribution of this paper mainly includes
three aspects: 1) A formation control protocol based on
static consensus protocol is designed. 2) The optimal
control law of a single agent is studied while theperformance function is optimal. 3) Combining theformation control protocol and the optimal control law, a
distributed optimization formation control protocol isdesigned. The protocol combined the distributed
advantage of formation control and the optimization
advantage of optimal control. The system is able toperform formation tasks even if some agents cannot
communicate with each other.

As the capabilities of robots continue to be improved, theapplication fields of robots are also expanding at the sametime. However, just like humans, a single robot will show
lower abilities in many cases, and in this way, thecoordination of multiple agents is required to play a moresignificant role. The ability of distributed coordination andcooperation between agents is the foundation of the multiagent system (MAS).
The problem of multi-agent coordinated control
includes consensus control [1], rendezvous control [2],formation control [3], etc. Besides, there is optimization
control [4] based on the above-mentioned coordinatedcontrol. Optimal control is an important bridge fromtheory to engineering. However, the optimization of a
MAS has a strong dependence on the systemcommunication network. Because the optimal control law
needs to be able to obtain all the states of the collective
individuals in the system in real-time, otherwise theconditions of optimal control cannot be met. To deal withthis problem, we designed a distributed optimization
formation control protocol, which is not influenced by theCorresponding author’s E-mail: daifz@tust.edu.cn, URL: www.tust.edu.cn

374

Fig.1. The illustration of UAV kinematics

For the sake of easy computation, system (2) istransformed into the state-space format, listed in equation(3).
𝑋̇ = AX + BU
(3)

2. Preliminaries
This section introduces the preliminary knowledge forstudying UAV systems, including the use of graph theory
to describe the communication relationships within thesystem, the dynamic model of a single UAV, and the statespace equation of the UAV system.

where 𝑋 =[𝑥 𝑦 𝑧 𝑥̇ 𝑦̇ 𝑧̇ 𝑔𝜃 −𝑔𝜙 0 𝑔𝜃̇ −𝑔𝜙̇ 0]T ,𝑈 = [𝑢𝜙 𝑢𝜃 0]T ,0 1 0 000 0 1 00
𝐴=[] ⊗ 𝐼3 , 𝐵 = [ ] ⊗ 𝐼3 ,0 0 0 100 0 0 01⊗ is the Kronecker product.
When there have N UAVs, the state-space equation (3)can transform into the following form.
̃𝑋̃̇ = 𝐼𝑁 ⊗ 𝐴 𝑋̃ + 𝐼𝑁 ⊗ 𝐵𝑈
(4)

2.1. Graph theory
In order to describe the relationship of UAV systems,
graph theory is used [1]. 𝒜 = [𝑎𝑖𝑗 ∈ {0,1}] is theadjacency matrix of the graph 𝒢, indicating whether there
is information exchange from node 𝑗 to 𝑖 or not. Thematrix 𝒟 is the in-degree matrix. The neighbors of node
𝑖 are 𝑁𝑖 . The Laplacian matrix is defined as
L = 𝒟–𝒜(1)

̃ = [𝑈1T , 𝑈2T , … , 𝑈𝑁T ]T , 𝐼𝑁
where 𝑋̃ = [𝑋1T , 𝑋2T , … , 𝑋𝑁T ]T , 𝑈
represent an 𝑁-dimensional identity matrix.
The desired formation state ℎ is defined as followsℎ𝑥ℎ = [ℎ 𝑦 ] ∈ ℝ3
(5)
ℎ𝑧

2.2. UAV model description
As shown in Fig.1, the UAV model can be simplified asequation (2) [5].
𝑥̈ = 𝑔𝜃𝑦̈ = −𝑔𝜙𝑧̈ = 𝑢ℎ /𝑚 − 𝑔𝜙̈ = 𝑢𝜙 /𝐼𝑥(2)̈𝜃 = 𝑢𝜃 /𝐼𝑦𝜓̈ = 𝑢𝜙 /𝐼𝑧
where 𝑥, 𝑦, 𝑧 are positions in the ground coordinate
system along 𝑋𝑔 , 𝑌𝑔 , 𝑍𝑔 , respectively, 𝜙, 𝜃, 𝜓 are rollangle, pitch angle, yaw angle of the UAV, respectively.
𝐼𝑥 , 𝐼𝑦 , 𝐼𝑧 are inertia moments along 𝑋𝑏 , 𝑌𝑏 , 𝑍𝑏 in the bodycoordinate system, respectively. And 𝑢ℎ is the lift fromfour propellers.

Take the formation states (5) into the position states, three
new error position states come naturally as (6).
𝑥 − ℎ𝑥𝛿𝑥𝑦[𝛿 ] = [𝑦 − ℎ 𝑦 ](6)𝛿𝑧𝑧 − ℎ𝑧
Then the formation control problem is turned intõ to drive the error vector 𝛿 to zero,finding a protocol 𝑈
which indicates thatlim ‖𝛿𝑖 − 𝛿𝑗 ‖ = 0, lim ‖𝑣𝑖 ‖ = 0,𝑡→∞𝑡→∞
lim ‖Ω𝑖 ‖ = 0,lim ‖Ω̇𝑖 ‖ = 0 , 𝑖 = 1,2, ⋯ , 𝑁. (7)𝑡→∞375𝑡→∞

3.

Main results

This section starts with the design of the formation control
protocol. After that, the optimal control law of a single
UAV is provided in detail. Finally, the optimal control law
is extended to the formation control protocol in collection.
Lemma 1 [6]. For a 𝑁 × 𝑁 Laplacian matrix 𝐿 ,−𝐿𝑡e , ∀𝑡 ≥ 0 is a random matrix with positive diagonalelements. If 𝐿 has a unique zero eigenvalue, Rank(𝐿) =𝑁 − 1 , then its left eigenvector has 𝑣 =[𝑣1 𝑣2 ⋯ 𝑣𝑁 ]T ≥ 0 and 1T𝑁 𝑣 = 1, 𝐿T 𝑣 = 0 , where𝑡 → ∞, e−𝐿𝑡 → 1𝑁 𝑣 T .

(a) The necessary conditions for formation control

3.1. Formation control
Referring to the previous work, the consensus protocols
can be divided into two types: One is static and the otherone is dynamic. Based on the static consensus protocol, thefollowing formation control protocol is employed here as(8)𝑢𝑖 = 𝛼 ∑𝑗∈𝑁𝑖 (𝛿𝑗 − 𝛿𝑖 ) − 𝛽𝑣𝑖 − 𝛾1 Ω𝑖 − 𝛾2 Ω̇𝑖 (8)(b) The necessary conditions for optimal control
Fig.2. System communication topology conditions

Through the optimal control theory, the optimal control
law of single agent is𝑢𝑖∗ = −𝑅 −1 𝐵𝑇 𝑃𝑋̅
(10)

where 𝛼, 𝛽, 𝛾1 and 𝛾2 all of them are positive gains.
T𝑦𝛿𝑖 = [𝛿𝑖𝑥 , 𝛿𝑖 , 𝛿𝑖𝑧 ] , 𝑣𝑖 = [𝑥̇ 𝑖 , 𝑦̇ 𝑖 , 𝑧̇𝑖 ]T ,TΩ𝑖 = [𝑔𝜃𝑖 , −𝑔𝜙𝑖 , 0]T , Ω̇𝑖 = [𝑔𝜃̇𝑖 , −𝑔𝜙̇𝑖 , 0] .

where 𝑃 is the solution of the Riccati equation (11).
𝐴T 𝑃 + 𝑃𝐴 − 𝑃𝐵𝑅 −1 𝐵T 𝑃 + 𝑄 = 0(11)

Theorem 1. Assume 𝐺 is the connected undirected
graph. The system (4) can realize the formation as definedin (7) if protocol (8) satisfies the following conditions:
𝛼 > 0, 𝛽 > 0, 𝛾1 > 0, 𝛾2 > 0, 𝛽 ≫ 𝛼,𝛾1 , 𝛾2 > 𝛽 ≫ 𝛼, 𝛽𝛾1 𝛾2 > 𝛽 2 + 𝛾22 𝛼
Proof. Please referring to Reference [5].
□3.3. Distributed optimization formation control
Through the above calculation, the optimal control law(10) can be obtained. Let 𝐾 = 𝑅 −1 𝐵𝑇 𝑃, the dimension of𝐾 must be 3 × 12, and it also has the following form
𝐾 = [𝑘1 𝑘2 𝑘3 𝑘4 ] ⊗ 𝐼3 =𝑘1 0 0 𝑘2 0 0 𝑘3 0 0 𝑘4 0 0 T[ 0 𝑘1 0 0 𝑘2 0 0 𝑘3 0 0 𝑘4 0 ]0 0 𝑘1 0 0 𝑘2 0 0 𝑘3 0 0 𝑘4

3.2. Optimal control

The formation control just requires the communication
The multi-agent system of UAV is isomorphic, that is,
topology is strongly connected, as shown in Fig.2(a). As
the dynamic performance of all agents is the same, so theshown in Fig.2(b), the optimal control needs each agent tooptimal control law can be directly applied to the UAV
be able to communicate with all agents in the system,system. Therefore, the optimal formation control law iswhich is often not satisfied in the multi-agent system.
obtained as follows
Fortunately, the state of each agent itself is fully available.
𝑢𝑖 = 𝑘1 ∑𝑗∈𝑁𝑖 (𝛿𝑗 − 𝛿𝑖 ) − 𝑘2 𝑣𝑖 − 𝑘3 Ω𝑖 − 𝑘4 Ω̇𝑖 (12)Therefore, the optimal control law in a single agent can bestudied to determine the optimal control law of the system.
where 𝑘1 , 𝑘2 , 𝑘3 , 𝑘4 come from matrix 𝐾.
Before giving the performance function, let 𝑋̅ =T𝑥𝑦𝑧[𝛿𝛿𝛿𝑥̇ 𝑦̇ 𝑧̇ 𝑔𝜃 −𝑔𝜙 0 𝑔𝜃̇ −𝑔𝜙̇ 0], then the performance function is defined as∞
𝐽𝑖 = ∫0 [𝑋̅𝑖T (𝑡)𝑄𝑋̅𝑖 (𝑡) + 𝑢𝑖T (𝑡)𝑅𝑢𝑖 (𝑡)] 𝑑𝑡(9)Since the states of UAV on different coordinate axes areindependent, we need to set weight matrices 𝑄 = 𝑞 ∗𝐼12 , 𝑅 = 𝑟 ∗ 𝐼3 , where 𝑞 > 0, 𝑟 > 0.

376

Theorem 2. Assume 𝐺 is connected undirected graph.
The UAV system (4) can complete the formation while
making the performance function (9) optimal if it uses theprotocol (12).
Proof. Please referring to Reference [7].
□4. Simulations
In this section, numerical simulation and virtual platform
simulation were carried out to verify the effectiveness ofthe designed formation control protocol.
4.1. Experiment 1
The numerical experiments have verified the formation
control and distributed optimization formation control
respectively, as shown in Fig.3 and Fig.4.

Fig.4. System states based on distributed optimization formation
control

4.2. Experiment 2
The simulation software CoppeliaSim/Vrep was also used
to verify the flight of the UAV formation. To keep thesystem in view during the operation, we assume that a
UAV remains stationary. Observing the position of thesystem at different time instants, as shown in Fig.5, it canbe seen that the system can finally complete the triangularformation. The full version of the experiment video can beseen athttps://www.bilibili.com/video/BV1Br4y1D7VJ/.

Fig.3. System states formation based on formation control

In the formation control, it can be observed that thesystem can complete the triangular formation, but there isa large error from the set position during the completion of
the formation, and there is still an error at the 5th second.
When using distributed optimal formation control, it canbe observed that the system can quickly complete thetriangular formation, and the error between the actual
position and the set value is small.
In addition, it is surprising that the optimal control not
only reduces the loss of the system but also accelerates theconvergence speed of the system, which is of great help in
reducing the time it takes for the system to form aformation.

(a) t = 0s(b) t = 1s(c) t = 3s(d) t = 5s
Fig.5. The position of three UAVs at different time instants

In this paper, the protocol is only verified in a simulatedenvironment. The biggest challenge between thesimulation environment and the real environment is thatthe real environment has many kinds of disturbances. Our
next step is to improve the robustness and interference
immunity of the protocol.

377

5. Conclusions
Authors Introduction

In this paper, a multi-UAV system is established byanalyzing the dynamic model of UAV. Based on the staticconsensus protocol, the formation control protocol isdesigned. An optimal control law in a single UAV isdesigned for solving the problem that UAV in the systemcannot obtain the status information of all other UAVs. A
distributed optimization formation control protocol for
multi-UAV system is designed is designed by combining
the formation control protocol and the optimal control law.
The protocol is not interfered with by the communication
topology of the multi-agent system. Finally, thecoeffective of the protocol is verified by the numericalsimulation and virtual platform.
In the future, we plan to study the optimal control ofheterogeneous system.

Mr. Jichao Zhao
He is a third-year master candidate in
Tianjin University of Science andTechnology. His research area isabout multi-agent system, controlengineering.

Dr. Fengzhi Dai
He received an M.E. and Doctor ofEngineering (PhD) from the Beijing
Institute of Technology, China in
1998 and Oita University, Japan in
2004 respectively. His main researchinterests are artificial intelligence,pattern recognition and robotics. He
worked in National Institute ofTechnology, Matsue College, Japanfrom 2003 to 2009. Since October 2009, he has been thestaff in College of Electronic Information and Automation,Tianjin University of Science and Technology, China.

Acknowledgements
This work was supported in part by the National Natural
Science Foundation of China (No. 61340041,61374079
and 61903126) and the Natural Science Foundation ofHenan Province (182300410112).

Dr. Yunzhong Song

References

He is affiliated with the School ofElectricalEngineeringandAutomation, Henan Polytechnic
University as a full professor. He
received a PhD degree from theDepartment of Systems Science,
Zhejiang University, P. R. China in
2006. His research interest covers
multi-agent systems analysis andcontrol, complex networks as well as nonlinear systems
analysis and control. Up to now, he has published about
100 papers in peer referred journals and conference
proceedings. To be specific, the work initiated by Professor
SONG such as consensus of multi-agent systems withhybrid order dynamics, and social role based multi-agent
systems flocking as well as hybrid-mode multi-agent
systems regulation have been recognized.

1. J. Zhao, F. Dai, Y. Song, Analysis of the consensus protocolof heterogeneous agents with time-delays, in: 2021
Proceedings of International Conference on Artificial Lifeand Robotics (ICAROB), 2021, pp. 592-595.
2. Song Y, Zhao W. Multi-agent system rendezvous via
refined social system and individual roles. Wseas
transactions on systems and control, 2014, 9(1): 526-532.
3. Wang J, Han L, Dong X, et al. Distributed sliding mode
control for time-varying formation tracking of multi-UAV
system with a dynamic leader. Aerospace Science andTechnology, 2021, 111: 106549.
4. Ren Y, Wang Q, Duan Z. Optimal Distributed Leaderfollowing Consensus of Linear Multi-agent Systems: ADynamic Average Consensus-based Approach. IEEETransactions on Circuits and Systems II: Express Briefs,2021.
5. J. Zhao, F. Dai, Y. Song, Consensus of Heterogeneous
Mixed-Order Multi-agent Systems Including UGV andUAV, Proceedings of 2021 Chinese Intelligent Systems
Conference 805 (2022) 202–210.
6. Zhu J, Tian Y P, Kuang J. On the general consensus protocolof multi-agent systems with double-integrator dynamics.
Linear Algebra and its Applications, 2009, 431(5-7): 701715.
7. J. Zhao, F. Dai, Y. Song, A Distributed Optimal Formation
Control for Multi-Agent System of UAVs, in: 2022
Proceedings of International Conference on Artificial Lifeand Robotics (ICAROB), 2022, pp. 803-807.

378