# A Mechanism for Recognizing and Suppressing the Emergent Behavior of UAV Swarm

Authors: Qiang Liu; Ming He; Daqin Xu; Ning Ding; Yong Wang
Journal: Mathematical Problems in Engineering
 (2018)

---

## Abstract

Similar to social animals in nature, UAV swarm is also a complex system that can produce emergent behavior. The emergent behavior of UAV swarm in specific airspace is undoubtedly the act that the defense side does not expect to see; therefore, recognition and suppression of the emergent behavior of UAVs swarm are needed. Based on the analysis of the UAV swarm emergent behavior mechanism, by adoptingf-divergence method, UAV swarm emergent behavior was quantified, and a rapid recognition mechanism of emergent behavior has been established, thus, making preparation for the suppression of the emergent behavior. In the academic circle, for the first time, in accordance with heuristic rules governing the algorithms of UAV swarm suppression, principle of emergent behavior suppression has been proposed, failure judgment model of UAV swarm control under interference conditions has been constructed, the stability of UAV swarm has been analyzed, and the combat command process of UAV swarm based on OODA loop has been put forward. Through the simulation, the comparison of information entropy and f-divergence based emergence measurement method has been made, and f-divergence based method has some advantages for measuring the emergence of UAV swarm. From the analysis and discussion of the inhibitory effect on swarm flocking behavior under different interference intensity and timing, conclusion has been drawn that comprehensive suppression on the premise of correct recognition of flocking behavior is the best strategy fighting against UAV swarm emergent behavior.


---

## Body

## 1. Introduction

A complex system, the individuals within it interacting with one another according to certain rules, at the macro level, exhibits a totally different function which the sum of all individual functions does not possess (such as structure of time, space, and function), known as emergence, resulting in emergent behavior. Whether in nature or in human society, emergent behavior can often be seen in all kinds of complex systems, such as migratory birds, foraging fish stocks, traffic networks, and the Internet. However, some kinds of emergent behavior are not what we expect, such as traffic jam and cyber-attack. We need to understand the laws and mechanisms behind emergent behavior and seek appropriate ways to manage them, so as to achieve the inhibition of harmful emergent behavior.Most of the scholars [1–3] once stated that the emergent behavior is produced by a number of local interactions among individuals. Forrest [4] also made the following definition: “a group of individuals create incidental phenomenon at macro level during interaction process, and the incidental phenomenon is explained by computation results”. The whole process is called emergence computation. It is necessary to recognize emergent behavior in a computation manner before inhibiting harmful emergence. Presently, the computation methods of recognizing emergent behavior are mainly variable based, formal language based, and event based methods [5]. It needs to be emphasized that there is no uniform computation method to measure emergence at present, and it is necessary to make specific analysis on different complex systems. At the same time, some input variables in certain computation methods can be the individual property parameters in other complex systems, such as in the measurement of emergence of complex systems, Angelis, Birdsey, and Szabo et al. [6–8], respectively, put forward a logical operator and Hausdorff distance, the degree of interactivity, and other specific metric variables; these variables should be calculated with the knowledge of the interaction radius between individuals within the system. Through the following analysis and discussion, we can see that these methods do not apply to the specific application environment studied in this article. The emergence measurement method in this paper is to study the emergent behavior from observer’s perspective; based on the data collected by external sensor as original data, a statistical analysis is used to compare the differences in spatial structure of the objects at different time, so as to recognize emergent behavior and create prerequisites for subsequent suppression of emergent behavior.The swarm of UAV is a branch of swarm robotics. Its design is inspired by the living habits of social animals. Such robots have no structure for centralized control. Through local interaction between UAVs and the interaction between UAVs and external environment, the group behavior presents group action like the emergent behavior of flocking birds as designers expected. Therefore, the UAV swarm can also be regarded as a complex system with the characteristics of emergence.Presently, UAV swarm technology has been highly considered by countries around the world. In the military field, the Department of Defense Advanced Research Planning (DARPA), the Strategic Capabilities Office (SCO), Navy, Air Force, and other departments have started the “Gremlins”, “Perdix”, the Low-Cost UAV Swarming Technology (LOCUST), Offensive Swarm-Enabled Tactics (OFFSET), and other projects, each of which complementing and focusing on their functions [9–11]. At the same time, terrorism organizations also began to use UAV swarm technology to carry out terrorism attacks on important targets. Just around the new year, Russian Ministry of Defense claimed that it had successfully resisted terrorist attacks of dozens of unmanned aerial vehicles (UAVs) carried out by terrorists on its naval base in Syria, marking the arrival of the era of UAVs swarm attack [12].As a new technology, UAV swarm is a double-edged sword. Attacker can gain overwhelming advantage due to its low cost and various ways of application, while the defense side needs to disintegrate the swarm from the mechanism perspective and reduce the advantage of large-scale group combat superiority so as to protect the important target of its own. For the defense side, the emergent behavior of UAV swarm is no doubt harmful. It needs to be recognized and suppressed on the basis of the quantification of its emergence.No matter how changeable the application of UAV swarm is, group flight control is indispensable to UAV swarm, and the control of this kind of mass behavior is called flocking control, which is the core method to produce emergent behavior. In recent years, scholars at home and abroad have created many algorithms about flocking control. Inspired by the migration and foraging behavior of the fish, birds, and ants in nature, early in 1987, Reynolds [13] proposed a group behavior model to simulate the swarm movement and explained that all the individuals within the swarm should follow three heuristic rules, namely, cohesion, separation, and alignment. Subsequently, Vicsek, Couzin, Gazi, and other experts put forward their own models on the basis of the research work of Reynolds. Vicsek [14] applied statistical mechanics to the study of the permutation of particle swarm in two-dimensional space and gave a classical swarm motion model description. Couzin et al. [15], with the assumption of individual’s constant speed movement in three-dimensional space, defined three zones affecting individual movement mode, namely, attraction zone, repulsion zone, and orientation zone, successfully achieving the simulation of four kinds of collective behavior. Similar to Couzin’s model, Gazi et al. [16] introduced attractiveness and repellent function in the network of swarm and constructed attraction/repulsion cluster model of swarm in two-dimensional space. In addition to the above classical algorithms, Olfati-Saber [17] also proposed the most typical framework in the field of flocking control at present. According to Lyapunov’s stability theory, Olfati-Saber made the mathematical definition of swarm, and the flocking rules proposed by Reynolds were embedded in these algorithms. It is worth noting that flocking control is a specific application scene of consensus theory, which can guide multiagent system to reach a consensus on a certain state, such as flying in huge flocks like birds. The previous studies on consensus theory mainly focus on the effect of node states on the consensus of multiagent system. However, the effect of interaction states among nodes on the consensus of system is usually ignored, namely, the effect of edge dynamics on multiagent system. Subsequently, some scholars found that edge dynamics played an important role in structural controllability and dynamic evolution predication of complex network [18]. At present, Su’s research team [19, 20] from Shanghai Jiao Tong University has made an initial exploration about this filed. They proposed a new concept named positive edge-consensus and gave the mathematical derivation of sufficient and necessary condition reaching positive edge-consensus. They also pointed out that the control input of positive linear system is allowed to be negative. Therefore, positive edge-consensus lays foundation for establishing effective close-loop control of positive linear system. In the future, edge-consensus methods may be useful for flocking control research.The above algorithms can effectively control group flight of the UAV swarm under ideal conditions, but when confronted with human interference, the above algorithms fail to consider possible destruction of the network topology of the swarm. Thus, from the perspective of defense side, this paper proposed the suppression principle of the emergent behavior of the UAV swarm on the basis of the inherent control mechanism hidden behind the flocking control algorithm and established the failure judgment model of the flocking control of UAV swarm under the condition of interference. At present, the research work on emergent behavior control or inhibition is relatively rare; the main reason is that there is yet no consensus on the definition of the emergence of complex system in academic circle, as there are various ways of measuring and computing the emergence, so we can only find a suitable entry point for specific application areas to study the emergence. In the field of robots swarm research, based on the analysis of the characteristics of the UAV swarm technology, this paper takes the lead in the research on the recognition and suppression of the emergent behavior of the UAV swarm, which lays foundation for the follow-up research of the UAV swarm theory and technology.This article proceeds as follows: Section2 describes UAV swarm system and explains how the emergent behavior of UAV swarm is generated. Section 3 is devoted to constructing f-divergence based emergence measurement method according to swarm motion state information. From the nature of the self-organizing process, Section 4 adopts two measurement indexes to reflect the orderliness or disorderliness of swarm state before and after interference. In Section 5, based on the heuristic rules that flocking control algorithm follows, the suppression principle of the orderly movement of swarm is proposed, the failure judgment model of flocking control of UAV swarm under the condition of interference is constructed, the stability of UAV swarm is analyzed, and OODA loop based operational command and control (C2) process for downing UAV swarm is put forward. In Sections 6 and 7, against the background of antiterrorism combat, a prototype system simulating attack and defense of UAV swarm is established; the advantages and disadvantages of information entropy and f-divergence based emergence measurement method are pointed out with the simulation results generated by the prototype system; the effect of interference intensity and timing on the emergence of swarm is analyzed and discussed; and the empirical conclusions guiding antiterrorism combat are obtained. Finally, we summarize the theories and methods put forward in this paper and make expectation for future research work in Section 8.


## 2. UAV Swarm and Its Emergence Mechanism

### 2.1. UAV Swarm System

UAV swarm is composed of multiple homogeneous or heterogeneous micro or small UAVs, and every swarming UAV has its own payload, such as active or passive radar seeker, optical infrared device, and electronic jammer. They often adopt their own sensors to detect external environment, and share target information through interaction among them. UAV swarm is used to carry out diversified tasks in highly complex and confrontational conditions, such as wide area surveillance, close reconnaissance, or saturation attack. For terrorism organizations, they generally prefer low-cost, consumer-level UAV equipped with GPS navigation devices and improvised explosive devices, achieving the purpose of penetrating and attacking high value targets through the superiority of quantity. Figure1 shows a DJI quadcopter used by IS, adopting small ammunition FPV telecontrol tactics at early stage, and a fixed-wing UAV, attacking the Russian air force base in Syria recently [21, 22].Figure 1
UAV platforms used by terrorist organizations.
(a)
DJI quadcopter (b)
Rough fixed-wing UAV


### 2.2. Emergence of Swarm’s Flocking Behavior

As shown in Figure2, the flocking behavior of swarm is formed through interaction among individuals within it, and these swarming UAVs often exchange control information of flight state with each other, such as respective position, speed, and heading. At the same time, every individual needs to follow the same rules: (1) separation, avoiding collision with its neighboring UAVs; (2) cohesion, keeping tight with its neighboring UAVs; (3) alignment, keeping the same speed with its neighboring UAVs. At the micro level, all swarming UAVs act on their behavior rules. At the macro level, the entire swarm takes on an orderly movement group clustered in a certain space.Figure 2
The mechanism for swarm’s emergent behavior generation.


## 2.1. UAV Swarm System

UAV swarm is composed of multiple homogeneous or heterogeneous micro or small UAVs, and every swarming UAV has its own payload, such as active or passive radar seeker, optical infrared device, and electronic jammer. They often adopt their own sensors to detect external environment, and share target information through interaction among them. UAV swarm is used to carry out diversified tasks in highly complex and confrontational conditions, such as wide area surveillance, close reconnaissance, or saturation attack. For terrorism organizations, they generally prefer low-cost, consumer-level UAV equipped with GPS navigation devices and improvised explosive devices, achieving the purpose of penetrating and attacking high value targets through the superiority of quantity. Figure1 shows a DJI quadcopter used by IS, adopting small ammunition FPV telecontrol tactics at early stage, and a fixed-wing UAV, attacking the Russian air force base in Syria recently [21, 22].Figure 1
UAV platforms used by terrorist organizations.
(a)
DJI quadcopter (b)
Rough fixed-wing UAV


## 2.2. Emergence of Swarm’s Flocking Behavior

As shown in Figure2, the flocking behavior of swarm is formed through interaction among individuals within it, and these swarming UAVs often exchange control information of flight state with each other, such as respective position, speed, and heading. At the same time, every individual needs to follow the same rules: (1) separation, avoiding collision with its neighboring UAVs; (2) cohesion, keeping tight with its neighboring UAVs; (3) alignment, keeping the same speed with its neighboring UAVs. At the micro level, all swarming UAVs act on their behavior rules. At the macro level, the entire swarm takes on an orderly movement group clustered in a certain space.Figure 2
The mechanism for swarm’s emergent behavior generation.


## 3. Emergent Behavior Recognition

Since emergent behavior cannot be divided into individual behavior at the micro level, before recognizing the emergent behavior of complex systems, it is necessary to observe the complex system from a macro perspective. The defense side ought to use its own distributed sensor network to detect, identify, and track adverse UAV swarm in order to obtain the motion state information, such as position, speed, and heading. It is necessary to use these information as the original data from the macro perspective to make the analysis of emergence behavior, to calculate the degree of the emergence of the system at different time, namely, emergence measurement. By setting up the threshold of emergent behavior recognition, it is determined whether the system is in the phase of producing emergence. In the phase, swarming UAVs will communicate with their neighboring UAVs, which open the timing window so that the defense side can take some countermeasures to counter UAV swarm. Therefore, recognizing emergent behavior of swarm creates prerequisites for subsequent suppression.In probability theory,f-divergence is an indicator function of measuring the divergence degree of two probability distributions. For the UAV swarm, f-divergence is used to compare the difference of spatial distributions of swarm at different time, so as to analyze the emergent behavior of swarm at the macro level. Therefore, in the academic circle, we firstly put forward the f-divergence based emergence measurement method for UAV swarm.


### 3.1.f-Divergence

Equation (1) shows the definition formula about emphf-divergence, proposed by Csiszar and Ali [23, 24]:(1)DfPx∣Qx=∫ΩfpxqxqxdμxwhereP(x),Q(x) are two probability distributions over probability space Ω, and they are both absolutely continuous with respect to a reference distribution μ. Accordingly, p(x), q(x) are their probability densities which satisfy dP(x)=p(x)dμ(x) and dQ(x)=q(x)dμ(x).For convenience of calculation, the discrete form off-divergence is often used in engineering applications; namely,(2)DfPx∣Qx=∑i=1Nqifpiqiwhere pi and qi are probability value of probability distribution sets P=p1,p2,…,pN and Q=q1,q2,…,qN over corresponding space Ω, and pi and qi satisfy(3)∑i=1Npi=1,pi≥0∑i=1Nqi=1,qi≥0Referring to [25], we choose Hellinger divergence and Jensen-Shannon divergence as measurement function whose formulas are(4)DfPx∣Qx=1-∑i=1Npiqi(5)DfPx∣Qx=∑i=1Npiln⁡2pipi+qi+qiln⁡2qipi+qi.


### 3.2. Multivariate Kernel Density Estimation

For UAV swarm, the key to applyingf-divergence to emergence measurement is to estimate its spatial distributions p^(x) and q^(x) over probability space at specific time. Without prior information and assumption about observation sample data, we employ nonparametric multivariate kernel density estimation method [26] to estimate p^(x) and q^(x), as shown below:(6)f^hx=1N∑i=1NKx-Xihwhere f^h(x) is probability density function to be estimated, X1,X2,…,XN are samples ofd-dimensional random variables,h is smoothing coefficient,N is sample size, and K(·) is kernel function; namely, K(x-Xi/h)=∏k=1d‍1/2πhexp(-(x-Xik)2/2h2).In the specific engineering application, probability space needs to be discretized, assuming that the probability density value in the discretized cell is constant. In practice, probability space is divided into multiple subregions that will be converted into corresponding probability value. Thus, we can calculate the joint probability distribution of the probability space, such asP=p1,p2,…,pN. Figure 3 shows the aforementioned concrete steps.Figure 3
The schematic diagram of joint probability distribution conversion.
(a)
Discretization (b)
Partition (c)
Calculation


### 3.3. Mechanism of Slide Time Window

During the automatic identification of UAV swarm’s emergent behavior, we need to continuously sense every swarming UAV’s movement state in order to generate observed samples with time stamp. According to the sample data, we can estimate spatial distribution of swarm at different time. As shown in Figure4, Fisch [27] established two sliding time window mechanisms to measure emergence. Because every swarming UAV’s position will change greatly at long observation time series, we choose the second sliding time window mechanism as shown in Figure 4(b) in order to compare the difference of spatial swarm distribution in the context of fixed duration.Figure 4
Two sliding time window mechanisms.
(a) (b)


## 3.1.f-Divergence

Equation (1) shows the definition formula about emphf-divergence, proposed by Csiszar and Ali [23, 24]:(1)DfPx∣Qx=∫ΩfpxqxqxdμxwhereP(x),Q(x) are two probability distributions over probability space Ω, and they are both absolutely continuous with respect to a reference distribution μ. Accordingly, p(x), q(x) are their probability densities which satisfy dP(x)=p(x)dμ(x) and dQ(x)=q(x)dμ(x).For convenience of calculation, the discrete form off-divergence is often used in engineering applications; namely,(2)DfPx∣Qx=∑i=1Nqifpiqiwhere pi and qi are probability value of probability distribution sets P=p1,p2,…,pN and Q=q1,q2,…,qN over corresponding space Ω, and pi and qi satisfy(3)∑i=1Npi=1,pi≥0∑i=1Nqi=1,qi≥0Referring to [25], we choose Hellinger divergence and Jensen-Shannon divergence as measurement function whose formulas are(4)DfPx∣Qx=1-∑i=1Npiqi(5)DfPx∣Qx=∑i=1Npiln⁡2pipi+qi+qiln⁡2qipi+qi.


## 3.2. Multivariate Kernel Density Estimation

For UAV swarm, the key to applyingf-divergence to emergence measurement is to estimate its spatial distributions p^(x) and q^(x) over probability space at specific time. Without prior information and assumption about observation sample data, we employ nonparametric multivariate kernel density estimation method [26] to estimate p^(x) and q^(x), as shown below:(6)f^hx=1N∑i=1NKx-Xihwhere f^h(x) is probability density function to be estimated, X1,X2,…,XN are samples ofd-dimensional random variables,h is smoothing coefficient,N is sample size, and K(·) is kernel function; namely, K(x-Xi/h)=∏k=1d‍1/2πhexp(-(x-Xik)2/2h2).In the specific engineering application, probability space needs to be discretized, assuming that the probability density value in the discretized cell is constant. In practice, probability space is divided into multiple subregions that will be converted into corresponding probability value. Thus, we can calculate the joint probability distribution of the probability space, such asP=p1,p2,…,pN. Figure 3 shows the aforementioned concrete steps.Figure 3
The schematic diagram of joint probability distribution conversion.
(a)
Discretization (b)
Partition (c)
Calculation


## 3.3. Mechanism of Slide Time Window

During the automatic identification of UAV swarm’s emergent behavior, we need to continuously sense every swarming UAV’s movement state in order to generate observed samples with time stamp. According to the sample data, we can estimate spatial distribution of swarm at different time. As shown in Figure4, Fisch [27] established two sliding time window mechanisms to measure emergence. Because every swarming UAV’s position will change greatly at long observation time series, we choose the second sliding time window mechanism as shown in Figure 4(b) in order to compare the difference of spatial swarm distribution in the context of fixed duration.Figure 4
Two sliding time window mechanisms.
(a) (b)


## 4. Orderliness

Haken [28] once defined orderliness, when he put forward synergetics theory, as consisting “of many subsystems, system form self-organizing structure taking on some functions by nonlinear interaction and coordination among subsystems. When the system presents an orderly structure of space, time, or function at the macro level, it will be in the new orderly state.” In order to maintain the morphology of flocking flight, swarming UAVs need to interact with their neighboring UAVs, which can achieve the transition from disorderly space-time structure to orderly one at the macro level. This indicates that UAV swarm possesses the characteristics of orderliness just as Haken said, in addition to emergence as we mentioned above. Generally, order parameters are used to describe the degree of orderliness, and their values are used to measure the difference of individual orderliness. Thus, order parameters can reflect the orderly state changes of UAV swarm before and after suppression of its flocking behavior, so as to assistedly evaluate the effectiveness with which defense sides suppress the emergent behavior of adverse UAV swarm. As shown in (7) and (8), we adopt two order parameters [14, 29], including heading consistency and average relative distance, so as to measure the orderliness of swarm flight.(7)ϕ=1N∑i=1Nvivi(8)davg=1N∑i=1N∑j=1NXi-Xjwhere vi is the ith swarming UAV’s velocity vector, and other symbols have the same meaning as above. Apparently, ϕ∈[0,1]. When ϕ=0, swarm is in a full disorder state. The larger the heading consistency ϕ is, the higher the orderliness of swarm is. When ϕ→1, swarm will achieve a stable and orderly state, and value of average relative distance davg is nearly constant. When the emergent behavior of swarm is suppressed, value of heading consistency ϕ begins to decrease, and the value of average relative distance davg becomes greater.


## 5. Principles for Suppressing Emergent Behavior

### 5.1. Suppression Mechanism

Qu [25, 30] pointed out that environmental stimulus is the extrinsic cause of emergence, while interaction among individuals is the intrinsic cause of emergence. The most obvious technical defect in UAV swarm is its serious dependency on information transmission system, with which stimulus information can spread the entire network of swarm. It is said that communication is the cornerstone of swarm’s emergent behavior. By contrast, the interference with swarm communication network will inevitably lead to the suppression on the emergence of flocking behavior. Most of classical flocking algorithms [13, 17, 31, 32] follow the three heuristic rules proposed by Reynolds, indicated in more detail in Section 2.2. As shown in Figure 5, when communication receiver of certain UAV is jammed, UAV will not receive movement state information from its adjacent UAVs and fail to cooperate with its adjacent UAVs in movement, such as cohesion, separation, and alignment. When the entire communication network of the swarm is fully suppressed, internal navigation control of the swarm will be in disorder, and the swarm will also be disintegrated.Figure 5
The principle diagram of swarm’s emergent behavior suppression.
(a)
Before suppression (b)
After suppression


### 5.2. Failure Judgment Model

Currently, in the research field of multiagent swarm control, the algorithm proposed by Olfati-Saber [17] is the most representative one, from which a series of significant and practical multiagent flocking tracking control algorithms [31, 33, 34] derive in the research field of distributed mobile sensor networks. The algorithm has become a standard flight control model framework of UAV swarm. Therefore, based on this algorithm, a model of flocking control for UAV swarm under interference conditions was put forward, namely, failure judgment model.For simplicity of notation, the UAV swarm is abstracted as a complex network, and every UAV is a node in this network. Communication between UAVs is symbolized asEij. The set of all swarming UAVs is marked as V=V1,V2,…,VN, and the set of edges in the network is marked as E=Eij=Vi,Vj:Vi,Vj∈V. Thus, these jammed UAVs are marked as D and D⊆V. The set of adjacent UAVs of the ith UAV is denoted as Ni=Vj∈V:i,j∈E.The collective flight control algorithm framework presented by Olfati-Saber consists of three control terms; that is,ui=uig+uid+uiγ. Among these terms, uig is called location control term, which controls separation and cohesion of swarming UAVs; uid is called velocity control term, which keeps pace with other swarming UAVs; and uiγ is called feedback guidance term, which guides UAV to target. The concrete expressions of aforementioned algorithm are shown in (9) and (10), and the meaning of every symbol in the equations can be seen in [17].(9)uig=∑j∈NiϕαXj-XiσXj-Xi1+ϵXj-Xi2uid=∑j∈NiaijXvi-vjuiγ=c1Xr-Xi+c2vr-vi(10)X˙i=viv˙i=uiWhen partial swarming UAVs’ communication devices are jammed, they cannot exchange flight control information with other swarming UAVs and would return to the initial waypoint. According to (9), we set up the failure judgment model of flocking control, as shown in (11). (11)ui′=∑j∈Ni/DϕαXj-XiσXj-Xi1+ϵXj-Xi2︸uig+∑j∈Ni/DaijXvi-vj︸uid+uiγwhere ui′ represents the flocking control vector of the ith UAV not jammed. Under this condition, it can only exchange position and speed information with other swarming UAVs not jammed. With the decrease in the number of nodes exchanging information, correspondingly, the possibility of realizing cooperative motion with other swarming UAVs also reduces.


### 5.3. Stability Analysis

For nonlinear dynamic systems, such as UAV swarm, we could adopt LaSalle’s invariance principle [35] to carry out stability analysis of them. According to mathematical expressions of three control terms shown in (9), we make the definition of the total energy of UAV swarm system; namely,(12)Q=12∑i=1NUi+Kiwhere Ui is the ith UAV’s total potential energy, namely, Ui=∑j=1,j≠iN‍Ψα(Xj-Xiσ)+c1(Xi-Xr)T(Xi-Xr), and Ki is relative kinetic energy between the ith swarming UAV and virtual leader, namely, Ki=(vi-vr)T(vi-vr).Then, settingXi~=Xi-Xr, vi~=vi-vr, we substitute Xi~, vi~ into (9), (10), and (12), respectively, resulting in (13), (14), and (15).(13)ui~=-∑j∈Ni∇Xi~ΨαX~ijσ-∑j∈NiaijXvi~-v~j-c1Xi~+c2vi~(14)X~˙i=v~iv~˙i=u~i(15)Q=12∑i=1N∑j=1,j≠iNΨαX~ijσ+c1X~iTX~i+v~iTv~iDue to the symmetry of potential function Ψα and adjacency matrix A, as shown in (16)∂ΨαX~ijσ∂X~ij=∂ΨαX~ijσ∂X~i=-∂ΨαX~ijσ∂X~jwe can obtain partial derivative of total potential energy Q; namely, Q˙=1/2∑i=1N‍(Ui˙+Ki˙)=∑i=1N‍(∑j∈Ni‍∇Xi~Ψα(X~ijσ)+c1v~iTX~i+v~iTv~i˙). According to (13) and (14), we get (17)Q˙=-v~TLt+c2IN⊗INv~where v~=col(v~1,v~2,…,v~N) and IN is identity matrix.BecauseL(t) is a positive semidefinite matrix, L(t)+c2IN is also a positive semidefinite matrix. Thus, Q˙≤0; correspondingly, Q(t) is a nonincreasing function. For any time t≥0, there exists Q(t)≤Q0(Q0=Q(X~(0),v~(0))). Therefore, the energy of UAV swarm system decreases by time, and it is shown that system is stable.At present, due to good robustness and strong adaptability of flying ad hoc networks (FANET) protocol [36], it is often used for network UAV swarm. When partial swarming UAVs are jammed, FANET protocol can reorganize these communication nodes not jammed, so as to maintain communication network of swarm. Simultaneously, wireless communication scheme based on WiFi technology may be applied to UAV swarm, which leads to covering all communication nodes in certain airspace, since maximum communication distance of UAV exceeds 1 kilometer [37]. Therefore, UAVs not jammed still continue to flight in cohesive manner followed by flocking control rules, except that the scale of current swarm will reduce. Then, N shown in (16) is substituted by N′, where N′ is the number of UAV not jammed. As a result, stability of swarm will not be affected.


### 5.4. OODA Loop Based Operational C2 Process

When combating large-scale UAV swarm attacks launched by terrorism organization, how to quickly identify air threats and establish an effective response mechanism are the most important for successfully completing counterterrorism mission. The Observe, Orient, Decide, Act (OODA) loop model proposed by Boyd [38], the US Air Force Colonel, can succinctly describe every stage of C2 process in dynamic and complex environment. The model has the advantages of cyclicity, timeliness, and recursiveness, which is very suitable for describing the C2 process of antiterrorism operations in the context of large-scale UAV swam attacks.Figure6 shows the flow chart of OODA loop based operational C2 process for countering UAV swarm. In the stage of observation, we can use various types of sensors to detect and observe surrounding targets, especially low, slow, and small (LSS) targets, and then collect target information, including their position, state, and attributes. In the stage of orientation, we can fuse all kinds of information and intelligence collected in the previous stage so as to form the current UAV swarm’s threat status, recognize emergent behavior, and make relevant analysis. In the stage of decision, we can formulate action plan according to current situation, such as the determination of optimal communication interference strategy for UAV swarm. In the stage of action, we can take measure to suppress the communication network of the UAV swarm according to the action plan. After cyclic iteration, the suppression of UAV swarm flocking behavior would be finally achieved.Figure 6
OODA loop based operational C2 process.


## 5.1. Suppression Mechanism

Qu [25, 30] pointed out that environmental stimulus is the extrinsic cause of emergence, while interaction among individuals is the intrinsic cause of emergence. The most obvious technical defect in UAV swarm is its serious dependency on information transmission system, with which stimulus information can spread the entire network of swarm. It is said that communication is the cornerstone of swarm’s emergent behavior. By contrast, the interference with swarm communication network will inevitably lead to the suppression on the emergence of flocking behavior. Most of classical flocking algorithms [13, 17, 31, 32] follow the three heuristic rules proposed by Reynolds, indicated in more detail in Section 2.2. As shown in Figure 5, when communication receiver of certain UAV is jammed, UAV will not receive movement state information from its adjacent UAVs and fail to cooperate with its adjacent UAVs in movement, such as cohesion, separation, and alignment. When the entire communication network of the swarm is fully suppressed, internal navigation control of the swarm will be in disorder, and the swarm will also be disintegrated.Figure 5
The principle diagram of swarm’s emergent behavior suppression.
(a)
Before suppression (b)
After suppression


## 5.2. Failure Judgment Model

Currently, in the research field of multiagent swarm control, the algorithm proposed by Olfati-Saber [17] is the most representative one, from which a series of significant and practical multiagent flocking tracking control algorithms [31, 33, 34] derive in the research field of distributed mobile sensor networks. The algorithm has become a standard flight control model framework of UAV swarm. Therefore, based on this algorithm, a model of flocking control for UAV swarm under interference conditions was put forward, namely, failure judgment model.For simplicity of notation, the UAV swarm is abstracted as a complex network, and every UAV is a node in this network. Communication between UAVs is symbolized asEij. The set of all swarming UAVs is marked as V=V1,V2,…,VN, and the set of edges in the network is marked as E=Eij=Vi,Vj:Vi,Vj∈V. Thus, these jammed UAVs are marked as D and D⊆V. The set of adjacent UAVs of the ith UAV is denoted as Ni=Vj∈V:i,j∈E.The collective flight control algorithm framework presented by Olfati-Saber consists of three control terms; that is,ui=uig+uid+uiγ. Among these terms, uig is called location control term, which controls separation and cohesion of swarming UAVs; uid is called velocity control term, which keeps pace with other swarming UAVs; and uiγ is called feedback guidance term, which guides UAV to target. The concrete expressions of aforementioned algorithm are shown in (9) and (10), and the meaning of every symbol in the equations can be seen in [17].(9)uig=∑j∈NiϕαXj-XiσXj-Xi1+ϵXj-Xi2uid=∑j∈NiaijXvi-vjuiγ=c1Xr-Xi+c2vr-vi(10)X˙i=viv˙i=uiWhen partial swarming UAVs’ communication devices are jammed, they cannot exchange flight control information with other swarming UAVs and would return to the initial waypoint. According to (9), we set up the failure judgment model of flocking control, as shown in (11). (11)ui′=∑j∈Ni/DϕαXj-XiσXj-Xi1+ϵXj-Xi2︸uig+∑j∈Ni/DaijXvi-vj︸uid+uiγwhere ui′ represents the flocking control vector of the ith UAV not jammed. Under this condition, it can only exchange position and speed information with other swarming UAVs not jammed. With the decrease in the number of nodes exchanging information, correspondingly, the possibility of realizing cooperative motion with other swarming UAVs also reduces.


## 5.3. Stability Analysis

For nonlinear dynamic systems, such as UAV swarm, we could adopt LaSalle’s invariance principle [35] to carry out stability analysis of them. According to mathematical expressions of three control terms shown in (9), we make the definition of the total energy of UAV swarm system; namely,(12)Q=12∑i=1NUi+Kiwhere Ui is the ith UAV’s total potential energy, namely, Ui=∑j=1,j≠iN‍Ψα(Xj-Xiσ)+c1(Xi-Xr)T(Xi-Xr), and Ki is relative kinetic energy between the ith swarming UAV and virtual leader, namely, Ki=(vi-vr)T(vi-vr).Then, settingXi~=Xi-Xr, vi~=vi-vr, we substitute Xi~, vi~ into (9), (10), and (12), respectively, resulting in (13), (14), and (15).(13)ui~=-∑j∈Ni∇Xi~ΨαX~ijσ-∑j∈NiaijXvi~-v~j-c1Xi~+c2vi~(14)X~˙i=v~iv~˙i=u~i(15)Q=12∑i=1N∑j=1,j≠iNΨαX~ijσ+c1X~iTX~i+v~iTv~iDue to the symmetry of potential function Ψα and adjacency matrix A, as shown in (16)∂ΨαX~ijσ∂X~ij=∂ΨαX~ijσ∂X~i=-∂ΨαX~ijσ∂X~jwe can obtain partial derivative of total potential energy Q; namely, Q˙=1/2∑i=1N‍(Ui˙+Ki˙)=∑i=1N‍(∑j∈Ni‍∇Xi~Ψα(X~ijσ)+c1v~iTX~i+v~iTv~i˙). According to (13) and (14), we get (17)Q˙=-v~TLt+c2IN⊗INv~where v~=col(v~1,v~2,…,v~N) and IN is identity matrix.BecauseL(t) is a positive semidefinite matrix, L(t)+c2IN is also a positive semidefinite matrix. Thus, Q˙≤0; correspondingly, Q(t) is a nonincreasing function. For any time t≥0, there exists Q(t)≤Q0(Q0=Q(X~(0),v~(0))). Therefore, the energy of UAV swarm system decreases by time, and it is shown that system is stable.At present, due to good robustness and strong adaptability of flying ad hoc networks (FANET) protocol [36], it is often used for network UAV swarm. When partial swarming UAVs are jammed, FANET protocol can reorganize these communication nodes not jammed, so as to maintain communication network of swarm. Simultaneously, wireless communication scheme based on WiFi technology may be applied to UAV swarm, which leads to covering all communication nodes in certain airspace, since maximum communication distance of UAV exceeds 1 kilometer [37]. Therefore, UAVs not jammed still continue to flight in cohesive manner followed by flocking control rules, except that the scale of current swarm will reduce. Then, N shown in (16) is substituted by N′, where N′ is the number of UAV not jammed. As a result, stability of swarm will not be affected.


## 5.4. OODA Loop Based Operational C2 Process

When combating large-scale UAV swarm attacks launched by terrorism organization, how to quickly identify air threats and establish an effective response mechanism are the most important for successfully completing counterterrorism mission. The Observe, Orient, Decide, Act (OODA) loop model proposed by Boyd [38], the US Air Force Colonel, can succinctly describe every stage of C2 process in dynamic and complex environment. The model has the advantages of cyclicity, timeliness, and recursiveness, which is very suitable for describing the C2 process of antiterrorism operations in the context of large-scale UAV swam attacks.Figure6 shows the flow chart of OODA loop based operational C2 process for countering UAV swarm. In the stage of observation, we can use various types of sensors to detect and observe surrounding targets, especially low, slow, and small (LSS) targets, and then collect target information, including their position, state, and attributes. In the stage of orientation, we can fuse all kinds of information and intelligence collected in the previous stage so as to form the current UAV swarm’s threat status, recognize emergent behavior, and make relevant analysis. In the stage of decision, we can formulate action plan according to current situation, such as the determination of optimal communication interference strategy for UAV swarm. In the stage of action, we can take measure to suppress the communication network of the UAV swarm according to the action plan. After cyclic iteration, the suppression of UAV swarm flocking behavior would be finally achieved.Figure 6
OODA loop based operational C2 process.


## 6. Simulation Experiments

By applying the C++ language, we constructed the UAV swarm attack and defense prototype system and built the simulation model of the UAV, the swarm flock control, the communication, and the communication interference. In order to dynamically display the suppression process of UAV swarm’s emergent behavior, we set a scene as shown in Figure7. The terrorists Group A attempted to destroy Country B’s important military infrastructure. (Red denotes Group A, while Blue denotes Country B). The Red had got exact position of the Blue’s important military infrastructure in advance through its intelligence network. Dozens of Red UAVs took off at a distance of 100 km away from the targets and formed a swarm of UAV in specific airspace to implement saturated attacks on Blue’s targets. Blue relied on some sensors deployed nearby its own highly protected targets to monitor peripheral airspace. Once ill-disposed UAVs found, Blue would immediately take countermeasures against them.Figure 7
The scenario of antiterrorism.Initial simulation scene was set as shown in Figure7, Red UAVs were randomly deployed in the airspace located at (34∘42′52′′N, 86∘43′32′′W). The flight speed of Red UAVs was 50 m/s, and flight height was 1000 m. Meanwhile, Blue’s important military targets were located at (34∘28′33′′N, 86∘35′32′′W), and Blue’s communication jammers were deployed at (34∘28′29′′N, 86∘36′51′′W). Blue’s low altitude blind compensation radars were deployed in the front position to monitor the surrounding airspace. Other simulation parameters were set as follows, r=110m, d=100m, a=b=5, ϵ=0.1, h=0.6, c1=0.02, c2=0.2, and dslide=7.5s. In order to compare it with aforementioned f-divergence based emergence measurement method, we also introduced information entropy based method into simulation experiments, so as to validate the effectiveness and performance of these methods. At present, information entropy based method mainly includes discrete entropy difference (DED) method proposed by Minf [39] and continuous entropy difference (CED) method by Cheng [40]. The essence of two methods is that degree of emergence is calculated by entropy difference. Generally, the former method calculates entropy value by discrete estimation, such as histogram method, while the latter method calculates entropy value by precise continuous estimation, like kernel density estimation. Readers interested in the two methods may refer to [39, 40]. It is noteworthy that DED measurement method requires measuring the emergence of every attribute for multiattribute systems. As shown in Figure 8, DED-X and DED-Y represent measurement of emergence of UAV swarm in 2-dimensional space.Figure 8
The measurement curves of different scale swarms’ emergence.
(a)
N = 10 (b)
N = 50 (c)
N = 100As can be seen from Figure8, without communication interference, the values of Hellinger and Jensen-Shannon divergence obviously increase after t=20s. This means the swarm’s emergent behavior is forming under the condition of local interactions among UAVs. Moreover, the swarm spatially presents a similar form of birds’ gathering flight. When the UAV swarm continued the flight for a period of time (after t=30s), the values of Hellinger and Jensen-Shannon divergence would reach the maximum and tend to be stable. It is shown that spatial distributions of swarm at different time keep invariant. In this case, swarm forms a stable and orderly flight formation. Figure 9 showed the spatial distributions of swarm at different time. We can see the whole transition process from disorderly to orderly state of swarm. The result agrees with the simulation results in Figure 8.Figure 9
The contrast diagram of swarm’s emergent behavior generation.
(a)
t=7s (b)
t=30sAs shown in Figure8, DED and CED methods could not effectively recognize the emergent behavior of swarm. The fundamental reason is that the modes of emergence measured by information entropy and f-divergence based emergence measurement method are different. Information entropy based method is inclined to measure emergence from the perspective of orderliness. However, orderliness is not absolutely equated with emergence. This view is still in dispute in the academic circle, and some scholars hold that orderliness of complex system will appear along with emergence phenomenons [1]. It is a remarkable fact that the measurement of entropy of multiattribute system is closely related to characteristics of practical system. Then, the comprehensive assessment of degree of emergence of multiattribute system could not be obtained only by simple methods, such as weighted average method. This is also limitation of DED method. For f-divergence based method, it is inclined to measure emergence from the perspective of statistical distributions, and degree of emergence is calculated by the difference of distributions of observed samples. This method is not affected by the multiattribute problem; therefore, it is more natural in measuring the emergence of UAV swarm.Obviously, compared with information entropy based method,f-divergence based method performs well on the aspect of measurement of UAV swarm’s emergence. However, we still empirically set the threshold of the “warming point” to judge the emergence of swarm’s flocking behavior, such as eHel=0.15, eJS=0.2. Only in this way, we can find the best time window to counter UAV swarm.Due to different weapon and equipment performance of the attack and defense sides, practical defense effect will also vary. To achieve desired suppression effect on swarm communication network, we need to complete the space alignment, frequency alignment, and energy alignment. To simplify complex model constructing work, we use a probabilistic model to simulate actual communication countermeasure effect. Figure10 shows the suppression effect of swarm emergent behavior when the rate of jamming is 20%, 50%, or 75%, respectively. Because UAV swarm is a highly redundant self-organizing combat network, some UAVs not jammed could still interact with each other and then achieve spatial cohesion and cooperative flight in an orderly manner, as shown in the black circles in Figure 10. Meanwhile, other UAVs would return to the initial waypoint due to the long time communication jamming, and then they become isolated combat units, as shown in the black squares in Figure 10.Figure 10
The effect diagram of swarm’s emergent behavior suppression in condition of different jamming intensity.
(a)
P j a m = 0.2 (b)
P j a m = 0.5 (c)
P j a m = 0.75


## 7. Discussion

It can be seen from the above simulation experiments that thef-divergence based emergence measurement method provides a judgment mechanism for the defense side to automatically recognize swarm’s emergent behavior. This also provides the defense side opportunity to take electronic countermeasures against the swarm. Next, we will explore the effects of early and hysteretic suppression on the UAV swarm’s emergent behavior.


### 7.1. Early Suppression Effect

In fact, in the launching phase of swarming UAV, the swarm has not yet formed, and most of swarming UAVs are always in disorderly state. If the defense side can find out the situation at this stage, they could manage to launch an electronic attack on adverse swarm in advance to kill emergent behavior in the cradle. For this case, we also carried out corresponding simulation experiments. The starting time of communication jammingtjam was set as 5 seconds. As shown in Figure 11, Hellinger divergence value basically kept steady. Heading consistency value was around 0.063, while the value of average relative distance davg became greater and greater. This case showed that the swarm was always in disorderly state. It should be noted that although the jamming effect is theoretically better in this way, considering that the swarming UAVs are more spatially dispersed at this stage, we need to dispatch more jamming resources to achieve desired jamming effect. Compared with the aforementioned jamming strategy, taking communication jamming measures after identification of swarm’s emergent behavior will receive a better effect practically.Figure 11
The effect of early suppression on the emergent behavior.
(a)
f-divergence curve (b)
Heading consistency curve (c)
Average relative distance curve (d)
2D simulation scene graph


### 7.2. Hysteretic Suppression Effect

If the defense side takes countermeasures relatively late, the swarm has been in a relatively stable flight formation, and we assume that every swarming UAV still fly to the mission area even though they are jammed, what is the consequence in this condition? Theoretically, every swarming UAV’s heading, speed, and relative position shall be basically consistent after the swarm reaches steady state, and swarming UAVs no longer need to exchange more motion state information. At this time, it is futile to apply electronic suppression to the swarm’s emergent behavior. As shown in Figure12, the flight formation of swarm has not been disturbed. However, from the perspective of information fusion, the jammed swarming UAVs cannot transmit the information acquired by its own sensors to its neighboring UAVs, which lead to the failure of establishing information sharing mechanism. Consequently, the swarming UAVs would take “go-it-alone” approach and fail to implement saturated attacks on important targets. We also noticed that this jamming strategy is meaningful to certain extent, but this goes beyond the scope of this article. In the future, we will continue to investigate the effect of local interaction on the evolutionary dynamics of self-organizing systems.Figure 12
The effect of hysteretic suppression on the emergent behavior.
(a)
f-divergence curve (b)
Heading consistency curve (c)
2D simulation scene graphIn summary, it is necessary to apply appropriate jamming strategy in face of a large number of UAV swarm attacks. Either too early or hysteretic countermeasure is not an ideal scheme. Furthermore, the measurement method based onf-divergence can utilize the motion state information of UAV swarm to recognize emergent behavior, which lays foundation for establishing a rapid antiswarm response mechanism, opening optimal coping time window. Considering the highly redundant self-organizing combating network of UAV swarm, the function failure of single UAV can cause damage to the complex system, but it would not destroy the function of the whole system instantly; that is, the emergence of UAV swarm would not disappear immediately. We should use more electronic interference assets to suppress the UAV swarm. Only in this way, UAVs within the swarm can be isolated, and the operational superiority of swarm’s collective attack can also be reduced to some extent.


## 7.1. Early Suppression Effect

In fact, in the launching phase of swarming UAV, the swarm has not yet formed, and most of swarming UAVs are always in disorderly state. If the defense side can find out the situation at this stage, they could manage to launch an electronic attack on adverse swarm in advance to kill emergent behavior in the cradle. For this case, we also carried out corresponding simulation experiments. The starting time of communication jammingtjam was set as 5 seconds. As shown in Figure 11, Hellinger divergence value basically kept steady. Heading consistency value was around 0.063, while the value of average relative distance davg became greater and greater. This case showed that the swarm was always in disorderly state. It should be noted that although the jamming effect is theoretically better in this way, considering that the swarming UAVs are more spatially dispersed at this stage, we need to dispatch more jamming resources to achieve desired jamming effect. Compared with the aforementioned jamming strategy, taking communication jamming measures after identification of swarm’s emergent behavior will receive a better effect practically.Figure 11
The effect of early suppression on the emergent behavior.
(a)
f-divergence curve (b)
Heading consistency curve (c)
Average relative distance curve (d)
2D simulation scene graph


## 7.2. Hysteretic Suppression Effect

If the defense side takes countermeasures relatively late, the swarm has been in a relatively stable flight formation, and we assume that every swarming UAV still fly to the mission area even though they are jammed, what is the consequence in this condition? Theoretically, every swarming UAV’s heading, speed, and relative position shall be basically consistent after the swarm reaches steady state, and swarming UAVs no longer need to exchange more motion state information. At this time, it is futile to apply electronic suppression to the swarm’s emergent behavior. As shown in Figure12, the flight formation of swarm has not been disturbed. However, from the perspective of information fusion, the jammed swarming UAVs cannot transmit the information acquired by its own sensors to its neighboring UAVs, which lead to the failure of establishing information sharing mechanism. Consequently, the swarming UAVs would take “go-it-alone” approach and fail to implement saturated attacks on important targets. We also noticed that this jamming strategy is meaningful to certain extent, but this goes beyond the scope of this article. In the future, we will continue to investigate the effect of local interaction on the evolutionary dynamics of self-organizing systems.Figure 12
The effect of hysteretic suppression on the emergent behavior.
(a)
f-divergence curve (b)
Heading consistency curve (c)
2D simulation scene graphIn summary, it is necessary to apply appropriate jamming strategy in face of a large number of UAV swarm attacks. Either too early or hysteretic countermeasure is not an ideal scheme. Furthermore, the measurement method based onf-divergence can utilize the motion state information of UAV swarm to recognize emergent behavior, which lays foundation for establishing a rapid antiswarm response mechanism, opening optimal coping time window. Considering the highly redundant self-organizing combating network of UAV swarm, the function failure of single UAV can cause damage to the complex system, but it would not destroy the function of the whole system instantly; that is, the emergence of UAV swarm would not disappear immediately. We should use more electronic interference assets to suppress the UAV swarm. Only in this way, UAVs within the swarm can be isolated, and the operational superiority of swarm’s collective attack can also be reduced to some extent.


## 8. Conclusion

Based on the actual antiterrorism demands, we analyzed the induced mechanism of UAV swarm’s emergent behavior and employedf-divergence based measurement method to recognize the emergent behavior of swarm. According to the heuristic rules followed by common flocking control algorithms, we put forward the suppression principle of swarm collective motion, constructed a failure judgment model of swarm’s flocking control under interference conditions, and analyzed the stability of UAV swarm. Through simulation experiments, we made a comparison of information entropy and f-divergence based emergence measurement method and analyzed and discussed the effect of interference intensity and interference timing on the swarm emergence behavior. It is also pointed out that comprehensive suppression on the emergent behavior of UAV swarm should be based on the premise of correctly recognizing the emergent behavior of UAV swarm, which helps to achieve the optimal counterstrategy. To sum up, we completed the modeling, observation, calculation, prediction, and control of the dynamic emergent behavior of UAV swarm and realized the effective identification and suppression of the emergence of UAV swarm, which laid a theoretical and technical foundation for the antiterrorism combat department to formulate counteracting scheme. In this paper, the defense side discusses the recognition method of emergent behavior based on the comprehensive detection of the motion state information of the UAV swarm with its distributed sensors network, which is essentially an emergence measurement method based on complete information. In the future, we will further discuss the method of emergence measurement under incomplete information, which is closer to reality.



---

*Source: 6734923-2018-09-13.xml*