# Cooperative Evolution Mechanism of Unmanned Swarm within the Framework of Public Goods Game

Authors: Minggang Yu; Ming He; Ziyu Ma; Mingguang Zou; Lei Wan; Kai Kang
Journal: Mathematical Problems in Engineering
 (2021)

---

## Abstract

One of the key advantages of unmanned swarm operation is its autonomous cooperation. When the communication is interrupted or the centralized control manner is lost, the cooperative operation can still be carried out orderly. This work proposed a cooperative evolution mechanism within the framework of multiplayer public goods game to solve the problem of autonomous collaboration of unmanned swarm in case of failure of centralized control. It starts with the requirement analysis of autonomous cooperation in unmanned swarm, and then, the evolutionary game model of multiplayer public goods based on aspiration-driven dynamics is established. On this basis, the average abundance function is constructed by theoretical derivation, and furthermore, the influence of cost, multiplication factor, and aspiration level on the average abundance is simulated. Finally, the evolutionary mechanism of parameter adjustment in swarm cooperation is revealed via case study, and deliberate proposals are suggested to provide a meaningful exploration in the actual control of unmanned swarm cooperation.


---

## Body

## 1. Introduction

With the continuous advancing of the third wave of artificial intelligence, “group evolutionary intelligence” developed from “single-agent autonomous intelligence” has become one of the important characteristics of the new generation of artificial intelligence. Particularly in the military field, unmanned swarm (unmanned vehicle cluster [1], unmanned boat cluster [2], and unmanned aerial vehicle cluster [3]) operations have received unprecedented attention over the past two years. The US military has listed unmanned swarm operations as a “subversive technology” that can change the rules of war.There are mainly two kinds of control modes of unmanned swarm: centralized control and autonomous collaboration. Under the premise of good communication, the Command and Control (C2) center can implement centralized control on the swarm. However, in the complex electromagnetic environment of the battlefield, there is a real risk of communication failure [4]. In such a predicament, the centralized control mode fails, and the unmanned swarm must make effective response on the spot according to the external situation and achieve self-management and self-coordination. An issue that has led to considerable interest is how unmanned swarms autonomously and cooperatively complete established military operations. The sketch of autonomous cooperation of unmanned swarm is shown in Figure 1.Figure 1
Sketch map of autonomous cooperation of unmanned swarm.Overall planning and reallocation of operation resources (communication, firepower, intelligence, etc.) within the unmanned swarms is required when autonomous collaboration occurs. However, in the process mentioned above, there are often contradictions between individual partiality and swarm needs, which are difficult to reconcile. For example, in the fire strike task, the “rational” unmanned units with intelligence and decision-making ability will choose to “contribute” ammunition to the swarm as little as possible in order to maintain its combat effectiveness, while on the other hand, the more ammunition each unit contributes to the swarm, the higher the survival rate and the greater the combat effectiveness of the whole swarm will be. The contradiction between the two will lead to “tragedy of the commons” [5]; therefore, how to increase the number of units’ willing to positively contribute ammunition to the swarm and avoid the tragedy has become a crucial and urgent problem in both technology research and practical application of unmanned swarm.


## 2. Related Work

Evolutionary game theory [6–9] combines “equilibrium” in economics with “adaptability” in biology to depict the process that individuals adapt to the external environment through learning, imitation, and trial-and-error under boundary rationality and asymmetric information. And the evolutionary game of public goods (PGG) [10] provides a basic theoretical framework for revealing the cooperative evolution mechanism and coping with the tragedy of the commons. PGG reflects that investors (collaborators) and hitchhikers (betrayers) play strategic games over time based on cost, multiplication factor, selection intensity, etc., which makes the proportion of collaborators and betrayers in the population change dynamically, and finally tends to an evolutionarily stable state (ESS). The research focus of PGG is to calculate the mathematical expectation of the proportion of collaborators in a population after multiround game, that is, the average abundance, and then analyze the relationship between average abundance and parameters (cost, multiplication factor, selection intensity, etc.) to achieve the ultimate purpose of manual control.At present, there are two main research directions to solve the problem of swarm cooperation with evolutionary game theory: one is to study the evolutionary dynamics process and cooperation mechanism of spatial structured population such as complex network based on graph theory [11, 12], and the other is to study the evolutionary stability state of well-mixed population and the dominant condition of cooperation based on Markov stochastic process [13, 14].For the former, the team of Professor Nowak from Harvard University theoretically deduced the evolution of population in spatial structure such as circle, random graph, and scale-free network and creatively proposed the relationship between the ratiob/c and the network average degree k. They pointed out that the smaller the network connectivity is, the more conducive the cooperation in natural selection is [15]. Then, they used the pair approximation theory to theoretically deduce the cooperation phenomenon on the regular lattice and obtained the boundary conditions for the generation and expansion of cooperation [16]. On the basis of the above achievements, further comparative analysis was made on the differences between homogeneous and heterogeneous networks in promoting cooperative behavior, and simulation results show that weak connection can better promote cooperative behavior on heterogeneous networks [17]. At the same time, other researchers studied the dynamic process of multiparty game on the graph, and simulation results show that spatial structured population can promote the occurrence of cooperation better compared with unstructured population. In the recent two years, the team of Nowak has applied the evolution dynamics of cooperation in spatial structure to social network, analyzed the critical conditions of cooperation behavior in human society [18], initially explored the trade-off between the evolution convergence probability and the evolution convergence time [19], and extended the cooperative evolution on structural population to weighted graph [20]. Other representative studies include literature [21, 22] on the specific model of multiplayer snowdrift game, the relationship curves between the ratio b/c and cooperation level on the well-mixed population and the structured population, respectively, and the significant differences between the homogeneous/heterogeneous network and the unstructured population in the promotion of cooperation are compared.For the latter, the representative researches are as follows: Wang at Wuhan University obtained the average abundance function of the snowdrift evolutionary game using the stable distribution of Markov chain and simulated the effect of parameters on the average abundance [23]. Based on the research work of Tarnita et al. [24], Du at Peking University obtained the inequality of strategic dominance conditions in two-party evolutionary game through strict mathematical derivation; besides, the simulation results show that the average abundance under weak selection intensity is independent of aspiration level [25].The aforementioned work on swarm cooperation is of great theoretical and engineering value. We have also conducted an exploration on the cooperation mechanism of unmanned swarm, and the relevant results can be referred to in [26–30]. Nevertheless, there are still two shortcomings in the above achievements in solving the issue of cooperative evolution of unmanned swarms: first, it has not focused on the public goods game; that is, although there are similarities between the snowdrift game and public goods game [31], there are essential differences in the game mechanism; in addition, the cooperative evolution of unmanned swarm is the multiple interaction of combat units; that is, the evolution result is not only related to the strategy selection of single unit, but also depends on the strategy of other units in the swarm, which is characterized by multiplayer games [32, 33]. So far, the academic community has mastered the payoff matrix [34] of the public goods game with multiplayer and has simulated the influence of different selection intensity [35–37] and threshold values [31] on the cooperation level. In particular, in literature [34], the authors derived a general average abundance formula of multiparty games in a finite population under aspiration-driven dynamics, which can be applicable to any multiparty game under the aspiration-driven dynamics of a finite population. However, the average abundance formula for specific public goods game is not mentioned, so the important work of this study is to obtain the average abundance analytic expression of public goods game based on the existing payment matrix.Furthermore, the generalized evolutionary game model can be simplified as Markov chain + strategy update rules in finite population, so the average abundance function is also closely related to the strategy update rules. By studying strategy update rules in the framework of evolutionary game theory, one can differentiate between imitation processes and aspiration-driven dynamics [38]. In the former case, individuals imitate the strategy of a more successful peer [39]. In the latter case, individuals adjust their strategies based on a comparison between their own payoff and the value they aspire, called the level of aspiration [40]. Unlike the imitation processes of pairwise comparison, aspiration-driven updates do not require additional information about the strategic environment and can thus be interpreted as being more spontaneous [41, 42]. In the complex battlefield environment, the information acquisition is incomplete, asymmetric, which requires the swarm to achieve self-management and self-coordination, and this requirement just coincides with the aspiration-driven dynamics. Moreover, the existing results show that, in both prisoner’s dilemma game and public goods game, the dynamic mechanism driven by aspiration can improve the average abundance value and promote cooperation more than the traditional imitation dynamics [43, 44].Aiming at the cooperative evolution mechanism of the unmanned swarm, we modelled the evolution process based on multiplayer public goods game framework and aspiration-driven update rule and then deduced the average abundance function of the model by analyzing the stable distribution of the Markov chain; on this foundation, we studied the influence of relevant parameters on the average abundance through theoretical analysis and numerical calculation; finally, we studied the effect of parameter adjusting on swarm cooperation via case study and discussed the corresponding solutions and advice to avoid “the tragedy of the commons.”


## 3. Model Hypothesis

In essence, the autonomous collaboration of unmanned swarms is a game process of multiparty and multiround, which focuses on the autonomous allocation of public resources. Therefore, we use multiplayer public goods evolutionary game to model the cooperative evolution of unmanned swarms. The mapping between the concepts of cooperative evolution in unmanned swarms and multiplayer public goods evolutionary game is listed in Table1.Table 1
Mapping between the concepts of cooperative evolution and multiplayer public goods game.
Cooperative evolution in unmanned swarmsMultiplayer public goods evolutionary gameUnmanned swarmsMixed homogeneous populationPublic operational resources (communication, firepower, etc.)Public goodsMultiple unmanned units participating in autonomous collaborationMultiplayerSingle combat unitIndividualsSingle combat unit as research objectFocal individualInvestors (collaborators)Strategy AHitchhikers (betrayers)Strategy BTotal resources that individuals ultimately obtain after each round of gamePayoffsPayoff-based strategy transformation between unitsGameDynamic change of proportion of units with different strategies in multiround gameEvolutionaryThe proportion is stable and the game ends after multiround gameEvolutionary stable state


### 3.1. Framework of Multiplayer Public Goods Evolutionary Game

It is set that the autonomous cooperation of unmanned units takes place in a well-mixed swarm of sizeN, and every unit has two alternative strategies, A and B. Every d units interact simultaneously to get their payoffs; i.e., they are in a two-strategy and d-player game. The strategy update procedure is as follows:(i)
Select any focal individualX in the population of N and select d−1 individuals from the remaining N−1 individuals to form a group of dd<N. A focal individual can be of type A or B, and encounters a group containing k0≤k≤d−1 other players of type A and d−k−1 players of type B.(ii)
The focal individualX plays games with the rest d−1 individuals in strategy space A,B. Denote by ak and bk the payoffs of strategy A and B a player obtains, respectively, when facing i other A individuals within the group of size d, where i ranges from 0 to d−1.(iii)
At the end of each round of the game, the focal individualX evaluates the benefits under different strategy choices and then updates its strategy according to aspiration-driven dynamics. The above process is repeated until the proportion of a certain strategy tends to be stable in the whole population. Obviously, the value of k determines the payoffs—ak and bk. All possible payoffs of a focal individual are uniquely defined by the number of A in the group, and the payoff matrix is as follows.For any group engaging in a one-shot game, we can obtain each member’s payoff according to Table2. When X chooses strategy A, the total contribution by individuals to the swarm is kc+c, the total gain of the swarm is rkc+c multiplied by the profit coefficient r, and the gain of each individual is rkc+c/d. As the cost of X is c, the net gain of X is rkc+c/d−c. When X chooses strategy B, the total contribution by individuals to the swarm is kc, the total gain of the swarm is rkc, and the gain of each individual is rkc/d. As there is no cost in such a case, the net gain of X is rkc/d. Thus, the payoffs for A and B are (1)ak=rkc+cd−c,(2)bk=rkcd.Table 2
Payoff in multiplayer evolutionary game.
d−1...k...0Aad−1...ak...a0Bbd−1...bk...b0


### 3.2. Expected Payoff for Strategies A and B

In a finite well-mixed population of sizeN, groups of size d are assembled randomly, so the probability of choosing a group that consists of another k players of type A and d−k−1 players of type B is given by a hypergeometric distribution [45]. For example, the probability that an A player is in a group of k other A’s is given by PAN,i;d,k=Ci−1kCN−id−k−1/CN−1d−1, where i is the number of A players in the population. The symbol Cnk denotes a combinatorial notation, which is the number of ways to choose a k element subset from an n element set.The expected payoffs for any A or B in a population of sizeN, with i players of type A and N−i players of type B, are given by (3)πAi=∑k=0d−1Ci−1kCN−id−k−1CN−1d−1ak,(4)πBi=∑k=0d−1CikCN−i−1d−k−1CN−1d−1bk.


### 3.3. Aspiration-Driven Dynamics

There are several typical strategy update rules in evolutionary game, such as unconditional imitation [35], Fermi rule [46, 47], etc. Aspiration-driven dynamics focuses on comparing the payoff with aspiration level to make new decisions. Players need not see any particular payoffs but their own, which they compare with an aspiration value. The aspiration-driven dynamics coincides with the requirement of self-management and self-coordination of unmanned swarm in the case of incomplete information acquisition in complex battlefield. The level of aspiration, α, is a variable that influences the stochastic strategy updating. The probability of switching strategy is random when individuals’ payoffs are close to the level of α, reflecting the basic degree of uncertainty in the population. When payoff exceeds α, strategy switching is unlikely. At high values of α compared with payoff, switching probabilities are high.To model stochastic aspiration-driven switching (from strategy A to B), we can use the following probability function:(5)PA⟶B=11+eωπAi−α.The aspiration level,α, provides the benchmark used to evaluate how “greedy” an individual is. Higher aspiration levels mean that individuals aspire to higher payoffs. The intensity of selection, ω, provides a measure of how important individuals deem the impact of the actual game on their update. Let Δ=πA−α; if Δ=0, then PA⟶B=1/2, which means that individuals have the same preference for strategies A and B; if Δ>0 (i.e., the individual payoff πA is higher than aspiration level α), then PA⟶B<1/2, which means individuals prefer strategy A; if Δ<0 (i.e., individual payoff πA is lower than aspiration level α), then PA⟶B>1/2, which means individuals prefer strategy B. As for whether an individual updates strategy or keeps strategy unchanged in a certain round of game, it can be further determined by other algorithms, such as roulette algorithm.In the same way, the probability of the focal individual updating from strategy B to A is(6)PB⟶A=11+eωπBi−α.In the aspiration-driven dynamics, at each time step, the number of strategy A, i.e.,i, can only increase by one, decrease by one, or stay the same. When the number of strategy A increases by one, two subsequent events happen: first, a B strategy individual is selected from the population; then it does not satisfy with the payoff it obtains and switches to the strategy A. A similar process holds for the number of strategy B. Therefore, the probability that the number of A individuals changes at one time step is(7)Pi⟶i−1=Ti−=iN11+eωπAi−α,(8)Pi⟶i+1=Ti+=N−iN11+eωπBi−α,(9)Pi⟶i=1−Ti−−Ti+.Because there is a stable distribution in the Markov chain without absorbing state, the average abundance function of multiplayer evolutionary game can be derived based on the above state transformation equation.


## 3.1. Framework of Multiplayer Public Goods Evolutionary Game

It is set that the autonomous cooperation of unmanned units takes place in a well-mixed swarm of sizeN, and every unit has two alternative strategies, A and B. Every d units interact simultaneously to get their payoffs; i.e., they are in a two-strategy and d-player game. The strategy update procedure is as follows:(i)
Select any focal individualX in the population of N and select d−1 individuals from the remaining N−1 individuals to form a group of dd<N. A focal individual can be of type A or B, and encounters a group containing k0≤k≤d−1 other players of type A and d−k−1 players of type B.(ii)
The focal individualX plays games with the rest d−1 individuals in strategy space A,B. Denote by ak and bk the payoffs of strategy A and B a player obtains, respectively, when facing i other A individuals within the group of size d, where i ranges from 0 to d−1.(iii)
At the end of each round of the game, the focal individualX evaluates the benefits under different strategy choices and then updates its strategy according to aspiration-driven dynamics. The above process is repeated until the proportion of a certain strategy tends to be stable in the whole population. Obviously, the value of k determines the payoffs—ak and bk. All possible payoffs of a focal individual are uniquely defined by the number of A in the group, and the payoff matrix is as follows.For any group engaging in a one-shot game, we can obtain each member’s payoff according to Table2. When X chooses strategy A, the total contribution by individuals to the swarm is kc+c, the total gain of the swarm is rkc+c multiplied by the profit coefficient r, and the gain of each individual is rkc+c/d. As the cost of X is c, the net gain of X is rkc+c/d−c. When X chooses strategy B, the total contribution by individuals to the swarm is kc, the total gain of the swarm is rkc, and the gain of each individual is rkc/d. As there is no cost in such a case, the net gain of X is rkc/d. Thus, the payoffs for A and B are (1)ak=rkc+cd−c,(2)bk=rkcd.Table 2
Payoff in multiplayer evolutionary game.
d−1...k...0Aad−1...ak...a0Bbd−1...bk...b0


## 3.2. Expected Payoff for Strategies A and B

In a finite well-mixed population of sizeN, groups of size d are assembled randomly, so the probability of choosing a group that consists of another k players of type A and d−k−1 players of type B is given by a hypergeometric distribution [45]. For example, the probability that an A player is in a group of k other A’s is given by PAN,i;d,k=Ci−1kCN−id−k−1/CN−1d−1, where i is the number of A players in the population. The symbol Cnk denotes a combinatorial notation, which is the number of ways to choose a k element subset from an n element set.The expected payoffs for any A or B in a population of sizeN, with i players of type A and N−i players of type B, are given by (3)πAi=∑k=0d−1Ci−1kCN−id−k−1CN−1d−1ak,(4)πBi=∑k=0d−1CikCN−i−1d−k−1CN−1d−1bk.


## 3.3. Aspiration-Driven Dynamics

There are several typical strategy update rules in evolutionary game, such as unconditional imitation [35], Fermi rule [46, 47], etc. Aspiration-driven dynamics focuses on comparing the payoff with aspiration level to make new decisions. Players need not see any particular payoffs but their own, which they compare with an aspiration value. The aspiration-driven dynamics coincides with the requirement of self-management and self-coordination of unmanned swarm in the case of incomplete information acquisition in complex battlefield. The level of aspiration, α, is a variable that influences the stochastic strategy updating. The probability of switching strategy is random when individuals’ payoffs are close to the level of α, reflecting the basic degree of uncertainty in the population. When payoff exceeds α, strategy switching is unlikely. At high values of α compared with payoff, switching probabilities are high.To model stochastic aspiration-driven switching (from strategy A to B), we can use the following probability function:(5)PA⟶B=11+eωπAi−α.The aspiration level,α, provides the benchmark used to evaluate how “greedy” an individual is. Higher aspiration levels mean that individuals aspire to higher payoffs. The intensity of selection, ω, provides a measure of how important individuals deem the impact of the actual game on their update. Let Δ=πA−α; if Δ=0, then PA⟶B=1/2, which means that individuals have the same preference for strategies A and B; if Δ>0 (i.e., the individual payoff πA is higher than aspiration level α), then PA⟶B<1/2, which means individuals prefer strategy A; if Δ<0 (i.e., individual payoff πA is lower than aspiration level α), then PA⟶B>1/2, which means individuals prefer strategy B. As for whether an individual updates strategy or keeps strategy unchanged in a certain round of game, it can be further determined by other algorithms, such as roulette algorithm.In the same way, the probability of the focal individual updating from strategy B to A is(6)PB⟶A=11+eωπBi−α.In the aspiration-driven dynamics, at each time step, the number of strategy A, i.e.,i, can only increase by one, decrease by one, or stay the same. When the number of strategy A increases by one, two subsequent events happen: first, a B strategy individual is selected from the population; then it does not satisfy with the payoff it obtains and switches to the strategy A. A similar process holds for the number of strategy B. Therefore, the probability that the number of A individuals changes at one time step is(7)Pi⟶i−1=Ti−=iN11+eωπAi−α,(8)Pi⟶i+1=Ti+=N−iN11+eωπBi−α,(9)Pi⟶i=1−Ti−−Ti+.Because there is a stable distribution in the Markov chain without absorbing state, the average abundance function of multiplayer evolutionary game can be derived based on the above state transformation equation.


## 4. Average Abundance Function

At present, most of the research on average abundance is based on digital simulation, but no strict mathematical expression is given. In this part, we first give the definition of the average abundance of unmanned swarm and then derive its mathematical expression by analyzing the stable distribution of the nonabsorbent Markov chain to support the subsequent simulation analysis in Section 5.


### 4.1. Average Abundance

Definition 1.
(average abundance of unmanned swarm). Set the proportionj/N of unmanned units with strategy A in a swarm as a random variable. Let νj be the probability distribution of j/N; then the expected value of j/N is defined as the average abundance of unmanned units with strategy A.
Therefore, the definition of average abundanceXAj can be expressed as (10)XAj=∑j=0NjNvj.
The key to calculating the average abundance is to determine the probability distributionνj. For Markov chains without absorbing state, νj is just the stable distribution φjj∈0,N, and it satisfies the detailed balance condition [48]: φjTj+=φj+1Tj+1−.
Equation (10) is just a definition formula, which cannot be directly applied to the actual calculation and analysis. Next, we will theoretically deduce the average abundance formula based on the detailed balance condition so as to reveal the quantitative relationship between the average abundance and related parameters (cost, multiplication factor, selection intensity, etc.) and provide a theoretical calculation basis for the subsequent characteristic analysis.


### 4.2. Function Deduction

It can be derived from the detailed balance condition:(11)φ1=T0+T1−φ0,(12)φ2=T1+T2−φ1=T0+T1+T1−T2−φ0,(13)φ3=T2+T3−φ2=T0+T1+T2+T1−T2−T3−φ0.Further, we induce and summarize the above formulas; then we get(14)φj=∏i=0j−1Ti+∏i=1jTi−φ0=∏i=0j−1hiφ0j≥1,where hi=Ti+/Ti+1− is the strategy dominant function. If hi>1, that is, the increasing probability of strategy A is greater than the decreasing probability, it means that strategy A is dominant in the swarm; otherwise, strategy B is dominant.Since∑j=0Nφj=1, (15)∑j=0Nφj=φ0+∑j=1N∏i=0j−1hiφ0=1.Then, we haveφ0=1/1+∑j=1N∏i=0j−1hi. Inserting φ0 into formula (14), we have (16)φj=∏i=0j−1hi1+∑j=1N∏i=0j−1hij≥1.Inserting (16) into (10), XAj can be written as (17)XAj=1N∑j=1Nj∏i=0j−1hi1+∑j=1N∏i=0j−1hi=1N∑j=1Nj∏i=0j−1hi1+∑j=1N∏i=0j−1hi,(18)hi=Ti+Ti+1−=N−i1+eωπAi+1−αi+11+eωπBi−α.Equations (17) and (18) are just a general expression for the average abundance of multiplayer evolutionary games under aspiration-driven dynamics, and the specific application depends on ak and bk. Therefore, the combination of equations (1)–(4), (17), and (18) forms the average abundance function of unmanned swarm under the framework of multiplayer public goods evolutionary game.


## 4.1. Average Abundance

Definition 1.
(average abundance of unmanned swarm). Set the proportionj/N of unmanned units with strategy A in a swarm as a random variable. Let νj be the probability distribution of j/N; then the expected value of j/N is defined as the average abundance of unmanned units with strategy A.
Therefore, the definition of average abundanceXAj can be expressed as (10)XAj=∑j=0NjNvj.
The key to calculating the average abundance is to determine the probability distributionνj. For Markov chains without absorbing state, νj is just the stable distribution φjj∈0,N, and it satisfies the detailed balance condition [48]: φjTj+=φj+1Tj+1−.
Equation (10) is just a definition formula, which cannot be directly applied to the actual calculation and analysis. Next, we will theoretically deduce the average abundance formula based on the detailed balance condition so as to reveal the quantitative relationship between the average abundance and related parameters (cost, multiplication factor, selection intensity, etc.) and provide a theoretical calculation basis for the subsequent characteristic analysis.


## 4.2. Function Deduction

It can be derived from the detailed balance condition:(11)φ1=T0+T1−φ0,(12)φ2=T1+T2−φ1=T0+T1+T1−T2−φ0,(13)φ3=T2+T3−φ2=T0+T1+T2+T1−T2−T3−φ0.Further, we induce and summarize the above formulas; then we get(14)φj=∏i=0j−1Ti+∏i=1jTi−φ0=∏i=0j−1hiφ0j≥1,where hi=Ti+/Ti+1− is the strategy dominant function. If hi>1, that is, the increasing probability of strategy A is greater than the decreasing probability, it means that strategy A is dominant in the swarm; otherwise, strategy B is dominant.Since∑j=0Nφj=1, (15)∑j=0Nφj=φ0+∑j=1N∏i=0j−1hiφ0=1.Then, we haveφ0=1/1+∑j=1N∏i=0j−1hi. Inserting φ0 into formula (14), we have (16)φj=∏i=0j−1hi1+∑j=1N∏i=0j−1hij≥1.Inserting (16) into (10), XAj can be written as (17)XAj=1N∑j=1Nj∏i=0j−1hi1+∑j=1N∏i=0j−1hi=1N∑j=1Nj∏i=0j−1hi1+∑j=1N∏i=0j−1hi,(18)hi=Ti+Ti+1−=N−i1+eωπAi+1−αi+11+eωπBi−α.Equations (17) and (18) are just a general expression for the average abundance of multiplayer evolutionary games under aspiration-driven dynamics, and the specific application depends on ak and bk. Therefore, the combination of equations (1)–(4), (17), and (18) forms the average abundance function of unmanned swarm under the framework of multiplayer public goods evolutionary game.


## 5. Evolutionary Game Analysis

On the basis of the average abundance of unmanned swarm obtained above, we will analyze the impact of costc, multiplication factor r, and aspiration level α on it. Set the basic parameters N=100, d=15, c=1, r=1.3, and α=1, and when calculating the impact of one parameter, others remain unchanged. In addition, in order to highlight the different influence degree of parameters on average abundance under different selection intensities, ω=0,5,10,15,20 is selected in each simulation scenario.


### 5.1. Average Abundance With Respect to Cost

It can be easily proved through mathematical induction from equations (1) and (2) that increasing c will increase ak and bk and then increase πAi and πBi, resulting in the decrease of both Ti+ and Ti−. Since hi=Ti+/Ti+1−, in the case of increasing c, the change of hi and XA are difficult to directly determined. Next, we will give a set of numerical solutions to intuitively observe the interaction within a certain range, so as to reveal the influence of c on XA through simulation.Select the intervalc∈0.9,1.8 and draw the average abundance curve of the strategy A as follows.As shown in Figure2, as c increases, XA will monotonically decrease; when ω=0, XA=0.5 (i.e., the proportion of collaborators and betrayers in the swarm is balanced), while ω≠0, XA increases with ω; moreover, with the decrease of ω, the influence degree of c on XA increases: ΔXAω=20≈0.028, while ΔXAω=5≈0.063.Figure 2
Average abundance with respect to cost.


#### 5.1.1. Conclusion 1

The increasing of cost will decrease the average abundance, especially when selection intensity is small.


### 5.2. Average Abundance with Respect to Multiplication Factor

Similarly, in the case ofr increasing, the change of hi and XAcannot be determined only by deduction. Select the interval r∈0.9,1.8 and draw the average abundance curve of the strategy A under different selection intensities as shown in Figure 3.Figure 3
Average abundance with respect to multiplication factor.As the multiplication factor increases,XA will monotonically decrease, which means the phenomenon of “free riding” appears, resulting in the weakening of cooperation and the decline of XA; moreover, with the decrease of ωω≠0, the influence of r on XA increases: ΔXAω=20≈0.014, while ΔXAω=5≈0.047.


#### 5.2.1. Conclusion 2

The increasing of multiplication factor will decrease the average abundance, especially when selection intensity is small.


### 5.3. Average Abundance with respect to Aspiration Level

Select the intervalα∈0.9,1.4 and draw the average abundance curve of the strategy A under different selection intensities as shown in Figure 4.Figure 4
Average abundance with respect to aspiration level.As the aspiration level increases,XA will monotonically increase, which means the rising of aspiration level makes it more difficult for betrayers to satisfy their expectations, and thus more betrayers transfer to cooperators; moreover, with the decrease of ωω≠0, the influence degree of α on XA increases: ΔXAω=20≈0.002, while ΔXAω=5≈0.037.


#### 5.3.1. Conclusion 3

The increasing of aspiration level will increase the average abundance, especially when selection intensity is small.According to the simulation results,c, r, and α have an impact on the curve trend of average abundance. When c and r increase, the average abundance decreases monotonically, while, with the increase of α, the average abundance increases monotonically. The conclusions from the simulation provide a theoretical basis for the regulation of swarms in practical application. Based on the conclusions above, in the following section, a case study is provided to further reveal the cooperative evolution mechanism of the unmanned swarm.


## 5.1. Average Abundance With Respect to Cost

It can be easily proved through mathematical induction from equations (1) and (2) that increasing c will increase ak and bk and then increase πAi and πBi, resulting in the decrease of both Ti+ and Ti−. Since hi=Ti+/Ti+1−, in the case of increasing c, the change of hi and XA are difficult to directly determined. Next, we will give a set of numerical solutions to intuitively observe the interaction within a certain range, so as to reveal the influence of c on XA through simulation.Select the intervalc∈0.9,1.8 and draw the average abundance curve of the strategy A as follows.As shown in Figure2, as c increases, XA will monotonically decrease; when ω=0, XA=0.5 (i.e., the proportion of collaborators and betrayers in the swarm is balanced), while ω≠0, XA increases with ω; moreover, with the decrease of ω, the influence degree of c on XA increases: ΔXAω=20≈0.028, while ΔXAω=5≈0.063.Figure 2
Average abundance with respect to cost.


### 5.1.1. Conclusion 1

The increasing of cost will decrease the average abundance, especially when selection intensity is small.


## 5.1.1. Conclusion 1

The increasing of cost will decrease the average abundance, especially when selection intensity is small.


## 5.2. Average Abundance with Respect to Multiplication Factor

Similarly, in the case ofr increasing, the change of hi and XAcannot be determined only by deduction. Select the interval r∈0.9,1.8 and draw the average abundance curve of the strategy A under different selection intensities as shown in Figure 3.Figure 3
Average abundance with respect to multiplication factor.As the multiplication factor increases,XA will monotonically decrease, which means the phenomenon of “free riding” appears, resulting in the weakening of cooperation and the decline of XA; moreover, with the decrease of ωω≠0, the influence of r on XA increases: ΔXAω=20≈0.014, while ΔXAω=5≈0.047.


### 5.2.1. Conclusion 2

The increasing of multiplication factor will decrease the average abundance, especially when selection intensity is small.


## 5.2.1. Conclusion 2

The increasing of multiplication factor will decrease the average abundance, especially when selection intensity is small.


## 5.3. Average Abundance with respect to Aspiration Level

Select the intervalα∈0.9,1.4 and draw the average abundance curve of the strategy A under different selection intensities as shown in Figure 4.Figure 4
Average abundance with respect to aspiration level.As the aspiration level increases,XA will monotonically increase, which means the rising of aspiration level makes it more difficult for betrayers to satisfy their expectations, and thus more betrayers transfer to cooperators; moreover, with the decrease of ωω≠0, the influence degree of α on XA increases: ΔXAω=20≈0.002, while ΔXAω=5≈0.037.


### 5.3.1. Conclusion 3

The increasing of aspiration level will increase the average abundance, especially when selection intensity is small.According to the simulation results,c, r, and α have an impact on the curve trend of average abundance. When c and r increase, the average abundance decreases monotonically, while, with the increase of α, the average abundance increases monotonically. The conclusions from the simulation provide a theoretical basis for the regulation of swarms in practical application. Based on the conclusions above, in the following section, a case study is provided to further reveal the cooperative evolution mechanism of the unmanned swarm.


## 5.3.1. Conclusion 3

The increasing of aspiration level will increase the average abundance, especially when selection intensity is small.According to the simulation results,c, r, and α have an impact on the curve trend of average abundance. When c and r increase, the average abundance decreases monotonically, while, with the increase of α, the average abundance increases monotonically. The conclusions from the simulation provide a theoretical basis for the regulation of swarms in practical application. Based on the conclusions above, in the following section, a case study is provided to further reveal the cooperative evolution mechanism of the unmanned swarm.


## 6. Case Study

Fire strike is a typical task in unmanned swarm operation. Limited by the ammunition loading/mounting capacity, when the unmanned swarm carries out the fire strike task in case of failure of centralized control mode, the “rational” unmanned units with intelligence and decision-making ability will strictly control the ammunition launching/delivery quantity with “free riding” mentality, while from the perspective of the whole swarm combat effectiveness, we hope that each unit can provide as much ammunition as possible to ensure the overall strike effectiveness on enemy. The key to coping with this contradiction is how to raise the proportion of cooperators in the swarm through self-regulation and self-coordination.Consistent with the above section, setN=100, d=15, c=1, r=1.3, and α=1 and draw the basic curve (see Figure 5). Since XA<0.5, this case is a nondominant case; that is, most units choose strategy B. Therefore, we try to regulate relevant parameters to increase the average abundance of unmanned swarm and promote cooperation.Figure 5
Effects of different factors on average abundance.As in Figure5, reducing the cost or increasing the aspiration level can raise the proportion of cooperators. However, increasing the multiplication factor will cause the average abundance curve to deviate downward from the basic curve, which is because increasing payoff of cooperators and betrayers by the same margin will make the “free riding” situation more serious. Consequently, we try to separate the multiplication factor of cooperators from that of betrayers, only increase the multiplication factor rA of cooperators (the multiplication factor rB of betrayers remains unchanged), and find that the average abundance curve deviates upward from the basic curve.Furthermore, we simulate the average abundance under differentrA (see Figure 6). When rA=2, the average abundance is approximately equal to 0.5, which indicates that the proportion of cooperators and betrayers in the swarm is basically balanced. With the further increase of rA, when rA=2.65, the average abundance will be greater than 0.5 at ω≈10, while when rA=2.9, the average abundance will be greater than 0.5 at ω≈5. Thus, we can reach the following conclusions.(1)
The adjustment onrA can switch the dominant strategy, making the average abundance of strategy A greater than 0.5.(2)
The lower theω is, the more stringent requirement for rA will be, and the higher the ω is, the looser requirement for rA will be: XAω=5,rA=2.9>0.5, while XAω=10,rA=2.65>0.5.Figure 6
Effects of differentrA on average abundance.In order to investigate the regulation sensitivity of different parameters, we simulate the affecting degree of unit variation ofc, α, and rA on the average abundance. We select the simulation results with ω=0,5,15 to be discussed, as shown in Figures 7(a)–7(c), respectively.(1)
Whenω=0, the average abundance is identically equal to 0.5, and thus the parameter regulation loses its effect (see Figure 7(a)).(2)
Whenω≠0 and unit variation of parameters (i.e., Δ) is small (note that the threshold of Δ is related to ω: Δ≈1.70|ω=5, Δ≈1.53|ω=15), the change in value of average abundance caused by adjusting α and c is much greater than adjusting rA (see Figures 7(b) and 7(c)). The regulation of α and c is more sensitive than that of rA.(3)
Whenω≠0 and Δ is large, the regulation effect of rA is much better than that of α and c. And the larger ω is, the more sensitive rA is; i.e., a small ΔrAleads to a large increasing in average abundance: ΔXAω=5,ΔrA=1.95≈0.43, while ΔXAω=20,ΔrA=1.55≈0.48 (see Figures 7(b) and 7(c)).Figure 7
Effects of unit variation in parameters on average abundance. (a)ω=0. (b) ω=5. (c) ω=15.
(a)(b)(c)To improve the average abundance, the ideal measure is to increase the multiplication factor, reduce the cost of cooperators, or both. However, in order to ensure the effectiveness of the operation in the actual battlefield, the cost is difficult to reduce or even increase. Therefore, it is necessary to consider increasing bothrA and c. Figure 8 shows the change of average abundance when rA and c increase at the same time (c increases by 50%, and rA increases by 69% and 73%, respectively). Accordingly, as long as rA increases by more than 73%, not only can the adverse effect of cost increasing on average abundance be offset, but also the cooperation in swarm can be promoted.Figure 8
Effect of increase in cost and multiplication factor on average abundance.Unfortunately, the above regulation can only achieve a limited increase in the average abundance; that is, it cannot make the average abundance greater than 0.5. According to the conclusions from Figures5 and 6, the conversion of dominant strategies (a large increase in average abundance) depends on a large selection intensity ω and a large unit variation ΔrA, and thus we further increase rA under the premise of increasing c by 50% (see Figure 9). According to the results in Figure 9, when rA=2.52 and ω≈15, XA will be greater than 0.5; when rA=2.65 and ω≈5, XA will be greater than 0.5.Figure 9
Strategy alternation by increasing cost and multiplication factor.The increase ofrA means that the hitchhiker will no longer get as much payoff as the cooperator, and the decrease of payoff will directly increase the strategy update probability PB⟶A, so then more units tend to cooperate (more betrayers transfer to cooperators).According to the above simulation results and conclusions, we can consider the following measures from two dimensions of management and technology in the actual control of unmanned swarm cooperation:(1)
Increase the multiplication factor valuerA of cooperators as much as possible. For example, with the help of advanced management means, for each combat unit in the swarm, its investment (i.e., cost c) in previous operations can be accumulated, and those with higher cumulative investment will be given more supplies (e.g., ammunition) or higher supply priority in the follow-up operations.(2)
Minimize the costc for each operation. For example, with the help of advanced technology means, improve the reliability and survivability of combat units or the strike accuracy and damage-power of ammunition.In addition, sincec,r, and α are closely related to specific operation tasks, it is also necessary to discuss specific control measures in combination with operation tasks under the limitation of parameter value range.


## 7. Conclusion

The advantage of unmanned swarm operation lies in its autonomy that it can continually conduct cooperative operation efficiently in case of combat unit damage or communication failure. This work aims at the autonomous collaboration of the unmanned swarm under the failure of centralized control mode and proposes a cooperative evolution mechanism within the framework of multiplayer public goods evolutionary game. We get the average abundance function by theoretical derivation and then simulate the influence of different parameters (i.e.,c, r, and α) on the abundance. The simulation results of unmanned swarm fire attack show that increasing the multiplication factor rA and reducing the cost c can improve the average abundance of cooperators; furthermore, when the unit variation Δ is large, rA not only has a high regulation sensitivity, but also can realize the switching of the dominant strategy. Finally, we suggest some proposals to provide an exploration for the transformation from theory to application.The evolution of cooperation is a fascinating topic that has been studied from different perspectives and theoretical approaches. Our approach by means of multiplayer public goods evolutionary game sheds new light on how to study and analyze the evolution of cooperation in the unmanned swarm. In our work, we assume that the units in the swarm are homogeneous, which indicates a globally consistentα in the process of strategy updating. However, in reality, different units (firepower units, intelligence units, etc.) probably have various requirements for α. Thus, how to get the average abundance and explore the cooperative evolution mechanism when multiple α coexist will be our further work.



---

*Source: 5575815-2021-04-07.xml*