# Optimal Tag-Based Cooperation Control for the “Prisoner’s Dilemma”

Authors: Rui Dong; Xinghong Jia; Xianjia Wang; Yonggang Chen
Journal: Complexity
 (2020)

---

## Abstract

A long-standing problem in biology, economics, and social sciences is to understand the conditions required for the emergence and maintenance of cooperation in evolving populations. This paper investigates how to promote the evolution of cooperation in the Prisoner’s Dilemma game (PDG). Differing from previous approaches, we not only propose a tag-based control (TBC) mechanism but also look at how the evolution of cooperation by TBC can be successfully promoted. The effect of TBC on the evolutionary process of cooperation shows that it can both reduce the payoff of defectors and inhibit defection; although when the cooperation rate is high, TBC will also reduce the payoff of cooperators unless the identified rate of the TBC is large enough. An optimal timing control (OTC) of switched replicator dynamics is designed to consider the control costs, the cooperation rate at terminal time, and the cooperator’s payoff. The results show that the switching control (SC) between an optimal identified rate control of the TBC and no TBC can properly not only maintain a high cooperation rate but also greatly enhance the payoff of the cooperators. Our results provide valuable insights for some clusters, for example, logistics parks and government, to regard the decision to promote cooperation.


---

## Body

## 1. Introduction

Animal Dispersion in Relation to Social Behavior was published by Wynne-Edwards and looked specifically at the “Evolution of cooperation” [1]. Why should an individual help another person who is a potential competitor in the struggle for survival? This question is listed in “Science” Magazine as one of the 25 core problems of the 125 scientific challenges proposed by scientists from all over the world.The evolution of cooperation is an enduring conundrum in biology, mathematics, and social sciences. Natural selection opposes cooperation unless some mechanisms are at work to promote the evolution of cooperation. The Prisoner’s Dilemma game (PDG), which represents an extreme case, has emerged as one of the most promising mathematical areas in the study of cooperation.In the PDG, two players simultaneously decide whether to cooperateC or to defect D. If one player cooperates, the other player can choose between cooperation which yields R (the reward for mutual cooperation) or defection which yields T (the temptation to defect). On the other hand, if one player defects, the other player can choose between cooperation which yields S (the sucker’s payoff) or defection which yields P (the punishment for mutual defection), where T>R and P>S. The donor-recipient game (DRG) is a special case of the PDG. In the DRG, a cooperator is someone who pays a cost, c, for another individual to receive a benefit, b. A defector has no cost and does not deal out benefits. By comparing the PDG and DRG, we obtain R=b−c, T=b, S=−c, and P=0. If the game is only played once, then each player gets a higher payoff from D than from C, regardless of what the other player does. So, natural selection implies competition and, therefore, opposes cooperation unless a specific mechanism is at work.Based on Hamilton’s rule [2], natural selection can favor cooperation if the donor and the recipient of an altruistic act are genetic relatives [3–7]. The donor may preferentially donate a benefit to the recipient through “kin recognition”. Explanations of cooperation between nonkin include “direct reciprocity” [8–13], “indirect reciprocity” [14–16], “network reciprocity” [17], “group selection” [18], “optional participation mechanism” [19, 20], and “punishment mechanism” [21–23].Direct reciprocity was proposed by Trivers [8] and developed as a mechanism for the evolution of cooperation [9, 24]. In the repeated Prisoner’s Dilemma, the same two individuals repeatedly encounter each other for some rounds. If one cooperates now, the other may cooperate later. Hence, he or she might cooperate [10]. Axelrod and Hamilton proposed a model of the evolution of cooperation based on the iterated Prisoner’s Dilemma [11]. In two computer tournaments, Axelrod discovered that the “winning strategy” was the simplest of all, tit-for-tat (TFT). TFT is a program where one player uses C on the first move of the game and, then, plays whatever the other player chose in the previous move. This simple concept captured the fascination of many enthusiasts of the repeated Prisoner’s Dilemma, and a number of empirical and theoretical studies were inspired by Axelrod’s groundbreaking work [12, 13, 25]. The presence of the decoy increased willingness of volunteers to cooperate in the first step of each game, leading to subsequent propagation of such willingness by (noisy) tit-for-tat [25]. Reference [26] pointed that resilient cooperators could sustain cooperation indefinitely in the finitely repeated Prisoner’s Dilemma. The Prisoner’s Dilemma experiment showed that the termination rules and the (expected) length of the game significantly increased cooperation in [27].Indirect reciprocity does not rely on repeated encounters between the same two individuals. An individual can establish a good reputation by helping someone which can increase the chance of receiving help from others. Indirect reciprocity has substantial cognitive demands. For example, language is needed to gain the information and spread the gossip associated with indirect reciprocity so that only humans seem to engage in the full complexity of the game. Differing from indirect reciprocity based on reputation, kin can be recognized through familiarity based on environmental or learned cues or through pattern matching based on some inherited trait. If individuals display a heritable marker or a tag, they preferentially cooperate with partners who share their own marker. So, tag-based donation does not have the substantial cognitive demands that a reputation mechanism based on indirect reciprocity has. Cooperation can also arise when individuals donate to others who are sufficiently similar to themselves in some arbitrary characteristic. Such a characteristic, or “tag,” for example, a green beard [28, 29], can be observed by others. Hamilton illustrated how this mechanism worked even when individuals were not genealogical kin [2]. This is known as the green beard effect. For tag-based donations, it is not necessarily required to remember past encounters. In the recent years, the green beard effect has become a favorite topic among both evolutionary biologists and sociologists [30–36]. For example, Tian et al. and Tian et al. [30, 31] proposed Vcash as a reputation framework for identifying denial of traffic service to resolve the trustworthiness problem, where the result of verified traffic event notification acted as a “tag”. Taylor and Nowak proposed nonuniform interaction rates (interaction rates dependent on the strategies) that allowed the coexistence of cooperators and defectors in the Prisoner’s Dilemma [37]. The analytical models of evolution of cooperation were given when nonuniform interaction rates were introduced [38]. Both“kin recognition” and “tag-based donation” lead to nonuniform interaction rates.By combining evolutionary processes with differential equations, the links between stability and optimization have been researched and a mathematical framework has been built [39, 40]. In multiagent systems, an individual is regarded as an agent who autonomously regulates his behavior according to his benefit. Agents play games with their neighbors through local interaction. The evolution of cooperation is an adaptive coordinated control process which is a controllable, intelligent, and autonomous decision process. From the perspective of evolutionary games, the following objectives, inter alia, have been achieved through the designing of the control law: optimizing individual cost function [39], designing consensus control of stochastic multiagent systems [41], and promoting the evolution of cooperation in social dilemmas [23, 38].Motivated by the abovementioned discussion, through combining direct reciprocity, indirect reciprocity, tag-based donation, and optimal control theory, a tag-based control (TBC) is proposed. The goal of this paper is to study the feasibility of promoting cooperation by TBC and designing an appropriate identified rate for operators. As part of this, we discuss the different effectiveness and drawbacks between by identifying cooperation and by identifying defection for promoting the evolution of cooperation. Using the optimal control theory, we examine whether an operator should design TBC mechanisms for the evolution of cooperation or give up TBC temporarily for increasing the group’s payoff. We show that an operator can design an optimal timing control (OTC) to optimize the control costs, the cooperation rate, and the cooperator’s payoff.The main contributions of this paper are as follows. (1) For the first time, a TBC that promotes the evolution of cooperation in the Prisoner’s Dilemma is proposed. (2) Advantages and disadvantages of identifying cooperation and identifying defection in TBC are investigated. (3) An OTC is designed to better promote cooperation rates and greatly enhance the group’s payoff. (4) From a research standpoint, this work contributes to evolutionary game theory and optimal control theory.The paper is organized as follows. In Section2, we obtain the payoff matrix and the replicator dynamics of the Prisoner’s Dilemma with TBC. By applying the results of Section 2, we discuss the problem of evolution of cooperation in the Prisoner’s Dilemma with TBC in Section 3. In the next two sections, the optimal TBC is designed. We design the optimal identified rate of promoting cooperation in Section 4. The effectiveness of TBC is illustrated and an OTC is given in Section 5. We provide some concluding remarks and discussion in the last section.


## 2. The Replicator Dynamics of the Prisoner’s Dilemma with TBC

In the Prisoner’s Dilemma game, two individuals can each either cooperateC or defect D. The payoff matrix of the Prisoner’s Dilemma is as follows:(1)where(2)T>R>P>S,(3)2R>T+S.No matter what the other does, the selfish choice of defection yields a higher payoff than cooperation, so a player will choose defection no matter whether his opponent chooses cooperation or defection. However, if both defect, both getP rather than the larger value of R that they both could have gotten if they had both cooperated. In other words, if both defect, both do worse than if both had cooperated. Hence, the game poses a dilemma. In repeated Prisoner’s Dilemma games, if constraint (3) cannot be satisfied, the social efficiency that all players always choose cooperation is less than that they agree to choose cooperation and defection in turn.LetΩ=1,2,…,N be a group of game participants, where N is a sufficiently large natural number. Consider the symmetric static games of complete information between two individuals. The pure strategy set for each individual in group Ω is denoted by Λ′=C,D. For notational convenience, we will label every individual’s pure strategies by positive integers. Hence, the pure-strategy set of each individual is written as Λ=1,2. A vector of pure-strategy profile is denoted as s=s1,s2, where si is a pure strategy for individual i. The pure strategy space is Λ×Λ, where Λ×Λ is the Cartesian product of the pure strategy set Λ. A probability distribution x,y over the pure-strategy set Λ of an individual i is defined as a hybrid strategy for the individual i, where x and y are the probabilities assigned to the individual’s pure strategy C and D, respectively. We denote z=x,y. The hybrid strategy space of each individual is the simplex Δ=z∈R+2:x+y=1, where R+2 is a subset of the two-dimensional vectors with all elements being positive. At any point of time t, let xt and yt be the rates at which individuals choose strategy C and D in group Ω, respectively. Then, the corresponding group state is zt=xt,yt, where xt,yt∈0,1 and xt+yt=1. Therefore, the group state is formally equivalent to the hybrid strategy. Because the state zt is completely determined by its component xt or yt, this paper discusses the variable xt only.In the groupΩ, the individuals who choose strategy D have a higher average fitness than the others. Therefore, selection acts to increase the relative abundance of strategy D. After sometime, cooperation will vanish from the group. So, strategy D is the only evolutionarily stable strategy (ESS) unless a specific mechanism is at work.


### 2.1. The Game under Labeling

This paper studies the special mechanisms for the evolution of cooperation in some clusters in the economic society when faced with the Prisoner’s Dilemma, for example, in logistics parks. For such a large number of individuals from different sources, it is difficult to build mutual trust because of the high cost of fully understanding and accurately knowing each other’s willingness to cooperate. In this case, the mechanisms for promoting the evolution of cooperation, such as direct reciprocity, indirect reciprocity, punishment mechanism, and nonuniform interaction, are difficult to meet. A third party in the cluster, though, can play the role of collecting information and rewarding or punishing enterprise behavior to promote mutual recognition efficiently. Such examples might be park managers, platform promoters, park management committees, industry associations, and so on.In view of this, we assume that there exists a controller (an operator) outside of groupΩ who can identify defection and cooperation. The controller labels “cooperator” or “defector” for everyone. The tags labeled by the controller affect the strategy choice of individuals in the following games until the next identification.Definition 1.
A person or an organization is called a controller if he/she can affect the choice of strategy of individuals through identifying the actions of individuals in past games and reveal the results of identification in groupΩ.
The identification implemented by the controller can be regarded as tagging individuals. Strategy choices of all individuals in groupΩ are decided by their tags in the following games.Definition 2.
One round game is defined as a game set which consists of a game identified by a controller and all games tagged by this identification.
Assume the number of rounds is infinite. In one round, every individual playsm+1 period games, where m is a positive integer. The first period is identified by the controller in one round. All individuals are tagged before the second period game begins. In the following m periods, all individuals know their tags and can distinguish tags of others.Assumption 1.
The individuals are bounded rationality and any individual encounters other individuals randomly with equal probability.
This paper assumes that the individuals (enterprises or persons) are in certain clusters (such as logistics parks) or bilateral platforms in the economic society. The indirect reciprocity has substantial cognitive demands for players. The tag mechanism for the evolution of cooperation demands that individuals own and display a heritable marker or a tag. Due to the large number of individuals in the cluster, the interaction between them is random, and it is difficult to form long-term repeated games. Therefore, it is impossible to establish a direct reciprocity mechanism based on repeated games. In the early stage of the formation of clusters or bilateral platforms, due to the lack of mutual understanding and trust relationship between individuals, the cost of collecting information and displaying individual reputation is very high, so indirect reciprocity and tag mechanism cannot work unless a person or organization helps the players. We define this person or organization as the controller (Definition1). The identification implemented by the controller can be regarded as tagging individuals. In the first period of a round game, the controller identifies individuals and displays the results.
Based on Assumption1, for each individual, the probability of interacting with a cooperator or a defector is xt or 1−xt, respectively.Definition 3.
Define an individual as a cooperator if he/she chooses cooperation without control.
At timet, the rate of cooperators in the group Ω is xt. So, the state xt is defined as the group cooperation rate. The cooperator defined by Definition 3 can be equated with the individual who always chooses cooperation without control based on Assumption 1. Obviously, an individual is a defector if he is not a cooperator under Definition 3.Assumption 2.
Assume that the identification service is not perfect. The probabilities that the cooperators and defectors are correctly identified by the controller areβ1 and β2, respectively, where βi∈0,1,i=1,2.
Based on Assumption2, β1 is called the cooperation identified rate and β2 is called the defection identified rate. We denote the tags of “cooperator” and “defector” as “C′” and “D′“. In one round, Ω is divided into 4 parts noted by Ωii=1,2,3,4. Ω1 and Ω2 are sets of cooperators and defectors labeled C′, respectively. Analogously, Ω3 and Ω4 are sets of cooperators and defectors labeled as D′, respectively. Furthermore, ∪i=14Ωi=ΩΩi∩Ωj=Φ, where Φ is an empty set. By denoting individual i labeled j′ as j′|ii,j=C,D, one can know that C′|C∈Ω1, C′|D∈Ω2, D′|D∈Ω3, and D′|C∈Ω4. According to Assumption 2, pC′|C=β1, pD′|C=1−β1, pD′|D=β2, and pC′|D=1−β2, where p· indicates the probability of event · happening.Assumption 3.
In every round game, cooperators chooseC and defectors choose D in the first period. In the following m periods, the cooperators labeled D′ and all defectors always choose D. Cooperators labeled C′ choose C if he/she encounters an individual labeled C′ and chooses D if he/she encounters an individual labeled D′.Definition 4.
The control of a controller dominating individuals’ strategy choice through tagging is called TBC. The individual identified as the cooperator is tabled asC′, and the individual identified as the defector is tabled as D′ by the controller. The TBC rule is designed as follows: the cooperators labeled D′ and all defectors always choose D. Cooperators labeled C′ choose C if he/she encounters an individual labeled C′ and chooses D if he/she encounters an individual labeled D′.
Based on Assumption3, the controller can dominate an individual’s strategy choice through tagging individuals. At time t, the cooperation rate is xt in group Ω. In the first period, the ratio of C to total strategies is equal to the cooperation rate xt. In the following m periods, an individual’s strategy is decided by tags according to Assumption 3. The ratio of C to total strategies is not the cooperation rate xt but β1xβ1x+1−β21−x. One can know that β1xβ1x+1−β21−x=β1x1−x1−β1−β21−x<x.
Assumption3 is reasonable by Assumption 1 and Definition 1. From the abovementioned discussion, we know that C not only denotes cooperation but also denotes a cooperator and D not only denotes defection but also denotes a defector. Numbers of individuals in Ω1,Ω2,Ω3, and Ω4 are β1xN, 1−β21−xN, β21−xN, and 1−β1xN, respectively. Based on Assumption 2 and 3, in the first period, cooperators choose C and defectors choose D. In the following m periods, two individuals choose C, if and only if they are in Ω1. The strategy pair C,C is contributed. When one player from Ω1 encounters a player from Ω2, a cooperator chooses C and a defector chooses D. In addition to the abovementioned two cases, the strategy pair D,D will be formed. According to Taylor and Nowak [31] and Dong et al. [32], the probabilities of C,C, D,D, and C,D are β12x2, 1−β12x2−2β11−β2x1−x, and β11−β2x1−x with TBC. The larger β1 and β2 means the higher probability of C,C and the lower probability C,D. So, the larger β1 and β2 are in accordance with the better effect of promoting the evolution of cooperation. But, under uniform interaction rates, the corresponding probabilities are x2 and 1−x2, respectively, so an individual’s encounter constrained by TBC no longer abides by uniform interaction rates.
Based on Assumptions2 and 3 and Definition 4, cooperation identified rate β1 and defection identified rate β2 reflect the accuracy of TBC. We denote β′ as the probability of another encounter between the same two individuals in the repeated Prisoner’s Dilemma and β″ as the probability of knowing someone’s reputation in indirect reciprocity. According to [31], the condition that direct reciprocity (indirect reciprocity) can lead to the evolution of cooperation depends on the parameters of payoff matrix (1) and β′β″. But, by Assumption 1, β′ is very small because the group size, N, is large enough. On the other hand, the high cost of fully understanding and accurately knowing each other’s willingness to cooperate leads to that β″ is very small also. So, direct reciprocity and indirect reciprocity cannot promote cooperation in the setting that this paper considers. In every round game, cooperation identified rate β1 and defection identified rate β2 can act as β′ of direct reciprocity or β″ of indirect reciprocity.Assumption 4.
The evolution of group statex is carried out according to the replicator. An individual replication occurs after each round game ends and before the next round begins. An individual’s payoff refers to the average total payoff in this round game.
Assumption4 means the offspring of cooperators are still cooperators and the offspring of defectors are still defectors. So, the offsprings of C or D are disconnected with their parent’s tag. By Assumption 3, defectors always choose D, but the cooperator does not always choose C.
“Tag” does a mappingσ between Ω and C′,D′, i.e., σ:Ω⟶C′,D′. By Assumption 2, pσC=C′=β1, pσC=D′=1−β1, pσD=D′=β2, and pσD=C′=1−β2.
By calculation, the payoff of cooperator (defector) in thepth period of the rth round game is independent of r but determined by the cooperation rate x. Let vpth,Cxvpth,Dx denote the payoff of a cooperator (defector) in the pth period. Let wCxwDx denote total payoff of the cooperator (defector) in one round game; we get(4)wCx=v1th,Cx+∑p=2m+1vpth,C,wDx=v1th,Dx+∑p=2m+1vpth,D.
Letvpth,j′/Cxvpth,j′/Dx denote the payoffs of the cooperator (defector) labeled j′j′=C,D in the pth period game; we get(5)vpth,Cx=pσC=C′vpth,C′|C+pσC=D′vpth,D′|C,vpth,Dx=pσD=C′vpth,C′|D+pσD=D′vpth,D′|D,p=2,3,…,m+1.
By Assumptions1 and 2, the probability of anyone encountering a cooperator labeled C′ is β1x and the probability of anyone encountering a cooperator labeled D′ is 1−β1x; the probability of anyone encountering a defector labeled C′ or D′ is 1−β21−x or β21−x. One can get(6)v1th,Cx=Rx+S1−x,vpth,C′/Cx=β1xR+1−β1x+β21−xP+1−β21−xS,vpth,D′/Cx=P,p=2,3,…,m+1,(7)v1th,Dx=xT+1−xP,vpth,C′/Dx=β1xT+1−β1xP,vpth,D′/Dx=P,p=2,3,…,m+1.
Together, (4)–(7) determine wCx and wDx as follows:(8)wCx=1+mβ12xR+β11−β1−β2x+1−β11−β2mP+n!r!n−r!1+mβ11−β21−xS,wDx=v1,Dx+mβ2vpth,D′/Dx+1−β2vpth,C′/Dx,=1+mβ11−β2xT+1−x+m1−β11−β2xP.
By Assumption3, an individual’s average payoff in the rth round game is xwCx+1−xwDx.
Summing up the abovementioned discussion, the total payoff matrix (1) of a round game under TBC is(9)1+mβ12R+1−β12mP1+mβ11−β2S+1−β11−β2mP1+mβ11−β2T+1+β11−β2mPmP.
In fact, in the latterm periods, the hybrid strategy and the group state are not equivalent because a cooperator constrained by Assumption 3 does not always choose C. So, the payoff aij in payoff matrix (9) does not indicate the payoff of individual i encountering strategy j, where i,j=C,D. However, we still use payoff matrix (9) for the following two reasons: (1) the payoff wix of individual Ω1 is still linear with the group state xt; (2) a cooperator’s average total payoff under TBC is 1+mβ12R+1−β12mP if he enters a group of cooperators or is 1+mβ11−β2S+1−β11−β2mP if he enters a group of defectors. Similarly, a defector’s average total payoff under TBC is 1+mβ11−β2T+1−β11−β2mP if he enters a group of cooperators or is P if he enters a group of defectors.
Ifβ1=1,β2=0, i.e., all individuals are identified as cooperators, the game with payoff matrix (9) is equivalent to a Prisoner’s Dilemma repeated m+1 times with payoff matrix (1). By Assumption 3, cooperators always choose C and defectors always choose D.


### 2.2. The Replicator Dynamics with TBC

At timet, we suppose the number of cooperators is M and the number of defectors is N−M. In the next Δt, under Assumption 4, the cooperators reproduce wCMΔt offspring and the defectors reproduce wDN−MΔt offspring. The total number of death is wCM+wDN−MΔt. The number of cooperators death is wCM+wDN−MxΔt, and the number of defectors death is wCM+wDN−M1−xΔt. Then, Δx=wCMΔt−wCM+wDN−MxΔt/N=wC−wDx1−xΔt. Then, the replicator dynamics under TBC is as follows:(10)x˙t=wCx−wDx1−xx,=R−S−T+Px+S−P1−xx+mβ1R+2−β1−2β2P−1−β2S+Tx−1−β2P−Sβ11−xx.Assumption 5.
The Prisoner’s Dilemma tagged repeats enough periods, i.e.,m≫1.
By Assumption5, the total payoff matrix (9) in a round game approximately equals the following matrix:(11)mβ12R+1−β12mP1+β11−β2mp+mβ11−β2S1+β11−β2mPβ11−β2mTmP.
The replicator dynamics (10) approximates the following dynamics:(12)x˙t=mβ1R+2−β1−2β2P−1−β2S+Tx−1−β2P−Sβ11−xx.Definition 5.
(See [42]). A state x˜,y˜∈Δ is an ESS if for all other states x,y∈Δ either (i) ux,y,x˜,y˜<ux˜,y˜,x˜,y˜ or (ii) ux,y,x˜,y˜=ux˜,y˜,x˜,y˜ and ux,y,x,y<ux˜,y˜,x,y, where u·1,·2 denotes the individual’s payoff with strategy ·1 under the strategy pair ·1,·2.Definition 6.
(See [43]). A fixed point x˜,y˜∈Δ in replicator dynamics is an evolutionary equilibrium (EE) if it is locally asymptotically stable, i.e., every open neighborhood Κ⊂Δ of the point x˜,y˜ has the property that every path starting sufficiently close to x˜,y˜ remains in Κ and converges asymptotically to x˜,y˜.Definition 7.
(See [43]). The largest open set of points whose evolutionary paths converge to a given EE is called its basin of attraction.
In order to simplify the calculation,P=0 is assumed in the following discussion. Doing so would result in a simpler model, without significant qualitative or directional changes to our key results about evolution of cooperation. Obviously, the parameters T,R, and S satisfy T>R>0>S. Based on (11), the payoff matrix for a period under TBC is transformed as follows:(13)
Regardless of time scale,m does not affect either ESS or EE and does not even affect the evolutionary path of a replicator’s dynamics, so we can rewrite the replicator dynamics (12) as follows:(14)x˙t=β1R−1−β2S+Tx+1−β2Sβ11−xx.
Two specific instances are given in the following.Case1. Only the defectors are identifies, and let pD′/D=β2. Ω is divided into two parts. One is the set of defectors tagged D′. The other is the set of individuals without tag. The corresponding part of Assumption 3 is modified as follows: a cooperator chooses C if he encounters an individual without tag but chooses D if he encounters an individual tagged D′. Defectors always choose defect. In this case, the payoff matrix is as follows:(15)Case 2. Only the cooperators are identified, and let pC′/C=β1. Ω is divided into two parts. One is the set of cooperators tagged C′. The other is the set of individuals without a tag. The corresponding part of Assumption 3 is modified as follows: the cooperator chooses C if he encounters an individual with tag C′ and chooses D if he encounters an individual without a tag. Defectors always choose defect. In this case, we obtain the payoff matrix as follows:(16)
By comparing matrices (13) and (15), we see that Case 1 is equivalent to cooperation identified rate β1=1 in matrix (13), i.e., the controller accurately identifies cooperators. It follows that Case 1 is in accordance with control based on identifying defection. Correspondingly, Case 2 means the controller identifies defectors accurately.
From the operator’s perspective, the relevant decisions are (1) whether to design TBC; (2) identifying which individual is more effective, cooperator or defector; and (3) how to design cooperation identified rateβ1 and defection identified rate β2.


## 2.1. The Game under Labeling

This paper studies the special mechanisms for the evolution of cooperation in some clusters in the economic society when faced with the Prisoner’s Dilemma, for example, in logistics parks. For such a large number of individuals from different sources, it is difficult to build mutual trust because of the high cost of fully understanding and accurately knowing each other’s willingness to cooperate. In this case, the mechanisms for promoting the evolution of cooperation, such as direct reciprocity, indirect reciprocity, punishment mechanism, and nonuniform interaction, are difficult to meet. A third party in the cluster, though, can play the role of collecting information and rewarding or punishing enterprise behavior to promote mutual recognition efficiently. Such examples might be park managers, platform promoters, park management committees, industry associations, and so on.In view of this, we assume that there exists a controller (an operator) outside of groupΩ who can identify defection and cooperation. The controller labels “cooperator” or “defector” for everyone. The tags labeled by the controller affect the strategy choice of individuals in the following games until the next identification.Definition 1.
A person or an organization is called a controller if he/she can affect the choice of strategy of individuals through identifying the actions of individuals in past games and reveal the results of identification in groupΩ.
The identification implemented by the controller can be regarded as tagging individuals. Strategy choices of all individuals in groupΩ are decided by their tags in the following games.Definition 2.
One round game is defined as a game set which consists of a game identified by a controller and all games tagged by this identification.
Assume the number of rounds is infinite. In one round, every individual playsm+1 period games, where m is a positive integer. The first period is identified by the controller in one round. All individuals are tagged before the second period game begins. In the following m periods, all individuals know their tags and can distinguish tags of others.Assumption 1.
The individuals are bounded rationality and any individual encounters other individuals randomly with equal probability.
This paper assumes that the individuals (enterprises or persons) are in certain clusters (such as logistics parks) or bilateral platforms in the economic society. The indirect reciprocity has substantial cognitive demands for players. The tag mechanism for the evolution of cooperation demands that individuals own and display a heritable marker or a tag. Due to the large number of individuals in the cluster, the interaction between them is random, and it is difficult to form long-term repeated games. Therefore, it is impossible to establish a direct reciprocity mechanism based on repeated games. In the early stage of the formation of clusters or bilateral platforms, due to the lack of mutual understanding and trust relationship between individuals, the cost of collecting information and displaying individual reputation is very high, so indirect reciprocity and tag mechanism cannot work unless a person or organization helps the players. We define this person or organization as the controller (Definition1). The identification implemented by the controller can be regarded as tagging individuals. In the first period of a round game, the controller identifies individuals and displays the results.
Based on Assumption1, for each individual, the probability of interacting with a cooperator or a defector is xt or 1−xt, respectively.Definition 3.
Define an individual as a cooperator if he/she chooses cooperation without control.
At timet, the rate of cooperators in the group Ω is xt. So, the state xt is defined as the group cooperation rate. The cooperator defined by Definition 3 can be equated with the individual who always chooses cooperation without control based on Assumption 1. Obviously, an individual is a defector if he is not a cooperator under Definition 3.Assumption 2.
Assume that the identification service is not perfect. The probabilities that the cooperators and defectors are correctly identified by the controller areβ1 and β2, respectively, where βi∈0,1,i=1,2.
Based on Assumption2, β1 is called the cooperation identified rate and β2 is called the defection identified rate. We denote the tags of “cooperator” and “defector” as “C′” and “D′“. In one round, Ω is divided into 4 parts noted by Ωii=1,2,3,4. Ω1 and Ω2 are sets of cooperators and defectors labeled C′, respectively. Analogously, Ω3 and Ω4 are sets of cooperators and defectors labeled as D′, respectively. Furthermore, ∪i=14Ωi=ΩΩi∩Ωj=Φ, where Φ is an empty set. By denoting individual i labeled j′ as j′|ii,j=C,D, one can know that C′|C∈Ω1, C′|D∈Ω2, D′|D∈Ω3, and D′|C∈Ω4. According to Assumption 2, pC′|C=β1, pD′|C=1−β1, pD′|D=β2, and pC′|D=1−β2, where p· indicates the probability of event · happening.Assumption 3.
In every round game, cooperators chooseC and defectors choose D in the first period. In the following m periods, the cooperators labeled D′ and all defectors always choose D. Cooperators labeled C′ choose C if he/she encounters an individual labeled C′ and chooses D if he/she encounters an individual labeled D′.Definition 4.
The control of a controller dominating individuals’ strategy choice through tagging is called TBC. The individual identified as the cooperator is tabled asC′, and the individual identified as the defector is tabled as D′ by the controller. The TBC rule is designed as follows: the cooperators labeled D′ and all defectors always choose D. Cooperators labeled C′ choose C if he/she encounters an individual labeled C′ and chooses D if he/she encounters an individual labeled D′.
Based on Assumption3, the controller can dominate an individual’s strategy choice through tagging individuals. At time t, the cooperation rate is xt in group Ω. In the first period, the ratio of C to total strategies is equal to the cooperation rate xt. In the following m periods, an individual’s strategy is decided by tags according to Assumption 3. The ratio of C to total strategies is not the cooperation rate xt but β1xβ1x+1−β21−x. One can know that β1xβ1x+1−β21−x=β1x1−x1−β1−β21−x<x.
Assumption3 is reasonable by Assumption 1 and Definition 1. From the abovementioned discussion, we know that C not only denotes cooperation but also denotes a cooperator and D not only denotes defection but also denotes a defector. Numbers of individuals in Ω1,Ω2,Ω3, and Ω4 are β1xN, 1−β21−xN, β21−xN, and 1−β1xN, respectively. Based on Assumption 2 and 3, in the first period, cooperators choose C and defectors choose D. In the following m periods, two individuals choose C, if and only if they are in Ω1. The strategy pair C,C is contributed. When one player from Ω1 encounters a player from Ω2, a cooperator chooses C and a defector chooses D. In addition to the abovementioned two cases, the strategy pair D,D will be formed. According to Taylor and Nowak [31] and Dong et al. [32], the probabilities of C,C, D,D, and C,D are β12x2, 1−β12x2−2β11−β2x1−x, and β11−β2x1−x with TBC. The larger β1 and β2 means the higher probability of C,C and the lower probability C,D. So, the larger β1 and β2 are in accordance with the better effect of promoting the evolution of cooperation. But, under uniform interaction rates, the corresponding probabilities are x2 and 1−x2, respectively, so an individual’s encounter constrained by TBC no longer abides by uniform interaction rates.
Based on Assumptions2 and 3 and Definition 4, cooperation identified rate β1 and defection identified rate β2 reflect the accuracy of TBC. We denote β′ as the probability of another encounter between the same two individuals in the repeated Prisoner’s Dilemma and β″ as the probability of knowing someone’s reputation in indirect reciprocity. According to [31], the condition that direct reciprocity (indirect reciprocity) can lead to the evolution of cooperation depends on the parameters of payoff matrix (1) and β′β″. But, by Assumption 1, β′ is very small because the group size, N, is large enough. On the other hand, the high cost of fully understanding and accurately knowing each other’s willingness to cooperate leads to that β″ is very small also. So, direct reciprocity and indirect reciprocity cannot promote cooperation in the setting that this paper considers. In every round game, cooperation identified rate β1 and defection identified rate β2 can act as β′ of direct reciprocity or β″ of indirect reciprocity.Assumption 4.
The evolution of group statex is carried out according to the replicator. An individual replication occurs after each round game ends and before the next round begins. An individual’s payoff refers to the average total payoff in this round game.
Assumption4 means the offspring of cooperators are still cooperators and the offspring of defectors are still defectors. So, the offsprings of C or D are disconnected with their parent’s tag. By Assumption 3, defectors always choose D, but the cooperator does not always choose C.
“Tag” does a mappingσ between Ω and C′,D′, i.e., σ:Ω⟶C′,D′. By Assumption 2, pσC=C′=β1, pσC=D′=1−β1, pσD=D′=β2, and pσD=C′=1−β2.
By calculation, the payoff of cooperator (defector) in thepth period of the rth round game is independent of r but determined by the cooperation rate x. Let vpth,Cxvpth,Dx denote the payoff of a cooperator (defector) in the pth period. Let wCxwDx denote total payoff of the cooperator (defector) in one round game; we get(4)wCx=v1th,Cx+∑p=2m+1vpth,C,wDx=v1th,Dx+∑p=2m+1vpth,D.
Letvpth,j′/Cxvpth,j′/Dx denote the payoffs of the cooperator (defector) labeled j′j′=C,D in the pth period game; we get(5)vpth,Cx=pσC=C′vpth,C′|C+pσC=D′vpth,D′|C,vpth,Dx=pσD=C′vpth,C′|D+pσD=D′vpth,D′|D,p=2,3,…,m+1.
By Assumptions1 and 2, the probability of anyone encountering a cooperator labeled C′ is β1x and the probability of anyone encountering a cooperator labeled D′ is 1−β1x; the probability of anyone encountering a defector labeled C′ or D′ is 1−β21−x or β21−x. One can get(6)v1th,Cx=Rx+S1−x,vpth,C′/Cx=β1xR+1−β1x+β21−xP+1−β21−xS,vpth,D′/Cx=P,p=2,3,…,m+1,(7)v1th,Dx=xT+1−xP,vpth,C′/Dx=β1xT+1−β1xP,vpth,D′/Dx=P,p=2,3,…,m+1.
Together, (4)–(7) determine wCx and wDx as follows:(8)wCx=1+mβ12xR+β11−β1−β2x+1−β11−β2mP+n!r!n−r!1+mβ11−β21−xS,wDx=v1,Dx+mβ2vpth,D′/Dx+1−β2vpth,C′/Dx,=1+mβ11−β2xT+1−x+m1−β11−β2xP.
By Assumption3, an individual’s average payoff in the rth round game is xwCx+1−xwDx.
Summing up the abovementioned discussion, the total payoff matrix (1) of a round game under TBC is(9)1+mβ12R+1−β12mP1+mβ11−β2S+1−β11−β2mP1+mβ11−β2T+1+β11−β2mPmP.
In fact, in the latterm periods, the hybrid strategy and the group state are not equivalent because a cooperator constrained by Assumption 3 does not always choose C. So, the payoff aij in payoff matrix (9) does not indicate the payoff of individual i encountering strategy j, where i,j=C,D. However, we still use payoff matrix (9) for the following two reasons: (1) the payoff wix of individual Ω1 is still linear with the group state xt; (2) a cooperator’s average total payoff under TBC is 1+mβ12R+1−β12mP if he enters a group of cooperators or is 1+mβ11−β2S+1−β11−β2mP if he enters a group of defectors. Similarly, a defector’s average total payoff under TBC is 1+mβ11−β2T+1−β11−β2mP if he enters a group of cooperators or is P if he enters a group of defectors.
Ifβ1=1,β2=0, i.e., all individuals are identified as cooperators, the game with payoff matrix (9) is equivalent to a Prisoner’s Dilemma repeated m+1 times with payoff matrix (1). By Assumption 3, cooperators always choose C and defectors always choose D.


## 2.2. The Replicator Dynamics with TBC

At timet, we suppose the number of cooperators is M and the number of defectors is N−M. In the next Δt, under Assumption 4, the cooperators reproduce wCMΔt offspring and the defectors reproduce wDN−MΔt offspring. The total number of death is wCM+wDN−MΔt. The number of cooperators death is wCM+wDN−MxΔt, and the number of defectors death is wCM+wDN−M1−xΔt. Then, Δx=wCMΔt−wCM+wDN−MxΔt/N=wC−wDx1−xΔt. Then, the replicator dynamics under TBC is as follows:(10)x˙t=wCx−wDx1−xx,=R−S−T+Px+S−P1−xx+mβ1R+2−β1−2β2P−1−β2S+Tx−1−β2P−Sβ11−xx.Assumption 5.
The Prisoner’s Dilemma tagged repeats enough periods, i.e.,m≫1.
By Assumption5, the total payoff matrix (9) in a round game approximately equals the following matrix:(11)mβ12R+1−β12mP1+β11−β2mp+mβ11−β2S1+β11−β2mPβ11−β2mTmP.
The replicator dynamics (10) approximates the following dynamics:(12)x˙t=mβ1R+2−β1−2β2P−1−β2S+Tx−1−β2P−Sβ11−xx.Definition 5.
(See [42]). A state x˜,y˜∈Δ is an ESS if for all other states x,y∈Δ either (i) ux,y,x˜,y˜<ux˜,y˜,x˜,y˜ or (ii) ux,y,x˜,y˜=ux˜,y˜,x˜,y˜ and ux,y,x,y<ux˜,y˜,x,y, where u·1,·2 denotes the individual’s payoff with strategy ·1 under the strategy pair ·1,·2.Definition 6.
(See [43]). A fixed point x˜,y˜∈Δ in replicator dynamics is an evolutionary equilibrium (EE) if it is locally asymptotically stable, i.e., every open neighborhood Κ⊂Δ of the point x˜,y˜ has the property that every path starting sufficiently close to x˜,y˜ remains in Κ and converges asymptotically to x˜,y˜.Definition 7.
(See [43]). The largest open set of points whose evolutionary paths converge to a given EE is called its basin of attraction.
In order to simplify the calculation,P=0 is assumed in the following discussion. Doing so would result in a simpler model, without significant qualitative or directional changes to our key results about evolution of cooperation. Obviously, the parameters T,R, and S satisfy T>R>0>S. Based on (11), the payoff matrix for a period under TBC is transformed as follows:(13)
Regardless of time scale,m does not affect either ESS or EE and does not even affect the evolutionary path of a replicator’s dynamics, so we can rewrite the replicator dynamics (12) as follows:(14)x˙t=β1R−1−β2S+Tx+1−β2Sβ11−xx.
Two specific instances are given in the following.Case1. Only the defectors are identifies, and let pD′/D=β2. Ω is divided into two parts. One is the set of defectors tagged D′. The other is the set of individuals without tag. The corresponding part of Assumption 3 is modified as follows: a cooperator chooses C if he encounters an individual without tag but chooses D if he encounters an individual tagged D′. Defectors always choose defect. In this case, the payoff matrix is as follows:(15)Case 2. Only the cooperators are identified, and let pC′/C=β1. Ω is divided into two parts. One is the set of cooperators tagged C′. The other is the set of individuals without a tag. The corresponding part of Assumption 3 is modified as follows: the cooperator chooses C if he encounters an individual with tag C′ and chooses D if he encounters an individual without a tag. Defectors always choose defect. In this case, we obtain the payoff matrix as follows:(16)
By comparing matrices (13) and (15), we see that Case 1 is equivalent to cooperation identified rate β1=1 in matrix (13), i.e., the controller accurately identifies cooperators. It follows that Case 1 is in accordance with control based on identifying defection. Correspondingly, Case 2 means the controller identifies defectors accurately.
From the operator’s perspective, the relevant decisions are (1) whether to design TBC; (2) identifying which individual is more effective, cooperator or defector; and (3) how to design cooperation identified rateβ1 and defection identified rate β2.


## 3. Evolution of Cooperation for the Prisoner’s Dilemma with TBC

### 3.1. Short-Term Effects Caused by TBC

Based on Assumption3, an individual chooses a strategy according to a tag. Can a tag improve the correctness of the choice?Proposition 1.
Leti/j′ denote the individual i labeled j′, where i,j=C,D. We denote pC/C′ as the probability that the individual tagged C′ is a cooperator and pD/D′ as the probability that the individual tagged D′ is a defector. Then, pC/C′>pC and pD/D′>pD, if and only if β1+β2>1.Proof.
According to Assumption2, pC′/C=β1 and pD′/D=β2. Denote the probability of an individual labeled i′ as pi′. Obviously, pC′=xβ1+1−x1−β2 and pD′=x1−β1+1−xβ2. So, one can obtain posterior probability pC/C′ and pD/D′ as follows:(17)pCC′=xβ1xβ1+1−x1−β2,pDD′=1−xβ2x1−β1+1−xβ2.
Thus,pC/C′>pC and pD/D′>pD, if and only if β1+β2>1.
Now, we discuss an individual’s payoff with TBC. Letfix,β1,β2 be the payoff of individual i with TBC and gix be the payoff of individual j without TBC, where i,j=C,D.Proposition 2.
Ifβ1+β2>1, then fCx,β1,β2−fDx,β1,β2>gCx−gDx.Proof.
From matrices (1) and (13), one can obtain(18)gCx=R−Sx+S,fCx,β1,β2=β12R−β11−β2Sx+β11−β2S,(19)gDx=Tx,fDx,β1,β2=β11−β2Tx.
It follows from (18) and (19) that fCx,β1,β2−gCx and fDx,β1,β2−gDx are all linear to x. Since fC0,β1,β2−gC0=−S1−β11−β2 and fD0,β1,β2−gD0=0, by −S>0, it can be derived that(20)fC0,β1,β2−gC0>fD0,β1,β2−gD0.
ByfC1,β1,β2−gC1=β12−1R and fD1,β1,β2−gD1=β11−β2−1T, if β1+β2>1, then β11−β2−1<β12−1<0. So, we get(21)fC1,β1,β2−gC1>fD1,β1,β2−gD1.
In summary,fCx,β1,β2−gCx>fDx,β1,β2−gDx if β1+β2>1.
Based on Proposition1, TBC can help cooperators not only identify cooperators but also reduce the risk of unilateral fraud by defectors if β1+β2>1. According to Proposition 2, for any cooperation rate x∈0,1, TBC satisfying β1+β2>1 can inhibit defection. But, Propositions 1 and 2 do not mean TBC can promote a cooperator’s payoff even when β1+β2>1. This judgment is confusing. Propositions 3 and 4 will demonstrate.Proposition 3.
We denote∗ix=∂∗i/∂x, where ∗=f,g; i=C,D. With increasing cooperation rate x, some properties of the individual’s payoff are as follows:(i)
∗ix>0(ii)
fixx,β1,β2−gixx<0Proof.
From (18) and (19), according to T>R>0>S, we get ∗ix and fCxx,β1,β2−gCxx=β12−1R+1−β11−β2S. Combining with R>0,S<0 and β1,β2∈0,1, we derive fCxx,β1,β2−gCxx<0. Similarly, we can get fDxx,β1,β2−gDxx<0. So, with increases in cooperation rate x, the payoff of all individuals with TBC increases less than their payoff without TBC.
By Proposition3, the fact that all individual’s payoff increases in cooperation rate x has nothing to do with TBC or not. But, the rate of increase of an individual’s payoff with TBC is lower than its payoff without TBC. In this sense, TBC reduces group efficiency when the rate of group cooperation rate increases. Why do we design TBC in the Prisoner’s Dilemma? In fact, without special mechanisms for promoting the evolution of cooperation, the group cooperation rate will always gradually decrease until cooperators are completely expelled from the group. So, the effect of TBC on the payoff of individuals and the evolution of cooperation is complex. The effect of TBC on the cooperator’s payoff and cooperation evolution involves β1,β2, parameters R,T,S, and group cooperation rate x. We denote x1β1,β2≜1−β11−β2−S/1−β12R+1−β11−β2−S, x2β1,β2≜1−β2−S/2β1R+1−β2−S.Proposition 4.
The effects of TBC on group cooperation ratex and an individual's payoff are as follows:(i)
For anyx∈0,1, the defector’s payoff with TBC is always lower than its payoff without TBC.(ii)
A cooperator's payoff with TBC is higher than its payoff without TBC when the group cooperation ratex is lower than x1β1,β2 but is lower than its payoff without TBC when x is higher than x1β1,β2, i.e., signx−x1β1,β2fCx,β1,β2−gCx<0, where sign· is a sign function of ·.(iii)
For anyx∈0,1, the cooperator's payoff increases in β2 and the defector's payoff increases in β1 but decreases in β2, i.e., fCβ2x,β1,β2>0, fDβ1x,β1,β2>0, and fDβ2x,β1,β2<0; where fiβj=∂fi/∂βj,i=C,D,j=1,2. However, the cooperator's payoff increases in β1 when the group cooperation rate x is higher than x2β1,β2 but decreases when x is lower than x2β1,β2, i.e., signx−x2β1,β2fCβix,β1,β2>0.(iv)
Ifβ1+β2>1, then x1β1,β2>x2β1,β2.Proof.
From (18) and (19), for any β1,β2∈0,1 and x∈0,1, fDβ1,β2,x<gDx and fCx,β1,β2−gCx=−1−β12R−1−β11−β2−Sx+1−β11−β2−S. fCx,β1,β2<gCx when x>x1β1,β2 and fCx,β1,β2>gCx when x<x1β1,β2. Conclusions (I) and (II) are proved.
Furthermore, one can obtainfCβ2x,β1,β2=1−x−Sβ1, fDβ1x,β1,β2=1−β2Tx, and fDβ2x,β1,β2=−β1Tx. From fCβ1x,β1,β2=2Rβ1+−S1−β2x+S1−β2, fCβ1x,β1,β2>0 when x>x2β1,β2 and fCβ1x,β1,β2<0 when x<x2β1,β2 Based on T>R>0>S, if β1+β2>1, then x1β1,β2>x2β1,β2. Conclusions (III) and (IV) are proved.
By applying Proposition4, for any β1,β2∈0,1, the TBC increases or decreases the cooperator’s payoff depending on the group cooperation rate and x1β1,β2. Fortunately, the critical point x1β1,β2 satisfies ∂x1β1,β2/∂βi>0i=1,2. Furthermore, we obtain(22)limβ1⟶1x1β1,β2=1.
So, the TBC can always improve the cooperator’s payoff if cooperation identified rateβ1 is large enough. Furthermore, for any x∈0,1, we can design an appropriate β1, such that x1. Applying (I) and (II) of Proposition 4, an appropriate TBC can raise the cooperator’s payoff and reduce the defector’s payoff, thus increasing the group cooperation rate rapidly.
According to (III) of Proposition4, for any x∈0,1, with β2 increasing, the cooperator’s payoff increases and defector’s payoff reduces. An interesting phenomenon is fDβ1x,β1,β2>0 for any x∈0,1, i.e., a defector’s payoff will always increase in β1. Proposition 4 shows that the effect of cooperation identified rate β1 on an individual’s payoff is more complex than defection identified rate β2. The cooperator’s payoff increases in β1 when the group cooperation rate C is higher than the critical value x2β1,β2 but decreases in β1 when x<x2β1,β2. Based on (II), TBC raises the cooperator’s payoff only when x<x1β1,β2. So, only when x∈x2β1,β2,x1β1,β2, TBC promotes the cooperator’s payoff, and larger β1 results in larger cooperator’s payoff.
According to (VI), ifβ1+β2>1, x2β1,β2<x1β1,β2. Furthermore, limβ2⟶1x2β1,β2=0, i.e., the cooperator’s payoff increases in cooperation identified rate β1 if defection identified rate β2 is large enough. At the same time, if β1>−S/2R1/x−1, then the cooperator’s payoff increases in β1 for any β2∈0,1. On the other hand, increasing β1 can also increase the defector’s payoff because the probability of the strategy pair D,C is β11−β2x1−x. Thus, increasing β1 increases the probability that a defector gains the payoff T!
According to (III) of Proposition4, for a given defection identified rate β2, if the group cooperation rate is very low, for example, x<1−β2−S/2R+1−β2−S, then increasing cooperation identified rate β1 will not only bring harm to cooperators but also help defectors to get more payoffs. A larger β1 will doubtlessly cause great damage to the group if the defection identified rate β2 and the group cooperation rate x are all little.
Denoting the group’s expected payoff without TBC asHx and with TBC as H′x,β1,β2, we get the following proposition.Proposition 5.
IfS+T>0, then, for any given x∈0,1 and any β1β2∈0,1, the group’s expected payoff without TBC is higher than that with TBC, i.e.,(23)H′x,β1,β2<Hx.Proof.
Applying the payoff matrix (13), we get H′x,β1,β2=xfCx,β1,β2+1−xfDx,β1,β2 and Hx=xgCx+1−xgDx. If S+T>0, it yields(24)H′x,β1,β2−Hx=−1−β12Rx2−1−β11−β2S+Tx−x2<0.
This completes the proof.
Proposition5 is simple but interesting. It shows that the TBC reduces the group’s average payoff when the cooperator’s loss −S, caused by defection, is less than the defector’s gain T. Furthermore, this conclusion is not affected by x, R, β1 and β2. Furthermore, we obtain(25)∂H′x,β1,β2−Hx∂β1=2β1R−1−β2S+Tx2−1−β2S+Tx.
Whenβ1+β2>1, based on (3), 2β1R−1−β2S+T>0. By letting x3β1,β2≜1−β2S+T/2β1R+1−β2S+T, we can obtain signx−x3β1,β2∂H′x,β1,β2−Hx/∂β1>0. This is consistent with the result of Proposition 2. By the fact that limβ2⟶1x3β1,β2=0, when β2 is large enough, increasing β1 can improve the group's average payoff, but it is still lower than that without TBC.
We assume thatS+T<0. Denoting x4β1,β2≜−1−β11−β2S+T/1−β12R−1−β11−β2S+T, we get signx−x4β1,β2H′x,β1,β2−Hx<0. In other words, in the situation where two players’ total payoff with strategy pair C,D is lower than the total payoff with D,D, the TBC can increase the group’s average payoff for a lower cooperation rate.Remark 1.
Propositions 2–5 are all based on the payoff matrix (13) that is obtained by m≫1 and P=0. The assumption m≫1 is realistic. For example, confirmation by the supervisory department of, amongst other things, an enterprise’s creditor product quality, cannot be eliminated in short term. The values of xiβ1,β2 will be more complicated if P≠0,j=1,2,3,4. But, the hypothesis that P=0 does not result in significant qualitative or directional changes to conclusions of Propositions 2–5. Furthermore, the ESS and EE of the game do not alter by the assumptions m≫1 and P=0.Remark 2.
Propositions2–5 are only static analysis of the impact of TBC on an individual’s payoff.
Propositions2–5 show that TBC will bring down a group’s payoff unless the cooperation identified rate β1 and the defection identified rate β2 are all large enough. From the point of static analysis, it is hard to understand that TBC can promote cooperation. However, from the evolutionary point of view, without TBC, cooperation will fail in evolution and all benefits based on cooperation will disappear. Now, we discuss the ESS and EE of the Prisoner’s Dilemma with TBC.


### 3.2. Evolution of Cooperation for the Replicator Dynamics

Proposition 6.
Defection is always ESS of the Prisoner’s Dilemma for anyβ1,β2∈0,1.Proof.
For any statex,y∈Δ and any β1,β2∈0,1, by payoff matrix (13), one can get ux,y,0,1=β11−β2xS<0 and u0,1,0,1=0. So, defection is always ESS.
According to Friedman [43], the Prisoner’s Dilemma was discussed as being a linear game; the point x,y=0,1 is an EE of replicator dynamics (14). Based on Proposition 6, a small group of cooperators cannot invade a group of defectors, no matter how high identified rates β1 and β2 are.Proposition 7.
Cooperation is an ESS of the Prisoner’s Dilemma with TBC, if and only if the following condition is satisfied:(26)β11−β2>TR.By [42], Proposition 7 can be obtained.Based on Propositions6 and 7, if and only if (26) is satisfied, the Prisoner’s Dilemma with TBC is a symmetric coordination game. The pure strategies “cooperation” and “defection” are all ESS. The fixed points 1,0 and 0,1 are all EE. Furthermore, the basin of attraction of the point 1,0 is x5β1,β2,1, where x5β1,β2≜1−β2−S/β1R−1−β2T+S. As ∂x5β1,β2/∂βi<0,i=1,2, the basin of attraction of cooperation can rise if we increase the cooperation identified rate β1 or the defection identified rate β2. For any β1∈0,1, limβ2⟶1x5β1,β2=0. This shows that a large-enough defection identified rate β2 can successfully promote the evolution of cooperation for any β1. Furthermore, the basin of attraction of cooperation can be sufficiently large through the designing of an appropriate defection identified rate.If the defection identified rate satisfiesβ2>T−R/T, cooperation can evolve successfully. By denoting limβ1⟶1x5β1,β2=1−β2−S/R−1−β2T+S≜x6β2, the largest basin of attraction of cooperation is x6β2,1 for given β2. Otherwise, if β2<T−R/T, defection is the only ESS. If we consider T−R/T as the cost-to-benefit ratio of the cooperation, this conclusion is in good agreement with the work of Nowak [44]. In [44], it is shown that direct reciprocity can lead to the evolution of cooperation only if the probability of another encounter between the same two individuals exceeds the cost-to-benefit ratio of cooperation, i.e., β′>T−R/T, and indirect reciprocity can only promote cooperation if the probability of knowing someone’s reputation exceeds the cost-to-benefit ratio of the cooperation, i.e., β″>T−R/T.By Proposition7, when condition (26) is satisfied, cooperation is the ESS of the Prisoner's Dilemma and the average payoff of an individual in a group full of cooperators is β12R. But, without TBC, defection is the only ESS and the average payoff of an individual will be 0! To sum up the abovementioned discussion, from a dynamic point of view, TBC can promote a cooperator’s payoff. Further observation shows that defection identified rate β2 plays a more important role in promoting the evolution of cooperation than cooperation identified rate β1, but β1 determines an individual’s payoff when the cooperation rate is high.Now, we only discuss the cooperator’s payoff and evolution of cooperation.Proposition 8.
Designing TBC with cooperation identified rateβ1 and the defection identified rate β2, we obtain the following conclusions:(i)
When condition (26) is satisfied, but β1+β2−β1β2+1/2β1β22<1, this TBC can promote the evolution of cooperation, but for given cooperation rate x in the basin of attraction of cooperation, the cooperator's payoff with TBC is lower than its payoff without TBC(ii)
WhenR/T>1−β21−β11−β2/β1+β2−1, this TBC can not only expel defectors but also increase the cooperator's payoff for x∈x5β1,β2,x1β1,β2Proof.
If condition (26) is satisfied, based on Proposition 7, the TBC can promote the evolution of cooperation. From the condition β1+β2−β1β2+1/2β1β22<1, we obtain that 1−β21−β11−β2/β1+β2−1>1 and x5β1,β2>x1β1,β2. The basin of attraction of cooperation is x5β1,β2,1, and the cooperator’s payoff with TBC is higher than its payoff without TBC only when the group cooperation rate fCx,β1,β2−gCx is lower than x1β1,β2. So, the TBC can expel defectors, but the cooperator’s payoff with TBC is lower than its payoff without TBC when x∈x5β1,β2,1.
If the conditionR/T>1−β21−β11−β2/β1+β2−1 is satisfied, we get β1+β2−β1β2+1/2β1β22>1 and x5β1,β2<x1β1,β2. So, the TBC can not only expel defectors but also increase the cooperator’s payoff for x∈x5β1,β2,x1β1,β2.
The condition of (II) of Proposition8 is stricter than (26). So, based on (II) of Proposition 8, the TBC can not only expel defectors if x∈x1β1,β2,1 but also increase the cooperator’s payoff for x∈0,x5β1,β2. But, only when x∈x5β1,β2,x1β1,β2, the TBC can result in these two advantages.
Cooperation identified rateβ1 and defection identified rate β2 are the input of TBC. They are feedback of cooperation rate x. At any time t, cooperation rate is variable xt. So, β1 and β2 are functions of time t. We denote β1t and β2t as the cooperation identified rate and the defection identified rate at time t in the next part of this paper.


## 3.1. Short-Term Effects Caused by TBC

Based on Assumption3, an individual chooses a strategy according to a tag. Can a tag improve the correctness of the choice?Proposition 1.
Leti/j′ denote the individual i labeled j′, where i,j=C,D. We denote pC/C′ as the probability that the individual tagged C′ is a cooperator and pD/D′ as the probability that the individual tagged D′ is a defector. Then, pC/C′>pC and pD/D′>pD, if and only if β1+β2>1.Proof.
According to Assumption2, pC′/C=β1 and pD′/D=β2. Denote the probability of an individual labeled i′ as pi′. Obviously, pC′=xβ1+1−x1−β2 and pD′=x1−β1+1−xβ2. So, one can obtain posterior probability pC/C′ and pD/D′ as follows:(17)pCC′=xβ1xβ1+1−x1−β2,pDD′=1−xβ2x1−β1+1−xβ2.
Thus,pC/C′>pC and pD/D′>pD, if and only if β1+β2>1.
Now, we discuss an individual’s payoff with TBC. Letfix,β1,β2 be the payoff of individual i with TBC and gix be the payoff of individual j without TBC, where i,j=C,D.Proposition 2.
Ifβ1+β2>1, then fCx,β1,β2−fDx,β1,β2>gCx−gDx.Proof.
From matrices (1) and (13), one can obtain(18)gCx=R−Sx+S,fCx,β1,β2=β12R−β11−β2Sx+β11−β2S,(19)gDx=Tx,fDx,β1,β2=β11−β2Tx.
It follows from (18) and (19) that fCx,β1,β2−gCx and fDx,β1,β2−gDx are all linear to x. Since fC0,β1,β2−gC0=−S1−β11−β2 and fD0,β1,β2−gD0=0, by −S>0, it can be derived that(20)fC0,β1,β2−gC0>fD0,β1,β2−gD0.
ByfC1,β1,β2−gC1=β12−1R and fD1,β1,β2−gD1=β11−β2−1T, if β1+β2>1, then β11−β2−1<β12−1<0. So, we get(21)fC1,β1,β2−gC1>fD1,β1,β2−gD1.
In summary,fCx,β1,β2−gCx>fDx,β1,β2−gDx if β1+β2>1.
Based on Proposition1, TBC can help cooperators not only identify cooperators but also reduce the risk of unilateral fraud by defectors if β1+β2>1. According to Proposition 2, for any cooperation rate x∈0,1, TBC satisfying β1+β2>1 can inhibit defection. But, Propositions 1 and 2 do not mean TBC can promote a cooperator’s payoff even when β1+β2>1. This judgment is confusing. Propositions 3 and 4 will demonstrate.Proposition 3.
We denote∗ix=∂∗i/∂x, where ∗=f,g; i=C,D. With increasing cooperation rate x, some properties of the individual’s payoff are as follows:(i)
∗ix>0(ii)
fixx,β1,β2−gixx<0Proof.
From (18) and (19), according to T>R>0>S, we get ∗ix and fCxx,β1,β2−gCxx=β12−1R+1−β11−β2S. Combining with R>0,S<0 and β1,β2∈0,1, we derive fCxx,β1,β2−gCxx<0. Similarly, we can get fDxx,β1,β2−gDxx<0. So, with increases in cooperation rate x, the payoff of all individuals with TBC increases less than their payoff without TBC.
By Proposition3, the fact that all individual’s payoff increases in cooperation rate x has nothing to do with TBC or not. But, the rate of increase of an individual’s payoff with TBC is lower than its payoff without TBC. In this sense, TBC reduces group efficiency when the rate of group cooperation rate increases. Why do we design TBC in the Prisoner’s Dilemma? In fact, without special mechanisms for promoting the evolution of cooperation, the group cooperation rate will always gradually decrease until cooperators are completely expelled from the group. So, the effect of TBC on the payoff of individuals and the evolution of cooperation is complex. The effect of TBC on the cooperator’s payoff and cooperation evolution involves β1,β2, parameters R,T,S, and group cooperation rate x. We denote x1β1,β2≜1−β11−β2−S/1−β12R+1−β11−β2−S, x2β1,β2≜1−β2−S/2β1R+1−β2−S.Proposition 4.
The effects of TBC on group cooperation ratex and an individual's payoff are as follows:(i)
For anyx∈0,1, the defector’s payoff with TBC is always lower than its payoff without TBC.(ii)
A cooperator's payoff with TBC is higher than its payoff without TBC when the group cooperation ratex is lower than x1β1,β2 but is lower than its payoff without TBC when x is higher than x1β1,β2, i.e., signx−x1β1,β2fCx,β1,β2−gCx<0, where sign· is a sign function of ·.(iii)
For anyx∈0,1, the cooperator's payoff increases in β2 and the defector's payoff increases in β1 but decreases in β2, i.e., fCβ2x,β1,β2>0, fDβ1x,β1,β2>0, and fDβ2x,β1,β2<0; where fiβj=∂fi/∂βj,i=C,D,j=1,2. However, the cooperator's payoff increases in β1 when the group cooperation rate x is higher than x2β1,β2 but decreases when x is lower than x2β1,β2, i.e., signx−x2β1,β2fCβix,β1,β2>0.(iv)
Ifβ1+β2>1, then x1β1,β2>x2β1,β2.Proof.
From (18) and (19), for any β1,β2∈0,1 and x∈0,1, fDβ1,β2,x<gDx and fCx,β1,β2−gCx=−1−β12R−1−β11−β2−Sx+1−β11−β2−S. fCx,β1,β2<gCx when x>x1β1,β2 and fCx,β1,β2>gCx when x<x1β1,β2. Conclusions (I) and (II) are proved.
Furthermore, one can obtainfCβ2x,β1,β2=1−x−Sβ1, fDβ1x,β1,β2=1−β2Tx, and fDβ2x,β1,β2=−β1Tx. From fCβ1x,β1,β2=2Rβ1+−S1−β2x+S1−β2, fCβ1x,β1,β2>0 when x>x2β1,β2 and fCβ1x,β1,β2<0 when x<x2β1,β2 Based on T>R>0>S, if β1+β2>1, then x1β1,β2>x2β1,β2. Conclusions (III) and (IV) are proved.
By applying Proposition4, for any β1,β2∈0,1, the TBC increases or decreases the cooperator’s payoff depending on the group cooperation rate and x1β1,β2. Fortunately, the critical point x1β1,β2 satisfies ∂x1β1,β2/∂βi>0i=1,2. Furthermore, we obtain(22)limβ1⟶1x1β1,β2=1.
So, the TBC can always improve the cooperator’s payoff if cooperation identified rateβ1 is large enough. Furthermore, for any x∈0,1, we can design an appropriate β1, such that x1. Applying (I) and (II) of Proposition 4, an appropriate TBC can raise the cooperator’s payoff and reduce the defector’s payoff, thus increasing the group cooperation rate rapidly.
According to (III) of Proposition4, for any x∈0,1, with β2 increasing, the cooperator’s payoff increases and defector’s payoff reduces. An interesting phenomenon is fDβ1x,β1,β2>0 for any x∈0,1, i.e., a defector’s payoff will always increase in β1. Proposition 4 shows that the effect of cooperation identified rate β1 on an individual’s payoff is more complex than defection identified rate β2. The cooperator’s payoff increases in β1 when the group cooperation rate C is higher than the critical value x2β1,β2 but decreases in β1 when x<x2β1,β2. Based on (II), TBC raises the cooperator’s payoff only when x<x1β1,β2. So, only when x∈x2β1,β2,x1β1,β2, TBC promotes the cooperator’s payoff, and larger β1 results in larger cooperator’s payoff.
According to (VI), ifβ1+β2>1, x2β1,β2<x1β1,β2. Furthermore, limβ2⟶1x2β1,β2=0, i.e., the cooperator’s payoff increases in cooperation identified rate β1 if defection identified rate β2 is large enough. At the same time, if β1>−S/2R1/x−1, then the cooperator’s payoff increases in β1 for any β2∈0,1. On the other hand, increasing β1 can also increase the defector’s payoff because the probability of the strategy pair D,C is β11−β2x1−x. Thus, increasing β1 increases the probability that a defector gains the payoff T!
According to (III) of Proposition4, for a given defection identified rate β2, if the group cooperation rate is very low, for example, x<1−β2−S/2R+1−β2−S, then increasing cooperation identified rate β1 will not only bring harm to cooperators but also help defectors to get more payoffs. A larger β1 will doubtlessly cause great damage to the group if the defection identified rate β2 and the group cooperation rate x are all little.
Denoting the group’s expected payoff without TBC asHx and with TBC as H′x,β1,β2, we get the following proposition.Proposition 5.
IfS+T>0, then, for any given x∈0,1 and any β1β2∈0,1, the group’s expected payoff without TBC is higher than that with TBC, i.e.,(23)H′x,β1,β2<Hx.Proof.
Applying the payoff matrix (13), we get H′x,β1,β2=xfCx,β1,β2+1−xfDx,β1,β2 and Hx=xgCx+1−xgDx. If S+T>0, it yields(24)H′x,β1,β2−Hx=−1−β12Rx2−1−β11−β2S+Tx−x2<0.
This completes the proof.
Proposition5 is simple but interesting. It shows that the TBC reduces the group’s average payoff when the cooperator’s loss −S, caused by defection, is less than the defector’s gain T. Furthermore, this conclusion is not affected by x, R, β1 and β2. Furthermore, we obtain(25)∂H′x,β1,β2−Hx∂β1=2β1R−1−β2S+Tx2−1−β2S+Tx.
Whenβ1+β2>1, based on (3), 2β1R−1−β2S+T>0. By letting x3β1,β2≜1−β2S+T/2β1R+1−β2S+T, we can obtain signx−x3β1,β2∂H′x,β1,β2−Hx/∂β1>0. This is consistent with the result of Proposition 2. By the fact that limβ2⟶1x3β1,β2=0, when β2 is large enough, increasing β1 can improve the group's average payoff, but it is still lower than that without TBC.
We assume thatS+T<0. Denoting x4β1,β2≜−1−β11−β2S+T/1−β12R−1−β11−β2S+T, we get signx−x4β1,β2H′x,β1,β2−Hx<0. In other words, in the situation where two players’ total payoff with strategy pair C,D is lower than the total payoff with D,D, the TBC can increase the group’s average payoff for a lower cooperation rate.Remark 1.
Propositions 2–5 are all based on the payoff matrix (13) that is obtained by m≫1 and P=0. The assumption m≫1 is realistic. For example, confirmation by the supervisory department of, amongst other things, an enterprise’s creditor product quality, cannot be eliminated in short term. The values of xiβ1,β2 will be more complicated if P≠0,j=1,2,3,4. But, the hypothesis that P=0 does not result in significant qualitative or directional changes to conclusions of Propositions 2–5. Furthermore, the ESS and EE of the game do not alter by the assumptions m≫1 and P=0.Remark 2.
Propositions2–5 are only static analysis of the impact of TBC on an individual’s payoff.
Propositions2–5 show that TBC will bring down a group’s payoff unless the cooperation identified rate β1 and the defection identified rate β2 are all large enough. From the point of static analysis, it is hard to understand that TBC can promote cooperation. However, from the evolutionary point of view, without TBC, cooperation will fail in evolution and all benefits based on cooperation will disappear. Now, we discuss the ESS and EE of the Prisoner’s Dilemma with TBC.


## 3.2. Evolution of Cooperation for the Replicator Dynamics

Proposition 6.
Defection is always ESS of the Prisoner’s Dilemma for anyβ1,β2∈0,1.Proof.
For any statex,y∈Δ and any β1,β2∈0,1, by payoff matrix (13), one can get ux,y,0,1=β11−β2xS<0 and u0,1,0,1=0. So, defection is always ESS.
According to Friedman [43], the Prisoner’s Dilemma was discussed as being a linear game; the point x,y=0,1 is an EE of replicator dynamics (14). Based on Proposition 6, a small group of cooperators cannot invade a group of defectors, no matter how high identified rates β1 and β2 are.Proposition 7.
Cooperation is an ESS of the Prisoner’s Dilemma with TBC, if and only if the following condition is satisfied:(26)β11−β2>TR.By [42], Proposition 7 can be obtained.Based on Propositions6 and 7, if and only if (26) is satisfied, the Prisoner’s Dilemma with TBC is a symmetric coordination game. The pure strategies “cooperation” and “defection” are all ESS. The fixed points 1,0 and 0,1 are all EE. Furthermore, the basin of attraction of the point 1,0 is x5β1,β2,1, where x5β1,β2≜1−β2−S/β1R−1−β2T+S. As ∂x5β1,β2/∂βi<0,i=1,2, the basin of attraction of cooperation can rise if we increase the cooperation identified rate β1 or the defection identified rate β2. For any β1∈0,1, limβ2⟶1x5β1,β2=0. This shows that a large-enough defection identified rate β2 can successfully promote the evolution of cooperation for any β1. Furthermore, the basin of attraction of cooperation can be sufficiently large through the designing of an appropriate defection identified rate.If the defection identified rate satisfiesβ2>T−R/T, cooperation can evolve successfully. By denoting limβ1⟶1x5β1,β2=1−β2−S/R−1−β2T+S≜x6β2, the largest basin of attraction of cooperation is x6β2,1 for given β2. Otherwise, if β2<T−R/T, defection is the only ESS. If we consider T−R/T as the cost-to-benefit ratio of the cooperation, this conclusion is in good agreement with the work of Nowak [44]. In [44], it is shown that direct reciprocity can lead to the evolution of cooperation only if the probability of another encounter between the same two individuals exceeds the cost-to-benefit ratio of cooperation, i.e., β′>T−R/T, and indirect reciprocity can only promote cooperation if the probability of knowing someone’s reputation exceeds the cost-to-benefit ratio of the cooperation, i.e., β″>T−R/T.By Proposition7, when condition (26) is satisfied, cooperation is the ESS of the Prisoner's Dilemma and the average payoff of an individual in a group full of cooperators is β12R. But, without TBC, defection is the only ESS and the average payoff of an individual will be 0! To sum up the abovementioned discussion, from a dynamic point of view, TBC can promote a cooperator’s payoff. Further observation shows that defection identified rate β2 plays a more important role in promoting the evolution of cooperation than cooperation identified rate β1, but β1 determines an individual’s payoff when the cooperation rate is high.Now, we only discuss the cooperator’s payoff and evolution of cooperation.Proposition 8.
Designing TBC with cooperation identified rateβ1 and the defection identified rate β2, we obtain the following conclusions:(i)
When condition (26) is satisfied, but β1+β2−β1β2+1/2β1β22<1, this TBC can promote the evolution of cooperation, but for given cooperation rate x in the basin of attraction of cooperation, the cooperator's payoff with TBC is lower than its payoff without TBC(ii)
WhenR/T>1−β21−β11−β2/β1+β2−1, this TBC can not only expel defectors but also increase the cooperator's payoff for x∈x5β1,β2,x1β1,β2Proof.
If condition (26) is satisfied, based on Proposition 7, the TBC can promote the evolution of cooperation. From the condition β1+β2−β1β2+1/2β1β22<1, we obtain that 1−β21−β11−β2/β1+β2−1>1 and x5β1,β2>x1β1,β2. The basin of attraction of cooperation is x5β1,β2,1, and the cooperator’s payoff with TBC is higher than its payoff without TBC only when the group cooperation rate fCx,β1,β2−gCx is lower than x1β1,β2. So, the TBC can expel defectors, but the cooperator’s payoff with TBC is lower than its payoff without TBC when x∈x5β1,β2,1.
If the conditionR/T>1−β21−β11−β2/β1+β2−1 is satisfied, we get β1+β2−β1β2+1/2β1β22>1 and x5β1,β2<x1β1,β2. So, the TBC can not only expel defectors but also increase the cooperator’s payoff for x∈x5β1,β2,x1β1,β2.
The condition of (II) of Proposition8 is stricter than (26). So, based on (II) of Proposition 8, the TBC can not only expel defectors if x∈x1β1,β2,1 but also increase the cooperator’s payoff for x∈0,x5β1,β2. But, only when x∈x5β1,β2,x1β1,β2, the TBC can result in these two advantages.
Cooperation identified rateβ1 and defection identified rate β2 are the input of TBC. They are feedback of cooperation rate x. At any time t, cooperation rate is variable xt. So, β1 and β2 are functions of time t. We denote β1t and β2t as the cooperation identified rate and the defection identified rate at time t in the next part of this paper.


## 4. The Optimal Identified Rate of Promoting the Cooperation Rate

### 4.1. The Optimal Identified Rate Model

We defineuit as(27)uit=ln1βit−1,i=1,2.Based onβi∈0,1, one can obtain ui∈−∞,+∞ and βit=1/1+euit, i=1,2. The performance index is introduced:(28)J=1−xtf+∫0tfq1β1t+q2β2tdt,where tf is the terminal time of the replicator dynamic (14), xtf denotes the terminal cooperation rate, 1−xtf is the effective cost of promoting the cooperation rate, and ∫0tfβitdt is the control cost of the entire TBC process. The bigger xtf corresponds to the better effect of promoting cooperation and the lower effective cost. The parameter qi>0 is used to adjust the difference between the effective cost and the control cost in the dimension and the weight. Furthermore, qi>0 reflects the controller’s balance of the TBC for cooperation and for defection.We rewrite replicator dynamic (14) and performance index (28) under transformation (27) as the follows:(29)x˙t=11+eu1R−eu21+eu2S+Tx+eu21+eu2S11+eu11−xx,(30)J=1−xtf+∫0tfq11+eu1+q21+eu2dt.We suppose the initial time of replicator dynamic (14) is t=0, and the initial value of system is(31)x0=x0,0<x0<1.The initial value (31) indicates that, at the beginning of the evolution, the population is in a state in which cooperation and defection are mixed. We denote ut=u1t,u2tT. We consider ut as a control variable of the dynamic (29).The feedback control law of the statext is designed so that the total cost defined by the performance index (30) is minimized. The optimal identified rate problem can be expressed as follows: under dynamic constraint (29) and initial value constraint (31), the optimal control u∗t and the optimal state trajectory x∗t are designed to minimize (30).


### 4.2. The Optimal Identified Rate Designing Process

There is no analytic solution for the optimal control problem (29)–(31). Now, we give a numerical algorithm to design an optimal identified rate.Theorem 1.
Consider the optimal identified rate design described by replicator dynamic (14) with performance index (30). Under (27) and (31), the optimal identified rate βi∗t is existent and unique, and βi∗t is expressed in the following form:(32)β1∗t=limh⟶∞limk⟶∞q2λh,kt1−xh,kxh,khxh,k,β2∗t=limh⟶∞limk⟶∞1+q1hxh,k−2Rq2xh,kλh,k1−xh,kxh,kh2xh,k,where λh,kt is the solution of the following adjoint variable differential equation:(33)λh,0t≡1,λ˙h,kt=q22Rx∗h23hx∗h−2T+q1q2hx∗h−3x∗h+1hx∗h+Tx∗hλh,k−1x∗h21−x∗h2h3x∗h,λh,ktf=1,h,k=1,2,…,where x∗0t=x0,x∗ht=limk⟶∞xh,kt, and xh,kt is the solution of the following state differential equation:(34)xh,0t≡x0,λ∗0t≡1,x˙h,kt=q1q2hxh,k−1−Rq22xh,k−1λ∗h−12xh,k−11−xh,k−1h2xh,k−1,t∈0,tf,xh,k0=x0,h,k=1,2,…,where λ∗ht=limk⟶∞λh,kt.Proof.
By introducing the adjoint variableλt and letting hx=S+Tx−S, if the optimal control and the optimal state are u∗t and x∗t, then there exists the adjoint variable λ∗t such that x∗0=x0, λ∗tf=1, and(35)q1+λt1−xx−2Rx1+eu1+eu21+eu2hx|λ=λ∗,x=x∗,u=u∗=0,(36)q2−λt1−xxhx11+eu1|λ=λ∗,x=x∗,u=u∗=0.
From (36), we obtain x∗t, λ∗t, and ui∗t satisfying the following equations:(37)11+eu1|λ=λ∗,x=x∗,u=u∗=q2λt1−xxhx|λ=λ∗,x=x∗,u=u∗,(38)11+eu2|λ=λ∗,x=x∗,u=u∗=1+q1λt1−xxhx−2Rq2xλt1−xxh2x|λ=λ∗,x=x∗,u=u∗.
The adjoint variable and its terminal condition with the optimal controlu∗t are as follows:(39)−λ˙t=λtRx2−3x1+eu12−eu21+eu211+eu1S+Tx2−3x−S1−2x,(40)λtf=1.
Substituting (37) and (38) into (29) and (39) yields the following two-point boundary value (TPBV) problem:(41)x˙t=q1q2hx−Rq22xλ2x1−xh2x,−λ˙t=q22Rx23hx−2T+q1q2hx−3x+1hx+Txλx21−x2h3x,x0=x0,λtf=1.
Note that the TPBV problem (41) is nonlinear; for a nonlinear TPBV problem, there is no analytic solution. From (37), (38), and (27), one can obtain(42)β1∗t=q2λ∗t1−x∗x∗hx∗,β2∗t=1+q1λ∗1−x∗x∗hx∗−2Rq2x∗λ∗1−x∗x∗h2x∗,where λ∗t and x∗t is the solution to the TPBV problem (41). A sequence of state differential equations is constructed:(43)x1,0t≡x0,x˙1,kt=q1q2hx1,k−1−Rq22x1,k−1x1,k−11−x1,k−1h2x1,k−1,t∈0,tf,x1,k0=x0,k=1,2,….
For certaink, x1,k−1t is known, so equation (43) is a sequence of differential equations. Furthermore, x1,kt can be solved from (43). According to Tang and Dong [45], the solution sequence defined by (43) is uniformly convergent to the solution of the following equation:(44)x˙t=q1q2hx−Rq22xx1−xh2x,t∈0,tf,x0=x0.
We defineλ∗1t=limk⟶∞x1,kt and construct the following sequence:(45)λ1,0t≡1,λ˙1,kt=q22Rx∗123hx∗1−2T+q1q2hx∗1−3x∗1+1hx∗1+Tx∗1λ1,k−1x∗121−x∗12h3x∗1,λ1,0tf=1,h,k=1,2,….
In thekth iteration, λ1,k−1t and fCxx,β1,β2−gCxx<0 are known, so (45) is also a sequence of differential equations. Based on [45], sequence (45) is uniformly convergent to the solution of the following equation:(46)λ˙t=q22Rx∗123hx∗1−2T+q1q2hx∗1−3x∗1+1hx∗1+Tx∗1λx∗121−x∗12h3x∗1,λtf=1.
We denoteλ∗1t=limk⟶∞λ1,kt. By constructing sequences (33) and (34) and repeating the previous iteration process, we can obtain the sequences xh,kt and λh,kt. The sequences xh,kt and λh,kt are uniformly convergent to the solution of TPBV (39), i.e.,(47)limh⟶∞limk⟶∞λh,kt=λ∗t,limh⟶∞limk⟶∞xh,kt=x∗t.
This completes the proof.
The number of iterationsX and Y can be determined by the accuracy requirement for the performance index (30). We give a practical algorithm calculating the X,Yth suboptimal optimal identified rate as Algorithm 1Algorithm 1:Finding a suboptimal optimal identified rate.
Input: Given positive real numbersδ and ε.Output:X,Yth suboptimal βi∗xt,i=1,2.Step 1: Initializeh=1, λ∗0t=1.Step 2: Initializek=1, xh,0t=x0.Step 3: Calculateh,kth state variable xh,kt according to equation (34).Step 4: Ifxh,kt−xh,k−1t<δ, let x∗ht=xh,kt; otherwise, let k=k+1 and jump to Step 3.Step 5: Initializek=1, λh,0t=1.Step 6: Calculateh,kth adjoint variable λh,kt according to equation (33).Step 7: Ifλh,kt−λh,k−1t<δ, let λ∗ht=λh,kt; Otherwise, let k=k+1 and jump to Step 6.Step 8: LettingX=h, calculate .. according to the following equation:JH=1−xtf+∫0tfq1β1∗H+q2β2∗HdtStep 9: IfJX−JX−1/JX<ε stop the algorithm and calculate the suboptimal βit,i=1,2 according to equation (32); otherwise, let h=h+1 and jump to Step 2.


## 4.1. The Optimal Identified Rate Model

We defineuit as(27)uit=ln1βit−1,i=1,2.Based onβi∈0,1, one can obtain ui∈−∞,+∞ and βit=1/1+euit, i=1,2. The performance index is introduced:(28)J=1−xtf+∫0tfq1β1t+q2β2tdt,where tf is the terminal time of the replicator dynamic (14), xtf denotes the terminal cooperation rate, 1−xtf is the effective cost of promoting the cooperation rate, and ∫0tfβitdt is the control cost of the entire TBC process. The bigger xtf corresponds to the better effect of promoting cooperation and the lower effective cost. The parameter qi>0 is used to adjust the difference between the effective cost and the control cost in the dimension and the weight. Furthermore, qi>0 reflects the controller’s balance of the TBC for cooperation and for defection.We rewrite replicator dynamic (14) and performance index (28) under transformation (27) as the follows:(29)x˙t=11+eu1R−eu21+eu2S+Tx+eu21+eu2S11+eu11−xx,(30)J=1−xtf+∫0tfq11+eu1+q21+eu2dt.We suppose the initial time of replicator dynamic (14) is t=0, and the initial value of system is(31)x0=x0,0<x0<1.The initial value (31) indicates that, at the beginning of the evolution, the population is in a state in which cooperation and defection are mixed. We denote ut=u1t,u2tT. We consider ut as a control variable of the dynamic (29).The feedback control law of the statext is designed so that the total cost defined by the performance index (30) is minimized. The optimal identified rate problem can be expressed as follows: under dynamic constraint (29) and initial value constraint (31), the optimal control u∗t and the optimal state trajectory x∗t are designed to minimize (30).


## 4.2. The Optimal Identified Rate Designing Process

There is no analytic solution for the optimal control problem (29)–(31). Now, we give a numerical algorithm to design an optimal identified rate.Theorem 1.
Consider the optimal identified rate design described by replicator dynamic (14) with performance index (30). Under (27) and (31), the optimal identified rate βi∗t is existent and unique, and βi∗t is expressed in the following form:(32)β1∗t=limh⟶∞limk⟶∞q2λh,kt1−xh,kxh,khxh,k,β2∗t=limh⟶∞limk⟶∞1+q1hxh,k−2Rq2xh,kλh,k1−xh,kxh,kh2xh,k,where λh,kt is the solution of the following adjoint variable differential equation:(33)λh,0t≡1,λ˙h,kt=q22Rx∗h23hx∗h−2T+q1q2hx∗h−3x∗h+1hx∗h+Tx∗hλh,k−1x∗h21−x∗h2h3x∗h,λh,ktf=1,h,k=1,2,…,where x∗0t=x0,x∗ht=limk⟶∞xh,kt, and xh,kt is the solution of the following state differential equation:(34)xh,0t≡x0,λ∗0t≡1,x˙h,kt=q1q2hxh,k−1−Rq22xh,k−1λ∗h−12xh,k−11−xh,k−1h2xh,k−1,t∈0,tf,xh,k0=x0,h,k=1,2,…,where λ∗ht=limk⟶∞λh,kt.Proof.
By introducing the adjoint variableλt and letting hx=S+Tx−S, if the optimal control and the optimal state are u∗t and x∗t, then there exists the adjoint variable λ∗t such that x∗0=x0, λ∗tf=1, and(35)q1+λt1−xx−2Rx1+eu1+eu21+eu2hx|λ=λ∗,x=x∗,u=u∗=0,(36)q2−λt1−xxhx11+eu1|λ=λ∗,x=x∗,u=u∗=0.
From (36), we obtain x∗t, λ∗t, and ui∗t satisfying the following equations:(37)11+eu1|λ=λ∗,x=x∗,u=u∗=q2λt1−xxhx|λ=λ∗,x=x∗,u=u∗,(38)11+eu2|λ=λ∗,x=x∗,u=u∗=1+q1λt1−xxhx−2Rq2xλt1−xxh2x|λ=λ∗,x=x∗,u=u∗.
The adjoint variable and its terminal condition with the optimal controlu∗t are as follows:(39)−λ˙t=λtRx2−3x1+eu12−eu21+eu211+eu1S+Tx2−3x−S1−2x,(40)λtf=1.
Substituting (37) and (38) into (29) and (39) yields the following two-point boundary value (TPBV) problem:(41)x˙t=q1q2hx−Rq22xλ2x1−xh2x,−λ˙t=q22Rx23hx−2T+q1q2hx−3x+1hx+Txλx21−x2h3x,x0=x0,λtf=1.
Note that the TPBV problem (41) is nonlinear; for a nonlinear TPBV problem, there is no analytic solution. From (37), (38), and (27), one can obtain(42)β1∗t=q2λ∗t1−x∗x∗hx∗,β2∗t=1+q1λ∗1−x∗x∗hx∗−2Rq2x∗λ∗1−x∗x∗h2x∗,where λ∗t and x∗t is the solution to the TPBV problem (41). A sequence of state differential equations is constructed:(43)x1,0t≡x0,x˙1,kt=q1q2hx1,k−1−Rq22x1,k−1x1,k−11−x1,k−1h2x1,k−1,t∈0,tf,x1,k0=x0,k=1,2,….
For certaink, x1,k−1t is known, so equation (43) is a sequence of differential equations. Furthermore, x1,kt can be solved from (43). According to Tang and Dong [45], the solution sequence defined by (43) is uniformly convergent to the solution of the following equation:(44)x˙t=q1q2hx−Rq22xx1−xh2x,t∈0,tf,x0=x0.
We defineλ∗1t=limk⟶∞x1,kt and construct the following sequence:(45)λ1,0t≡1,λ˙1,kt=q22Rx∗123hx∗1−2T+q1q2hx∗1−3x∗1+1hx∗1+Tx∗1λ1,k−1x∗121−x∗12h3x∗1,λ1,0tf=1,h,k=1,2,….
In thekth iteration, λ1,k−1t and fCxx,β1,β2−gCxx<0 are known, so (45) is also a sequence of differential equations. Based on [45], sequence (45) is uniformly convergent to the solution of the following equation:(46)λ˙t=q22Rx∗123hx∗1−2T+q1q2hx∗1−3x∗1+1hx∗1+Tx∗1λx∗121−x∗12h3x∗1,λtf=1.
We denoteλ∗1t=limk⟶∞λ1,kt. By constructing sequences (33) and (34) and repeating the previous iteration process, we can obtain the sequences xh,kt and λh,kt. The sequences xh,kt and λh,kt are uniformly convergent to the solution of TPBV (39), i.e.,(47)limh⟶∞limk⟶∞λh,kt=λ∗t,limh⟶∞limk⟶∞xh,kt=x∗t.
This completes the proof.
The number of iterationsX and Y can be determined by the accuracy requirement for the performance index (30). We give a practical algorithm calculating the X,Yth suboptimal optimal identified rate as Algorithm 1Algorithm 1:Finding a suboptimal optimal identified rate.
Input: Given positive real numbersδ and ε.Output:X,Yth suboptimal βi∗xt,i=1,2.Step 1: Initializeh=1, λ∗0t=1.Step 2: Initializek=1, xh,0t=x0.Step 3: Calculateh,kth state variable xh,kt according to equation (34).Step 4: Ifxh,kt−xh,k−1t<δ, let x∗ht=xh,kt; otherwise, let k=k+1 and jump to Step 3.Step 5: Initializek=1, λh,0t=1.Step 6: Calculateh,kth adjoint variable λh,kt according to equation (33).Step 7: Ifλh,kt−λh,k−1t<δ, let λ∗ht=λh,kt; Otherwise, let k=k+1 and jump to Step 6.Step 8: LettingX=h, calculate .. according to the following equation:JH=1−xtf+∫0tfq1β1∗H+q2β2∗HdtStep 9: IfJX−JX−1/JX<ε stop the algorithm and calculate the suboptimal βit,i=1,2 according to equation (32); otherwise, let h=h+1 and jump to Step 2.


## 5. Simulation Examples

In the Prisoner’s Dilemma with payoff matrix (1), let tf=5, T=5,R=3,P=0,S=−4. The replicator dynamic with TBC is(48)x˙t=3β1t+β2t−1xt−41−β2tβ1t1−xtxt,t∈0,5.The replicator dynamic without TBC is(49)x˙′t=2x′t−21−x′tx′t,t∈0,5.By simulation, we study the following three problems.


### 5.1. Effect Analysis of TBC

Letx0=0.4, β=β1t=β2t=0.7 and 0.9, and t∈0,5, respectively. In the previous discussion, we denote fCx,β1,β2 as the payoff of the cooperator with TBC, denote fDx,β1,β2 as the payoff of the defector with TBC, denote gCx as the payoff of the cooperator without TBC, and denote gDx as the payoff of the defector without TBC, denoting the group’s expected payoff without TBC as Hx and with TBC as H′x,β1,β2. In Figure 1, to simplify legends and discuss the effect of β on promoting cooperation, we redefine the symbol as follows: at t=t0, let H′β and fiβ be the payoffs of group and individual i with TBC β form t=0 to t=t0, respectively, Hβ and giβ be obtained after TBC has just been withdrawn, i.e., TBC β is used from t=0 to t=t0−τ, where τ is positive and small enough, xβ is the cooperation rate with TBC β, gi is the payoff function curve of individual i, H is the payoff function curve of the group, and x is the cooperation rate curve without TBC, respectively, where i=C,D.Figure 1
(a). The payoffs of the cooperator. (b). The payoffs of the defector. (c). The payoff of the group. (d). Evolution of the cooperation under TBC.
(a)(b)(c)(d)Based on Figures1(a) and 1(b), from the curves of gC and gD, we can know the payoffs of the cooperator and defector are all decreasing and cooperation fails to evolve without TBC. Comparing Figures 1(a) and 1(d), we can see that cooperators obtain more payoff than without TBC, but cooperation rate x0.7 decreases and cooperators are expelled still because the identified rate is low when β=0.7. According to the parameters in this example, x0=0.4 is in the basin of attraction of cooperation only when β>11/14. So, as a result, payoff of cooperator decreases and cooperation fails to evolve. When β=0.9, payoff of cooperators increases and cooperation evolves successfully; payoff of defectors increases but defection fails to evolve.Here, we try to show what will happen if we withdraw TBC for a short time. First of all, we emphasize the symbolsH′0.9, H0.9, and H of Figure 1(c). For a given time t0, H0.9 and H′0.9 all correspond to the cooperation rate xt0 defined by the replicator dynamics (48) with TBC β=0.9, but H corresponds to the cooperation rate x′t0 defined by the replicator dynamics without TBC (49). As shown in Figure 1(b), gDβ<fDβ for any identified rate β. When β=0.7, condition (26) is not satisfied and x=0.861, so the point x0=0.4 is not in the basin of attraction of cooperation and fC0.7>gC0.7. But, by designing a large identified rate β=0.9, we obtain the basin of attraction of cooperation is 0.154,1 and x1=0.861. When t>2.213, one gets x>0.861 and fC0.9>gC0.9 so that TBC inhibits the growth rate of the cooperator’s payoff. If we withdraw TBC for a short time, cooperators will obtain more payoffs.Combined with the evolution of cooperation, we come to the following conclusion:when the cooperation ratextis low, TBC results in the payoff for all individuals being higher than the payoff without TBC; when the cooperation ratextis large enough, the payoff for all individuals will be lower than the payoff without TBC under samext. On one hand, Figures 1(b) and 1(c) show that TBC decreases not only the defector’s payoff but also the group’s payoff. Furthermore, the higher the cooperation rate is, the more obvious the inhibition is. On the other hand, we observe the positive effects of TBC on promoting both the cooperation rate and the group’s payoff. Only under a bigger βit,i=1,2 does the cooperation rate xt increases. For example, although the group payoff H′0.9 under TBC is always smaller than H0.9, it is always greater than H.In summary, Figure1 shows that when the group cooperation rate xt defined by (48) reaches a satisfactory value, temporary withdrawal of TBC can increase the payoff of all individuals in the group for a short time. However, if without TBC for a long time, the group cooperation rate βit,i=1,2 defined by (49) will drop to a very low level and all individual benefits will decrease.


### 5.2. Analysis of Rationality of the Optimal Identified Rate

Lettingx0=0.4, q1=q2=q, we obtain the optimal identified rate βi∗t,i=1,2 under different q as Figure 2 and the optimal state path as Figure 3.Figure 2
The optimal identified rateβ∗t under differentq.
(a)(b)Figure 3
Cooperation ratex∗t withβ∗t.Given a sufficiently large constantΓ>0 and a sufficiently small constant γ>0, if uit>Γ, we define βit=0, and if 0<uit<γ, we define βit=1, where i=1,2. Figure 2 shows that βi∗t,i=1,2 approximates bang-bang control with switching time tis. If t<tis, then βi∗t=1; otherwise, βi∗t=0. The smaller q corresponds to the bigger switching time. If qi is large enough, then βi∗t=0 and x∗t=x0,t∈0,tf.Now, we explainβi∗t=0. According to the definition of TBC, βi∗t=0 means all cooperators are labeled as “defectors” and all defectors are labeled as “cooperators”. This is impossible and inefficient. Here, βi∗t=0 can be understood as no effective TBC; all individuals choose “defection” strategy for self-protection. In this case, the control cost is 0. Under βit=0, the cooperation rate xt is a constant value. By designing the bang-bang control as Figure 2, we can achieve the goal of successfully promoting the evolution of cooperation. This meets the requirement of minimizing the performance index (30). This is, however, unreasonable for the group in terms of revenue. Probably, the more reasonable approach would be to withdraw TBC when the cooperation rate xt is satisfactory. Now, let us discuss the TBC design from the perspective of revenue.Figure3 shows that there is a suitable TBC to make the cooperation evolve successfully. Furthermore, the smaller q results in the better effect of promoting cooperation.By designingβi∗t defined by Theorem 1, the corresponding expressions of the total payoff of the cooperator, defector, and group is ∫053β12t+4β1t1−β2txt−4β1t1−β2tdt, ∫055β1t1−β2txtdt, and ∫05β1t3β1t+β2t−1x2t+β1t1−β2txtdt, respectively. If we do not design the TBC, from t=0 to t=5, the total payoffs of the cooperator, defector, and group can be expressed as ∫057x′t−4dt, ∫055x′tdt, and ∫052x′2t+x′tdt, respectively, where x′t is described by (49). Let q1=q2=0.1 and q1=0.1,q2=0.05, respectively. The calculation results are shown in Table 1.Table 1
Payoffs of the cooperator, defector, and group under different.qi
Cooperator’s payoffDefector’s payoffGroup’s payoffβi∗t with q1=q2=0.14.58103.618βi∗t with q1=0.1,q2=0.055.11104.203No TBC−18.9930.7190.208Table1 shows that βi∗t improves the cooperator’s payoff. Furthermore, as qi decreases, the switching time tis moves backwards and the cooperator’s payoff increases.


### 5.3. Optimal Timing Control (OTC) of Switched Replicator Dynamics

System (48) and (49) is a switched nonlinear system. In the past decades, switched systems have received significant research attention, see, e.g., [46–48]. In this paper, we design the switching control (SC) as follows: for a given q1 and q2, let ts=t1s=t2s, and we design TBC with the identified rate as(50)βit=1,t∈0,ts,i=1,2.However, whent>ts, we give up the TBC. The replicator dynamics with SC is as follows:(51)x˙″t=31−x″tx′′2t,t∈0,ts,x″t−21−x″tx″t,t∈ts,5,x″0=x0,limt⟶ts−x″t=x″ts.Assumingx0=0.4, the switching time ts is decided by the optimal identified rate problem (29)–(31) according to different weight coefficients qi. Based on a switched system (51), the total payoffs of the cooperator, defector, and group can be expressed as ∫0ts3x″tdt+∫ts57x″t−4dt, ∫ts55x″tdt, and ∫0ts3x′′2tdt+∫ts52x′′2t+x″tdt, respectively. The simulation results are shown in Table 2, and the state path with SC is shown in Figure 4.Table 2
The payoff of the cooperator, defector, and group with SC.
Cooperator’s payoffDefector’s payoffGroup’s payoffSC withq1=0.1,q2=0.24.5718.6108.111SC withq1=0.1,q2=0.057.9164.8587.067Figure 4
Cooperation ratext with SC.By comparing Tables1 and 2, we find the impact of control on the average payoff of a group is complex. Table 1 shows that the smaller coefficient qi corresponds to a large group’s payoff under optimal TBC, but Table 2 shows the opposite result. Combining with Figures 3 and 4, the group has a larger average payoff under SC, but the cooperation rate is lower at t=tf=5.So, how to choose an appropriate control, TBC or SC, and how to choose the switching timets depend on what we pay more attention to.Now, we design the cost functionJSC with OTC as follows:(52)JSC=x″5−2qts+q′∫0ts3x″tdt+∫ts57x″t−4dt,where x5 is the terminal cooperation rate, 2ts is the control cost, ∫0ts3x″tdt+∫ts57x″t−4dt is the cooperator’s average payoff, and q>0 is its weight coefficient. We search the optimal switching time ts to maximize the function (52), i.e., we design an OTC for promoting evolution of the Prisoner’s Dilemma and increasing the cooperator’s average payoff.Theorem 2.
Consider the OTC described by the switched replicator dynamic (51) with performance index (52). φ1ts and φ2t,ts are solutions to algebra equation (53) and implicit function (54), respectively:(53)φ1ts1−φ1tse−1/φ1ts=x01−x0e−1/x0e3ts,(54)12−φ2t,tsφ2t,ts=1+1−φ1ts22−φ2t,tsφ1tse2t−ts,t∈ts,5.
The optimal switching timets is decided by(55)dJSCdts=dφ25,tsdts−2q′−4qφ1ts+7q∫ts5∂φ2t,ts∂tsdt=0.Proof.
The following two implicit functions can be obtained from the switched replicator dynamics (51):(56)x2t1−x2te−1/x2t=x01−x0e−1/x0e3t,t∈0,ts,(57)12−x2tx2t=1+1−x2ts22−x2tsx2tse2t−ts,t∈ts,5.
Based on existence theorem of an implicit function, we getx2t=φ1t,t∈0,tsφ2t,ts,t∈ts,5 from (56) and (57). So, φ1t,t∈0,ts satisfies (53) and φ2t,ts,t∈ts,5 satisfies (54). The cost function xt+yt=1 can be rewritten as follows:(58)JSC=φ25,ts−2qts+q′∫0ts3φ1tdt+∫ts57φ2t,tsdt.
Becaused∫0tsφ1tdt/dts=φ1ts, d∫ts5φ2t,tsdt/dts=−φ2ts,ts+∫ts5∂φ2t,ts/∂tsdt, and φ2ts,ts=φ1ts, the optimal switching time ts is decided by (55).
Lettingq′=0.1, q1=0,0.01,0.1 and 1, respectively, we get the cooperation rate curve as in Figure 5 and the payoffs of the cooperator, defector, and group as in Table 3 by applying Theorem 2.
Figure5 shows that the cooperation rate xt is very high whenever the weight coefficient of the cooperator’s payoff q=0,0.01,0.1 or q=1 although the switching time ts changes (ts varies from 2.472 to 2.967). It is because we choose a large coefficient of cooperation rate at terminal time x5. In fact, the main goal of this paper is to promote the evolution of cooperation by designing TBC. So, we design the cost function JSC to promote the cooperation rate and to reconcile control cost and the cooperator’s payoff. Observing Table 3, we note that the switching time ts increases as the cooperator’s payoff weight coefficient q gets bigger.
The OTC has the following advantages when Table3 is compared to Tables 1 and 2. Firstly, it has the same advantage as TBC: it restrains the payoffs of defectors. Secondly, it greatly enhances the payoffs of cooperators compared to TBC. Thirdly, it is the optimal way to promote the payoffs not only for cooperators but also for groups. Lastly, it can effectively promote the cooperation rate xt.Figure 5
Cooperation ratext with OTC.Table 3
Switching time and payoffs of individuals under OTC.
Switching timetsCooperator’s payoffDefector’s payoffGroup’s payoffq=02.47213.18012.40913.275q=0.012.48813.20112.34313.289q=0.12.59713.30812.87813.364q=12.96713.45810.13513.471


## 5.1. Effect Analysis of TBC

Letx0=0.4, β=β1t=β2t=0.7 and 0.9, and t∈0,5, respectively. In the previous discussion, we denote fCx,β1,β2 as the payoff of the cooperator with TBC, denote fDx,β1,β2 as the payoff of the defector with TBC, denote gCx as the payoff of the cooperator without TBC, and denote gDx as the payoff of the defector without TBC, denoting the group’s expected payoff without TBC as Hx and with TBC as H′x,β1,β2. In Figure 1, to simplify legends and discuss the effect of β on promoting cooperation, we redefine the symbol as follows: at t=t0, let H′β and fiβ be the payoffs of group and individual i with TBC β form t=0 to t=t0, respectively, Hβ and giβ be obtained after TBC has just been withdrawn, i.e., TBC β is used from t=0 to t=t0−τ, where τ is positive and small enough, xβ is the cooperation rate with TBC β, gi is the payoff function curve of individual i, H is the payoff function curve of the group, and x is the cooperation rate curve without TBC, respectively, where i=C,D.Figure 1
(a). The payoffs of the cooperator. (b). The payoffs of the defector. (c). The payoff of the group. (d). Evolution of the cooperation under TBC.
(a)(b)(c)(d)Based on Figures1(a) and 1(b), from the curves of gC and gD, we can know the payoffs of the cooperator and defector are all decreasing and cooperation fails to evolve without TBC. Comparing Figures 1(a) and 1(d), we can see that cooperators obtain more payoff than without TBC, but cooperation rate x0.7 decreases and cooperators are expelled still because the identified rate is low when β=0.7. According to the parameters in this example, x0=0.4 is in the basin of attraction of cooperation only when β>11/14. So, as a result, payoff of cooperator decreases and cooperation fails to evolve. When β=0.9, payoff of cooperators increases and cooperation evolves successfully; payoff of defectors increases but defection fails to evolve.Here, we try to show what will happen if we withdraw TBC for a short time. First of all, we emphasize the symbolsH′0.9, H0.9, and H of Figure 1(c). For a given time t0, H0.9 and H′0.9 all correspond to the cooperation rate xt0 defined by the replicator dynamics (48) with TBC β=0.9, but H corresponds to the cooperation rate x′t0 defined by the replicator dynamics without TBC (49). As shown in Figure 1(b), gDβ<fDβ for any identified rate β. When β=0.7, condition (26) is not satisfied and x=0.861, so the point x0=0.4 is not in the basin of attraction of cooperation and fC0.7>gC0.7. But, by designing a large identified rate β=0.9, we obtain the basin of attraction of cooperation is 0.154,1 and x1=0.861. When t>2.213, one gets x>0.861 and fC0.9>gC0.9 so that TBC inhibits the growth rate of the cooperator’s payoff. If we withdraw TBC for a short time, cooperators will obtain more payoffs.Combined with the evolution of cooperation, we come to the following conclusion:when the cooperation ratextis low, TBC results in the payoff for all individuals being higher than the payoff without TBC; when the cooperation ratextis large enough, the payoff for all individuals will be lower than the payoff without TBC under samext. On one hand, Figures 1(b) and 1(c) show that TBC decreases not only the defector’s payoff but also the group’s payoff. Furthermore, the higher the cooperation rate is, the more obvious the inhibition is. On the other hand, we observe the positive effects of TBC on promoting both the cooperation rate and the group’s payoff. Only under a bigger βit,i=1,2 does the cooperation rate xt increases. For example, although the group payoff H′0.9 under TBC is always smaller than H0.9, it is always greater than H.In summary, Figure1 shows that when the group cooperation rate xt defined by (48) reaches a satisfactory value, temporary withdrawal of TBC can increase the payoff of all individuals in the group for a short time. However, if without TBC for a long time, the group cooperation rate βit,i=1,2 defined by (49) will drop to a very low level and all individual benefits will decrease.


## 5.2. Analysis of Rationality of the Optimal Identified Rate

Lettingx0=0.4, q1=q2=q, we obtain the optimal identified rate βi∗t,i=1,2 under different q as Figure 2 and the optimal state path as Figure 3.Figure 2
The optimal identified rateβ∗t under differentq.
(a)(b)Figure 3
Cooperation ratex∗t withβ∗t.Given a sufficiently large constantΓ>0 and a sufficiently small constant γ>0, if uit>Γ, we define βit=0, and if 0<uit<γ, we define βit=1, where i=1,2. Figure 2 shows that βi∗t,i=1,2 approximates bang-bang control with switching time tis. If t<tis, then βi∗t=1; otherwise, βi∗t=0. The smaller q corresponds to the bigger switching time. If qi is large enough, then βi∗t=0 and x∗t=x0,t∈0,tf.Now, we explainβi∗t=0. According to the definition of TBC, βi∗t=0 means all cooperators are labeled as “defectors” and all defectors are labeled as “cooperators”. This is impossible and inefficient. Here, βi∗t=0 can be understood as no effective TBC; all individuals choose “defection” strategy for self-protection. In this case, the control cost is 0. Under βit=0, the cooperation rate xt is a constant value. By designing the bang-bang control as Figure 2, we can achieve the goal of successfully promoting the evolution of cooperation. This meets the requirement of minimizing the performance index (30). This is, however, unreasonable for the group in terms of revenue. Probably, the more reasonable approach would be to withdraw TBC when the cooperation rate xt is satisfactory. Now, let us discuss the TBC design from the perspective of revenue.Figure3 shows that there is a suitable TBC to make the cooperation evolve successfully. Furthermore, the smaller q results in the better effect of promoting cooperation.By designingβi∗t defined by Theorem 1, the corresponding expressions of the total payoff of the cooperator, defector, and group is ∫053β12t+4β1t1−β2txt−4β1t1−β2tdt, ∫055β1t1−β2txtdt, and ∫05β1t3β1t+β2t−1x2t+β1t1−β2txtdt, respectively. If we do not design the TBC, from t=0 to t=5, the total payoffs of the cooperator, defector, and group can be expressed as ∫057x′t−4dt, ∫055x′tdt, and ∫052x′2t+x′tdt, respectively, where x′t is described by (49). Let q1=q2=0.1 and q1=0.1,q2=0.05, respectively. The calculation results are shown in Table 1.Table 1
Payoffs of the cooperator, defector, and group under different.qi
Cooperator’s payoffDefector’s payoffGroup’s payoffβi∗t with q1=q2=0.14.58103.618βi∗t with q1=0.1,q2=0.055.11104.203No TBC−18.9930.7190.208Table1 shows that βi∗t improves the cooperator’s payoff. Furthermore, as qi decreases, the switching time tis moves backwards and the cooperator’s payoff increases.


## 5.3. Optimal Timing Control (OTC) of Switched Replicator Dynamics

System (48) and (49) is a switched nonlinear system. In the past decades, switched systems have received significant research attention, see, e.g., [46–48]. In this paper, we design the switching control (SC) as follows: for a given q1 and q2, let ts=t1s=t2s, and we design TBC with the identified rate as(50)βit=1,t∈0,ts,i=1,2.However, whent>ts, we give up the TBC. The replicator dynamics with SC is as follows:(51)x˙″t=31−x″tx′′2t,t∈0,ts,x″t−21−x″tx″t,t∈ts,5,x″0=x0,limt⟶ts−x″t=x″ts.Assumingx0=0.4, the switching time ts is decided by the optimal identified rate problem (29)–(31) according to different weight coefficients qi. Based on a switched system (51), the total payoffs of the cooperator, defector, and group can be expressed as ∫0ts3x″tdt+∫ts57x″t−4dt, ∫ts55x″tdt, and ∫0ts3x′′2tdt+∫ts52x′′2t+x″tdt, respectively. The simulation results are shown in Table 2, and the state path with SC is shown in Figure 4.Table 2
The payoff of the cooperator, defector, and group with SC.
Cooperator’s payoffDefector’s payoffGroup’s payoffSC withq1=0.1,q2=0.24.5718.6108.111SC withq1=0.1,q2=0.057.9164.8587.067Figure 4
Cooperation ratext with SC.By comparing Tables1 and 2, we find the impact of control on the average payoff of a group is complex. Table 1 shows that the smaller coefficient qi corresponds to a large group’s payoff under optimal TBC, but Table 2 shows the opposite result. Combining with Figures 3 and 4, the group has a larger average payoff under SC, but the cooperation rate is lower at t=tf=5.So, how to choose an appropriate control, TBC or SC, and how to choose the switching timets depend on what we pay more attention to.Now, we design the cost functionJSC with OTC as follows:(52)JSC=x″5−2qts+q′∫0ts3x″tdt+∫ts57x″t−4dt,where x5 is the terminal cooperation rate, 2ts is the control cost, ∫0ts3x″tdt+∫ts57x″t−4dt is the cooperator’s average payoff, and q>0 is its weight coefficient. We search the optimal switching time ts to maximize the function (52), i.e., we design an OTC for promoting evolution of the Prisoner’s Dilemma and increasing the cooperator’s average payoff.Theorem 2.
Consider the OTC described by the switched replicator dynamic (51) with performance index (52). φ1ts and φ2t,ts are solutions to algebra equation (53) and implicit function (54), respectively:(53)φ1ts1−φ1tse−1/φ1ts=x01−x0e−1/x0e3ts,(54)12−φ2t,tsφ2t,ts=1+1−φ1ts22−φ2t,tsφ1tse2t−ts,t∈ts,5.
The optimal switching timets is decided by(55)dJSCdts=dφ25,tsdts−2q′−4qφ1ts+7q∫ts5∂φ2t,ts∂tsdt=0.Proof.
The following two implicit functions can be obtained from the switched replicator dynamics (51):(56)x2t1−x2te−1/x2t=x01−x0e−1/x0e3t,t∈0,ts,(57)12−x2tx2t=1+1−x2ts22−x2tsx2tse2t−ts,t∈ts,5.
Based on existence theorem of an implicit function, we getx2t=φ1t,t∈0,tsφ2t,ts,t∈ts,5 from (56) and (57). So, φ1t,t∈0,ts satisfies (53) and φ2t,ts,t∈ts,5 satisfies (54). The cost function xt+yt=1 can be rewritten as follows:(58)JSC=φ25,ts−2qts+q′∫0ts3φ1tdt+∫ts57φ2t,tsdt.
Becaused∫0tsφ1tdt/dts=φ1ts, d∫ts5φ2t,tsdt/dts=−φ2ts,ts+∫ts5∂φ2t,ts/∂tsdt, and φ2ts,ts=φ1ts, the optimal switching time ts is decided by (55).
Lettingq′=0.1, q1=0,0.01,0.1 and 1, respectively, we get the cooperation rate curve as in Figure 5 and the payoffs of the cooperator, defector, and group as in Table 3 by applying Theorem 2.
Figure5 shows that the cooperation rate xt is very high whenever the weight coefficient of the cooperator’s payoff q=0,0.01,0.1 or q=1 although the switching time ts changes (ts varies from 2.472 to 2.967). It is because we choose a large coefficient of cooperation rate at terminal time x5. In fact, the main goal of this paper is to promote the evolution of cooperation by designing TBC. So, we design the cost function JSC to promote the cooperation rate and to reconcile control cost and the cooperator’s payoff. Observing Table 3, we note that the switching time ts increases as the cooperator’s payoff weight coefficient q gets bigger.
The OTC has the following advantages when Table3 is compared to Tables 1 and 2. Firstly, it has the same advantage as TBC: it restrains the payoffs of defectors. Secondly, it greatly enhances the payoffs of cooperators compared to TBC. Thirdly, it is the optimal way to promote the payoffs not only for cooperators but also for groups. Lastly, it can effectively promote the cooperation rate xt.Figure 5
Cooperation ratext with OTC.Table 3
Switching time and payoffs of individuals under OTC.
Switching timetsCooperator’s payoffDefector’s payoffGroup’s payoffq=02.47213.18012.40913.275q=0.012.48813.20112.34313.289q=0.12.59713.30812.87813.364q=12.96713.45810.13513.471


## 6. Conclusions

The analysis in this paper provides value insights into a number of issues regarding some clusters in the economic society when faced with the Prisoner’s Dilemma, which can help operators of clusters decide to adopt TBC or not, identify cooperators or identify defectors, and design the value of the identified rate.First, operators should judge the harm of defection in the Prisoner’s Dilemma. We show that when the cooperation rate is high, to further increase cooperation rate, a large identified rate is needed. Thus, it is difficult to design a TBC to further promote cooperation and further improve the cooperator’s payoff. Furthermore, the cluster must pay for the TBC. So, it is not necessary to use TBC for a cluster when the cooperation rate is high. If operator considers TBC is necessary, we advice he/she depends more on the cooperation identification.By contrast, if the cooperation rate is too low, i.e., the group is almost full of defectors, it is difficult to design a TBC which can help cooperators invade the group of defectors successfully. In this case, any TBC all can increase the cooperator’s payoff. But, a large defection identified rate is essential to promote the cooperation rate. It is very important to note that defection identification is more effective than cooperation identification.We show the switch between adopting TBC and withdrawing TBC is necessary. TBC has an advantage in promoting evolution of cooperation. However, when the cooperation rate is high, TBC reduces the payoffs of the cooperators unless the identified rate is high enough. This paper gives OTC as a way to design switching law. The simulation results confirm OTC not only maintains a high cooperation rate but also increases the payoff of the cooperators.While this paper is a significant step in the economic analysis of the Prisoner’s Dilemma and in the optimal control of promoting cooperation, there are a number of interesting directions for future work in this area. There are a number of potential interesting extensions of our model through relaxation of some of our assumptions. For example, we assume the payoff of mutual defection is0 in this paper, i.e., P=0. In fact, we can assume an individual is exclusive [49] and every individual only accepts an encounter-tagged “cooperator”.Last but not the least, this paper does not really design an SC law for the switched replicator dynamics; the SC designed in this paper is only an OTC with one switching time. Further discussions for promoting cooperation and increasing the cooperator’s average payoff will aim to design an OTC with multiple switching times and general SC unrestrained by bang-bang control. Furthermore, the optimal tag-based cooperation control proposed in this paper can be applied in management practice, such as in e-commerce platforms. Because of information asymmetry about product quality in e-commerce platforms, there exists the PDG in seller groups and consumer groups. Platform manager can design tag-based cooperation control by offering an authentication service. A valuable extension of this research is to empirically examine the feasibility of TBC proposed in this paper.



---

*Source: 8498613-2020-12-11.xml*