# Self-organization Promotes the Evolution of Cooperation with Cultural Propagation

Luis Enrique Cortés-Berrueco, Carlos Gershenson, Christopher R. Stephens

► To cite this version:

Luis Enrique Cortés-Berrueco, Carlos Gershenson, Christopher R. Stephens. Self-organization Promotes the Evolution of Cooperation with Cultural Propagation. 7th International Workshop on Self-Organizing Systems (IWSOS), May 2013, Palma de Mallorca, Spain. pp.145-150, 10.1007/978-3-642-54140-7_13. hal-01291511

HAL Id: hal-01291511

https://inria.hal.science/hal-01291511

Submitted on 21 Mar 2016

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Self-Organization Promotes the Evolution of Cooperation with Cultural Propagation

Luis Enrique Cortés-Berrueco<fnref n="1" />, Carlos Gershenson<fnref n="2" />,<fnref n="4" />, and Christopher R. Stephens<fnref n="3" />,<fnref n="4" />

**Abstract.** In this paper three computational models for the study of the evolution of cooperation under cultural propagation are studied: Kin Selection, Direct Reciprocity and Indirect Reciprocity. Two analyzes are reported, one comparing their behavior between them and a second one identifying the impact that different parameters have in the model dynamics. The results of these analyzes illustrate how game transitions may occur depending of some parameters within the models and also explain how agents adapt to these transitions by individually choosing their attachment to a cooperative attitude. These parameters regulate how cooperation can self-organize under different circumstances. The emergence of the evolution of cooperation as a result of the agent's adapting processes is also discussed.

## 1 Introduction

Urban traffic problems have a complex behavior even in their most simplistic abstractions [2]. This behavior becomes more complex when driver interaction is added [10]. With these two premises, in this paper, three computational models, which were originally intended for studying driving behaviors and their impact in traffic, are presented. The aim is to explore the possibilities for controlling some of the complex characteristics of traffic, e.g. the consequences of driver interactions. The possibility of using them for other social problems is also discussed.

The models were conceived to study the evolution of cooperation [1], which studies how cooperative behaviors are preserved when selfish behaviors offer greater individual rewards, e.g. drivers who yield at lane changes or crossings, animal females taking care of their offspring, students who don't betray each other. Several

<footnote n="1">¹ Graduate Program in Computer Science and Engineering, Universidad Nacional Autónoma de México, Ciudad Universitaria, A.P.70-600, México D.F., México
[lecortesb@comunidad.unam.mx](mailto:lecortesb@comunidad.unam.mx)</footnote>

<footnote n="2">² Computer Science Department, Instituto de Investigaciones en Matemáticas Aplicadas y en Sistemas, Universidad Nacional Autónoma de México, Ciudad Universitaria, A.P.20-726, México D.F., México
[cgg@unam.mx](mailto:cgg@unam.mx)</footnote>

<footnote n="3">³ Gravitation Department, Instituto de Ciencias Nucleares, Universidad Nacional Autónoma de México, Ciudad Universitaria, A.P.70-543, México D.F., México
[stephens@nucleares.unam.mx](mailto:stephens@nucleares.unam.mx)</footnote>

<footnote n="4">⁴ Centro de Ciencias de la Complejidad, Universidad Nacional Autónoma de México</footnote>
abstractions for simulating different circumstances in which this phenomenon occurs have been proposed [6]. The models are based in three abstractions of different observed situations in which cooperation evolves [4]. Table 1 shows these abstractions as theoretical games in which players may choose between two behaviors: cooperative or selfish. When a player chooses to cooperate she has to pay a cost that her game partner is going to receive as a benefit. In each game, the decision of the players is influenced by a different probabilistic variable. This variable represents some feature that is exploited by the global behavior to favor the evolution of cooperation. For the kin selection strategy, relationships are exploited, the relation may be genetic, emotional or of any other type but as long as it is closer, cooperation will be more frequent. In the direct reciprocity case, the evolution of cooperation is linked with the probability of one player to play again with the same partner, and this condition is set because players are responding cooperation with cooperation and defection with defection. The feature exploited by the indirect reciprocity case is peer pressure. While more players get to know the actions of a determined player, she will obtain more benefits by cooperating and while fewer players get to know her actions more benefits she will obtain by defecting. Each strategy has a condition for the cooperation to become an evolutionary stable strategy (ESS, cooperators survive), another when cooperation becomes risk-dominant (RD, cooperators are near 1/2 of the population) and other when cooperators are advantageous (AD, cooperators are near 2/3 of the population).

<table><caption>**Table 1.** Rules for the three game strategies [4].</caption><thead><tr><th>Strategy</th><th colspan="2">Payoff table</th><th>ESS</th><th>RD</th><th>AD</th><th>Variables</th></tr></thead><tbody><tr><td rowspan="3">Kin Selection</td><td>C</td><td>D</td><td>b &gt; 1</td><td>b &gt; 1</td><td>b &gt; 1</td><td>b=benefit</td></tr><tr><td>C<br>(b-c)(1+r)</td><td>br-c</td><td>c &gt; r</td><td>c &gt; r</td><td>c &gt; r</td><td>c=cost<br>r=relatedness<br>probability</td></tr><tr><td>D<br>b-rc</td><td>0</td><td></td><td></td><td></td><td></td></tr><tr><td rowspan="3">Direct<br>Reciprocity</td><td>C</td><td>D</td><td>b &gt; 1</td><td>b &gt; 2-w</td><td>b &gt; 3-2w</td><td>b=benefit<br>c=cost</td></tr><tr><td>C<br>(b-c)/(1-w)</td><td>-c</td><td>c &gt; w</td><td>c &gt; w</td><td>c &gt; w</td><td>w=probability of<br>next round</td></tr><tr><td>D<br>b</td><td>0</td><td></td><td></td><td></td><td></td></tr><tr><td rowspan="3">Indirect<br>Reciprocity</td><td>C</td><td>D</td><td>b &gt; 1</td><td>b &gt; 2-q</td><td>b &gt; 3-2q</td><td>b=benefit<br>c=cost</td></tr><tr><td>C<br>b-c</td><td>-c(1-q)</td><td>c &gt; q</td><td>c &gt; q</td><td>c &gt; q</td><td>q=social<br>acquaintanceship</td></tr><tr><td>D<br>c(1-q)</td><td>0</td><td></td><td></td><td></td><td></td></tr></tbody></table>

## 2 The Models

We developed agent-based computational models, using the NetLogo [12] platform, to better understand three strategies for the evolution of cooperation: kin selection, direct reciprocity and indirect reciprocity [4]. Unlike the results shown in [4] these models are focused in cultural evolution (players may choose to cooperate or not under different circumstances instead of been born as players who always cooperate or always defect), therefore, the first characteristic these models share is that they have a constant population. The behavior of the agents is similar to the one of agents
described in Skyrm's Matching Pennies model [8]. Agents have a cooperation probability variable (cp) that determines their attachment to a cooperator strategy, while this variable gets a greater value the agent will cooperate more frequently. We will use this variable for deciding if the agents will cooperate or not in a particular game. Each agent is able to identify herself as a cooperator or as a defector:

$$ \text{If } cp(p_i) > 0.5 \text{ then } c_i = p_i \text{ and } d_i = 0 $$

$$ \text{If } cp(p_i) \le 0.5 \text{ then } d_i = p_i \text{ and } c_i = 0 $$

Where $p_i$ is the i-th agent or player, $c_i$ is the i-th cooperator and $d_i$ is the i-th defector.

As a consequence of this characteristic, a partition of the population can be made. The initial proportions of cooperative versus defective agents in the population (ipc and ipd) are relevant parameters of the model although they can be reduced to one as $ipc=1-ipd$. In order to identify agents as cooperators or defectors at the beginning of the simulations we give them an initial cp, thus, initial cp of defectors (iicpd) and initial cp of cooperators (icpc) are two more parameters. The agents move through a defined two-dimensional space. It has been seen [3] that in such cases population density is a key determinant of the dynamics, so the number of players (population) interacting in the defined space is another important parameter of the model.

Agents interact following the payoff tables corresponding to each strategy studied. As can be seen in Table 1, the three strategies share two variables: the benefit (b) one agent gives to another while cooperating and the cost (c) paid by the cooperator while cooperating. With the purpose of keeping track of the player's decisions, each one of them is assigned a fitness value from where the cost will be subtracted and the benefits will be added.

Appendix A of [13] shows the instructions executed by the agents during each iteration of the simulations. The behaviors of each game are detailed in appendix B of [13].

## 2.1 Tune cooperation probability

Agents adapt to the environment by modifying whether they are going to cooperate or defect in the next round. We implemented several exogenous tuning algorithms based on the literature [8][9][6][7][11] with no satisfactory results. We developed two self-organizing tuning alternatives and were tested with very good results:

* **Selfish fitness:** the agents only take their own fitness value as a parameter, unlike other algorithms that take into account the values of all the other players or values of the game partners. If the player cooperates or defects, and her fitness increases, then the agent increases or decreases her cp respectively. If the player cooperates or defects, and her fitness is reduced, then the agent decreases or increases her cp respectively. And if there is no change in the agent's fitness, then there is no change in her cp.
<figure><img src="image_2.png" /></figure>

<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /><figcaption>Fig. 1. Graphics of the payoff table behaviors for x's values in the range [0,1] for a) Kin Selection, b) Indirect Reciprocity and c) Direct Reciprocity (DR is in logarithmic scale).</figcaption></figure>

* **Selfish profit:** this is very similar to the previous one, but instead of comparing fitness values (sum of all profits obtained), agents act based in the comparison of the profit of their last game and the profit of the current round. Thus, if the player cooperates or defects and the actual profit is greater than the last one, then the agent increases or decreases her *cp*. If the player cooperates or defects and the actual profit is lower than the last one, then the agent decreases or increases her *cp*. Finally if the last profit is equal to the actual profit, then there is no change in the *cp* value.

## 3 Methods

Before general results were obtained, an analysis of the variables was conducted. A primary goal of this analysis was to determine the impact of the parameters on the dynamics of the models. A secondary goal was to obtain parameter values that best exemplify the model's behavior. Detailed information about the experiments may be found in appendix C of [13].

## 4 Results

Interesting results can be derived from a careful analysis of the graphics of the payoff table behavior as the corresponding probability takes values within the range [0,1] (Figure 1). As *x* has a higher value, the agents move to different games. Using the same notations as in [6] that designates R (reward) when two agents cooperate, P (punishment) when two players defect, T (temptation) when one player defects and
the other cooperates and S (sucker's payoff) when one player cooperates and the other defects; the move from game to game is described in appendix D of [13]. The values of *b* and *c* also determine the transition value of *x* that takes the agents from one game to another. It is important to notice that the agents respond to these transitions by interacting with the others and adapt by using only simple local rules with only two basic requirements: 1) the agents must know how to distinguish that an amount is bigger than other and 2) the agents must have the notion of more is better. Within the strategies studied, other requirements are implicitly given for each case and are described in the appendix E of [13].

## 4.1 ESS, RD and AD conditions

It was shown in Table 1, when the *x* variable reaches certain value, ESS, RD and AD behaviors emerge. Figure 2 shows the characteristic behavior obtained by the analysis described in the behavior part of the Methods section; in it, these expected behaviors are not noticeable. Analyzing the model variables, we could find that these conditions are preserved, but the effect can be observed by setting the starting population of cooperators as 2/3, 1/2 and 1/3 of the total population for ESS, RD and AD respectively and obtaining a graphic similar to Figure 2. This result is shown in appendix F of [13] along with the impact of all the parameters to each strategy.

<figure><img src="image_5.png" /><figcaption>Fig. 2. Percentage of cooperating population while the corresponding probability is varied.</figcaption></figure>

## 4.2 Self-organization

Self-organization seems to be a key element for the cultural evolution of cooperation. There are several studies [8][9][6][7][11] in which the agents just can choose between cooperate or defect. In the models presented here, the players can choose the degree of cooperation that they consider the best. Another important difference between our models and the others [6][7] is the inclusion of a probabilistic variable so that agents choose the degree of cooperation by comparing their actual state with a past one instead of comparing their state with the state of others. This is important
because each agent may be seen as an individual system adaptable to different envi-
ronments or contexts using the information given by the interactions with other simi-
lar, not necessarily equal, systems.

5 Conclusions

Three agent-based computational models for the study of the evolution of coopera-
tion under cultural propagation were described. It was shown that their behavior is the
result of transitions between games defined by Game Theory. The transitions are con-
sequences of the structure determined by the payoff matrixes of the three strategies
studied. Each of these strategies abstracts well-known real behaviors [4]; hence the
importance of creating computational models that let us experiment exhaustively
different circumstances for these phenomena, a difficult task with living organisms.
The impact of the parameters in each model was analyzed to better understand how to
manipulate the models and adapt them for more specific studies, such as social phe-
nomena (e.g. bulling, market formation, and migration), traffic problems (e.g. signal-
ing issues, conflicting use of roads) and biological processes (e.g. interactions be-
tween organisms and populations).

6 References

1. Axelrod, R., Hamilton, W.D.: The evolution of cooperation. Science, New Series, Vol.211, no.4489, pp. 1390-1396. (1981)

2. Nagatani, T.: Chaos and dynamical transition of a single vehicle induced by traffic light and speedup. Physica A, Vol. 348, pp. 5617-571. (2005)

3. Nagel, K., Schreckenberg, M.: A cellular automaton model for freeway traffic. Journal de Physique 1, Vol. 2, Issue 12, pp. 2221-2229. (1992)

4. Nowak, M.A.: Five rules for the evolution of cooperation. Science, Vol. 314, pp. 1560-1563. (2006)

5. Nowak, M.A.: Five rules for the evolution of cooperation supporting online material, Sci-
ence, http://www.sciencemag.org/content/314/5805/1560/suppl/DC1

6. Santos, F.C., Pacheco, J.M., Lenaerts, T.: Evolutionary dynamics of social dilemmas in structured heterogeneous populations. PNAS, Vol. 103, no. 9, pp. 3490-3494. (2006)

7. Santos, F.C., Santos, M.D., Pacheco, J.M.: Social diversity promotes the emergence of co-
operation in public goods games. Nature, Vol. 454, pp.213-216. (2008)

8. Skyrm, B.:The dynamics of rational deliberation. Cambridge MA, Harvard University Press. (1990)

9. Skyrms, B.: Dynamic models of deliberation and the theory of games. Proceeding of the 3rd conference of Theoretical aspects of reasoning about knowledge, pp. 185-200. (1990)

10. Wastavino, L.A., Toledo, B.A., Rogan, J., Zarama, R., Muñoz, V., Valdivia, J.A.: Modeling traffic on crossroads. Physica A, Vol.381, pp.411-419. (2007)

11. Weirich, P.: Computer simulations in game theory. http://philsci-archive.pitt.edu/2754/ (2006)

12. Wilensky, U. (1999). NetLogo. http://ccl.northwestern.edu/netlogo/. Center for Connected Learning and Computer-Based Modeling, Northwestern University, Evanston, IL.

13. arXiv:1302.6533 [cs.GT]