Cooperation applies the situations where two or more individuals obtain a net benefit by working together. Cooperation is widely spread in nature and takes several forms, ranging from behavioral coordination to sacrifice of one's own life for the benefit of the group. This latter form of cooperation is known as "true cooperation", or "altruism", and is found only in few cases. Truly cooperative robots would be very useful in conditions where unpredictable events may require costly actions by individual robots for the success of the mission. However, the interactions among robots sharing the same environment can affect in unexpected ways the behavior of individual robots, making very difficult the design of rules that produce stable cooperative behavior. It is thus interesting to examine under which conditions truly cooperative behavior evolves in nature and how those conditions can be translated into evolutionary algorithms that are applicable to a wide range of robotic situations. Building on earlier intuitions by Haldane (1955) , Hamilton (1964) suggested that altruism can evolve if the cooperator is genetically related to the recipient of help. In this case, even if the cooperator cannot propagate its own genes to the next generation, its altruistic act will increase the probability that a large portion of those genes will be propagated through the reproduction of the recipient of the altruistic act. The theory of kin selection (Maynard-Smith 1964), which developed from Hamilton's model, predicts that the ratio of altruistic individuals in a population is related to the degree of kinship, or genetic relatedness, among individuals. Another explanation for the evolution of altruistic cooperation is provided by the theory of group selection, which argues that altruistic cooperation may also evolve in groups of genetically unrelated individuals that are selected and reproduced together at a higher rate than the single individuals composing the group (Wynne-Edwards 1986) . This could happen in situations where the synergetic effect of cooperation by different individuals provides a higher fitness to the group with respect to other competing groups. In those situations, cooperating individuals can be seen as a superorganism that becomes the unit of selection. It has been suggested that group selection may be a driving force behind the transition from unicellular to multicellular organisms (Michod 1999). In the context of evolution experiments with robots (Floreano and Mattiussi, 2008; Nolfi and Floreano, 2000) , the hypotheses of kin selection and of group selection translate into the genetic composition of the group of robots and into the method used to select individuals for reproduction. If we consider only the extreme conditions, individuals in a team could be genetically homogeneous (clones) or heterogeneous (they differ from each other); and the fitness could be computed at the level of the team (in which case, the entire team of individuals is reproduced) or at the level of the individual (in which case, only individuals of the team are selected for reproduction). We compared the performance of genetically homogeneous teams evolved with individuallevel selection, genetically homogeneous teams evolved with team-level selection, genetically heterogeneous teams evolved with individual-level selection and genetically heterogeneous teams evolved with team-level selection (Waibel, Keller, and Floreano, 2009) . The experiments were conducted with teams of sugar-cube robots in a foraging task that could be manipulated to offer the opportunity for various degrees of cooperation (figure 1). The genomes encoded the synaptic connections of the robot neurocontrollers. The results indicated that the best combination of genetic composition and level of selection depend on the degree of cooperation requested by the task. When the task does not require costly cooperation, teams of heterogeneous individuals evolved under individual-level selection obtain the best performance. However, when the task involves a cost for the individual, genetic relatedness or team-level selection generate neural controllers with the best performance. In another set of experiments (figure 2), we compared the four combinations of genetic composition and level of selection with groups of "expressive" robots that are exposed to food and danger sources that cannot be uniquely be identified at distance (Floreano, Mitri, Magnenat, and Keller, 2007) . In these experiments, the evolutionary emergence of communication of the source type brings an advantage to the group at the expense of the individuals that decide to tell which is the food or poison. The results indicated that communication by color emerged in populations of robots that had been evolved under the condition of genetic relatedness or of team-level selection, but not in populations of robots that were genetically unrelated and evolved under individual-level selection. Finally, we turned to an application scenario where a swarm of Micro Aerial Vehicles (MAVs) are deployed in a catastrophic scenario to locate the rescuers on the ground and establish an ad hoc radio network that allows rescuers to communicate with each other (figure 3). The stringent mission requirements, which ruled out the use of Geo Positioning System and of vision sensors along with the unpredictable location of the rescuers on the ground, made it very difficult to come up with suitable control systems for the micro aerial vehicles. Consequently, we resorted to the evolutionary methods that we designed and evaluated in the previous experiments. In this situation, the fitness was proportional to the quantity of communication packets that could be exchanged among rescuers during a period of 30 minutes, which made it impossible to compute the contribution of individuals in the team. Furthermore, robots could incur a cost by searching for rescuers instead of participating to the maintenance of an already established radio network. Consequently, we resorted to teams of genetically related controllers evolved under team-level selection (Hauert, Zufferey, and Floreano, 2009) . The genomes of the robots encoded the synaptic weights of the neurocontrollers of the robots. Evolution experiments were carried out in simulations that modeled both the flight and the radio network dynamics. Evolved teams were capable of rapidly locating a rescuer located at an unknown position and maintaining a reliable voice connection with another rescuer. Current work is aimed at transferring the evolved control systems to a swarm of real MAVs and extending their use to multiple rescuers on the ground. Cooperation could be mediated by many other factors both in nature and in robotics. For example, if organisms and robots were capable of recognizing each other and memorizing the effects of their interactions, one could observe the emergence of reciprocal sharing and even of a reputation system. However, these experiments indicate that even robots whose control system cannot recognize other individuals, cannot learn, and cannot memorize events can still engage in sophisticated forms of altruistic cooperation.
At a higher level, the conflicting goals of individual versus team rewards were studied by Keller and Floreano REF , focusing on the importance of relatedness (kin selection) in robotic agents.
Methods for Artificial Evolution of Truly Cooperative Robots
IWANN
Computer Science