On-line, On-board Evolution of Reaction-Diffusion Control for Self-Adaptation

Evolutionary robotics (ER) aims to automatically develop sensor–actuator control of agents by using techniques of ar tificial evolution. The first review article in the field was published in the mid nineties by Matarić and Cliff (1996). In their article they are clear about a list of fundamental problems arising when using ER in simulation or in the real world. Unfortunately, 16 years after tackling the problems concisely, their hope expressed in the very last sentence of the paper did not fully become true yet: “If the challenges can be successfully addressed, the use of evolutionary tech niques may become a viable alternative to manual design. ” Several of these challenges were, however, treated in-dept h. Concerning the methodology of using a simulation environment to develop a controller followed by transferring it to the robot, the reality gap problem was addressed. One of the methods of how to bridge the reality gap was proposed by Jakobi (1997). The idea is to mask non relevant aspects of the environment by noise. When using real robot hardware the problem of keeping the robots’ autonomy for long periods arises. A technical solution for energy autonomy was proposed by Watson et al. (2002) in the form of electrified floors. Obviously this method is not applicable outside of the lab. However, an environment with unforeseen conditions is meant to be the actual operation ground of ER in the first place. A broader overview of methods which address the challenges reported by Matarić and Cliff (1996) can be found, for example, in Nolfi and Floreano (2000). One of the major goals of the projects SYMBRION and REPLICATOR is to create controllers for modular robots which autonomously dock to form robot organisms. Due to the combinatorial explosion of possible robot organism configurations the robot controllers cannot be predefined for particular organism shapes. Therefore, we are constrained to an approach of ER, which proved to be a promising method in the last years: on-line evolution, whereas we rely on the definitions of this category established by Watson et al. (2002) and Eiben et al. (in Levi and Kernbach (2010), ch. 5.2). The idea is to tune the controller of the robot while it is actively trying to achieve the given objective. Thus, environmental changes or changes in the task are immediately incorporated into the adaptation process of the robot controller. The method of on-line, on-board ER is combined with a bio-inspired controller called AHHS (Artificial Homeostatic Hormone Controller), see Schmickl et al. (2011); Hamann et al. (2012). AHHS was designed for high evolvability in multi-modular robotics. To our knowledge, the presented results represent the first investigations of usi ng a reaction-diffusion based robot controller in an on-line, o nboard experiment. We report experiments using both a real robot and a simulator which represents the applied robot in detail. The used simulation environment is Robot3D and the used hardware platform is the ‘backbone robot’ (Levi and Kernbach, 2010) from our projects. The number of evaluations was reduced from 2500 in simulation to 600 in the robot experiments. The complexity of the investigated task is limited because this is a first case study of our on-line, on-board evolution approach using AHHS on hardware. By restricting the actuator control values we limit the robot’s DOF to one: motion back and forth between two walls. We use a single input s from a front proximity sensor which is scaled to s ∈ [0, 1]. The fitness functionF of the task is