ALIFE 14 : Proceedings of the Fourteenth International Conference on the Synthesis and Simulation of Living Systems

The ESP method for evolving virtual creatures (Lessin et al., 2013) consisted of an encapsulation mechanism to preserve learned skills, a human-designed syllabus to build higherlevel skills by combining lower-level skills systematically, and a pandemonium mechanism to resolve conflicts between encapsulated skills in a single creature’s brain. Previous work with ESP showed that it is possible to evolve much more complex behavior than before, even when fundamental morphology (i.e., skeletal segments and joints) was evolved only for the first skill. This paper introduces a more general form of ESP in which full morphological development can continue beyond the first skill, allowing creatures to adapt their morphology to multiple tasks. This extension increases the variety and quality of evolved creature results significantly, while maintaining the original ESP system’s ability to incrementally develop complex behaviors from a sequence of simpler learning tasks. In the future, this method should make it possible to build EVCs with complex and believable behavior. Introduction Since their introduction two decades ago by Sims (1994), evolved virtual creatures (or EVCs; Figure 1) have had a significant impact on multiple fields, including graphics, evolutionary computation, artificial life, and robotics (Shim and Kim, 2003; Lehman and Stanley, 2011; Miconi, 2008; Lipson and Pollack, 2000). But despite their many incarnations in the intervening years, their behaviors have not grown more complex. Sims’ original work demonstrated light following. This level of complexity has been approximately matched multiple times since (Miconi, 2008; Pilat and Jacob, 2010), but was never clearly exceeded until the ESP system of Lessin et al. (2013). ESP was shown to construct EVC behaviors approximately twice as complex as any seen before, measured as the number of discriminable behaviors in the creature’s repertoire. Moreover, in principle there is no upper limit on behavioral complexity yet established within this new framework. The ESP system is named for its three principal components–encapsulation, syllabus, and pandemonium– defined as follows: (a) Initial locomoting creature. (b) Heavy smashing arms. (c) Smashing flail arms. (d) Jump with anti-tip limbs. (e) Smashing tail, stabilizers. (f) Jump with heavier body. Figure 1: Adapting EVC morphology to multiple tasks. (a) A creature adapted for locomotion. From this creature, creatures (b) through (f) were evolved using the extended ESP method described in this paper. Each of them has developed a new technique (with corresponding morphological changes) for accomplishing an additional task–in this case, delivering a strike to the ground–while still maintaining the ability to perform the initial skill (locomotion) to prescribed levels. In the original ESP system, these secondary adaptations would have been impossible. • Once a new skill is learned, it can be encapsulated, which preserves that ability and makes it easily accessible to future evolution. ALIFE 14: Proceedings of the Fourteenth International Conference on the Synthesis and Simulation of Living Systems • A human-designed syllabus is used to direct the sequential acquisition of these component skills toward the larger goal of achieving new, more complex behaviors, building up hierarchically from simpler ones. • A pandemonium-like mechanism (Selfridge, 1958) is employed to resolve conflicts between competing encapsulated skills in the increasingly complex brain. The initial ESP implementation did achieve its goal of breaking the behavioral-complexity barrier for EVCs. However, it applied only to a significantly restricted case–one in which some of the most important morphological changes (those to the creature’s skeletal segments and joints) were prohibited after the first skill’s evolution was complete. (Note, however, that muscle drives and photoreceptors– described below–could continue to be evolved throughout all evolutionary stages, even in the original system, since they can be added without disrupting existing abilities.) This is a serious limitation. For example, what if a creature is evolved for an initial skill such as locomotion, then is asked to adapt to a largely orthogonal skill such as reaching up to a high target? That creature may or may not have the required morphological capacity for performing the second task, as determined only by the accidents of evolution. This paper introduces a significantly extended version of ESP, in which a retesting and reconciliation scheme replaces previous absolute limitations on morphological evolution. Morphology is thus fully evolved to suit the requirements of more than just a single skill. In the following sections, this new ESP implementation is described. The results demonstrate a significant increase in the useful variety and quality of evolved creatures, while the ability to incrementally develop complex behaviors from a sequence of simpler learning tasks is maintained. The Underlying EVC Implementation Both the ESP implementation of Lessin et al. (2013)–called original ESP in this paper–and its improvement that is presented in later sections–called extended ESP–were built on a reimplementation of Sims’ original work (1994) that will be referred to as the underlying EVC implementation. That underlying EVC system–along with some important novel extensions–is described in the balance of this section, and a representative sample of its results is shown in Figure 2. Evolutionary Algorithm A conventional evolutionary algorithm is used, with elitism, fitness-proportionate selection, and rank selection (Mitchell, 1998). In addition, the most challenging tasks employ shaping (Urzelai et al., 1998; Gomez and Miikkulainen, 1997). Fitness is evaluated in a physically simulated virtual environment implemented with NVIDIA PhysX. Figure 2: Typical results from the underlying EVC system. These examples were all evolved to complete a forward locomotion task—a common baseline result for EVCs. Morphology As in Sims’ original work, creature morphology is described by a graph-based genotype, with graph nodes representing body segments, and graph edges representing joints between segments By starting at the root and traversing the graph’s edges, the phenotype is expressed. Reflexive edges as well as multiple edges between the same node pair are allowed, making it possible to define recursive and repeated body substructures easily. In addition, as in Sims’ work, reflection of body parts as well as body symmetry are made easily accessible to evolution, with only a single mutation required to produce each of them. In this implementation, all PhysX primitives are available for use as body segments: boxes, spheres, and capsules. Joints between segments may be of most of the types offered by PhysX, specifically: fixed, revolute, spherical, prismatic, and cylindrical. In contrast to the typical technique of separately evolving explicit joint limits, most limitations on joint movement are provided implicitly by creature structure through natural collisions between adjacent segments. In addition to the typical segments and joints, the implementation of the underlying EVC system also evolves muscle drives and photoreceptors, as described below. Muscles In a break with traditional evolved virtual creature systems, which typically use forces exerted directly at joints, the underlying EVC system of this paper uses simulated muscles as actuators. Each muscle ((b) in Figure 3) is defined by two attachment points on adjacent segments, along with a maximum strength value. In simulation, the muscle is implemented as a spring, with muscle activation modifying the spring constant. In addition to acting as an effector, each muscle also produces a proprioceptive feedback signal based ALIFE 14: Proceedings of the Fourteenth International Conference on the Synthesis and Simulation of Living Systems Figure 3: Photoreceptors (a) and muscles (b) bring sensing and actuation to creatures in the underlying EVC system. For both, function depends upon placement, so creature form develops meaningfully as capabilities are evolved. on its current length. For each muscle, two nodes are added to the brain: one that accepts an input to set the muscle’s activation, and another that makes the muscle’s proprioceptive output signal available to the rest of the brain. Muscle drives benefit EVCs in several ways: they can be used even on creatures without joints; they only need to exist where they are useful, not at every degree of freedom of every joint; and they have the potential to embody some degree of control intelligence, with benefits for both aesthetics and the reduction of cognitive load (Lessin et al., 2014). Photoreceptors For tasks involving light sensing, creatures are allowed to develop simple photoreceptors ((a) in Figure 3), defined only by a direction from the center of their parent segment. This direction indicates a location on the creature’s surface as well as an orientation for the receptor. The signal produced by the receptor is determined by light strength, distance, occlusion, and the difference between the direction to the light and the sensor’s orientation. Multiple lights are allowed. For each photoreceptor in the body, a corresponding brain node is added which makes the receptor’s output signal available to the rest of the brain. Control In a manner which is again very similar to that of Sims, creature control is provided by a brain composed of a set of nodes connected by wires (as in Figure 5a). Nodes receive varying numbers of input wires, and use their inputs to compute an output value (always in the range [0,1]) which may be sent to other wires. Signals originate from sensors in the body as well as certain types of internal brain nodes, travel through the network of internal nodes and wires, and ultimately control the operation of actuators (muscles) in the physically simulated body. For each step