Deep HyperNEAT : Evolving the Size and Depth of the Substrate Evolutionary

This report describes DeepHyperNEAT, an extension of HyperNEAT to allow it to alter the topology of its indirectlyencoded neural network (called the substrate) so that it can continue to grow and increase in complexity over evolution. This aim is accomplished by augmenting HyperNEAT’s compositional pattern producing networks (CPPNs) with new information that allows them to represent substrate topology, and by adding three novel mutations to HyperNEAT that exploit this new information. The purpose of this report is to detail the method and validate its ability to evolve, which is shown through a simple XOR-based test. Significantly more work will be needed to analyze the algorithmic and scientific implications of these new capabilities, but by releasing this report it becomes possible for others also to explore the opportunities that such an extension opens up. A link to associated source code is also provided at the end of this document. Introduction HyperNEAT [1; 2; 3] (Hypercube-based NeuroEvolution of Augmenting Topologies) is a method for evolving neural networks (NNs) through an indirect encoding based on the NEAT method [4] that came before it. The main idea in HyperNEAT is that a special genetic representation called a compositional pattern producing network (CPPN) [5] that is evolved by NEAT generates a pattern of weights that connect a fixed set of neurons positioned within a two-dimensional or three-dimensional space called the substrate. A longstanding limitation of the conventional HyperNEAT algorithm is that while the CPPN can increase in size, the number of neurons and layers in the substrate remains fixed from the beginning of evolution. While there exists one method called evolvable substrate HyperNEAT (ES-HyperNEAT) [6] that allows HyperNEAT to increase the complexity of its substrate that indeed offers opportunities not available in the original HyperNEAT, the new method proposed here, called Deep HyperNEAT is designed explicitly to evolve substrates reminiscent of architectures seen in deep learning today, with increasing numbers of layers and separate parallel pathways through the network. Deep HyperNEAT achieves its new ability by fitting more information into the standard CPPN of HyperNEAT, which now can mutate to describe new layers and new neurons in the substrate. This paper is a preliminary technical report on the method, released to allow others to experiment with its ideas quickly. Thus many intriguing scientific questions about the opportunities created by this new capability will not be answered here; instead the focus is on introducing the technical components of the method and showing that they work correctly (i.e. perform as expected) in a simple validation experiment on the XOR problem. Rigorous scientific investigation is left for the future. The paper is organized as follows: HyperNEAT and the CPPN indirect encoding are reviewed first. The main extensions to HyperNEAT and CPPNs that constitute Deep HyperNEAT are then described, and demonstrated running in a simple XOR test. The paper concludes with thoughts on implications and future work. A public repository containing all of the code for the method is linked at the end of the paper. Background: HyperNEAT HyperNEAT [1; 2; 3] is a neuroevolution method that utilizes indirect encoding to evolve connectivity patterns in neural networks (NN). HyperNEAT historically has found applications in a wide breadth of domains, including multiagent control [7], checkers [2; 3], quadruped locomotion [8; 9], Robocup Keepaway [10], and simulated driving in traffic [11]. The motivation for HyperNEAT was to capture the ability of natural genetic encoding (DNA) to reuse information to encode massive neural structures with orders of magnitude fewer genes. The key principle in HyperNEAT that enabled this kind of meaningful reuse is that neural geometry should be able to exploit problem structure. (This principle is similar to the inspiration behind incorporating convolution in deep learning, but its realization and implications in HyperNEAT are different.) For example, two hidden neurons that process adjacent parts of an input field (e.g. a visual field) should also be roughly adjacent within the neural geometry. That way, locality within an NN becomes a heuristic for relatedness. To situate neurons in an NN within a geometric context, each neuron in HyperNEAT is assigned a Cartesian coordinate. For example, if the NN is configured as a stack of two-dimensional layers of neurons, then the position of each neuron within a layer is simply (x,y). In HyperNEAT it is conventional that the coordinate system of each layer centers at (0,0) and ranges between [-1, 1] in both x and y, thereby forming a square filter or “sheet” of neurons. That way, each neuron n within a layer l can be identified by its position (x,y). Specifying neurons in the NN as a function of the neural geometry allows for an elegant formalism for encoding the connectivity patterns between layers. The connectivity pattern between one layer and another (e.g. all possible connections and weights between neurons in some layer i and another layer j) can be encoded by a function ConnectLayers that takes the positions of the source and target neurons as input. Instead of evolving the weights of every connection individually as in past approaches to neuroevolution like NEAT [12], HyperNEAT evolves a graphical representation of a function, ConnectLayers, that describes how the neurons are connected. In this way, ConnectLayers acts as a lower-dimensional encoding for how to generate connectivity patterns. For example, we might want to query ConnectLayers for the weight of a connection between a neuron a in layer 1 with coordinates (x1,y1) and a neuron b in layer 2 with coordinates (x2,y2). ConnectLayers then returns the weight wx1,y1,x2,y2 : wx1,y1,x2,y2 = ConnectLayers(x1,y1,x2,y2). (1) The main idea in HyperNEAT is to evolve ConnectLayers. In HyperNEAT, ConnectLayers is represented by a compositional pattern producing network (CPPN) [13] (figure 1). A CPPN is an evolvable directed graph representation for functions. Much like NNs, nodes are functions and connections have weights that are applied to the outputs of nodes before passing into other nodes. Importantly, CPPN node functions are heterogeneous in any given CPPN (they are sampled from a set of possible functions usually containing sigmoid, linear, Gaussian, and cosine) and there is no notion of layers. A thorough introduction or review of CPPNs is in [5]. The CPPN begins with a simple topology (usually just its input nodes and its output node with a sparse number of connections between them) and complexifies via evolution in HyperNEAT. Evolving the CPPN this way, through complexification, allows the CPPN to be biased towards compressed representations of connectivity patterns. This approach leads to increasingly intricate yet organized connectivity patterns. In practice CPPNs have proven effective and popular in part because they are designed to evolve increasing complexity with the original NEAT algorithm, which underlies evolution in HyperNEAT. The predicate “hyper” in HyperNEAT signifies that the CPPN is in effect encoding a pattern within a four-dimensional or six-dimensional hypercube (because ConnectLayers typically takes four or six inputs), which is then painted onto the connectivity of a network. In effect, HyperNEAT provides the unique ability to evolve a compressed function that can encode the connectivity of an arbitrarily large network as a function of its geometry. That way, it is possible to produce NNs whose connections display regular and compositional structure through the reuse of motifs encoded in the CPPN. The