I NTRINSICALLY M OTIVATED D ISCOVERY OF D IVERSE P ATTERNS IN S ELF -O RGANIZING S YSTEMS
Anonymous authors
Paper under double-blind review
A BSTRACT
In many complex dynamical systems, artiﬁcial or natural, one can observe selforganization of patterns emerging from local rules.
Cellular automata, like the
Game of Life (GOL), have been widely used as abstract models enabling the study of various aspects of self-organization and morphogenesis, such as the emergence of spatially localized patterns.
However, ﬁndings of self-organized patterns in such models have so far relied on manual tuning of parameters and initial states, and on the human eye to identify “interesting” patterns.
In this paper, we formulate the problem of automated discovery of diverse self-organized patterns in such high-dimensional complex dynamical systems, as well as a framework for experimentation and evaluation.
Using a continuous GOL as a testbed, we show that recent intrinsically-motivated machine learning algorithms (POP-IMGEPs), initially developed for learning of inverse models in robotics, can be transposed and used in this novel application area.
These algorithms combine intrinsicallymotivated goal exploration and unsupervised learning of goal space representations.
Goal space representations describe the “interesting” features of patterns for which diverse variations should be discovered.
In particular, we compare various approaches to deﬁne and learn goal space representations from the perspective of discovering diverse spatially localized patterns.
Moreover, we introduce an extension of a state-of-the-art POP-IMGEP algorithm which incrementally learns a goal representation using a deep auto-encoder, and the use of CPPN primitives for generating initialization parameters.
We show that it is more efﬁcient than several baselines and equally efﬁcient as a system pre-trained on a hand-made database of patterns identiﬁed by human experts.
I NTRODUCTION
Self-organization of patterns that emerge from local rules is a pervasive phenomena in natural and artiﬁcial dynamical systems (Ball, 1999).
It ranges from the formation of snow ﬂakes, spots and rays on animal’s skin, to spiral galaxies.
Understanding these processes has boosted progress in many ﬁelds, ranging from physics to biology (Camazine et al., 2003).
This progress relied importantly on the use of powerful and rich abstract computational models of self-organization (Kauffman, 1993).
For example, cellular automata like Conway’s Game of Life (GOL) have been used to study the emergence of spatially localized patterns (SLPs) (Gardener, 1970), informing theories of the origins of life (Gardener, 1970; Beer, 2004).
SLPs, such as the famous glider in GOL (Gardner et al., 1983), are self-organizing patterns that have a local extension and can exist independently of other patterns.
However, ﬁnding such novel self-organized patterns, and mapping the space of possible emergent patterns, has so far relied heavily on manual tuning of parameters and initial states.
Moreover, the dependence of this exploration process on the human eye to identify “interesting” patterns is strongly limiting further advances.
We formulate here the problem of automated discovery of a diverse set of self-organized patterns in such high-dimensional, complex dynamical systems.
This involves several challenges.
A ﬁrst challenge consists in determining a representation of patterns, possibly through learning, enabling to incentivize the discovery of diverse “interesting” patterns.
Such a representation guides exploration by providing a measure of (di-)similarity between patterns.
This problem is particularly challenging in domains where patterns are high-dimensional as in GOL.
In such cases, scientists have a limited intuition about what useful features are and how to represent them.
Moreover, low-dimensional representations of patterns are needed for human browsing and the visualization of the discoveries.
Representation learning shall both guide exploration, and be fed by self-collected data.
A second challenge consists in how to automate exploration of high-dimensional, continuous initialization parameters to discover efﬁciently “interesting” patterns, such as SLPs, with a limited budget of experiments.
Sample efﬁciency is important to enable the later use of such discovery algorithms for physical systems (Grizou et al., 2019), where experimental time and costs are strongly bounded.
For example, in the continuous GOL used in this paper as a testbed, initialization consists in determining the values of a real-valued, high-dimensional 256 × 256 matrix besides 7 additional dynamics parameters.
The possible variations of this matrix are too large for a simple random sampling to be efﬁcient.
More structured methods are needed.
To address these challenges, we propose to leverage and transpose recent intrinsically motivated learning algorithms, within the family of population-based Intrinsically Motivated Goal Exploration
Processes (POP-IMGEPs - denoted simply as IMGEPs below, Baranes & Oudeyer (2013); P´er´e et al.
They were initially designed to enable autonomous robots to explore and learn what effects can be produced by their actions, and how to control these effects.
IMGEPs self-deﬁne goals in a goal space that represents important features of the outcomes of actions, such as the position reached by an arm.
This allows the discovery of diverse novel effects within their goal representations.
It was recently shown how deep neuronal auto-encoders enabled unsupervised learning of goal representations in IMGEPs from raw pixel perception of a robot’s visual scene (Laversanne-Finot et al., 2018).
We propose to use a similar mechanism for automated discovery of patterns by unsupervised learning of a low-dimensional representation of features of self-organized patterns.
This removes the need for human expert knowledge to deﬁne such representations.
Moreover, a key ingredient for sample efﬁcient exploration of IMGEPs for robotics has been the use of structured motion primitives to encode the space of body motions (Pastor et al., 2013).
We propose to use a similar mechanism to handle the generation of structured initial states in GOL-like complex systems, based on specialized recurrent neural networks (CPPNs)
(Stanley, 2006).
In summary, we provide in this paper the following contributions:
• We formulate the problem of automated discovery of diverse self-organized patterns in high-dimensional and complex game-of-life types of dynamical systems.
• We show how to transpose POP-IMGEPs algorithms to address the associated joint challenge of (learning to) represent interesting patterns and discovering them in a sample efﬁcient manner.
• We compare various approaches to deﬁne or learn goal space representations for the sample efﬁcient discovery of diverse SLPs in a continuous GOL testbed.
• We show that an extension of a state-of-the-art POP-IMGEP algorithm, with incremental learning of a goal space using a deep auto-encoder, is equally efﬁcient than a system pretrained on a hand-made database of patterns.
R ELATED W ORK
Automated Discovery in Complex Systems Automated processes have been widely used to explore complex dynamical systems.
For example, evolutionary algorithms have been applied to search speciﬁc patterns or rules of cellular automata (Mitchell et al., 1996;
Sapin et al., 2003).
However, their objective is to optimize a speciﬁc goal instead of discovering a diversity of patterns.
Another line of experiments represent active inquiry-based learning strategies which query which set of experiments to perform to improve a system model, i.e. a mapping from parameters to the system outcome.
Such strategies have been used in biology (King et al., 2004; 2009), chemistry (Raccuglia et al., 2016; Reizman et al., 2016; Duros et al., 2017) and astrophysics (Richards et al., 2011).
However, these approaches have relied on expert knowledge, and focused on automated optimization of a pre-deﬁned target property.
Here, we are interested to automatically discover and map a diversity of unseen patterns without prior knowledge of the system.
An exception is the concurrent work of Grizou et al.
(2019), which showed how a simple POP-IMGEP algorithm could be used to automate discovery of diverse patterns in oil-droplet systems.
However, it used a low-dimensional input space, and a hand-deﬁned low-dimensional representation of goal spaces, identiﬁed as a major limit of the system.
Intrinsically motivated learning Intrinsically-motivated learning algorithms (Baldassarre &
Mirolli, 2013; Baranes & Oudeyer, 2013) autonomously organize an agent’s exploration curriculum in order to discover efﬁciently a maximally diverse set of outcomes the agent can produce in an unknown environment.
They are inspired from the way children self-develop open repertoires of skills and learn world models.
Intrinsically Motivated Goal Exploration Processes (IMGEPs) (Baranes &
Oudeyer, 2013; Forestier et al., 2017) are a family of curiosity-driven algorithms developed to allow efﬁcient exploration of high-dimensional complex real world systems.
Population-based versions of these algorithms, which leverage episodic memory, hindsight learning, and structured dynamic motion primitives to parameterize policies, enable sample efﬁcient acquisition of high-dimensional skills in real world robots (Forestier et al., 2017; Rolf et al., 2010).
Recent work (Laversanne-Finot et al., 2018; P´er´e et al., 2018) studied how to automatically learn the goal representations with the use of deep variational autoencoders.
However, training was done passively and in an early stage on a precollected set of available observations.
Recent approaches (Nair et al., 2018; Pong et al., 2019) introduced the use of an online training of variational autoencoders VAEs to learn the important features of a goal space similar to the methods in this paper.
However, these approaches focused on the problem of sequential decisions in MDPs, incurring a cost on sample efﬁciency.
This problem is observed in various intrinsically motivated RL approaches (Bellemare et al., 2016; Burda et al., 2019).
The approaches are orthogonal to the automated discovery framework considered here with independent experiments allowing the use of memory-based sample efﬁcient methods.
A related family of algorithms in evolutionary computation is novelty search (Lehman & Stanley, 2008) and quality-diversity algorithms (Pugh et al., 2016), which can be formalized as special kinds of population-based IMGEPs.
Representation learning We are using representation learning methods to learn autonomously goal spaces for IMGEPs.
Representation learning aims at ﬁnding low-dimensional explanatory factors representing high-dimensional input data (Bengio et al., 2013).
It is a key problem in many areas in order to understand the underlying structure of complex observations.
Many state-of-the-art methods (Tschannen et al., 2018) have built on top of Deep VAEs (Kingma & Welling, 2013), using varying objectives and network architectures.
However, studies of the interplay between representation learning and autonomous data collection through exploration of an environment have been limited so far.
A LGORITHMIC M ETHODS FOR A UTOMATED D ISCOVERY 3. P OPULATION - BASED I NTRINSICALLY M OTIVATED G OAL E XPLORATION P ROCESSES
An IMGEP is an algorithmic process generating a sequence of experiments to explore the parameters of a system by targeting self-generated goals (Fig. 1).
It aims to maximize the diversity of observations from that system within a budget of N experiments.
In population-based IMGEPs, an explicit memory of the history H of experiments and observations is used to guide the process.
The systems are deﬁned by three components.
A parameter space Θ corresponding to the controllable system parameters θ .
An observation space O where an observation o is a vector representing all the signals captured from the system.
For this paper, the observations are a time series of images which depict the morphogenesis of activity patterns.
Finally, an unknown environment dynamic D : Θ → O which maps parameters to observations.
To explore a system, an IMGEP uses a goal space T that represents relevant features of its observations, computed by an encoding function ˆ g = R ( o ) .
For an exploration of patterns, such features may describe their form or extension.
The exploration process iterates N times through: 1) sample a goal from a goal sampling distribution g ∼ G ( H ) ; 2) infer corresponding parameter θ using a parameter sampling policy Π
= Pr( θ ; g, H ) ; 3) roll-out an experiment with θ , observe outcome o , compute encoding R ( o ) ; 4) store ( θ, o, R ( o )) in history H .
The parameter sampling policy Π and in some cases the goal sampling distribution G take into account previous explorations stored in history
Deep Neural
Autoencoder 
Parameter Space Θ   =   1) sample goal: 2) select parameter: ∼ Π ( ,  )
̂  4) encode observation  =→
Goal Space   ̂
Initial System State
Parameter for Dynamics = 0.8 = 0.4 = 0.2  = 0.
= 0.
=[ , ,  ( )] ⋯ [ , ,  ( )]  1
History  … 3) perform experiment u p d a t e e n c o d e r e v e r y  s t e p s
Observation Space  ∼   ̂
CPPN   =  ( )     =   =   =   … …
System Rollout … … =
H .
Therefore, the history is ﬁrst populated through exploring N init randomly sampled parameters.
After this initial phase the described intrinsically motivated goal exploration process starts.
Different goal and parameter sampling mechanisms can be used within this architecture (Baranes & Oudeyer, 2013; Forestier & Oudeyer, 2016).
In the experiments below, goals are sampled uniformly over a hyperrectangle deﬁned in T . The hyperrectangle is chosen large enough to allow a sampling of a large goal diversity. The parameters are sampled by 1) given a goal, selecting the parameter from the history whose corresponding outcome is most similar in the goal space; 2) then mutating it by a random process.
3. O NLINE L EARNING OF G OAL S PACES WITH D EEP A UTO -E NCODERS
For IMGEPs the deﬁnition of the goal space T and its corresponding encoder R are a critical part, because it biases exploration of the target system.
One approach is to deﬁne a goal space by selecting features manually, for example by using computer vision algorithms to detect the positions of a pattern and its form.
The diversity found by the IMGEPs will then be biased along these pre-deﬁned features.
A limit of this approach is its requirement of expert knowledge to select helpful features, particularly problematic in environments where experts do not know in advance what features are important, or how to formulate them.
Another approach is to learn goal space features by unsupervised representation learning.
The aim is to learn a mapping R ( o ) from the raw sensor observations o to a compact latent vector z ∈ R d .
This latent mapping can be used as a goal space where a latent vector z = g is interpreted as a goal.
Previous IMGEP approaches already learned successfully their goal spaces with VAEs (LaversanneFinot et al., 2018; P´er´e et al., 2018).
However, the goal spaces were learned before the start of the exploration from a prerecorded dataset of observations from the target environment.
During the exploration the learned representations were kept ﬁxed.
A problem with this pretraining approach is that it limits the possibilities to discover novel patterns beyond the distribution of pretraining examples, and requires expert knowledge.
In this paper we attempt to address this problem by continuously adapting the learned representation to the novel observations encountered during the exploration process.
For this purpose, we propose an online goal space learning IMGEP (IMGEP-OGL), which learns the goal space incrementally
Initialize goal space encoder VAE R with random weights for i ← 1 to N do if i < N init then //
Initial random iterations to populate H
Sample θ ∼ U (Θ) else //
Intrinsically motivated iterations
Sample a goal g ∼ G ( H ) based on space represented by R
Choose θ ∼ Π( g, H )
Perform an experiment with θ and observe o
Encode reached goal ˆ g =
R
( o )
Append ( θ, o, ˆ g ) to the history H if i mod K == 0 then //
Periodically train the network for E epochs do
Train R on observations in H with importance sampling for ( θ, o, ˆ g ) ∈ H do // Update the database of reached goals
H [ˆ g ] ← R ( o ) during the exploration process (Algorithm 1). The training procedure of the VAE is integrated in the goal sampling exploration process by ﬁrst initializing the VAE with random weights (Appendix B.6).
The VAE network is then trained every K explorations for E epochs on the observation collected in the history H .
Importance sampling is used to give more weight to recently discovered patterns by using a weighted random sampler.
It samples for 50% of the training batch samples patterns from the last K iterations and for the other 50% patterns from all other previous iterations.
3. S TRUCTURING THE PARAMETER SPACE IN IMGEP S : FROM DMP S TO CPPN S
A key role in the generation of patterns in dynamical systems is their initial state
A t =1 .
IMGEPs sample these initial states and apply random perturbations to them during the exploration.
For the experiments in this paper this state is a two-dimensional grid with 256 × 256 cells.
Performing directly a random sampling of the 256 × 256 grid cells results in initial patterns that resemble white noise.
Such random states result mainly in the emergence of global patterns that spread over the whole state space, complicating the search for spatially localized patterns.
This effect is analogous to a similar problem in the exploration of robot controllers.
Direct sampling of actions for individual actuators at a microscopic time scale is usually inefﬁcient.
A key ingredient for sample efﬁcient exploration has been the use of structured primitives (dynamic motion primitives - DMPs) to encode the space of possible body motions (Pastor et al., 2013).
We solved the sampling problem for the initial states by transposing the idea of structured primitives.
Indeed, “actions” consist here in deciding the parameters of an experiment, including the initial state.
We propose to use compositional pattern producing networks (CPPNs) (Stanley, 2006) to produce structured initial patterns similar do DMPs.
CPPNs are recurrent neural networks that allow the generation of structured initial states. (Appendix B.4, Fig. 14).
They are deﬁned by their network structure (number of neurons, connections between neurons) and their connection weights.
Thus, instead of using directly an initial state as part of the parameters θ to control the dynamical system, a CPPN is used. If a system roll-out for the parameters θ are performed, then the initial state A t = is generated by the CPPN. Moreover, instead of sampling and mutating directly an initial state, the weights and structure of the CPPN are randomly generated and mutated.
Please note, the number of parameters in θ is therefore not ﬁxed and open-ended (yet starts small) because the structure and the number of weights of randomly sampled and mutated CPPNs differ. [Figure] discovered by IMGEP-OGL ( [Figure]
Lenia animals discovered by Chan (2019)
[Figure]
E XPERIMENTAL METHODS
We describe here the continuous Game of Life (Lenia) we use as a testbed representing a large class of high-dimensional dynamical systems, as well as the experimental procedures, the evaluation methods used to measure diversity and detect SLPs, and the used algorithmic baselines and ablations.
Implementation details and parameter settings for all procedures are given in Appendix B.
4. C ONTINOUS G AME OF L IFE AS A TESTBED
Lenia (Chan, 2019) is a continuous cellular automaton (Wolfram, 1983) similar to Conway’s Game of Life (Gardener, 1970).
Lenia, in particular, represents a high-dimensional complex dynamical system where diverse visual structures can self-organize and yet are hard to ﬁnd by manual exploration.
It features the richness of Turing-complete game-of-life models.
It is therefore well suited to test the performance of pattern exploration algorithms for unknown and complex systems.
The fact that GOL models have been used widely to study self-organization in various disciplines, ranging from physics to biology and economics (Bak et al., 1989), also supports their generality and potential of reuse of our approach for discovery in other computational or wet high-dimensional systems.
Lenia consists of a two-dimensional grid of cells A ∈ [0 , 1] 256 × 256 where the state of each cell is a real-valued scalar activity
A t ( x ) ∈ [0 , 1] .
The state of cells evolves over discrete time steps t (Fig. 2, a).
The activity change is computed by integrating the activity of neighboring cells.
Lenia’s behavior is controlled by its initial pattern A t =1 and several settings that control the dynamics of the activity change ( R, T, µ, σ, β 1 , β 2 , β 3 ).
Lenia can be understood as a self-organizing morphogenetic system.
Its parameters for the initial pattern A t =1 and dynamics control determine the development of morphological patterns.
Lenia can produce diverse patterns with different dynamics (stable, non-stable or chaotic).
Most interesting, spatially localized coherent patterns that resemble in their shapes microscopic animals can emerge (Fig. 2, b, c).
These pattern types, which we will denote “animals” as a short name, are a key reason scientists have used GOL models to study theories of the origins of life (Gardener, 1970; Beer, 2004).
Therefore, in our evaluation method based on measures of diversity (see below), we will in particular study the performance of IMGEPs, and the impact of using various approaches for goal space representation, on ﬁnding a diversity of animal patterns.
We implemented for this purpose different pattern classiﬁers to analyze the exploration results.
Initially we differentiate between dead and alive patterns.
A pattern is dead if the activity of all cells are either 0 or 1 .
Alive patterns are separated into animals and non-animals.
Animals are a connected areas of positive activity which are ﬁnite, i.e. which do not inﬁnitely cross several borders.
All other patterns are non-animals whose activity usually spreads over the whole state space.
4. E VALUATION BASED ON THE DIVERSITY OF P ATTERNS
The algorithms are evaluated based on their discovered diversity of patterns.
Diversity is measured by the spread of the exploration in an analytic behavior space .
This space is externally deﬁned by the experimenter as in previous evaluation approaches in the IMGEP literature.
For example, in P´er´e et al.
(2018) is the diversity of discovered effects of a robot that manipulates objects measured by binning the space of object positions and counting the number of bins discovered.
A difference here is that the experimenter does not have access to an easily interpretable hand-deﬁned low-dimensional representation of possible patterns, equivalent to the cartesian coordinate of rigid objects.
The space of raw observations O , i.e. the ﬁnal Lenia patterns A t =200 , is also too high-dimensional for a meaningful measure of spread in it.
We constructed therefore an external evaluation space.
First, a latent representation space was build through the training of a β -VAE (Higgins et al., 2017) to learn the important features over a large dataset of 42500 Lenia patterns identiﬁed during the many experiments over all evaluated algorithms.
This large dataset enabled to cover a diversity of patterns orders of magnitude larger than what could be found in any single algorithm experiment, which experimental budget was order of magnitude smaller.
We then augmented that space by concatenating hand-deﬁned features (the same features were used for a hand-deﬁned goal space IMGEP).
For each experiment all explored patterns were projected into the analytic behavior space.
The diversity of the patterns is then measured by discretizing the space into bins of equal size by splitting each dimension into 7 sections (results were found to be robust to the number of bins per dimension, see Appendix B.7).
This results in 7 13 bins.
The number of bins in which at least one explored entity falls is used as a measure for diversity.
We also measured the diversity in the space of parameters Θ by constructing an analytic parameter space .
The 15 features of this space consisted of Lenia’s parameters ( R , T , µ , σ , β 1 , β 2 , β 3 ) and the latent representation of a β -VAE.
The β -VAE was trained on a large dataset of initial Lenia states ( A t =1 ) used over the experimental campaign.
This diversity measures also used 7 bins per dimension.
4. A LGORITHMS
The exploration behaviors of different IMGEP algorithms were evaluated and compared to a random exploration.
The IMGEP variants differ in their way how the goal space is deﬁned or learned.
Random exploration: The IMGEP variants were compared to a random exploration that sampled randomly for each of the N exploration iterations the parameters θ including the initial state A t =1 .
IMGEP-HGS - Goal exploration with a hand-deﬁned goal space:
The ﬁrst IMGEP uses a handdeﬁned goal space that is composed of 5 features used in Chan (2019).
Each feature measures a certain property of the ﬁnal pattern A t =200 that emerged in Lenia: 1) the sum over the activity of all cells, 2) the number of activated cells, 3) the density of the activity center, 4) an asymmetry measure of the pattern and 5) a distribution measure of the pattern.
IMGEP-PGL - Goal exploration with a pretrained goal space: For this IMGEP variant the goal space was learned with a β -VAE approach on training data before the exploration process started.
The training set consisted of 558 Lenia patterns: half were animals that have been manually identiﬁed by Chan (2019); the other half randomly generated with CPPNs (see Section 4.4).
IMGEP-OGL - Goal exploration with online learning of the goal space: Algorithm 1.
IMGEP-RGS - Goal exploration with a random goal space: An ablated IMGEP using a goal space based on the encoder of a VAE with random weights.
4. E XPERIMENTAL P ROCEDURE AND HYPERPARAMETERS
For each algorithm 10 independent repetitions of the exploration experiment were conducted.
Each experiment consisted of N = 5000 exploration iterations.
This number was chosen to be compatible with the application of the algorithms in physical experimental setups similar to Grizou et al.
(2019), planned in future work.
For IMGEP variants the ﬁrst N init = 1000 iterations used random parameter sampling to initialize their histories H .
For the following 4000 iterations each IMGEP approach sampled a goal g via an uniform distribution over its goal space.
The ranges for sampling in the hand- deﬁned goal space (HGS) are deﬁned in Table 3 (Appendix B.5).
The ranges for the β -VAE based goal spaces (PGL, OGL) were set to [ − 3 , 3] for each of their latent variables.
Then, the parameter θ k from a previous exploration in H was selected whose reached goal ˆ g k had the minimum euclidean distance to the current goal g within the goal space.
This parameter was then mutated to generate the parameter θ that was explored.
The parameters θ consisted of a CPPN (Section 3.3) that generates the initial state A t =1 for Lenia and the settings deﬁning Lenia’s dynamics: θ =
[ CPPN → A t =1 , R, T, µ, σ, β 1 , β 2 , β 3 ] .
The
CPPNs were initialized and mutated by a random process that deﬁnes their structure and connection weights as done by Stanley (2006).
The random initialization of the other Lenia settings was done by an uniform distribution and their mutation by a Gaussian distribution around the original values.
The meta parameters to initialize and mutate the parameters were the same for all algorithms.
They were manually chosen without optimizing them for a speciﬁc algorithm.
R ESULTS
We address several questions evaluating the ability of IMGEP algorithms to identify a diverse set of patterns, and in particular diverse “animal” patterns (i.e. spatially localized patterns).
Does goal exploration outperform random parameter exploration?
In robotics/agents contexts where scenes are populated with rigid objects, various forms of goal exploration algorithms outperform random parameter exploration (Laversanne-Finot et al., 2018).
We checked whether this still holds in continuous GOL which have very different properties.
Measures of the diversity in the analytic behavior space conﬁrmed the advantage of IMGEPs with hand-designed (HGS) or learned goal spaces (PGL/OGL) over random explorations (Fig. 3, b).
The organization resulting from goal exploration is also visible through the diversity in the space of parameters.
IMGEPs focus their [Figure] a) Diversity in
Parameter Space
[Figure]
) Diversity in Behavior Space
[Figure]
[Figure]
exploration on subspaces that are useful for targeting new goals.
In contrast, random parameter exploration is unguided, resulting in a higher diversity in the parameter space (Fig. 3, b).
What is the impact of learning a goal space vs. using random or hand-deﬁned features?
We compared also the performance of random VAE goal spaces (RGS) to learned goal spaces (PGL/OGL).
For reinforcement learning problems, using intrinsic reward functions based on random features of the observations can result in a high or boosted performance (Burda et al., 2018; 2019).
In our context however, using random features (RGS) collapsed the performance of goal exploration, and did not even outperform random parameter exploration for all kinds of behavioral diversity (Fig. 3).
Results also show that hand-deﬁned features (HGS) produced signiﬁcantly less global diversity and less “animal” diversity than using learned features (PGL/OGL).
However, HGS found an equal diversity of “non-animals”.
These results show that in this domain, the goal-space has a critical inﬂuence on the type and diversity of patterns discovered.
Furthermore, unsupervised learning is an efﬁcient approach to discover a diversity of diverse patterns , i.e. both efﬁcient at ﬁnding diverse animals and diverse non-animals.
Is pretraining on a database of expert patterns necessary for efﬁcient discovery of diverse animals?
A possibility to bias exploration towards patterns of interest, such as “animals”, is to pretrain a goal space with a pattern dataset hand-made by an expert.
Here PGL is pretrained with a dataset containing 50% animals.
This leads PGL to discover a high diversity of animals.
However, the new online approach (OGL) is as efﬁcient as PGL to discover diverse patterns (Fig.
3, b,c,d).
Taken together, these results uncover an interesting bias of using learned features with a VAE architecture, which strongly incentivizes efﬁcient discovery of diverse spatially localized patterns.
How do goal space representations differ?
We analyzed the goal spaces of the different IMGEP variants to understand their behavior by visualizing their reached goals in a two-dimensional space.
T-SNE (Maaten & Hinton, 2008) was used to reduce the high-dimensional goal spaces.
The hand-deﬁned (HGS) and learned (OGL) goal spaces show strong differences between each other (Fig. 4).
We believe this explains their different abilities to ﬁnd either a high diversity of non-animals or animals (Fig. 3, c, d).
The goal space of the HGS shows large areas and several clusters for nonanimal patterns (Fig. 4, a).
Animals form only few and nearby clusters.
Thus, the hand-deﬁned features seem poor to discriminate and describe animal patterns in Lenia.
As a consequence, when goals are uniformly sampled within this goal space during the exploration process, then more goals are generated in regions that describe non-animals.
This can explain why HGS explored a higher diversity of non-animal patterns but only a low diversity of animal patterns.
In contrast, features learned by OGL capture better factors that differentiate animal patterns.
This is indicated by the several clusters of animals that span a wide area in its goal space (Fig. 4, b).
We attribute this effect to the difﬁculty of VAEs to capture sharp details (Zhao et al., 2017).
They therefore represent mainly the general form of the patterns but not their ﬁne-grained structures.
Animals differ often in their form whereas non-animals occupy often the whole cell grid and differ in their ﬁne-grained details.
The goal spaces learned by VAEs seem therefore better suited for exploring diverse sets of animal patterns.
C ONCLUSION
We formulated a novel application area for machine learning: the problem of automatically discovering self-organized patterns in complex dynamical systems with high-dimensions both in the action space and in the observation space.
We showed that this problem calls for advanced methods requiring the dynamic interaction between sample efﬁcient autonomous exploration and unsupervised representation learning.
We demonstrated that population-based IMGEPs are a promising algorithmic framework to address this challenge, by showing how it can discover diverse self-organized patterns in a continuous GOL.
In particular, we introduced a new approach of learning a goal space representation online via data collected during the exploration process.
It enables sample efﬁcient discovery of diverse sets of animal-like patterns, similar to those identiﬁed by human experts and yet without relying on such prior expert knowledge (Fig. 2).
We also analyzed the impact of goal space representations on the diversity and types of discovered patterns.
The continuous game of life shares many properties with other artiﬁcial or natural complex systems, explaining why GOL models have been used in many disciplines to study self-organization, see Bak et al.
We therefore believe this study shows the potential of IMGEPs to automated discovery in other systems encountered in physics, chemistry or even computer animation.
In further work, we aim to apply this approach in roboticized wet experiments such as the one presented in Grizou et al.
and addressing fundamental understanding of how proto-cells can self-organize.
A A DDITIONAL R ESULTS AND F IGURES
We provide here visualizations of examples of discovered patterns, as well as t-SNE and PCA visualizations of examples of learned goal-spaces. A dataset and an interactive visualization of all discovered patterns can be found at https://intrinsically-motivated-discovery. github.io/ .
The visualization serves as a support in qualitatively evaluating the diversity of patterns discovered by each algorithm.
Integrated into an interactive interface, these graphs are also useful for a potential human end-user to easily explore and visualize the different types of found patterns.
In addition we provide additional statistical results quantifying the proportion of patterns of different types in the set of patterns discovered by different algorithms
[Figure]
[Figure]
[Figure] [Figure] [Figure] [Figure]
A. D ISCOVERED P ATTERNS
Fig.
5 to 9 illustrate examples of discovered pattern per class (animal, non-animal, dead) and their ratio for each algorithm. The patterns have been randomly sampled from the results of a single exploration repetition experiment of each algorithm. Please note that the shown examples represent only a small fraction of all discovered patterns. A database with all patterns can be found on the website for the paper.
This section provides further visualizations of goal spaces complementary to Fig. 4. The goal spaces of all IMGEP algorithms are visualized (Fig. 10) via two-dimensional reductions: PCA (Jolliffe, 1986) and t-Distributed Stochastic Neighbor Embedding (t-SNE) (Maaten & Hinton, 2008).
The visualizations were constructed by using for each algorithm its goal space representations of all patterns it explored from a single repetition experiment.
All goal representations were normalized so that the overall minimum value became 0 and the maximum value 1 for each goal space dimension.
PCA was performed to detect the two principle components.
T-SNE was executed using the default standard Euclidean distance metric and perplexity set to 50. A. P ROPORTION OF D ISCOVERED P ATTERNS OF D IFFERENT C LASSES The main paper used the measure of diversity of the found patterns per algorithm to compare their performance. Another measure to compare the algorithms is the average proportion of dead, animals, non-animals patterns discovered by each algorithm (Fig. 11).
For animal patterns the results follow the diversity results (Fig. 3), i.e. OGL and PGL ﬁnd the highest proportion of animal patterns, followed by HGS, then RGS and Random.
A corollary is that RGS, random and HGS ﬁnd in proportion a higher percentage of non-animal patterns than OGL and PGL.
As the number of different non-animal patterns is as high for OGL and PGL, this shows the higher sample efﬁciency of OGL and PGL to ﬁnd diverse patterns. In contrast, Random, RGS and HGS approaches tend to ﬁnd non-animal patterns that are more similar to each other on average. [Figure] p r opo r ti o n
B I MPLEMENTATION D ETAILS AND H YPERPARAMETER S ETTINGS
This section provides implementation details and hyperparameter settings of all experiments. The content is organized in the following manner: • B.1: Settings of Lenia • B.2: Description of the classiﬁers for dead, animal and non-animal Lenia patterns • B.3: Description of the statistical measures about Lenia patterns used as features for the IMGEP-HGS goal space and the analytic behavior space.
• B.4: Sampling mechanisms for Lenia’s initial state via CPPNs and Lenia’s other dynamic parameters.
• B.5: Details about the IMGEP-HGS
• B.6: Details about IMGEPs utilizing deep variational autoencoders (RGS, PGL, OGL).
• B.7: Description of the measurement method of diversity in the analytic parameter and behavior space
B. L ENIA S ETTINGS
A full description of Lenia can be found in (Chan, 2019).
For all experiments the following conﬁgurations of Lenia were used:
• Grid size: 256 × 256 ( A ∈ [0 , 1] {+256 × 256 ) • Number of steps: M =• Exponential growth mapping: G ( u ; µ, σ ) = 2 exp (cid:16) − ( u − µ ) 2 
+}
σ 2 (cid:17) −• Exponential kernel function: K C ( r ) = exp (cid:16) α − α 4 r (1 − r ) (cid:17) , with α =• Kernel shell: K S ( r ; β ) =
β (cid:98)
Br (cid:99) K C ( Br mod 1) , with β = ( β 1 , β 2 , β 3 )
The controllable parameters of Lenia are the kernel size R ∈ N , time step T ∈ N , µ ∈ R and σ ∈ R that control the growth mapping, and β 1 , β 2 , β 3 ∈ R that control the kernel shell.
Additionally the initial state A t =1 ∈ [0 , 1] 256 × 256 controls the system dynamics.
B. C LASSIFICATION OF L ENIA P ATTERNS
We classiﬁed 3 types of Lenia patterns: dead, animals and non-animals.
The categories were used to analyze if the exploration algorithms showed differences in their behaviors by identifying different types of patterns.
The classiﬁers only categorize the ﬁnal pattern into which the Lenia system morphs after M = 200 time steps.
Dead Classiﬁer: For dead patterns the activity of all cells are either 0 or 1 .
Animal Classiﬁer: A pattern is classiﬁed as an animal if it is a ﬁnite and connected pattern of activity.
Cells x , y are connected as a pattern if both are active ( A ( x ) ≥ 0 .
1 and A ( y ) ≥ 0 . 1 )
and if they inﬂuence each other.
Cells inﬂuence each other when they are within their radius of the kernel K as deﬁned by the parameter R .
Furthermore, the connected pattern must be ﬁnite.
In Lenia ﬁnite and inﬁnite patterns can be differentiated because the opposite borders of Lenia’s cell grid are connected, so that the space is similar to a ball surface.
Thus, a pattern can loop around this surface making it inﬁnite.
We identify inﬁnite patterns by the following approach.
First, all connected patterns are identiﬁed for the case of assuming an inﬁnite grid cell, i.e. opposite grid cell borders are connected.
Second, all connected patterns for the case of a ﬁnite grid cell, i.e. opposite grid cell borders are not connected, are identiﬁed.
Third, for each border pair (north-south and east-west)
it is tested if cells within a distance of R from both borders exists, that are part of a connected pattern for the inﬁnite and ﬁnite grid cell case.
If such a pattern exists than it is assumed to be inﬁnite, because it loops around the grid cell surface of Lenia (Fig. 12, a).
All other patterns are considered to be ﬁnite (Fig. 12, b).
Please note that this method has a drawback.
It can not identify certain inﬁnite patterns that loop over several borders, for example, if a pattern connects the north to east and then the west to south border (see the third animal in Fig. 5 for an example).
Moreover, there are two additional constraints that an animal pattern must fulﬁll.
First, the cells of the connected pattern P = { x 1 , . . .
, x n } must have at least 80% of all activation, i.e. (cid:80) x ∈ P A ( x ) ≥ 0 .
8 (cid:80) ∀ y A ( y ) .
Second, a pattern must exists for the last two time steps ( t = M and t =
M − 1 ).
Both constraint are used to avoid that too small patterns or chaotic entities which change drastically between time steps are classiﬁed as animals.
See Figs.
5, 7, 8 and 9 for examples of animal patterns.
Non-Animal Classiﬁer: We also classiﬁed non-animal patterns which are all entities that were not dead and not an animal.
These patterns spread usually over the whole state space and are connected via borders.
See Figs.
5, 7, 8 and 9 for examples of non-animal patterns.
B. S TATISTICAL M EASURES FOR L ENIA P ATTERNS
We deﬁned ﬁve statistical measurements for the ﬁnal patterns A t =
M that emerge in Lenia.
The measures were used as features for hand-deﬁned goal spaces of IMGEPs and to deﬁne partly the analytic behavior space in which the results of the exploration experiments were compared.
Activation mass M A : Measures the sum over the total activation of the ﬁnal pattern and normalizes it according to the size of the Lenia grid:
[Equation]
where L 2 = 256 · 256 is the number of cells of the Lenia system.
Activation volume V A : Measures the number of active cells and normalizes it according to the size of the Lenia grid:
[Equation]
> (cid:15) (cid:9)(cid:12)(cid:12) with (cid:15) = 10 − 4 .
Activation density D A : Measures how dense activation is distributed on average over active cells:
[Equation]
Activation asymmetry A A : Measures how symmetrical the activation is distributed according to an axis that starts in the center of the patterns activation mass and goes along the last movement direction of this center.
This measure was introduced to especially characterize animal patterns.
The center of the activity mass is usually also the center of the animals and analyzing the activity along their movement axis measures how symmetrical they are.
As a ﬁrst step, the center of the activation mass is computed for every time step of the Lenia simulation and the Lenia pattern recentered to this location.
This ensures that the center is all the time (a) Inﬁnite Pattern [Figure] pattern inﬁnite segmentation ﬁnite segmentation pattern inﬁnite segmentation ﬁnite segmentation correctly computed in the case the animal moves and reaches one border to appear on the opposite border in the uncentered pattern.
The center (¯ x, ¯ y ) t for time step t is calculated by:
[Equation]
with M pq = (cid:88) x (cid:88)
y x p y q
A t ( x, y ) , where M pq measures the image moment (or raw moment) of order ( p + q ) for p, q ∈ N .
Based on the center (¯ x, ¯ y ) t the pattern A t is recentered to A tC by shifting the x and y indexes according to the center:
[Equation]
where L is width and length of the Lenia grid and the indexing is x, y = 0 , . .
, L − 1 .
After each time step the center is recomputed and the pattern recentered:
[Equation]
Please note, the simulations and all ﬁgures of patterns in the paper are done with the uncentered pattern.
The centered version is only computed for the purpose of statistical measurements.
The recenter step by (¯ x, ¯ y ) t deﬁnes also the movement direction of the activity center:
[Equation]
where x mid , y mid
=
L − 1 2 are the coordinates for the grid’s middle point.
A line can be deﬁned that starts in the midpoint (cid:0) x mid , y mid (cid:1) of the ﬁnal centered pattern A t = M C and goes in and opposite to the movement direction of the line amount
[Equation]
Activation centeredness C A : Measures how strong the activation is distributed around the activity mass center:
[Equation]
where d ( x, y ) = (cid:112) ( x − x mid )
+ ( y − y mid ) 2 is the distance from the point ( x, y ) to the center point (cid:0) x
mid , y mid (cid:1) .
A t = M C ( x, y ) is the centered activation that is updated every time step as for the asymmetry measure (Eq. 1).
The weights w xy decrease the farer a point is from the center.
Thus, patterns that are concentrated around the center have a high value for C A close to 1 .
Whereas, patterns whos activity is distributed throughout the whole grid have a smaller value.
For patterns that are equally distributed ( ∀ x,x (cid:48)
: A ( x ) =
A ( x (cid:48) ) ) is
C A
=
deﬁned as centeredness measure.
B. S AMPLING OF P ARAMETERS FOR L ENIA
All algorithms explore Lenia by sampling the parameters θ that control Lenia.
The parameters are comprised of the initial pattern A t =1 and the parameters which control the dynamic behavior ( R, T, µ, σ, β 1 , β 2 , β 3 ).
There are two operations to sample parameters: 1) random initialization and 2) mutating an existing parameter θ .
CPPNs are used for the random initialization and mutation of the initial pattern A t =1 .
The details of this process are described in the next section.
Afterwards, the initialization and mutation of Lenia’s parameter that control its dynamics are described.
B.4. S AMPLING OF S TART P ATTERNS FOR L ENIA VIA CPPN S
Compositional Pattern Producing Networks (CPPNs) are recurrent neural networks (Stanley, 2006) which we to generate and mutate the initial state of Lenia
A t =1 .
The CPPNs generate Lenia activity patterns cell by cell by taking as input a bias value, the x and y coordinate of the cell (mapped to
Pattern: CPPN: (  ,  )    2 - -  1 − |  |  x = [ − 2 , 2] and y =
[ − 2 , 2] ) and its distance d to the grid center (Fig. 13).
Their output p deﬁnes the activity of the cell ( A ( x, y )
= 1 − | p | ) between 0 and 1 for the given ( x, y ) coordinate.
CPPNs consist of several hidden neurons (typically 4 to 6 in our experiments) that can have recurrent and self connections.
Each CPPN has one output neuron.
Two activation functions were used for the hidden neurons and the output neuron.
The ﬁrst is Gaussian and the second is sigmoidal:
[Equation]
To randomly initialize a Lenia initial pattern A t =1 a CPPN is randomly sampled by sampling the number of hidden neurons, the connections between inputs and neurons and neurons to neurons, their connection weights and the activation functions for neurons.
Afterwards the initial pattern is generated by it.
In the history H of the IMGEPs is then the CPPN as part of the parameter θ added.
If the parameter is mutated, then the weights, connections and activation functions of the CPPN are mutated and the new initial pattern A t =1 generated by it.
A CPPN is deﬁned over its network structure (number of nodes, connections of nodes) and its connection weights.
We used the neat-python 2 package for the random generation and mutation of CPPNs.
It is based on the NeuroEvolution of Augmenting Topologies (NEAT) algorithm for the evolution of neural networks (Stanley & Miikkulainen, 2002).
The meta-parameters for the initialization and mutation of CPPNs are listed in Table 1.
The random sampling and mutation of CPPNs allows to generate complex patterns as illustrated in Fig.
Please note, the neat-python package allows also the setting and mutation of response and bias weights for each neuron.
Those settings were not used
Initialization
[Figure]
[-For each neuron the activation function is changed with probability 0 . 1 by randomly assigning a new activation function (either gauss or sigm). For each connection the weight is mutated by the following steps. With probability 0 . 
-]CPPNs can generate
[Table]
for the experiments.
Moreover, we adjusted the sigmoid and Gaussian function in the neat-python package to the ones deﬁned in Eq.
2 to be able to replicate similar images as in Stanley (2006).
B.4. S AMPLING OF L ENIA ’ S D YNAMIC P ARAMETERS
The parameters that control the dynamics of Lenia ( R, T, µ, σ, β 1 , β 2 , β 3 ) are initialized and mutated via uniform and Gaussian distributions.
Table 2 lists for each parameter the meta-parameters for their initialization and mutation.
Each parameter is initialized by an uniform sampling θ i ∼ U ( a, b ) with a and b as upper and lower border.
An existing parameter θ
i is mutated by the following equation:
[Equation]
where σ M is the mutation power and [ n ] ab = min(max( n, a ) , b ) is the clip function.
For natural numbers θ i ∈ N the resulting value is rounded.
[Table]
R
N [
[Table]
[Table]
T
N [1 , 20] 0 .µ
R [0 , 1] 0 .σ
R [0 . 001 , 0 . 3] 0 .
[Table]
B. IMGEP-HGS D ETAILS
The 5 statistical features that are given in Appendix B.3 were used to deﬁne the goal space of the
IMGEP-HGS approach (Algorihm 2).
Goals in this space were sampled from a uniform distribution within the ranges deﬁned in Table
[Table]
B. IMGEP S WITH D EEP V ARIATIONAL A UTOENCODERS B.6. VAE F RAMEWORK AND I MPLEMENTATION D ETAILS
We applied deep variational autoencoders (VAEs) to learn latent representations of Lenia patterns.
VAEs have two components: a neural encoder and decoder .
The encoder q ( z | x , χ ) represents a given data point x in a latent representation z .
In variational approaches the encoder describes a data point by a representative distribution in the latent space of reduced dimension d .
A standard
Gaussian prior p ( z ) = N (0 , I ) and a diagonal Gaussian posterior q ( z | x , χ ) =
N ( µ, σ ) are used.
Given a data point x , the encoder outputs the mean µ and variance σ of the representative distribution in the latent space.
The decoder p ( x | z , ψ ) tries to reconstruct the original data x from a sampled latent representation z for the distribution given by the encoder.
Under these assumptions, training is done by maximizing the computationally tractable evidence lower bound (with β = 1 ):
[Equation]
The ﬁrst term ( a ) represents the expected reconstruction accuracy while the second ( b ) is the KL divergence of the approximate posterior from the prior.
[Equation]
In particular, we used the β -VAE framework (Higgins et al., 2017) which re-weights the b term by a factor β > 1 , aiming to enhance the disentangling properties of the learned latent factors.
All βVAEs used for this paper use the same architecture (Table 4).
The encoder network has as input the
Lenia pattern and as outputs for each latent variable z
i the mean µ i and log-variance log( σ 2 i ) .
The decoder takes as input during the training for each latent variable a sampled value
z i ∼ N ( µ i , σ 2 i ) .
For validation runs and the generation of all reconstructed patterns shown in ﬁgures the decoder takes the mean z i = µ i as input.
Its output is the reconstructed pattern.
The training objective (Eq. 3) results in the following loss function for a batch:
[Equation]
where x are the input patterns, ˆ x are the reconstructed patterns, µ, σ are the outputs of the decoder network and d is the number of latent dimensions.
The reconstruction accurray part a of the loss is given by a binary cross entropy with logits:
[Equation]
, where the index j is for the single cells (pixel) of the pattern and n for the datapoint in the current batch, N is the batch size and σ ( x ) = [Equation] − x .
The KL divergence terms b i are given by:
[Equation]
[Table]
All β -VAEs were trained for 2000 epochs and initialized with pytorch default initialization.
We used the Adam optimizer (Kingma & Ba, 2014)
( lr = 1e − 3 , β 1 = 0 .
9 , β 2 = 0 .
999 , (cid:15) =
1e − 8 , weight decay= 1e − 5 ) with a batch size of 64.
The patterns from the datasets were augmented by random x and y translations (up to half the pattern size and with probability 0.3), rotation (up to 40 degrees and with probability 0.3), horizontal and vertical ﬂipping (with probability 0.2).
The translations and rotations were preceded by spherical padding to preserve Lenia spherical continuity.
B.6. IMGEP V ARIANTS
IMGEP-RGS (random goal space): IMGEP with a goal space deﬁned by an encoder network with random weights (Algorithm 2).
The network architecture of the encoder is the same that the one of the VAEs used for IMGEP with learned goal spaces.
In the other IMGEP algorithms (HGS/PGL/OGL), the goals are sampled uniformly within ﬁxed-range boundaries that are chosen in advance.
However, in the case of random goal spaces, we do not know in advance in which region of the space goals will be encoded.
Therefore, we set the sampling range for each latent variable to the minimum and maximum value of the latent representations of all so far explored patterns.
IMGEP-PGL (prelearned goal space): IMGEP with a goal space deﬁned by a β -VAE that was trained before the exploration starts (Algorithm 2).
The β -VAE was trained on a dataset consisting of 558 precollected Lenia patterns (training: 75%, validation: 10%, testing: 15%).
Half of the patterns (279) were manually identiﬁed animal patterns by Chan (2019).
The other half (279) were randomly initialized CPPN patterns (see Section B.4.1).
The best β -VAE model obtained during training (highest accuracy on the validation data) was used as goal space for the exploration.
During the exploration, goals were uniformly sampled in a [ − 3 , 3] 8 hypercube.
These values were chosen because the encoder of the β -VAE is trained to match a prior standard normal distribution. Therefore, we can assume that most area of the covered goal space will fall into that hypercube.
IMGEP-OGL (online learned goal space): IMGEP with a goal space that was online learned during the exploration via an β -VAE (Algorithm 1).
Every K = 100 explorations the β -VAE model was trained for 40 epochs resulting in 2000 epochs in total (less if there is not enough data after the ﬁrst T runs to start the training).
The datasets were constructed by collecting non-dead patterns during the exploration.
One pattern in every ten is added to the validation set (10%). All others are used for the training set.
At the initial period of training, the training dataset has approximatelypatterns and at the last period approximately 3425 patterns. The validation dataset only serves for checking purposes and has no inﬂuence on the learned goal space.
Importance sampling is used to give the patterns a different weight during the training.
A weighted random sampler is used that samples newly discovered patterns half of the time.
Each pattern that has been added to the training dataset during the last period of 100 explorations has a probability of 0 .
5 N to be sampled (N is the total number of new patterns in the dataset).
Older patterns are with probability [-
-]0 .
5 | D T |− N .
As a result, newer discovered patterns have a higher weight and a stronger inﬂuence on the training.
Algorithm
Initialize goal space encoder R with handdeﬁned features (HGS), random weights (RGS), or pretrained weights (PGL) for i ← 1 to N do if i < N init then //
Initial random iterations to populate H
Sample θ ∼ U (Θ) else //
Intrinsically motivated iterations
Sample a goal g ∼ G ( H ) based on space represented by R
Choose θ ∼ Π( g, H )
Perform an experiment with θ and observe o
Append ( o, θ, R ( o )) to the history H
B.6. VAE T RAINING R ESULTS
The β -VAE learning saturates for both the precollected dataset and the online collected dataset (Fig. 15).
Their ability to reconstruct patterns based on the encoded latent representation is also qualitatively similar.
For both datasets the β -VAEs are able to learn the general form of the activity pattern (Fig. 16).
Nonetheless, the compression of the images to a 8-dimensional vector results in a general blurriness in the reconstructed patterns.
As a result, the β -VAEs are not able to encode ﬁner details and textures of patterns.
[Figure]
[Figure]
B. M EASUREMENT OF D IVERSITY IN THE A NALYTIC P ARAMETER AND B EHAVIOR S PACE B.7. D IVERSITY M EASURE
Diversity is measured by the area that explored parameters or observations cover in their respective spaces.
The parameter space consisted of the initial start state of Lenia ( A t =1 ∈ [0 , 1] 256 × 256 ) and the settings for Lenia’s dynamics ( R, T, µ, σ, β 1 , β 2 , β 3 ).
The space consist therefore of 256dimensions, each for a single grid cell of the initial pattern, plus 7 dimensions for the dynamic settings.
The observation space consists of the ﬁnal patterns A t =200 ∈ [0 , 1] 256 × 256 resulting in 256 2 dimensions for the space.
Each single exploration results in a new point in those spaces.
The diversity measures how much area the algorithms explored in those spaces (Fig. 17).
The measurement is done by discretizing the space with a spatial grid and counting the number of discretized areas in which at least one point falls. For the discretization a minimum and maximum border is deﬁned for each space dimension.
Each dimension is then split in equally sized bins between those borders. The areas with values falling below the minimum or above the maximum border are counted as two additional bins. The number of dimensions of the original parameter and observation space are too large to measure diversity, because the initial and the ﬁnal pattern have 256 2 dimensions. We constructed therefore an analytic parameter and behavioral space where the latent representations of a β -VAE were used to reduce the high-dimensional patterns to 8 dimensions.
5 bins (7 with the out of range values) per dimension were used for the discretization of those spaces for all experiments in the paper. [Figure]
The analytic parameter space was constructed by the 7 Lenia parameters that control its dynamics and 8 latent representation dimensions of a β -VAE (Table 5). The β -VAE was trained on initial patterns A t =1 used during the experiments. The analytic behavior space was constructed by combining the 5 statistical measures for ﬁnal Lenia patterns (Section B.3) and also 8 latent representation dimensions of a β -VAE (Table 5).
This β -VAE was trained on ﬁnal patterns A t =200 observed during the experiments.
The datasets for both β -VAEs were constructed by randomly selecting 42500 patterns (37500 as training set, 5000 as validation set) from the experiments of all algorithms and each of their 10 repetitions.
The β -VAEs used the same structure, hyper-parameters, loss function and
Analytic Parameter Space Deﬁnition
[Table] [Figure] learning algorithm as described in Section B.6. They were trained for more than 1400 epochs.
The encoder which resulted in the minimal validation set error during the training was used.
According to their reconstructed patterns they can represent the general form of patterns but often not individual details such as their texture (Fig. 18). B.7. D EPENDENCE OF THE D IVERSITY ON THE N UMBER OF B INS PER D IMENSION
The diversity measure has as parameter the number bins per dimension. We analyzed the inﬂuences of this parameter on the diversity measure (Fig. 19).
Although the diversity difference between algorithms depends on the number of bins per dimension for each space, the order of the algorithms is generally invariant to it.
Only if the number of bins per dimension grows large ( > 10) the order of the algorithms changes in some cases.
The order starts to follow the same order as seen for the proportion of identiﬁed patterns (Fig. 11).
In this case the discretization of the space becomes too ﬁne.
Each pattern falls into its own discretized area.
We chose therefore a smaller number of bins per dimension of 7 to compare the algorithms in a meaningful way
[Figure]
Diversity
[Figure]
Animals
[Figure]