Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: The paper describes an algorithm to find diverse patterns in Lenia (a continuous CA system) by using a CPPN to generate initial states, and a stochastic exploration algorithm to mutate parameters of the CPPN + CA parameters. The fitness is the closeness of a generated set of latents to a set of latents produced through one of several possible processes; hand-design, pretraining, or online training on previously generated CA settings. 

The core results are in Figures 27 to 31 in an appendix. Initial inspection reveals that handdesigned goal states produce the most interesting non-animal patterns. With regard to animal forms, it appears to me that Online goal learning harms the diversity of animal forms considerably compared to PGL and perhaps HGS. High frequency spatial structure seems to be lost there. 

I would like to see a further analysis of maybe 10000s of such images generated, and an understanding of exactly why RGS produces the same kind of red linear patterns, and why HGS produces the distribution of pattern types in Figure 29, and why non-animal types differ in PGL vs HGS, and why high frequency spatial structure is lost in OGL. How robust are these over many runs? The results should NOT be shown just for the first repetition of the experiment but for all independent runs of the experiments, e.g averaged over 30 independent CPPN evolutions, for PGL, OGL, Random, and HGS! 


review_title: Official Blind Review #4

final_score: rating: 6: Weak Accept
review_content: The focus of the presented paper is on formulating the automated discovery of self-organized patterns in high-dimensional dynamic systems. The introduced framework uses cellular automata (game of life) as a testbed for experimentation and evaluation and existing machine learning algorithms (POP-IMGEPs). The goal of the paper is to show that these algorithms can be used to discover and represent features of patterns. Moreover, an extension of SOTA algorithms is introduced and several approaches to define goal space representations are compared. 

Overall, I have the impression this is an interesting paper that could be accepted to ICLR. The idea of applying IMGEPs to explore parameters of a dynamic system is novel and interesting, which could also simulate further research in this field. Furthermore, the paper well-written, technically sound, and the results are interesting. The overall contribution of the paper is in applying IMGEP algorithms to exploring parameters of dynamic systems and in comparing different algorithms along with an extensive set of experiments. As a point of criticism, a lot of (interesting) material was pushed to the Appendix. Resolving the references makes reading the paper harder. Moreover, given that this paper has more than 35 pages appendix material, it seems this work would better be suited for a journal as for a conference. There is a reason for papers to have a page limit and this work circumvents this limit by presenting a lot of additional material. Therefore, I am not willing to strongly support this work. 

Specific Comments: 

- Section 3.1: It is not clear how the initial system state is established. In Section 3.1. the text states that 'parameters are randomly sampled and explored' before the process starts, but it is not clear why a random sampling is used and what this means for the subsequent sampling. Later in the text (3.3) it becomes more clear, but here this appears too unclear.
-  Section 3.1: "distribution over a hypercube in \mathcal{T} chosen to be large enough to bias exploration towards the frontiers of known goals to incentivize diversity." This sentence is not clear and needs more details. How is the distribution chosen exactly?
- Section 3.2 appears a bit repetitive and could be more concise. I don't think it is necessary here to contrast manual vs learned features of the goal space.
- Section 3.2 (P3): the last sentence of this paragraph reads as if there exists no approaches for VEAs in online settings. This should be toned down or backed up by a reference.
- Section 3.2: (last sentence): it is not clear how the history is used exactly to train the network. Which strategy is used to sample from the history of observations?
- Section 3.3: What is meant by "The CPPNs are used of the parameters \{theta}"? The details provided after this sentence are not clear and need more details. 
- Section 4.2: Please provide more details what "very large" dataset means.
- Section 4.2: 'HGS algorithm' is not defined.
- Section 5: It seems unnecessary to explain what t-SNE does as a method. 
review_title: Official Blind Review #1

final_score: rating: 8: Accept
review_content: The paper uses the continuous Game of Life as a testing ground for algorithms that discover diverse behaviors. The problem is interesting, under-explored, and rich. The  combines a variety of interesting ideas including compositional pattern producing networks (CPPNs) to learn structured primitives. Although the authors do propose formal measures of behavioral diversity and so show performance improvements, at the end of the day this work, like much empirical work on generative adversarial networks, is drifting towards art -- where performance is ultimately judged by human eyes rather than quantiative metrics. 

Comments:
The paper refers to hand-designed goal spaces and talks, on p28, about “the statistical measures used to define the goal space”. At the same time, the analytic behavior space is also defined in terms of statistical measures, but it is *not* referred to as hand-designed. At this point, the profusion of spaces and measures means that I am no longer sure what counts as hand-crafted or not. Please clarify.
The hypothesis on p34, sec E.4.2 that the VAE’s 8-dim bottleneck helps focus on animals rather than non-animals (which are differentiated more in terms of textures and details) is important and should be checked. 
Some of the decisions about what to check and vary are unclear. For example, section E.1 considers the effect of different initializations (“pytorch”, “xavier” and “kaiming”). The choice of initialization is important mostly to do with improving gradients to improve the rate of convergence (or convergence at all) in deep nets. It’s not clear why initializations are an parameter to vary when considering diversity of solutions. Or, rather, why initializations are more interesting to consider various other architectural considerations. More broadly, looking at Fig 17, the x-axis doesn’t make much sense. The experiments along the x-axis vary according to initialization, but also according to the nature of the goal space and other features. It seems a bit incoherent. 

Overall I think this is a good paper. The results are novel and even better, they are fun. However, the paper is extremely long, and it feels as though the authors have to some extent lost control of the material. I could add more comments but TL;DR it needs a lot of editing and pruning. 

review_title: Official Blind Review #2

Meta-review: The authors introduce a framework for automatically detecting diverse, self-organized patterns in a continuous Game of Life environment, using compositional pattern producing networks (CPPNs) and population-based Intrinsically Motivated Goal Exploration Processes (POP-IMGEPs) to find the distribution of system parameters that produce diverse, interesting goal patterns.

This work is really well-presented, both in the paper and on the associated website, which is interactive and features source code and demos. Reviewers agree that it’s well-written and seems technically sound. I also agree with R2 that this is an under-explored area and thus would add to the diversity of the program.

In terms of weaknesses, reviewers noted that it’s quite long, with a lengthy appendix, and could be a bit confusing in areas. Authors were responsive to this in the rebuttal and have trimmed it, although it’s still 29 pages. My assessment is well-aligned with those of R2 and thus I’m recommending accept. In the rebuttal, the authors mentioned several interesting possible applications for this work; it’d be great if these could be included in the discussion. 

Given the impressive presentation and amazing visuals, I think it could make for a fun talk.

Decision: Accept (Talk)