Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: This paper proposes a simple extension to the training of emergent communication protocols in multi-agent settings. 
The central hypothesis is that learnability will favor more 'compressible' and therefor more compositional languages to emerge. 

This hypothesis is tested by training a population of listens and speakers in an emergent communication task and comparing a number of different strategies for reinitializing agents in the population. Since the new agents start from a random initialization, they provide a learning signal that reinforces protocols which can be learned quickly. 

Experiments:
While the experimental results largely confirm the hypothesis there are a few issues:
-all of the plots show mean and standard deviations, rather than error of the mean. This makes it difficult to understand which differences are statistically significant and which ones are not. 
- The replacement strategy 'epsilon-greedy' replaces agents based on their validation loss, which seems like an unfair advantage. 
- It is currently unclear how much we can learn from Section 5.2: Naturally, agents trained in the same population will develop more similar protocols than those trained independently. This result is obvious and it is unclear whether reinitializing the agents makes any significant difference to the similarity. Again, confidence intervals would help.  
- The experiments are also extremely toy. I would be more convinced if the authors tested their method on a more challenging task, though I am aware this is a common problem in this field. 

Novelty:
The single biggest issues with the current form of the paper is the related work section. In this section two papers [1,2] are mentioned as "concurrent" when at least one of them [1] has been available online since June 2019. I think it is important to clearly point out the novelty of the current work compared to those two previous papers. 
In particular [1] seems to be extremely close to the ideas and methods proposed here. Saying that these papers "confirm the hypothesis" simply is not enough. 

References:
[1]: "Ease-of-Teaching and Language Structure from Emergent Communication", Funshan Li et al
[2]: "Co-evolution of language and agents in referential games", Gautier Dagan et al 

[Updated score based on the rebuttal]

review_title: Official Blind Review #4

final_score: rating: 6: Weak Accept
review_content: This paper studies whether language composition may emerge by partially re-sampling new agents inside a pool of language agents. They set up a consistent experimental setting for assessing compositionality,  assess different agent architectures, e.g., memory vs. memoryless agents,  and explore how the language remains close to each other by re-sampling new agents.

The paper is well-motivated with substantial background literature on the cognitive science and emergent communication side. The claim is clear, the hypotheses are well-stated, and the experiments look solid (I particularly appreciated the paragraph on shortcoming evaluation). In the end, I enjoy reading the paper despite its density, and I could see that the authors made quite some effort in that direction.

Improvement direction, questions:
 - The authors made were careful not to take ownership of Kottur et al. 's works. Yet, the writing sometimes gives the feeling that their work is solely an extension of Kottur's work, which is not the case. It also gives the feeling that Kottur et al. is the only valid experimental setting, which is not the case (at the authors pointed out in the related work section). Thus, I would recommend to summarise at some point the similarity/difference between the two papers, or at least stop referring the paper every two lines! 
 - Replacement strategy: the authors use simple replacement strategies, and conclude that it has little impact. Althought It sounds reasonable in the current setting, the conclusion may be a bit premature. I would recommend to discuss further this result with complementary experiments could be the following: see the impact of epsilon, trying tournament strategies, why 8 populations (this sounds a bit arbitrary too). I would also like to put those observations in perspective with the evolutionary literature [1], and even provide a full paragraph in the related work section.
 - Population dynamics: I am missing a key element in the paper: an analysis of the population dynamics. Although the paper deals with generational transmissions, there are no experiments that analyze the evolution of language generations after generations. Most of the experiments deal with the final convergence state. Again, I would recommend having a look at the evolutionary literature to see which protocol they use to analyze such behavior.
 - Literature side: The authors did an excellent job on the emergent communication and cognitive science side. I think that it is worth extending the comparison further. For instance: 
    * generational transmission can be studied in the light of game theory [2] where compositionality can be seen as a Nash Equilibrium between agent. 
    * generational transmission is a form of dynamic distillation [3]
    * and evolutionary algorithms!
 - I had some difficulties in understanding Fig4, and the final-take away correctly. Would it be possible to give me one or two examples to correctly parse the table? More generally, I would recommend to add a few lines with some concrete and cherry-picked examples from the experiments to help the reader to have more intuition). 
 - In a similar spirit, it is hard to interpret the distance in Figure 3. What would correspond to an increase of 1pt of distance? Having said that, the experiment is sound, and it is insightful.
 - reproducible: having a final table in the array in the appendix could be very helpful 
 - crazy experiment: even if I am also a DRL addict, I would be curious to train one of the models with evolutionary algorithms (CMA-ES over parameters, for instance) to assess whether RL has an impact on compositionality (or it is solely the experimental protocol that matters). 
 - I may have missed this point, but how many seeds did you use to run your experiments? 
 - I may have also missed this point, what is the average length of the dialogue. Can you upload (non-understandable) dialogue example? 

Last point... but it does not undermine the soundness of the experimental protocol! 
 - In the end, Is 25 accuracy points really compositionality? What would be the score of simple strategies with overcomplete tokens? What is the score of the minimal vocab if we are only correct with one modality, two modalities?



Remarks:
 - in the introduction, you mention that previous old agents have grounded language, I am not sure whether we can speak of grounded language here, they have a predefined language, but it is not grounded. 
 - Please remove the bold sentence in the introduction :) The claim is clear!
 - P11: Alg undefined
 - P12: the legend cannot be read
 

Conclusion
I am familiar with this type of experimental protocols, and I am well aware that they are never-ending works. There are always more experiments to do, more parameters to analyze. The final question is the following: is this paper have enough of these never-ending experiments? I think that this paper is just above this threshold by a short margin, and I vouch for weak accept.

However, I am missing at least one dynamic figure (to see the impact of the population along time, which is one of the core concepts of the paper), and there are several links with other ML communities that still have to be highlighted (especially evolutionary algorithms). 
Besides, I somehow feel that the authors pursue two different goals in this paper: they both analyze memory/memoryless complete/overcomplete agents, which is somehow orthogonal to the general transmission hypothesis. Maybe, It would have made more sense to focus on one (or two) of the models and change the experimental setting on them (population size, training time, etc.) 

In the end, I would favor a weak accept. 
I am open to discussion regarding this scoring.

[1] Bäck, Thomas, and Frank Hoffmeister. "Extended selection mechanisms in genetic algorithms." (1991).
[2] Lanctot, Marc, et al. "A unified game-theoretic approach to multiagent reinforcement learning." Advances in Neural Information Processing Systems. 2017.
[3] Hinton, Geoffrey, Oriol Vinyals, and Jeff Dean. "Distilling the knowledge in a neural network." arXiv preprint arXiv:1503.02531 (2015).
review_title: Official Blind Review #3

final_score: rating: 1: Reject
review_content: The paper continues the line of work on emergent compositionality in dialogs, and here it is extended to handle groups of interacting agents that pass language in an evolutionary way from one generation to another. The key idea is that with group of interacting agents, if some agents are replaced with new ones, then the newbies would learn the same language as the group. 

General assessment: 

The setup of the paper is is interesting, and the paper covers a lot of ground. The paper makes bold claims like "cultural transmission induces compositionality" and "Variations in replacement strategy tend to not affect performance". Unfortunately, the paper does not systematic provide experimental evidence to adequately support its claims.

In my experience, the emergence of compositionality (or lack of) in the setup learned here is very sensitive to various aspects of the learning setup, hence are hard to reproduce. Specifically, they the task-and-talk paper by Kottur et al may yield different conclusions, if parameters of the original experiments are modified, even slightly. The current paper does improve the evaluation protocol of Kottur 2017 by reporting variance over runs, but does not explore the parameter space more systematically, hence I am concerned that it may suffer from similar fragility. 

In this light, significantly more evidence should be provided to convince that the experimental results are stable and can be reproduced.  First, one needs to show the experiments repeated over the full range of setup parameters. Including, the vocabulary sizes V_Q and V_A, the number of tasks, the number of attributes per task, and number of agents etc. Similarly, the current paper introduces a new compositional split, generated in one specific way. The effect of the split on what aspects of language emerge should be studied systematically, instead of using one "hard" and one random split. 
The "evolutionary" part has a similar issue. The paper draws conclusions from three anecdotal rules for replacing the population. There is no systematic analysis of the "evolution" process, not even studying a range of the parameter epsilon, which is set 0.8 (arbitrarily? to fit the story? we do not know).  Drawing conclusions based on anecdotal evidence is bad scientific practice, that ICLR should discourage.

While the ideas in this paper are innovative and exciting, the paper promises much more than its analysis supports, and the paper is not ready for publication.

Other comments: 
-- The paper states that "darker blue bars" in figure 2 are higher. The statistical analysis is not well explained, not even in the supplemental, so it is hard to tell which differences are significant. If data is paired, it would be useful to view data as a scatter plot, instead of a barplot which hides the pairing. BTW, p<0.05 is not a "strong support", but rather is the most permissive threshold. The results in the supplemental may be stronger. 
-- "Variations in replacement strategy tend to not affect performance." This is a key result of the paper and mush be quantified and analyzed. Authors should define some space of replacements strategies (e.g. in in parametric ways like  how often and how many agents are replaced), then compute performance difference as a function of grid search over the parameter space and show a figure. 
-- "We stop after 8 generations". Justify with data. 
-- Other parts of the paper make additional claims, that should similarly be systematically analyzed and supported with data-driven evidence.

review_title: Official Blind Review #1

Meta-review: This paper explores the emergence of language in environments that demand agents communicate, focusing on the compositionality of language, and the cultural transmission of language.

Reviewer 1 has several suggestions about new experiments that are possible. The AC does think there is value in many of the suggested experiments, if not to run, then just to acknowledge their possibility and leave for future work. The reviewers also point to some previous work that is very similar.  E.g. "Ease-of-Teaching and Language Structure from Emergent Communication", Funshan Li et al
Decision: Reject