Paper:
The principle of compositionality, which enables natural language to represent complex concepts via a structured combination of simpler ones, allows us to convey an open-ended set of messages using a limited vocabulary. If compositionality is indeed a natural property of language, we may expect it to appear in communication protocols that are created by neural agents in language games. In this paper, we propose an effective neural iterated learning (NIL) algorithm that, when applied to interacting neural agents, facilitates the emergence of a more structured type of language. Indeed, these languages provide learning speed advantages to neural agents during training, which can be incrementally amplified via NIL. We provide a probabilistic model of NIL and an explanation of why the advantage of compositional language exist. Our experiments confirm our analysis, and also demonstrate that the emerged languages largely improve the generalization of the neural agent communication.

Review:
This paper studies the emergence of compositional language in neural agents. They propose an iterated learning method that consists of three phases: a supervised learning phase for a randomly-initialized speaker and listener, a self-play phase (where both agents are updated together), and a phase where a new dataset is created based on the current speaker’s language. This dataset is then passed on to the next ‘generation’ of speaker and listener. The paper finds that this procedure, with the right hyperparameters, leads to the emergence of more compositional languages in a simple symbolic referential game.
The question of how to emerge a compositional language is indeed interesting. This paper does a good job of conducting a careful set of ablations and analyzing the results. In my mind, the main scientific contribution of this work is the empirical verification of the principle ‘compositional languages are easier to learn’. While this principle is intuitive, it’s good to see it confirmed via experiments. The paper’s description of the ‘interval of advantage’ --- the range of updates where a compositional language performs better on the task than a non-compositional language --- is insightful to me. 
I do have concerns for this paper around utility and novelty. As the paper mentions, it has already been shown that iterated learning procedures give rise to more compositional languages in non-neural models. While there are some things to consider in adapting this to neural networks, to my eye they seem rather straightforward (i.e. tuning the number of updates of the speaker and listener, the values I_a and I_b), contrary to the paper’s assertion.  From a utility perspective, the paper doesn’t go into how this might be practically applied in general to train neural agents to learn compositional languages in more complex environments (where they might be simultaneously speakers and listeners), as they stick to a very simple symbolic referential game. The main contribution of this paper is really: “studying how neural networks behave when trained in an iterated learning setting in a simple referential game“. I think this is a nice contribution, but the main question for me is whether this is enough for an ICLR acceptance. 
My other concern is around the length of the paper. In my opinion, while the paper is well-written, it’s quite bloated and there is a lot of repetition. I think the paper could easily be condensed to 8 pages and retain the same information. Alternatively, some of the graphs in the Appendix (which are quite nice) could be added to the main paper to give more insight about how neural networks behave in this iterated learning procedure. 
Finally, the paper shows that compositional languages generalize better to the held-out validation set. While this is also an intuitive result, it’s nice to have in the paper. I’d encourage the authors to remove the ‘zero-shot’ terminology though (which usually refers to predictions on new samples outside of the training distribution), and just stick to what is actually being shown, which is improved generalization. 
Overall, I like the paper, but due to the concerns mentioned above I think it’s borderline, with a slight lean towards rejection.