Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: This paper proposed a neural iterated learning algorithm to encourage the dominance of high compositional language in the multi-agent communication game. The author shows that the iterative training of two agents playing a referential game can incrementally increase the agent to use the language with high topological similarity. The authors also demonstrated that topological similarity is correlated with zero-shot performance. And Experiment results show the authors could propose alternative pre-training strategies for the neural agent can prefer high compositional language and achieve high task performance. 

Emerging the compositional language from uniform prior can be very challenging, as mentioned in the paper, "high-\rho language only represents a small portion of all possible unambiguous language" and "high-\rho do not seem to be directly preferred during the interaction phase, they can be favored by the neural agent during the learning phase." I agree with the authors with respect to the difficulties of generating high-\rho language, but I have questions about the designed learning phrase, especially how to avoid the mode collapse during training. 

With the first hypothesis, "high topological similarity improves the learning speed of the speaking neural agent", I agree that high topological language has less low sample complexity compared to random sampled low topological language. However, low topological language didn't necessarily lead to low sample complexity, for example, given a D consists of {a, a, a, a ...}, the sample complexity can be quite low and also with a high topological score. I wonder is the hypothesis still true in this case? 

On the second hypothesis, a high-\rho language will be faster to success choosing the right object using fewer samples. I agree with the authors that compositional language can and will lead to better generalization ability. However, from Algorithm1, it seems Bob receives the message only update with its parameters. There is no change of language generation. I wonder how the update of Bob will help Alice to speak the more compositional language? More explicitly, to avoid mode collapse. 

Exp 3 mainly tests the model with different \rho as the posterior probability. In Exp 4, I assume the posterior probability of the mapping is random (uniform), is that correct? It will be great if the confirm this since this is my major doubt when reading the paper. 

As mentioned above, I understand that high topological similarity language both benefit from the speaker and listener. However, there are some corner cases that mode collapse will happen and It seems the model will hardly recover from that. From Table 3, it seems the authors have fixed vocabulary size 8, I'm wondering what happens with a large vocabulary size? will the model still learn to emerge the compositional language with a large vocab size? 
review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: This paper studies the emergence of compositional language in neural agents. They propose an iterated learning method that consists of three phases: a supervised learning phase for a randomly-initialized speaker and listener, a self-play phase (where both agents are updated together), and a phase where a new dataset is created based on the current speaker’s language. This dataset is then passed on to the next ‘generation’ of speaker and listener. The paper finds that this procedure, with the right hyperparameters, leads to the emergence of more compositional languages in a simple symbolic referential game.

The question of how to emerge a compositional language is indeed interesting. This paper does a good job of conducting a careful set of ablations and analyzing the results. In my mind, the main scientific contribution of this work is the empirical verification of the principle ‘compositional languages are easier to learn’. While this principle is intuitive, it’s good to see it confirmed via experiments. The paper’s description of the ‘interval of advantage’ --- the range of updates where a compositional language performs better on the task than a non-compositional language --- is insightful to me. 

I do have concerns for this paper around utility and novelty. As the paper mentions, it has already been shown that iterated learning procedures give rise to more compositional languages in non-neural models. While there are some things to consider in adapting this to neural networks, to my eye they seem rather straightforward (i.e. tuning the number of updates of the speaker and listener, the values I_a and I_b), contrary to the paper’s assertion.  From a utility perspective, the paper doesn’t go into how this might be practically applied in general to train neural agents to learn compositional languages in more complex environments (where they might be simultaneously speakers and listeners), as they stick to a very simple symbolic referential game. The main contribution of this paper is really: “studying how neural networks behave when trained in an iterated learning setting in a simple referential game“. I think this is a nice contribution, but the main question for me is whether this is enough for an ICLR acceptance. 

My other concern is around the length of the paper. In my opinion, while the paper is well-written, it’s quite bloated and there is a lot of repetition. I think the paper could easily be condensed to 8 pages and retain the same information. Alternatively, some of the graphs in the Appendix (which are quite nice) could be added to the main paper to give more insight about how neural networks behave in this iterated learning procedure. 

Finally, the paper shows that compositional languages generalize better to the held-out validation set. While this is also an intuitive result, it’s nice to have in the paper. I’d encourage the authors to remove the ‘zero-shot’ terminology though (which usually refers to predictions on new samples outside of the training distribution), and just stick to what is actually being shown, which is improved generalization. 

Overall, I like the paper, but due to the concerns mentioned above I think it’s borderline, with a slight lean towards rejection. 

review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: The paper "Compositional languages emerge in a neural iterated learning model" address the problem of language emergence in two-players games. In particular, the authors proposed a neural iterated learning model which seeks comopsitional languages. Authors claim that compositional languages are easier to be learned and that they allow listeners to more easily understand provided messages. 

The problem of language emergence is interesting since it refers to the problem of finding efficient ways to communicate.  At first I was wondering why such compositional language messages would be desirable and was a bit negative on this work. But in Table 2 authors give results for zero-shot performance which emphasize the benefits resulting from finding such composition properties in language: compositional languages have greatly better generalization properties. I like the parallel that we can make with humans, that have to learn to understand language for achieving tasks when they are childs, which is simulated here in the reset and re-training performed  at the start of each generation, and which explains the natural emergence of such compositionality. This is not a big surprise, but I like the simple but clever idea of reset that the paper exploits. 

What I like less is the inequality (5) that not fully convinced me. I am not sure why this should hold. Moreover, authors claim that if Ia is too long, no improvement of the topology can be made. I cannot understand why. For instance if there are ambiguous messages in D, the interaction phase can radically change the language even if the pre-training has converged... And why should weak pre-training favor low-p languages? 

Also, the considered learning scheme is that in the transmitting phase Alice records messages for all objects in D. But is it realistic ? 

A study of the impact of the size of vocabulary would also be useful (since it must have a big impact on the results)   

At last, why considering such discrete messages while for agents communication  would be easier with continuous messages ? When considering continuous messages, the problem relates with disentanglement which is a current hot-topic: having one factor controling one specific aspect of the object would also be useful for improving zero-learning.     
review_title: Official Blind Review #1

Meta-review: This paper examines the correspondence between topological similarity of languages (correlation between the message space and object space) and ability to learn quickly in a situation of emergent communication between agents.

While this paper is not without issues, it does seem to present a nice contribution that all of the reviewers appreciated to some extent. I think it will spark further discussions in this area, and thus can recommend it for acceptance.
Decision: Accept (Poster)