Venue: ICLR 2018 conference

Reviews:

final_score: rating: 3: Clear rejection
review_content: This paper presents a technique for training a two-agent system to play a simple reference game involving recognition of synthetic images of a single object.  Each agent is represented by an RNN that consumes an image representation and sequence of tokens as input, and generates a binary decision as output. The two agents are initialized independently and randomly. In each round of training, one agent is selected to be the speaker and the other to be the listener. The speaker generates outputs by greedily selecting a sequence of tokens to maximize the probability of a correct recognition w/r/t the speaker's model. The listener then consumes these tokens, makes a classification decision, incurs a loss, and updates its parameters. Experiments find that after training, the two agents converge to approximately the same language, that this language contains some regularities, and that agents are able to successfully generalize to novel combinations of properties not observed during training.

While Table 3 is suggestive, this paper has many serious problems. There isn't an engineering contribution---despite the motivation at the beginning, there's no attempt to demonstrate that this technique could be used either to help comprehension of natural language or to improve over the numerous existing techniques for automatically learning communication protocols.  But this also isn't science: "generalization" is not the same thing as compositionality, and there's no testable hypothesis articulated about what it would mean for a language to be compositional---just the post-hoc analysis offered in Tables 2 & 3. I also have some concerns about the experiment in Section 3.3 and the overall positioning of the paper.

I want to emphasize that these results are cool, and something interesting might be going on here! But the paper is not ready to be published. I apologize in advance for the length of this review; I hope it provides some useful feedback about how future versions of this work might be made more rigorous.

WHAT IS COMPOSITIONALITY?

The title, introduction, and first three sections of this paper emphasize heavily the extent to which this work focuses on discovering "compositional" language. However, the paper doesn't even attempt to define what is meant by compositionality until the penultimate page, where it asserts that the ability to "accurately describe an object [...] not seen before" is "one of the marks of compositional language". Various qualitative claims are made that model outputs "seem to be" compositional or "have the strong flavor of" compositionality. Finally, the conclusion notes that "the exact definition of compositional language is somewhat debatable, and, to the best of our knowledge, there was no reliable way to check for the compositionality of an arbitrary language."

This is very bad.

It is true that there is not a universally agreed-upon definition of compositionality. In my experience, however, most people who study these issues do not (contra the citation-less 5th sentence of section 4) think it is simply an unstructured capacity for generalization.  And the implication that nobody else has ever attempted to provide a falsifiable criterion, or that this paper is exempt from itself articulating such a criterion, is totally unacceptable.  (You cannot put "giving the reader the tools to evaluate your current claims" in future work!)

If this paper wishes to make any claims about compositionality, it must _at a minimum_:

1. Describe a test for compositionality.

2. Describe in detail the relationship between the proposed test and other definitions of compositionality that exist in the literature.

3. If this compositionality is claimed to be "language-like", extend and evaluate the definition of compositionality to more complex concepts than conjunctions of two predicates.

Some thoughts to get you started: when talking about string-valued things, compositionality almost certainly needs to say something about _syntax_. Any definition you choose will be maximally convincing if it can predict _without running the model_ what strings will appear in the gray boxes in Figure 3.  Similarly if it can consistently generate analyses across multiple restarts of the training run.  The fact that analysis relies on seemingly arbitrary decisions to ignore certain tokens is a warning sign. The phenomenon where every color has 2--3 different names depending on the shape it's paired with would generally be called "non-compositional" if it appeared in a natural language.

This SEP article has a nice overview and bibliography: https://plato.stanford.edu/entries/compositionality/. But seriously, please, talk to a linguist.

MODEL

The fact that the interpretation model greedily chooses symbols until it reaches a certain confidence threshold would seem to strongly bias the model towards learning a specific communication strategy. At the same time, it's not actually possible to guarantee that there is a greedily-discoverable sequence that ever reaches the threshold! This fact doesn't seem to be addressed.

This approach also completely rules out normal natural language phenomena (consider "I know Mary" vs "I know Mary will be happy to see you"). It is at least worth discussing these limitations, and would be even more helpful to show results for other architectures (e.g. fixed-length codes or loss functions with an insertion penalty) as well.

There's some language in the appendix ("We noticed that a larger vocabulary and a longer message length helped the agents achieve a high communication accuracy more easily. But the resulting messages were challenging to analyze for compositional patterns.") that suggests that even the vague structure observed is hard to elicit, and that the high-level claim made in this paper is  less robust than the body suggests. It's _really_ not OK to bury this kind of information in the supplementary material, since it bears directly on your core claim that compositional structure does arise in practice. If the emergence of compositionality is sensitive to vocab size & message length, experiments demonstrating this sensitivity belong front-and-center in the paper.

EVALUATION

The obvious null hypothesis here is that unseen concepts are associated with an arbitrary (i.e. non-compositional) description, and that to succeed here it's enough to recognize this description as _different_ without understanding anything about its structure. So while this evaluation is obviously necessary, I don't think it's powerful enough to answer the question that you've posed. It would be helpful to provide some baselines for reference: if I understand correctly, guessing "0" identically gives 88% accuracy for the first two columns of Table 4, and guessing based on only one attribute gives 94%, which makes some of the numbers a little less impressive. 

Perhaps more problematically, these experiments don't rule out the possibility that the model always guesses "1" for unseen objects. It would be most informative to hold out multiple attributes for each held-out color (& vice-versa), and evaluate only with speakers / listeners shown different objects from the held-out set.

POSITIONING AND MOTIVATION

The first sentence of this paper asserts that artificial general intelligence requires the ability to communicate with humans using natural language. This paper has nothing to do with AGI, humans, or human language; to be blunt, this kind of positioning is at best inappropriate and at worst irresponsible. It must be removed. For the assertion that "natural language processing has shown great progress", the paper provides a list of citations employing neural networks exclusively and beginning in 2014 (!). I would gently remind the authors that NLP research did not begin with deep learning, and that there might be slightly earlier evidence for their claim.

The attempt to cite relevant work in philosophy and psychology is commendable!  However, many of these citations are problematic, and some psycho-/historico-linguistic claims are still missing citations. A few examples: Ludwig Wittgenstein died in 1951, so it is somewhat surprising to see him cited for a 2010 publication (PI appeared in 1953); similarly Zipf (2016).  The application of this Zipf citation is dubious; the sentence preceded by footnote 7 is false and has nothing to do with the processes underlying homophony in natural languages. I would encourage you to consult with colleagues in the relevant fields.
review_title: Review

final_score: rating: 6: Marginally above acceptance threshold
review_content: Pros:
1. Extend the input from disentangled feature to raw image pixels
2. Employ “obverter” technique, showing that it can be an alternative approach comparing to RL 
3. The authors provided various experiments to showcase their approach

Cons:
1. Comparing to previous work (Mordatch & Abbeel, 2018), the task is relatively simple, only requiring the agent to perform binary prediction.
2. Sharing the RNN for speaking and consuming by picking the token that maximizes the probability might decrease the diversity.
3. This paper lack original technical contribution from themselves.  

The paper is well-written, clearly illustrating the goal of this work and the corresponding approach. The “obverter” technique is quite interesting since it incorporates the concept from the theory of mind which is similar to human alignment or AGI approach. Authors provided complete experiments to prove their concept; however, the task is relatively easy compared to Mordatch & Abbeel (2018). Readers would be curious how this approach scales to a more complex problem.

While the author sharing the RNN for speaking and consuming by picking the token that maximizes the probability, the model loses its diversity since it discards the sampling process. The RL approach in previous works can efficiently explore sentence space by sampling from policy distribution. However, I can not see how the author tackles this issue. One of the possible reason is that the task is relatively easy, therefore, the agent does not need explicitly exploration to tackle this task. Otherwise, some simple technique like “beam search” or perform MC rollout at each time could further improve the performance.

In conclusion, this paper does not have a major flaw. The “obverter” approach is interesting; however, it is originally proposed in Batali (1998). Generating language based only on raw image pixels is not difficult. The only thing you need to do is replacing FC layers with CNN layers. Though this paper employs an interesting method, it lacks some technical contribution from themselves.  


[1] Igor Mordatch and Pieter Abbeel. Emergence of grounded compositional language in multi-agent populations. AAAI 2018

[2] John Batali. Computational simulations of the emergence of grammar. Approaches to the evolution of language: Social and cognitive bases, 405:426, 1998.
review_title: The paper is well-written, clearly illustrating the goal of this work and the corresponding approach. The “obverter” technique is quite interesting since it incorporates the concept from the theory of mind which is similar to human alignment or AGI approach. 

final_score: rating: 9: Top 15% of accepted papers, strong accept
review_content: This paper proposes to apply the obverter technique of Batali (1998) to a multi-agent communication game. The main novelty with respect to Batali's orginal work is that the agents in this paper have to communicate about images represented at the raw pixel level. The paper presents an extensive analysis of the patterns learnt by the agents, in particular in terms of how compositional they are.

I greatly enjoyed reading the paper: while the obverter idea is not new, it's nice to see it applied in the context of modern deep learning architectures, and the analysis of the results is very interesting.

The writeup is somewhat confusing, and in particular the reader has to constantly refer to the supplementary materials to make sense of the models and experiments. At least the model architecture could be discussed in more detail in the main text.

It's a pity that there is no direct quantitative comparison between the obverter technique and a RL architecture.

Some more detailed comments:

It would be good if a native speaker could edit the paper for language and style (I only annotated English problems in the abstract, since there were just too many of them).

Abstract:

distinguishing natures -> aspects

describe the complex environment -> describe complex environments

with the accuracy -> with accuracy

Introduction:

have shown great progress -> has...

The claim that language learning in humans is entirely based on communication needs hedging. See, e.g., cultures where children must acquire quite a bit of their language skills from passive listening to adult conversations (Schieffelin & Ochs 1983; Shore 1997).

Also, while it's certainly the case that humans do not learn from the neatly hand-crafted features favored in early language emergence studies, their input is most definitely not pixels (and, arguably, it is something more abstract than raw visual stimuli, see, e.g., the work on "object biases" in language acquisition).

Method:

The description of the experiment sometimes refers to the listener having to determine whether it is seeing the same image as the speaker, whereas the game consists in the listener telling whether it sees the same object as the speaker. This is important, since, crucially, the images can present the same object in different locations.

Section 2.2 is very confusing, since the obverter technique has not been introduced, yet, so I kept wondering why you were only describing the listener architecture. Perhaps, current section 2.3 should be moved before 2.2.

Also in 2.2: what is the "meaning vector" that BAtali's RNN hidden vector has to be close to?

It's confusing to refer to the binary 0 and 1 vectors as outputs of the sigmoid: they are rather the binary categories you want the agents to approximate with the sigmoid function.

The obverter technique should be described in more detail in the main text. In particular, with max message length 20 and a vocabulary of 5 symbols, you'll have an astronomical number of possible inputs to evaluate at each step: is this really what you're doing?

Experiments:

distinctness: better call it distinctiveness

Is training accuracy in Fig 4 averaged across both agents? And what were the values the Jaccard similarity was computed over?

It's nice that the agents discovered some non-strictly agglutinative morphology, consisting in removing symbols from the prefix. Also, it looks like they are developing some notion of "inflectional class" (Aronoff 1994) (the color groups), which is also intriguing. However, I did not understand rules such as: "remove two as and add one a"... isn't that the same as: "add one a"?

Discussion and future work:

I didn't follow the discussion about partial truth and using negations.

There is another ICLR submission that proposes a quantitative measure of compositionality you might find interesting: https://openreview.net/pdf?id=HJGv1Z-AW

review_title: very interesting paper, more details would help

Meta-review: This paper investigates emergence of language from raw pixels in a two-agent setting. The paper received divergent reviews, 3,6,9. Two ACs discussed this paper, due to a strong opinion from both positive and negative reviewers. The ACs agree that the score "9" is too high: the notion of compositionality is used in many places in the paper (and even in the title), but never explicitly defined. Furthermore, the zero-shot evaluation is somewhat disappointing. If the grammar extracted by the authors in sec. 3.2 did indeed indicate the compositional nature of the emergent communication, the authors should have shown that they could in fact build a message themselves, give it to the listener with an image and ask it to answer. On the other hand, "3" is also too low of a score. In this renaissance of emergent communication protocol with multi-agent deep learning systems, one missing piece has been an effort toward seriously analyzing the actual properties of the emergent communication protocol.  This is one of the few papers that have tackled this aspect more carefully. The ACs decided to accept the paper. However, the authors should take the reviews and comments seriously when revising the paper for the camera ready.
Decision: Accept (Poster)