Venue: ICLR 2017 workshop

Reviews:

final_score: rating: 7: Good paper, accept
review_content: This work presents an extension of Lazaridou et al., 2017 (another ICLR submission) to communication between agents with sequence of symbols. Due to the complexity of the problem (generating a sequence of symbols rather than a single symbol), the authors switch from using 1-hot symbols (and thus RL) to using Gumbel-softmax distribution, thus allowing for training the agents in an end-to-end fashion by backprop. Similar to Lazaridou et al., they attempt grounding the communication protocol to natural language (at this point it's not entirely clear to me whether the authors trained the sender on caption generation or the receiver on caption retrieval). Interestingly, when this happens, the induced communication protocol reflects properties of natural language  (as measured by the omission score) while at the same time decreasing the agents' communication performance (from 95% to 52%).

The fact that a very similar paper has already been accepted in the same venue takes away some of the novelty points. Moreover, the fact that the communicative success with the grounding task decreases so much hints that the proposed way of grounding is not an effective one (also from the text it's not crustal clear how is the grounding achieved as the KL is measured  between the probability of the messages as produced by the sender and some unspecified p_w(m) distribution). Perhaps, the authors need to look into the strength of the regularizer as it seems to be taking over.  The analysis of the appendix is really interesting, as well as the point about hierarchical coding.

Overall, this is a very intriguing line of research and, as the authors point in the conclusions, many open questions remain. That being said, the current work does feel a bit rushed; many parts are not clear (specifically details regarding the grounding part) and the proposed grounding approach doesn't seem to be effective in terms of communication.

pros:
- extending the rather limited setup of Lazaridou et al. to sequences of symbols, resembling more natural language
- to the best of my knowledge, the use of Gumbel-distribution for text generation is novel

cons:
- lack of clarity, especially in the section about grounding
- proposed grounding method is not effective with regards to communicative success
- rather limited novelty (given the emphasis of the ICLR workshop) as work is direct extension of previous work


review_title: null

final_score: rating: 7: Good paper, accept
review_content: Applying the straight-through Gumbel-softmax estimator for end to end differentiable sequence generation is likely an early instance of a trick we will see used more broadly. This has also not been applied to this task before with previous approaches either being continuous or RL. That the process is the same during training and testing is a welcome benefit.

The analysis into the "language" of the agents has fascinating qualities but is also likely representative of encoding the pre-existing knowledge of the VGG-19. With that said, the hierarchical aspect that is captured would not have been carried through the pre-existing knowledge and is a promising insight.

I'd have been interested in seeing one more example of qualitative analysis beyond 5747 simply due to curiousity and whether the "animal" aggregation would continue cleanly to other categories.

As noted by the other author, additional clarification regarding the grounding towards natural language would be welcome. For future work, other aspects of grounding are possible, such as penalizing longer resulting sequences, given that "redundancy in encodings" as noted in the paper doesn't seem a favourable trait for natural language.

While many questions are left open for future exploration I believe this work is interesting and introduces some new insights into communication between intelligent agents in such a setting.
review_title: null

Decision: Accept