Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: What is the specific question/problem tackled by the paper?

The paper studies whether discrete communication channels between agents are low-entropy. The claim is that agents that try to solve a prediction task subject to a communication bottleneck will exchange low entropy messages, even if these messages are not explicitly encouraged to have low entropy.

Is the approach well motivated, including being well-placed in the literature?

The paper is well motivated, though I am not an expert in the area.

Does the paper support the claims? This includes determining if results, whether theoretical or empirical, are correct and if they are scientifically rigorous.

The support for the claims is almost adequate. The entropy of the messages was only analyzed for the runs where agents have successfully learned to communicate in order to solve the task. Part of the paper's conclusion is that an entropy constraint on messages is not necessary, but maybe it still is necessary to increase the frequency of successful runs, or help faster learning.

This means that successful runs lead to low-entropy messages, but what about the unsuccessful runs? Do messages have low entropy as well? 

I am also somewhat confused by the second set of experiments. The discussion seems to suggest that setting higher temperature in GS creates pressure for lower-entropy messages. Buf if that's the case, then there's a controllable parameter that implicitly controls an entropy constraint and it's no longer clear to me that low-entropy is emerging.

Summarize what the paper claims to do/contribute. Be positive and generous.

I think the paper does an interesting analysis and makes an interesting point about the problem being studied. I am a bit confused by how the experimental setup supports the claims and their consequences. In particular, I have some doubts about the claim that entropy regularization is unnecessary.

Clearly state your decision (accept or reject) with one or two key reasons for this choice.

I am voting for acceptance.

Provide supporting arguments for the reasons for the decision.

The paper sets up a clear problem to study and focuses on increasing our understanding around the issue. After reading the paper a few times I am a bit confused about how the experimental setup supports the claims & conclusions. I think the results in Figs. 1-2 adequately support the claim, but the results in Fig. 3 make it unclear whether the temperature parameter is implicitly controlling entropy. The fact that unsuccessful runs were discarded for the first set of experiments limits the implications of the main claim that low entropy emerges, because an entropy regularization might still meaningfully improve the frequency of successful runs. 

Provide additional feedback with the aim to improve the paper. Make it clear that these points are here to help, and not necessarily part of your decision assessment.

I think if the paper will be improved if it resolves the lack of clarity around the temperature in GS being an implicit entropy-regularization parameter. Perhaps an entropy-regularized setup is a useful comparison to show that it provides marginal benefit over the setup studied, and this might resolve the lack of clarity around the implications of the claims made from the first set of experiments.

Apart from these issues I am happy with the choice of topic and execution of the paper. I also appreciate that due care has been taken to present the work as understanding a phenomenon, to avoid any misconceptions about a new method being proposed. 
review_title: Official Blind Review #2

final_score: rating: 1: Reject
review_content: This paper sets up a couple discrete communication games in which agents must communicate a discrete message in order to solve a classification task.  The paper claims that networks trained to perform this case show a tendency to use as little information as necessary to solve the task.

I vote to reject this paper.

The experiments are not very interesting and I don't at all agree with the assertion of the paper.  The paper claims the networks use only the entropy necessary to solve the task, but there are two main problems with this assertion.  (1) their own experiments don't support this all that strongly, as in the limit of few hidden bits (left half of the x axis in Figure 1), the networks all had noticeable excess information, and (2) and perhaps most damning the paper applies entropy regularization on the sender during training?  Could it perhaps instead be the fact that the entropy of the sender was penalized as an explicit regularization term that the entropy of the senders messages tended to be small?

I also find the experimental design puzzling.  Why both reinforce and the 'stochastic computation graph' approach?  Treating the receiver's output as binary and stochastic without using the log loss of the bernoulli observation model is just giving up on a good gradient as far as the receiver is concerned.

The experiments done are much to simple and the protocol flawed.

The second set of experiments in Figure 3 were not left to converge, so I'm not sure how we can derive a great deal of insight.  Additionally, relaxing the gumbel softmax channel to being continuous rather than discrete technically ruins any argument that there is an entropy bottleneck present anymore, as theoretically even a single dimensional continuous signal could store an arbitrary amount of information.  If the paper wanted to, it could have upper bounded the mutual information between the input and the message using a variational upper bound.

--------------- Response to Response ---------------------------------

I'm editing here in light of continuing to look at the paper and the responses from the author below.

I have to still argue for a rejection of this paper.  

I thank the authors for addressing my comments and I admit that at first I thought the paper was minimizing the entropy during training which would have been particularly bad.  While I was mistaken on that point, I still believe the paper is deeply flawed.

In particular, the paper makes a very bold claim, namely that "We find that, under common
training procedures, the emergent languages are subject to an entropy minimization pressure that has also been detected in human language, whereby the mutual
information between the communicating agent’s inputs and the messages is minimized, within the range afforded by the need for successful communication."  But if we are being honest here, the experiments are very lacking to support such a bold claim.

In particular there was one thing I was worried about upon reading the paper again, and is similar to the point raised by the other reviewers.  In Figure 1, we are shown the entropy only of those networks that have succeeded.  Naturally to succeed, the entropy i the message must be large enough to accomodate the size of the remaining bits we are trying to reconstruct.  That is why Figure 1 includes the dotted line, since the networks must be above that line to have good performance.  And the main evidence for the main claim of the paper is that the trained networks are above that line and arguably close to it.

Now, we know that there are clearly solutions to these tasks (in particular the Guess Number task) which could achieve good performance at noticeably higher entropy.  For instance we could take any minimal solution and simply split up each message into 8x different buckets, each of which had exactly the same behavior from the decoder.  This would give us a +3 in the entropy of our message space while having no effect whatsoever on the loss.  The claim of the paper is that under normal training procedures it seems like we don't find those solutions and instead seem to find minimal ones.

But after implementing a simplified version of the experiment in the paper (Notebook available here: https://nbviewer.jupyter.org/urls/pastebin.com/raw/ZF7g34GN ) I suspect something much simpler is going on.  The reason the solutions look minimal in Figure 1 is probably because the initialization chosen for the encoder they used in the paper tended to start at low entropies.  Imagine if all of the networks started out with an initial message entropy of around 3 bits.  Then Figure 1 could be explained by the problems with hidden bits ~< 3 bits simply preserved their entropy, which in order to solve the task with higher numbers of digits hidden we know requires some minimal budget, so they get sort of pushed up.  This could explain the figure, but we wouldn't claim this explains why we observe small entropy for the high number of hidden digits case.

In particular, if we initialized the encoders with higher entropy, we might expect that we fail to see this phenomenon.  That is exactly what I was able to show for myself in that notebook.  If you simply initialize the encoder to have high entropy, all of the solutions have high entropy and the observed effect goes away.

Overall, the paper as I said is low quality.  Several choices were made that don't make a lot of sense.  With the experiments being as small scale as they were, why not explicitly marginalize out the message (as I did in the notebook)?  Why use single layer neural networks to predict 256 x 1024 parameters? Why not just learn them directly?   If the paper aimed to mimic more standard setups and show that under those setups we observe this kind of minimal message entropy, then it would have to much better tease out the effects of all of these choices. 

Why does the decoder use a mean field sigmoid bernoulli observation model to try to predict something is in one of ~32 states?  The missing digits are not independent given the message, why model them as so?  Is that part of the purported reason why these models show minimal entropy (cause it isn't discussed).  

For such a simple problem, you could presumably analytically compute the gradient with respect to the loss and study whether that correlates with the gradient of the entropy.  There are several things I could imagine checking, none of which are checked in the paper.

The primary question the paper addresses is an interesting one.  But this paper does very little to carefully investigate that question.  I maintain my vote to reject.
review_title: Official Blind Review #4

final_score: rating: 6: Weak Accept
review_content: This paper investigates the phenomenon of entropy minimization in emergent languages. The paper studies two simple speaker-listener prediction tasks where the amount of information that’s needed to be encoded in the speaker’s message can be manipulated. They find, in both games and across training methods, that speaker agents generally learn to minimize the amount of information conveyed in their messages (i.e. minimize the entropy), such that they are still able to solve the task. The paper then conducts some experiments measuring the robustness to overfitting, and find that more discrete channels leads to better generalization when training on partially shuffled labels.

Overall, this paper is very well written. I think the main result, that speaker agents learn to only convey as much information as is needed to solve the task, is interesting and insightful. I do, however, have a slight concern about the novelty of this result. Given that the environments are very simple (consisting of a single message sent by a speaker and a prediction made by a listener), the line between ‘multi-agent emergent communication’ and ‘neural network with discrete representations’ is very blurred. This is addressed only briefly in the related work section, with a handful of post-2016 papers cited. I’d personally want to see a more expanded related work section that goes into some more depth into some of these papers, to be able to better judge the novelty of this contribution. 

I found the second result, that making the representations more discrete (by lowering the temperature in Gumbel-Softmax) leads to increased robustness to overfitting, to be less clearly explained, especially how it relates to the first result. It seems obvious to me that, if you can’t pass enough information from the speaker to the listener (when there’s a low temperature), then you won’t be able to solve the task at training time if a lot of information needs to be conveyed (as in the case of randomly shuffled labels). The authors describe this by saying: “With a low temperature (more closely approximating a discrete channel), this is hard, due to stronger entropy minimization pressure.” But this seems misleading to me, since it’s very different from the ‘entropy minimization pressure’ that was discovered in the first result (which comes about when both agents are able to solve the task, but the listener has redundant information). Thus, I don’t see how this result is either surprising or connected to the first result. 

Further, the main claim to novelty of this second result is that it tests the information bottleneck principle in a ‘language learning’ set-up. However, since now the communication is continuous, this setup resembles the classic ‘neural network prediction’ even more closely, and calling it a language learning setup seems down to semantics. Given this, I’m unsatisfied with the comparison to previous work on the information bottleneck (which is not my area of expertise). 

Given the above points, I’d say the paper is borderline it its current form, with a tendency towards rejection. However, I’d be willing to increase my score if the authors can clarify some of my confusion around the second experiment. 

UPDATE: I've increased my review to a 6 after the author rebuttal, although I still feel the paper is borderline. 

review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: The authors present a series of simple experiments to characterize the objective that emergent languages are optimizing and why we see various behaviors (not aligned with natural language) when training them to play these language games.  The paper is clearly written, the tasks are simple and minimalist (in a good way) and the experiments are often followed by the exact ablation I was just noting that I wanted to see. 

Questions:
1. What happens if the vocabulary is too small to completely communicate the space?  Do models accept that some concepts are not expressible, discover a random behavior that overloads a lexical item, or fail to learn entirely?

2. Why are values for lamda_r not investigated or plotted?

3. How many runs are averaged for each experiment? Should I assume one per seed? why are there different numbers of seeds for each experiment?

4. Page 5 -- "We conclude that in this case there is no apparent entropy minimization pressure on Sender simply because there is no communication" -- Was any analysis performed on the gradients to see if they are completely random or what kind of signal they are getting?

5. How important is the architectural design to these experiments? e.g. number of hidden layers, etc?
review_title: Official Blind Review #1

Meta-review: This paper studies the information-theoretic complexity for emergent languages when pairs of neural networks are trained to solve a two player communication game. One of the primary claims of the paper was that under common training protocols, networks were biased towards low entropy solutions. During the discussion period, one reviewer shared an ipython notebook investigating the experiments shown in Figure 1. There it was discovered that low entropy solutions were only obtained for networks which were themselves initialized at low entropy configurations. When networks are initialized at high entropy configurations, the converged solution would remain high entropy. This experiment raises questions about the validity of the claim that there was "pressure" towards low entropy solutions to the task. Therefore, a more careful analysis of the phenomenon is required. 
Decision: Reject