Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: This seems to be an ambitious paper that claims to replicate the phenomenon of “spontaneous” learning of compositional language (as in Choi 2018) under relaxed constraints.  It extends the two-agent description game, which agents swap between Teacher and Student roles (“obverter technique”) into a multi-agent game in which messages can be rejected.

There are problems with clarity, particularly after page 4 (Sections 3.2 and 3.3), starting with the paragraph about how their architecture differs from the preexisting architectures. The lack of line numbers in the submission makes it impractical to give detailed feedback.

Overall I have had a lot of difficulty understanding the proposed method, and would have needed more time (or more background knowledge) in order to properly evaluate this paper.
review_title: Official Blind Review #4

final_score: rating: 3: Weak Reject
review_content: This paper proposes a framework for emergent communication system using internal reflection, similar to obverter technique in Batali 1998 and Choi et al., 2018, while not requiring explicit supervision. Despite the nice idea, the dire quality of writing makes it extremely difficult if not impossible to understand the proposed model and the results. Also, some of the experimental results seem unnecessary. In Table 3., for example, (1) in §4.4.1 the authors merely list the rules without giving an overall conclusion. (2) I'm not certain what's the significant of the results in Table 3., as these are messages from one trained instance, and I'm sure a different training run will yield different messages.

Although I am strongly leaning towards reject, I am willing to discuss with my co-reviewers if they have a different opinion.

Cons
- There doesn't seem to be a proper baseline in any of the experiments conducted.
- The writing generally is unclear, full of ungrammatical sentences and extremely hard to read: e.g. see examples below. 
     - §3.2: "A language system can emerge by a pure machine, excluding the human viewpoint."
     - §3.2: "The structure, characteristics, and limitations of the messages generated by the machine can be determined."
     - §3.2: "The agents as a whole have a movement to create a system in which languages is unsupervised and one conceptual pact is formed."
     - §3.3: "Receive an image and generate a message based on self-knowledge."
     - §3.4: "This formal rule was from the human perspective".

- The evaluation section is very rambling and contains unnecessary parts that need not be included, e.g. convergence of mean squared error loss between GRU output and latent space z.

Questions
- The internal reflection function part, the core of this paper, is not properly explained until the end. For example, in §3.3, "when the comparison result is lower than the threshold value": what is the "comparison result"? How does one compare a message and another message? Please elaborate on this further.
- In the evaluation section §3.4, the authors keep referring to "dream": "correct answer rate of Dream", "We entered the message into an agent to evaluate the reconstructed Dream". But the authors don't provide a formal definition of "dream" except providing a citation to Ha and Schmidhuber, 2018. Are the readers expected to know what this means? 
- In §3.4, "Jaccard coefficient were applied to test the message similarity between labels". How does one apply Jaccard index to compute similarity between labels? This seems an important part of evaluation but not explained at all.

review_title: Official Blind Review #1

Meta-review: This paper is very different from most ICLR submissions, and appears to be addressing interesting themes.  However the paper seems poorly written, and generally unclear.  The motivation, task, method and evaluation are all unclear.  I recommend that the authors add explicit definitions, equations, algorithm boxes, and more examples to make their paper clearer.
Decision: Reject