Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The authors propose a modified training procedure for population-based training for emergent communication in which specific speaker-listener pairs are constructed, and replace the "free for all" mixing of pairs present in traditional methods.  Constraining the mixing in this way limits co-adaptation, and thus improves generalization and topographical similarity.  In addition, this solution also solves some more long-standing issues, like why scaling to larger populations in traditional training methods has not typically met with significant improvements in emergent structure or task accuracy, and the authors demonstrate how this originates in the loss of the standard training algorithm.

strengths_and_weaknesses: Strengths:

Overall I really enjoyed this paper.  It's very well written, and the way the authors narrow their focus to this one specific issue helps the argument structure come across very clearly.  I thought their argument was convincing, especially when the scope and argument applies to training dynamics in toy scenarios.  The paper contributes important theoretical understanding to population-based optimization for EC, and the effects of controlling population size, a co-adaptation "factor", and mutual intelligibilty.

Weaknesses:
I don't have any major criticism of the work within the scope the authors layout.  However, working against this paper is that the authors make only a passing effort to connect their work to previous related work, some of which may have very similar interpretations.  Two strands of work that I think should be discussed in much more detail and direct comparison are are neural iterated learning and cultural transmission models.  Both approaches are mentioned in the related work, but no direct comparison is made, despite that both approaches apply similar constraints to the population, essentially limiting who is talking to who, and likely reducing co-adaptation.  Considering the improved performance on communicating with novel partners, one would think some comparison to iterated learning, or the Easy-of-teaching (Li 2019) would have been warranted here.

I think by the phrasing of the last paragraph of the related work (L332/333) the authors are trying to position themselves outside of this work, but I don't find it a convincing excuse.  The motivations are similar, the solutions are quite similar, and it would be appropriate to connect to this research, both in discussion and in experimentation.  Ideally I would like to see existing work as baselines in direct comparison to this work, exploring the role of different partitioning patterns have.

Because in the end, I'm unsure of what these results would mean in terms of pushing the field forward.  If iterated learning is more effective on all of the proposed benchmarks, is this just a footnote in the progression of EC research or only relevant to those who are for some reason unwilling to consider known approaches to training that work better than the standard population-based training scheme?  If the authors more explicitly compared their partitioning approach to other population-based approaches, it would be clearer what the impact of the research would mean in a more practical sense.

Intuitively, to me, the other approaches seem more a bit more plausible from a sociolinguistic point-of-view.  Of course there is no need for the partitionings to exist in this circular pattern, but I'm not sure how these results relate to existing work when more realistic communication cliques are used.  Some discussion of partitioning as it relates to a plausible force in shaping human language development would be appreciated.

My review score reflects that I believe this paper has a lot of merit, but comes up short in connecting to existing research and acknowledging extent and similarities of those contributions to a sufficient extent.
questions: Questions are mostly implied by 'weaknesses', but to be clear: how does your proposed method relate to neural iterated learning, cultural transmission, and ease-of-teaching based approaches which study similar dynamics and reach similar conclusions -- typically higher task accuracy and topographic similarity?
limitations: I did not see any explicit discussion of limitations or societal impact, but it is hard to imagine such small scale experiments in this topic having important negative societal impact.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper proposes a partitioned way of training communication in receiver-sender games in a network of receiver-sender agents. A theoretical analysis is carried and the optimality of popoulation training is discussed. The results show that a partitioned approach performs better when compared to a standard fully connected communication approach. In addiiton, the quality of the language learned by the agents is analysed. 
strengths_and_weaknesses: Despite some grammar mistakes/misspelings (I document some below) this paper reads well.

Strengths:
-
The paper analyses theoretically a sender receiver system where there is exchange of messages and builds up to achieve the proposed partitioned setup. In addition, an interesting analysis of components such as language similarity/topographic similarity is carried.

Weaknesses:
-
- in line 74: “sampled from p”; p should be a distribution (or the input space X in this case); according to line 61, p is a probability value. Same problem in most equations throughout the paper, when writing for example “$E_{x\sim p}$”.
- in the objective of the sender in equation before line 111, why is $\pi$ conditioned on m in the second expectation $E_{m\sim\pi_{\theta_i^*(.|m)}}$? In all the previous definitions $\pi_i$ is always conditioned in the input, i.e., x.
- in line 138, the same derivation from section 2.2 is assumed to be valid to the partitioned setting. While I understand the logic for the standard setting (fully connected) I have concerns about whether the derivations still hold for the partitioned setting. I believe this requires some deeper explanation instead of simply stating it. For instance, in section 2 the objective of the sender is given as the average of the objectives of the neighbour senders of the sender i. But according to Fig 1, in the partitioned setting only the receivers have neighbour senders since the senders are only connected to receivers. Same question for receivers, since the neighbours of receivers are all senders according to Fig 1. I believe this should be made more clear.
- in line 104, from my understanding it is stated that the policy of the receiver is equal to the average policy of all the neighbour senders. It is unclear to me how this equality holds even when the receiver has enough capacity. At the optimal level I agree that the policy of the receiver could indeed be some mix of the policies of the neighbour senders but it is unclear to me why this mix is the average. Is this an assumption?

Generally, I have some concerns that I have outlined in this section and some questions below.
questions: - according to line 246, the high standard deviations are due to the fact that different behaviour may show up between agents that are far away in the population. Does it mean that agents are then limited to communicate with their neighbours if they would be placed in a new setting? Based on the results with the distances between pairs (Fig 3) I would say that the distance does not seem to be a big problem.
- in section 6.4 and Figure 5c, the values remain stable until very low values of beta and only drop when mutual intelligibility is almost non-existent, raising questions regarding the importance of a lot of mutual intelligibility. It seems true that the existence of mutual intelligibility is needed for better compositionality, but why is there almost no difference from 0.1 to 0.5? 
- in line 197, if for every sampled pair both objectives for sender and receiver i are calculated, from my understanding then it means that the neighbour senders of receiver i will also be updated (because it receives the message from the other receiver as in Fig 1) without updating their tied receiver. Will this have an impact on the language if many times only the sender of the pair will be updated, as it was initially mentioned to be a problem in line 190?
limitations: In the results, the compositionality demonstration could be improved; although the plots in Fig 5a give an idea of the argument, it would be interesting to visualise in some way the learned languages to a more clear perception of the compositionality. 


Some minor mistakes/misspelings:
- in line 66, blank space at the end, seems that something is missing.
- line 294: “Figure 5c” -> “Figure 5b”
- line 79: “an” -> “a”
- line 94: “analyses” -> “analysis”.
- line 65: C is defined as a set of length |C| including x; thus “containing x and |C| distractors” is incorrect since the number of distractors should be |C|-1.
- line 192: “neighbor” -> “neighbors”
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The research reevaluates the standard training procedure, highlights its shortcomings, and studied the communication of multi-agents at the population level. 
The primary contributions: the paper propose an alternative training procedure that partitions sender-receiver pairs, restricts co-adaptation of receiver agents, and explicitly promotes mutual understanding between various agents. Moreover, the article discovers that languages created in partitioned populations are more compositional, and that population size has an effect, with bigger populations developing more structured languages.

strengths_and_weaknesses: Strengths:
In this study, multi-agent communication is examined from the viewpoint of the population. the idea is relatively creative, Moreover, population-level communication in sender-receiver Lewis games is analyzed, and a novel protocol is proposed, experiments also demonstrate the efficacy of the proposed protocol. 
Weaknesses: The analysis of 6.3 may be unclear. It would not be able to understand it without reading the supplementary materials, but in 6.3 it does not say that supplementary materials are needed.

questions: Can you make your code public?
limitations: Yes.
review_title: null

Meta-review: metareview: The paper investigates the effectiveness of population-level training of multi-agent communication strategies. Based on the finding that agents that interact with one another co-adapt, the paper proposes an alternative training process that "partitions" the population by constructing specific sender-receiver pairs in a manner that reduces co-adaptation. The paper shoes that this partition-based strategy gives rise to a new optimization objective that encourages alignment across the population. Experiments demonstrate the emergence of mutual understanding between agents that have never communicated, and that partition-based training results in language that is more compositional compared to alternative strategies.

The paper was reviewed by three researchers who discussed the merits of the paper with the AC. There is general agreement that the paper provides an interesting discussion of and important insights into the effect of population-based optimization for emergent communication. Based on these insights, the authors propose a novel training procedure that experiments show is effective. Several reviewers commented that the paper is very well written and was enjoyable to read. The reviewers raised several concerns/questions that the authors made a concerted effort to address. However, a notable limitation of the current version of the paper is the lack of qualitative and quantitative comparisons to previous work. While the proposal to update the related work discussion is helpful, the paper should also provide experimental evaluations that compare to existing work, without which the significance of this particular training procedure is unclear.
confidence: Certain
Decision: Reject