Emergent Communication of Generalizations
Authors: Jesse Mu
Noah Goodman
Venue: NeurIPS 2021 Poster (2021)
Keywords: emergent communication
multi-agent communication
language grounding
compositionality

Abstract: To build agents that can collaborate effectively with others, recent research has trained artificial agents to communicate with each other in Lewis-style referential games. However, this often leads to successful but uninterpretable communication. We argue that this is due to the game objective: communicating about a single object in a shared visual context is prone to overfitting and does not encourage language useful beyond concrete reference. In contrast, human language conveys a rich variety of abstract ideas. To promote such skills, we propose games that require communicating generalizations over sets of objects representing abstract visual concepts, optionally with separate contexts for each agent. We find that these games greatly improve systematicity and interpretability of the learned languages, according to several metrics in the literature. Finally, we propose a method for identifying logical operations embedded in the emergent languages by learning an approximate compositional reconstruction of the language.

Emergent Communication of Generalizations
Jesse Mu
Noah Goodman Stanford University
Stanford University muj aerj@attaord....
ngoodmanSstanford..c edu
Abstract
To build agents that can collaborate effectively with others, recent research has trained artificial agents communicate with each other in Lewis-style referential games. However, this often leads to successful but uninterpretable communication. We argue that this is due to the game objective: communicating about single object in a shared visual context is prone to overfitting and does not encourage language useful beyond concrete reference. In contrast, human language conveys a rich variety of abstract ideas. To promote such skills, we propose games that require communicating generalizations over sets of objects representing abstract visual concepts. optionally with separate contexts for each agent. We find that these games greatly improve systematicity and interpretability of the learned languages, according to several metrics in the literature. Finally, we propose method for identifying logical operations embedded the emergent languages by learning an approximate compositional reconstruction of the language.
1 Introduction
The communication systems that emerge when two agents are trained to cooperate offer a window on the evolution of human language, well promising avenue for improving the collaboration abilities of artificial agents. Much recent work studies Lewis-style 24 signaling games (Figure [b), where agents are trained refer to a single object shared visual context. However, a general consensus of this work is that without careful environmental pressures, agents develop successful but uninterpretable communication schemes distinctly unlike human language 6 16 20)
We argue that the reference games typically used in these studies are ill-suited to drive linguistic systematicity for two reasons. One is perceptual: agents exploit inscrutable patterns in single inputs, which leads to communication via spurious features [3]. The other reason cognitive: human language can convey abstract ideas, such kinds and causes, not only reference specific objects. Simple reference games are unlikely to drive emergence of such abstract language. In particular, generalizations over categories may be useful in the future. For example, we would like to teach our kin not just to avoid one specific lion, but avoid all lions, including those that have not yet been seen. Some have even argued that language emerged precisely from this need to teach hard-won generalizations to others [19]. With this idea in mind, can we design an experimental setting that better catalyzes these abilities?
crucial part of language (36], helping
transfer knowledge that
In this paper, we propose extensions Lewis-style signaling games to sets. the set reference (setref) student not just single object, but rather group of objects
game, teacher must communicate belonging to a concept (Figure Tp). In the concept game, each agent sees different examples of the concept (Figure l..) Inspired by human teaching (9. our insight is that requiring generalization to combinatorially ly large sets of (possibly unseen) objects encourages agents to learn and communicate rich abstractions across inputs (e.g. seagulls), instead of low-level features (e.g. color #FDA448). These tasks are more difficult than traditional reference games, and we will show with a variety of metrics that the learned languages more systematic, compositional, and interpretable. Finally, the rich compositional space of concepts explored in these games allows us to probe for specific logical
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

(a) reference
(b) set reference
(c) concept
Figure I: Communication games for the concept red triangle. Given a set of targets (red borders)
and distractors, teacher must send a message to help student identify the targets. In (a) reference games, targets are identical; (b) set reference (setref) games, there are multiple targets; and in (c) concept games, the agents see different inputs.
operators in the emergent language. We propose a method for doing so, thereby demonstrating how the emergent languages reflect the compositional structure of their inputs.
2 Communication Games
First imagine generic communication game between a teacher T and student S. Let G (c, XT,Y"
y be communication game, where X + {0, 1} is latent concept to be communicated, X = = xn") is a set of n inputs presented to the teacher, and Y = =Vvi ..... Vn is set of labels for the teachers' inputs, defined as v c(x?)) We call x larget if yf 1, which indicates that x²." is member of the concept otherwise a! is distractor and yu" 0. X and Y are defined similarly for the student. Given its targets and distractors (but the latent concept c), the teacher must send a message m student that allows them correctly identify their own targets, where m (m...... ,, is a discrete sequence over a fixed alphabet m; M. Now we can define variants of this communication game as follows:
Reference game. In basic reference games, the teacher and student see the same examples (XT X Y = =) and there is single (repeated) target: x! x for all i.j where yf y
Set reference (setref) game. Now we extend our game to sets: the teacher and student see the same examples, but there are multiple target images encoding the concept (e.g. different red triangles).
Concept game. Finally, we propose the more abstract concept game, where the teacher and student different examples (XT + X yT * YS) of the concept. When X and YT contain single positive and negative example, this is reference game with separate inputs for each agent, a setup which has been shown to encourage linguistic systematicity in some settings [ 21 22
3 Models
Now we will formalize our models of the teacher and student. Given a communication game G, teacher is defined distribution over messages given inputs p" (m X Y"), and student is distribution over targets given a message: p PYY X m) = II. P (y) ..)
Teacher. The teacher encodes all inputs with a convolutional neural network (CNN) fe embed- dings for targets and distractors averaged to form target and distractor prototype embeddings which then conditions recurrent neural network (RNN) used to produce the message. Let XT and X denote the sets of targets and distractors in X"', then define prototype embedding
33
'For the most consistent comparison across games, our reference game has multiple identical targets and student target decisions made independently for each input, instead of the single-target forced-choie setting. Appendix shows results with traditional games trained with cross entropy loss; conclusions are the same.
"Note that the teacher's job is one of representation learning for sets [42) and thus one can use any set repre- sentation learning method beyond the prototypical networks explored here As a more advanced implementation. we tried variant of Set Transformers [23]. but this did not give any tangible performance benefit.

---

blue OR rectangle
bill:cone AND
breast:red AND...
0 o
cose
teak
[e
[1
8
8]
[0
8]
Figure 2: Example games, with targets (red border) and distractors for the Shape World concept blue OR rectangle (left) and the Birds concept painted bunting (right). Concepts are represented as intensional logical formulas (top) or extensional sets of boolean input features (bottom), where each vector is the binary representation an individual member the concept (e.g. blue rectangle, or the labeled attributes of one particular bird). See Figure SS Appendix A for additional game examples.
x
IXI EmExT I8 (xx) (analogously for x)). Then p" (m | X", (Y) PRNN DECODE (m
proj([x[;; xI)) where proj is a linear projection to the RNN hidden state.
Student. The student takes a message and makes predictions about the labels of independently for each input v.S. Given the teacher message and an input image, define y (y x m) = (RNN-ENCODE(m)) f (x."), where f is separate CNN for the student.
We jointly train suucher-student pair, including the vision modules and communication protocol, via stochastic gradient descent to maximize the student likelihood of selecting the correct target images. Formally, the loss for single game is
CT, G) = Σ logr (y m)), in ~ p" (m X ,²,Y)..
(1)
To maintain backwards differentiabilityy we use the trraight-throue Gumbel-Softmax 14 trick with 7 = 1, simulating samples from the teacher distribution over tokens via softmax and discretizing in the forward pass only. For full model and training details and a link to code, see Appendix A
Tasks
We examine the languages developed for our proposed communication games over two datasets: first, an artificial shape dataset which allows us to evaluate communication over cleanly defined logical concepts; second, dataset of birds to test agents' ability to learn concepts from realistic visual input.
World. We use the Shape visual reasoning dataset [8] (Figure 2 left). For reference games, target images are single object, defined by a conjunction of a shape and color (e.g. red triangle, green square); of the 30 possible shapes, we hold out 20% for testing. For setref and concept games, concepts include the conjunctions tested in reference games, but also primitive concepts (e.g. blue shapes) and arbitrary disjunctions or conjunctions of (possibly negated) shapes and/or colors. This produces 312 concepts, 20% of which are held out for testing. These more general concepts cannot be tested in reference games, since a single object is always identified by a shape and color. This rules disjunctive concepts like red OR blue that only make sense across multiple objects. Similarly, since reference game targets must necessarily have both color and shape, we can never guarantee that a message for reference game only carries the semantics blue and not, for example, blue circle, if the target is a blue circle. By looking at sets, setref and concept games allow us to more precisely control the semantics of the concepts each game.
Each game consists of 10 targets depicting shapes satisfying the concept, and 10 distractors. We specifically sample "hard" targets and distractors to test understanding of conjunctions disjunctions (see Appendix B for details). Finally, we specify an agent vocabulary of 14 tokens and maximum length so that the communication channel has the same bandwidth the true concept formulas see Appendix C for experiments varying these parameters for both this dataset and the next one.
Birds. We next the Caltech- UCCD Birds dataset [40 which contains 200 classes of birds with 40-60 images (Figure right). As before, reference games involve a single target; setref and concept
II the true concept formulas there are 5 shapes, 6 colors, and the 3 AND/OR/NOT operators, i.e. 14 tokens; and the longest concept formulas have the form NOT AND NOT y, i.. length 5.
3

---

game targets are members of a specific bird class. We use 100 classes at train and 50 at test, sampling targets and 5 distractors per game. The dataset contains boolean attributes (e.g. beak, size) for individual birds and classes Thus, Thus, we represent reference game concepts as the feature vector of the target bird, and setref/concept game concepts as the feature vector of the class. In our evaluation, we will measure how well the languages capture these features. As there is no reference language for this task, we set the vocabulary size to 20 and the message length to 8 (though again see Appendix C
5 Evaluation
We first measure communication success, as defined by student accuracy on held-out games from seen and unseen concepts, with the unseen concepts testing a language's ability to generalize compositionally. Ultimately, however, we are interested in the systematicity of the learned languages, which we evaluate via the following measures:
Information theoretic measures. We first ignore the specific content of messages and concepts, and simply compute simple information theoretic quantities. by treating each distinct message and concept as a unique value and imagining probability distributions over these values. First, we measure the conditional entropy of teacher messages given concepts, H HM C)), averaged across seen and unseen games; lower entropy indicates that agents use more consistent language for fixed concept. However, (MM C') measures systematicity only in one direction; as more symmetric measure, we also use the adjusted mutual information
(2) Here, (M,, C)) is evaluated with respect to a hypergeometric model of randomness, where M and are assumed to be random permutations subject to the number of unique values in either set [39]. AMI E [0, 1] represents the mutual information between M and C, adjusted for chance to maintain comparability across different distributions of messages and concepts. A higher score indicates overall higher alignment between messages and concepts.
AMI(M, C) (I( (MI C))) (M)) H(C) B(I(M,))))
Topographic p. To more precisely measure the lexical compositionality of a language, a measure often used in the literature is topographic o 4 20 22 255, which quantifies the agreement between two representational systems a la Representational Similarity Analysis [17.. We define a distance metric between game concepts (G, C₃) and another between agent messages dy (M; my compute pairwise distances between concepts and between messages, and measure their alignment with Spearman's 's p. A high p indicates that teacher sends lexically similar messages for similar concepts.
For our distance function on messages, we use the Edit (i.e. Levenshtein) distance with equal insert/delete/replace costs. For distances between game concepts, we define two distances based on intensional and extensional representations of concepts (Figure First, we use the word-level Edit distance between string representations of logical formulas. Second, we use the Hausdorff distance dH, a distance function between sets of members of a concept. Let Z ; =4 be the set of feature-based boolean representations of inputs belonging to concept For ShapeWorld, these are two-hot vectors denoting the color and shape of all objects belonging to a specific formula; for example, for the concept red, we have vectors for red triangle, red circle, and so on. For Birds, these are boolean vectors of the attributes of each individual bird of a species. Then the Hausdorff distance du is the maximum distance from any point in one set to the closest point in the other: du (Z", Z²") (sx( 2(2 Zd), sup, d( a(2).- z))), where d(a, B)= = inf EditDistancc(a, b).
6 Results
Table T shows test accuracy, as measured by student classification accuracy (partial credit given), on communication games over seen and unseen concepts for 5 models trained in each condition. Reference game performance high across both datasets, and agents are able to generalize well to unseen games. Accuracy on setref and concept games is lower, with lower performance on novel games in both datasets Overall, communicating sets is a much harder task than specific reference.
FFeature vectors for individual birds a class vary due to the visibility of features the image: class vectors are averaged across all individual birds, then rounded to or O.
To calibrate setref and concept performance, Appendix D tests listeners ideal (human) languages.

---

Table 1: Student accuracy (seen and unseen concepts, where chance accuracy is 50%), conditional entropy of messages given concepts (lower is better), and adjusted mutual information score (higher better), with (SD) across runs.
Dataset
Game
Acc (Seen)
Acc (Unseen)
H ()
AMI(M, C))
ShapeWorld
Ref
97 (0.4)
98 (0.3)
7.3 (0.2)
0.04 (0.00)
Setref
92 (2.2)
87 (1.6)
3.9 (0.6)
0.59 (0.08)
Concept
88 (3.4)
75 (3.0)
2.4 (0.2)
0.66 (0.07)
Birds
Ref
93 (0.3)
89 (0.1)
(0.2)
0,05 (0.00)
Setref
89 (0.2)
78 (0.2)
(0.1)
0.17 (0.02)
Concept
88 (0.1)
73 (0.3)
4.1 (0.2)
0.26 (0.02)
The ability to communicate accurately, even unseen concepts, is not necessarily indicative of more systematic communication; generalization without compositional language a common finding in Instead, we find that the more difficult games produce more systematic language. For Shape World, concept game entropy over messages is lower than reference game entropy (2.4 vs. 7.3), and AMI is higher (0.66 vs. 0.04), with setref in the middle; this pattern also occurs in Birds. Furthermore, Figure Tigure]Jshows shows that topographic p between the languages and the (Edit and Hausdorff) concept distances is higher for concept and setref than ref, throughout training.
the literature 6 16 22
Figure 4 (more examples in Appendix F) shows messages generated by agents for concepts in both games, where arbitrarily assign letters to agent "words" to assist with interpretation. For example, for concept game teachers conveying the concept red AND triangle, the innermost cirele is largely red, indicating that the majority of messages sent for this concept begin with d; proceeding outward, see blue regions indicating the token e. Thus, concept agents consistently use the 5-token sequence deeee to refer red triangles (less commonly, edeee). In contrast, for different red triangles, the language of reference game agents strikingly inconsistent, as indicated by the extreme diversity of colors, while setref language is somewhere in the middle. The entropies over the message distributions for these concepts correlate with the "busyness" of the plots, reinforcing the idea that the setref and concept languages are more consistent.
6.1
Set Size
The difference between reference and setref/concept games can be interpreted as a continuum, ranging from referring to single object (reference) to referring to potentially infinite num- ber of objects. With a small number of objects, it may still be possible to communicate only low-level, non-generalizable features of the set, similar the strategies adopted by our refer- ence game agents. In contrast, increasingly large numbers of objects should put further pressures on the semantics of the messages to not refer to individual inputs, but rather entire categories.
In Figure we confirm this hypothesis, show- ing how increasing the number of targets n (with equal numbers of distractors) increases
ShapeWorld
Birds
0.6
Topographic
0.2
language systematicity. n has a statistically significant effect on topographic p for Shape- World setref (Spearman = 0.39,p- = 0.029)
0.1
0.2
and concept (p = 0.75, < 10-5) and Birds setref (p = 0.90, < 10-7) and concept (p 0.51, p 0.024). When n tref game is equivalent to a reference game with I target and distractor, and the concept game is similar, but with agents given separate inputs. Our results suggest that this decoupling, as pro- posed by Lazaridou et al. [1 and often used the emergent communication literature, pro- motes systematicity in some cases (Birds) but not others WorlpeWorld). We additionally show that (1) sets are an alternative way of encouraging systematicity without needing this separation, and (2) even with this separation, larger set sizes further improve the systematicity of the resulting language.
0.0
1
10
2
3
targets
# targets
the se-
Setref
Concept
Figure 5: Topographic (concept Edit distance) with varying number of targets (average over seen/unseen splits). Each point and Ref line is an independent run.
5

---

ShapeWorld
Birds
Seen
Unseen
Seen
Unseen
0.6
0.15
0.10
0.4
Edit
0.05
0.2
0.00
0.0
Ref
0.25
0.5
Setref
0.20
0.4
Hausdorft
Concept
0.15
0.3
0.10
0.2
0.05
0.1
0.00
0.0
25
50
75
100
25
50
75
100
25
50
75
100
25
75
100
Epoch
Epoch
Figure 3: Topographic between language and Edit (top) or Hausdorff (bottom) concept distances for seen and unseen games across both datasets. Results from runs plotted, with averages bold.
Ref
Setref
Concept
Ref
Setref
Concept
Laysan Albatross
red AND triangle
6.51
3.34
H 0.66
=
H =
= 5.93
Painted Bunting
red OR white
N/A
H 5.03
2.32
H=5.61
H 1.71
= 1.78
m
Figure 4: Distribution of 300 messages and entropies for game concepts ref, setref, and concept settings. Messages start at the center and proceed outwards, with each colored section corresponding to unique token and its frequency of occurrence that position the message. Empty spaces indicate end of sentence. For convenience, Shape World plots are partially labeled with tokens.
6.2 Generalizing across game types
To measure the generality of the agents' strategies, we evaluate their ability to generalize zero-shot across game types. Tiboes accuracy and systematicity metrics for agents evaluated on games of different type. Agents trained the setref and concept games able generalize reference games (yellow cells), producing systematic referring expressions, as they have already been biased towards generic language. Importantly, setref game agents can generalize to concept games with separate inputs (magenta cells), though to a lesser extent for Birds (75% vs 84%). This suggests that sets pressure agents learn generalizable features despite the shared input. In contrast, we see little generalization ability from reference games to setref and concept games (orange cells), suggesting that agents are not conveying generalizable features, rather spurious patterns in the input [
Probing for compositionality
The richer set of concepts afforded by our setref and concept games allow for more detailed analyses of the emergent languages beyond the standard metrics presented above. Broadly, we are interested in whether the compositional structure of concepts is reflected in the language. For example, our agents produce messages for the primitive concepts red and friangle, well as the conjunctive concept red AND triangle [ Natural language equipped with a composition operator, AND(m1, m22) = my AND m2, that operates solely on lexical forms and whose meaning is defined as the conjunction of the meanings of its arguments. Does similar operator exist in the emergent language (Figure 61)
We cannot do this analysis for reference games, since cannot test primitive concepts; recall Section

---

Table Accurey/Mi//Poorrapicc p (concept Edit distance) for agents trained on different game types (columns), then evaluated (zero-shot) on different game types (rows). Chance accuracy is 50%. Gray shaded cells indicate standard test-time evaluation; other cell colors are explained in the text. Note that for Shape World reference agents, we evaluate only on setref and concept games that use the 30 conjunctive concepts tested reference games (e.g. red triangle, blue square).
Train Ref
Train Setref
Train Concept
ShapeWorld
Eval Ref
(0.4).00 (.009.00 (.00)
91 (4.3)/44 (.09)/422 (.II)
(5.6).60 (.04M/63 (.08)
Eval Setref
56 (0. 10..1 (.00)/00 (.00)
90 (1.3)/559 (.08V.12 (.03)
83 (6.7) (.07M.09 (.01)
Eval Concept
50 (0.0)/00 (.00):00 (.00)
90 (4.6)/55 (.08V.12 (.03)
82 (3.0V.66 (.07).09 (.01)
Birds
Eval Ref
91 (0.8W.05 (.00).04 (.01)
85 (1.1)/.14 (.02).16 (.02)
82 (0.9).20 (.02M.15 (.01)
Eval Setref
64 (1.7M.03 (.01).15 (.01)
84 (1.1).17 (.02Y377 (.04)
78 (0.7V.26 (.02M.40 (.03)
Eval Concept
(0.9)/.03 (.01M.15 (01)
75 (0.9)/.17 (,02).37 (.04)
(0.6).26 (.02y.40 (.03)
We propose to learn such an operator by training model to compose messages in the emergent language to form their conjunctions. Like the English AND, this operator must be able to combine any of the concepts in the language, and must crucially generalize to novel combinations of features. If, given new concepts, we can reconstruct a message that induces the right behavior in the student, this suggests our model has learned some analog of AND in the language. The reconstruction accuracy of this model can then interpreted as a much more explicit measure of compositionality than the measures explored above: it reveals the degree to which the syntax of the emergent language operationalizes the specific logical operations present in the underlying space of concepts.
Our method is inspired by the Tree Reconstruction Error (TRE) metric proposed by Andreas [. which learns compositional approximation to representation space, assuming the latent compo- sitional structure is known However, there are several crucial differences in our method that are optimized for probing for linguistic structure. First, we learn a set of arbitrary composition operations, instead of imposing predefined operation (e.g. elementwise addition). Moreover, these learned composition operators are actually valid and interpretable linguistic transformations on messages, rather than operations (like addition) that work only internal representations of sub-concepts. And finally. we evaluate our operators on held-out concepts, examining how the reconstructed messages serve the ultimate communicative goal: inducing the correct generalization behavior in the student.
7.1 Learning an Approximate Compositional Reconstruction (ACRe)
Let us first formalize the learning problem: after training our agents, we have dataset of message and concept pairs T {mm, c generated by teacher for each game. Each concept is one of set of logical forms C(C) defined inductively over a set of primitive concepts (e.g. red, triangle) and composition operations Q as follows:
1. Every primitive concept is in L(C): C C(C).
2. Every composition of concepts is a concept: let Q, be the set of n-ary composition opera- tions. Then V'n (2,,02..... G.) L(C)", w E Ωn, we have w(( 02 ... Cr.) E L(C).
.c.)
T defines a probability distribution over messages given concepts: pr (m c). Our aim is to learn Approximate Compositional Reconstruction to these messages Pry (m c), composed of p-parameterized message distributions that factorize along the compositional structure of L(C))
(m) Ex ((mllee [P (m mi....., m.))
if c E C, i.e. is primitive if c = C())...
(m c)
We cannot apply TRE directly to our setting. Andreas ① applied TRE to standard reference game, where targets are represented conjunctions of shape and color features (e.g. blue square, red triangle). As we mention Section because reference games cannot test primitive concepts like and triangle, Andreas (I] proposed to learn representations for primitives which can then be composed via some (predefined) composition function. However, in our setting, it makes little sense to learn arbitrary representations for primitive concepts, when we actually have real messages for such concepts in the first place, hence the method we propose.
7

---

m,
NOT
m, AND m
pn
Pq
P'y
NOT
d
AND
red
m,
triangle b
ttiangle
AND
(a)
(b)
(c)
Figure 6: Our ACRe procedure. (a) Does lexical analog of AND exist in our emergent language?
(b) We first train primitive LMs to mimic the distribution of agent messages given a fixed concept. (c) We then train composition operations by sampling arguments from primitive LMs, then training a seq2seq model to mimic the agent message produced for a higher-order concept.
where Pry (m) is a model of the distribution of messages for primitive concept and Pi (m m...... mn) is a learned lexical analog to operation w that takes in messages and outputs distri- bution over composed messages. Given a concept c, we can sample a message from Pa (m c) by
either sampling directly from pr (if с is primitive) or recursively sampling messages m, from the constituents of c, then sampling from the corresponding (77-
We implement ir, and pr as small 2-layer transformer- based language models (LMs) B8 For each e, P,, is an unconditional LM (i.e. we have an LM for red, triangle, etc.). For n-ary operations w, Pyj is requence-to-sequeee
(seq2seq) model decoding from the concatenated arguments: 8 (m
my..... m.,) = p deccode (m] m, [SEP] m2 [SEP] m,)). Note that this formulation imposes few constraints on the mechanism of the composition operator: for example, we are not enforcing that there exists token denoting conjunction (AND), or that the arguments must copied verbatim into the output. These constraints, generally true of human languages, could be explored in future work.
To train ACRe, let To {(m,, ci) T c c} be the set of message-concept pairs with concept and Tw {(mi, c.) E T G; w(-)} be the set of messages where ei uses w. Now, for each primitive models, for n = 1,...,N, N, E Pn, train the composition model pi on Tw Specifically, given a pair (m, w(c1.... ). first sample messages for the sub-concepts c; from our frozen models: my ~ (m,, c Then, train the composition model to maximize P mi....., m,,) For example, given a message bca for the concept red AND triangle, we first sample messages for red and triangle from pad and Pi pipiagle then train AADD to decode bca from these messages (Figure 6).
train the LM pú, Te to approximate pT c). Then, freezing these
Full training and model details are in Appendix
7.2 Evaluation and results
After training, each / is a learned lexical analog to the composition operation w. To test whether approximation has learned the semantics of the language, we hold out 20% of conjunctive and disjunctive concepts e; during training and sample messages for these concepts from the learned (m,m <) according to Equation 3 We evaluate how well these reconstructed messages encode the concepts via (1) lexical overlap with the true teacher messages (as measured by BLEU) and (2)
student performance when given language and a game for the corresponding concept
Results are in Table 3 The upper bound on performance is given by the true Teacher language, PT Pm(m c). We also use language sampled randomly from (1) teacher messages for any concept PT (mm) (Random) and (2) teacher messages for the Closest concept as measured by Edit distance (breaking ties randomly). ACRe's performance is a measure of the degree of compositionality in the language, upper bounded by the teacher language and lower bounded by the random baseline. The results reveal some degree of compositional structure: ACRe reconstructs the training data well and crucially outperforms basclines in predicting messages for unseen concepts. Of course, our ACRe
As presented, this procedure is only possible if the composition models can he trained and frozen in a specific order without circular dependencies. For example, we cannot naively train P", on concepts of the form (w)()), since sampling from the inner w requires an already- trained Pi Learning from arbitrary concepts is possible by backpropagating through samples (e.g. via Guubell-Sfftaal
which we leave for future work. "In these experiments, we run ACRe on agents trained on the full set of 312 concepts in Shape World, since not testing compositional generalization of the agents, but of ACRe.
8

---

Table 3: ACRe evaluation. (SD) across 5 runs. In the highlighted cells we conduct paired t-tests, comparing ACRe to Closest; indicates significance at p < 0.05.
Train
Test BLEU-4
Game
Language
BLEU-1
BLEU-4
Student Acc
BLEU-1
Student Acc
Setref
Teacher
(0.0)
100 (0.0)
(2.7)
(0.0)
(0.0)
86 (5.2)
ACRe
96 (2.1)
73 (5.0)
81 (3.4)
91 (3.9)*
52 (10.0)*
65 (3.4)*
Closest
78 (4.6)
28 (6.2)
48 (0.7)
88 (3.8)
38 (9.0)
56 (0.7)
Random
71 (6.6)
22 (4.3)
50 (0.0)
73 (7.4)
24 (5.5)
50 (0.3)
Concept
Teacher
100 (0.0)
100 (0.0)
88 (4.2)
100 (0.0)
100 (0.0)
84 (4.4)
ACRe
95 (3.4)
80 (7.6)
82 (2.2)
87 (6.9)*
59 (12.2)*
70 (4.0)*
Closest
64 (3.7)
(6.2)
(1.6)
76 (3.2)
38 (6.7)
56 (1.3)
Random
54 (3.0)
19 (2.8)
50 (0.2)
56 (2.2)
20 (3.2)
50 (0.4)
Teacher
Teacher
ACRe
Closest
Random
yellow
yellow AND
triangle (ACRe train)
Student Acc
91%
75%
65%
48%
green
green AND triangle (ACRe test)
triangle
Student Acc
94%
68%
55%
49%
Figure 7: Composition in the emergent languages. A concept game teacher's messages for primitive concepts yellow, green, and triangle, conjunctions yellow AND triangle and green AND triangle (where green AND triangle is unseen combination for ACRe), and predicted messages according to ACRe and other baselines. Color key same as Figure
model trails Teacher language accuracy by 16-20 points. This gap could stem from a failure ACRe to find the correct compositional generalization, lack of compositionality in the language itself.
qualitative analysis supports the interpretation that the compositional operations are not always interpretable. Figure S an example of message distributions for the primitive concepts yellow, green, and triangle, as well as distributions for the conjunctive concepts yellow AND triangle and green AND triangle, with predictions from ACRe and baseline models. The message for yellow AND triangle intuitively composed out of tokens concatenated from both primitives similar to natural language [1]]. However, the message green AND triangle uses tokens from green, but none from triangle, thereby violating the mutual exclusivity principle language (28). Regardless, in both cases ACRe model is able to approximate such messages better than the other baselines. An exciting avenue for future work is encouraging models to develop operators more akin to human language, and evaluating their acquisition by searching among more restricted class of ACRe models Pry.
8 Related Work
Promoting compositionality in multi-agent communication. Compositionality and systematicity emergent languages have long been priori goals multi-agent communication. Such languages may be easier to interpret, teach, and integrate with human language via supervised learning [26)..
Towards this end, large body of work (see 200 for review) has explored what factors might encourage compositionality in emergent communication, such teachability and iterated learning agent capacity [31] and regularization (27). self-understanding (8) and game design [16] 22
25 30
However, most of this existing work operates within the limited paradigm of the Lewis [24 reference game. this paper, we propose to revisit and revise the fundamentally limited reference game objective: inspired by human teaching [ and generic language (36), we encourage our agents to communicate generalizations over objects, which significantly increases linguistic systematicity, orthogonal to any of the alternative pressures proposed in related work.

---

Measures of compositionality in languages. Crucial to the problem of promoting compositional- ity in emergent communication is how to measure it in the first place. The literature has seen a wide variety of methods D 2 31 claiming to more accurately align with human notions of composi- tionality, some of which are reported here. Most of this existing work focuses on outputting broad scalar quantity that represents the degree of compositionality in a language [e.g. topographic p: In contrast, ACRe is a much more granular attempt at measuring not just how much compositionality, but what kinds of compositionality emerge, by actually learning, evaluating, and interpreting each distinct compositional operation.
This brings our work more in line with more precise studies of the emergence of composition operations in emergent languages [4 35 and the analyses of van der Wal al. [37] and Andreas [i] In contrast to Steinert- Threlkeld [34 [5], who studies simpler settings where compositionality can be verified with manual inspection, we propose a way to measure compositionality in more complex languages that clearly do not exhibit perfect compositionality, but may still have learnable latent structure. contrast to van der Wal et al. [37. who use grammar induction techniques for syntactic analysis of emergent languages, a known semantic space of concepts. This lets us approximate concrete syntactic operations in the language, and evaluate how well the approximations capture the corresponding semantic operations. Lastly, our method ACRe builds off of the TRE metric developed by Andreas [. and we describe this relationship in Section 7
tailor our syntactic analysis to messages generated for
ACRe as program induction and grammatical inference. Finally, ACRe is reminiscent of a program induction or grammatical inference problem, where inputs are agent messages for primitive concepts, and outputs are the messages produced after some composition has been applied to the inputs. Our task is to discover the (ideally simple and interpretable) lexical programs that implement the corresponding transformation in semantic space. Because we have no priors over what lexical transformations, if any, the emergent languages might implement, we search for programs among a general class of seq2seq translation models. However, in this domain, human languages have much simpler lexical operators involving concatenation and infix notiation (c.g. x AND y), and in the future, we would like to push emergent languages towards stricter compositionality. One way of benchmarking more cleanly compositional languages is to restrict ACRe models to more constrained and interpretable programs learned with techniques from the program synthesis [1] grammar induction [10] literature.
9 Conclusion
We have proposed extensions of referential games to sets of objects, and found that the need to convey generalizable categories leads to the development of more systematic languages, whether inputs are shared (setref) or unshared (concept) across agents. Moving forward, the richer space of concepts afforded by our setref and concept games are a promising testbed for studying the emergence of higher-level linguistic phenomena, such as quantifiers probabilistic language. Finally, while ACRe reveals some compositional structure in the emergent languages as-is, the learned composition operations are not particularly interpretable. Future work should identify what kinds of environmental or architectural constraints might encourage more transparent composition in learned artificial languages. One challenging evaluation of compositionality along these lines is measure the ability of agents to extrapolate to longer and more complex concepts not seen during training (c.g. green OR (blue AND triangle)), and evaluating ACRe's ability to capture this recursive structure.
10 Broader Impact
Our work investigates agents communicating in artificial and isolated environments, and thus has limited immediate societal impact. However, we can imagine that with future advances in research and compute, agents may learn linguistic strategies to collaborate on real-world problems in increasingly high-stakes domains, and it will be important to ensure that the learned languages are interpretable, safe, and reliable. Our work has potential positive benefits these scenarios: our proposed games encourage agents to learn more human-like linguistic behavior, which might ease collaboration with humans; and ACRe is tool for evaluating and interpreting learned languages. However, future work is needed to see whether these tools remain reliable as we scale up our agents and tasks.
10

---

Acknowledgments and Disclosure of Funding
We thank Alex Tamkin, Elisa Kreiss, Josh Rozner, Gabriel Poesia, and Chris Potts for helpful comments and discussions, and our anonymous reviewers for extensive feedback on the paper. This research was supported by an NSF Graduate Research Fellowship for JM, the Stanford HAI- AWS Cloud Credits for Research program, and the Office of Naval Research grant ONR MURI
N00014-16-1-2007.
References
[1] J. Andreas. Measuring compositionality in representation learning. In International Conference on Learning Reprexentations 2019.
[2] J. Andreas and D. Klein. Analogs of linguistic structure in deep representations. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2893-2897, 2017.
[3] D. Bouchacourt and M. Baroni. How agents see things: On visual representations in an emergent language game. In Proceedings of the 2018 Conference Empirical Methods in Natural Language Processing, pages 981 981-985, 2018.
[4] H. Brighton and S. Kirby. Understanding linguistic evolution by visualizing the emergence of topographic mappings. Artificial Life, 12(2):229 242, 2006.
[5] R. Chaabouni, E. Kharitonov, E. Dupoux, and M. Baroni. Anti-efficient encoding in emergent communica- tion. Advances in Neural Information Procexsing Systems, 2019.
[6] R. Chaahouni, E. Kharitonov, D. Bouchacourt, Dupoux, and M. Baroni. Compositionality and gen- eralization emergent languages. Proceedings of the S8th Annual Meeting of Association for Computational Linguistics, pages 1427-4442 2020.
[7] K. Cho, B. van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Bengio. Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pages 1724-1734, 2014.
[8] E. Choi, A. Lazaridou, and N. de Freitas. Compositional obverter communication learning from raw visual input. In International Conference on Learning Representations, 2018.
[9] S. Chopra, M. H. Tessler, and N. D. Goodman. The first crank of the cultural ratchet: Learning and transmitting concepts through language. In Proceedings of the 41st Annual Meeting of the Cognitive Science Sociery, pages 226 232, 2019.
[10] O. Cicchello and S. C. Kremer. Inducing grammars from sparse data sets: survey of algorithms and results. Journal of Machine Learning Research, 4:603-632, 2003.
[11] S. Gulwani. Automating string processing in spreadsheets using input-output examples. ACM Sigplan Notices, 46(1):317-330, 2011.
[12] Haiman. The iconicity of grammar: Isomorphism and motivation. Language, pages 515 315-540, 1980.
[13] K. He, X. Zhang, S. Ren, and Sun. Deep residual learning for image recognition. In Proceedings of the IEFE Conference on Computer Vision and Pattern Recognation, pages 770-778, 2016.
[14] E. Jang, Gu, and Poole. Categorical reparameterization with gumbcl-softmax International Conference on Learning Representations, 2017.
[15] D. Kingma and Ba. Adam: method for stochastic optimization. International Conference on Learning Representations, 2014.
[16] S. Kottur, J. Moura, S. Lee, and Batra. Natural language does not emerge 'naturally' in multi-agent dialog. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 2017.
[17] N. Kriegeskorte, M. Mur, and P. A. Bandettini. Representational similarity analysis-conmecting the branches of systems neuroscience. Frontiers in Systems Neuroxcience, 2:4, 2008
[18] A. Kuhnle and A. Copestake. ShapeWorld a new test methodology for multimodal language understanding. arXiv preprint arXiv:1704.045177 2017.

---

[19] K. N. Laland. The origins of language in teaching. Psychonomic Bulletin & Review, 24(1):225-231) 2017.
[20] A. Lazaridou and M. Baroni. Emergent multi-agent communication in the deep learning era. arXiv preprint arXiv:2006.02219 2020.
[21] A. Lazaridou, A. Peysakhovich, and M. Baroni. Multi-agent cooperation and the emergence of (natural) language. In International Conference Learning Representations, 2017.
[22] A. Lazaridou, K. M. Hermann, K. Tuyls, and S. Clark. Emergence of linguistic communication from referential games with symbolic and pixel input. In International Conference Learning Representations. 2018.
[23] J. Lee, Y. Lee, J. Kim, A. Kosiorek, S. Choi, and Y. W. Tch. Set Transformer: A framework for attention- based permutation invariant neural networks. Proceedings of the 36th International Conference on Machine Learning, pages 3744 3753, 2019.
[24] D. Lewis. Convention: A Philosophical Study. Harvard University Press, Cambridge, MA, 1969.
[25] F. Li and M. Bowling. Ease-of-teaching and language structure from emergent communication. In Advances in Neural Information Processing Systems, 2019.
[26] R. Lowe, A. Gupta, Foerster, D. Kicla, and J. Pineau. On the interaction between supervision and self-play in emergent communication. In International Conference on Learning Representations, 2020.
[27] D. Luna, E. M. Ponti, D. Hupkes, and Bruni. Internal and external pressures on language emergence: Least effort, object constancy and frequency. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings, pages 4428-4437, 2020.
[28] E. M. Markman and G. F. Wachtel. Children's 's use of mutual exclusivity to constrain the meanings of words. Cognitive Psychology, 20(2):121-155 157, 1988.
[29] S. Reed, Z. Akata, H. Lee, and B. Schiele. Learning deep representations of fine- grtiegr visual descriptions. In Proceedings of the IEEE Conference Computer Vision and Patlern Recognition, pages 49-58, 2016.
[30] Y. Ren, S. Guo, M. Labeau, S. B. Cohen, and S. Kirby. Compositional languages emerge in neural iterated learning model. In International Conference on Learning Representations, 2020.
[31] C. Resnick, A. Gupta, J. Foerstor, A. M. Dai, and K. Cho. Capacity, bandwidth, and compositionality in emergent language learning. In Proceedings of the 19th International Conference on Autonomous Agents and Multi-Agent Systems, pages 1125-1133, 2020.
[32] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, 7. Huang, A. Karpathy, A. Khosla, M. Bernstein, al. ImageNet large scale visual recognition challenge. International Journal of Computer Vision, 115(3):11-252, 2015.
[33] Snell, K. Swersky, and R. Zemel. Prototypical networks for few-shot learning. Advances Neural Information Processing Systems, pages 4077-4087, 2017.
[34] S. Steinent-Threlkeld Compositional signaling in a complex world. Journal of Logic, Language and Information, 25(3):379-33 2016.
[35] S. Seinnet--Thrlkeldd Toward the emergence nontrivial compositionality. Philosophy of Science, 87(5): 897-909 2020.
[36] H. Tessler and N. D. Goodman. The language of generalization. Psychological Review, 126(3):395. 2019.
[37] O. van der Wal, S. Boer, F. Bruni, and D. Hupkes. The grammar of emergent languages. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 3339-3359, 2020.
[38] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems, 2017.
[39] N. X. Vinh, J. Epps, and J. Bailcy. Information theoretic measures for clusterings comparison: Variants, properties, normalization and correction for chance. Journal of Machine Learning Research, 11:2837-2854, 2010.
[40] C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. The Caltech-UCSD Birds-200 200-2011 dataset. Technical report, California Institute of Technology, 2011.
12

---

[41] T. Wolf, J. Chaumond. L. Debut, V. Sanh, C. Delangue, A. Moi, P. Cistac, M. Funtowicz. J. Davison, S. Shleifer, al. Transformers; State-of-the-att natural language processing. In Proceedings the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38-45, 2020.
[42] Zaheer, S. Kottur, S. Ravanbakhsh, B. Poczos, R. Salakhutdinov, A. Smola. Deep sets. Advances in Neural Information Processing Systems, 2017.
13

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] ACRe shows some progress towards compositionality in emergent languages, but as we mention Section | the languages developed by agents are not perfectly interpretable or human-like, and more work needs to be done to further increase the interpretability of the languages. Finally, in Section TO (Broader Impact) we caution about the applicability of these results to much larger studies of emergent communication.
(c) Did you discuss any potential negative societal impacts of your work? [Yes] Yes, in Section I0 (Broader Impact)
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A] No theoretical results
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material
URL)? [Yes] See Appendix (b) Did you specify all the training details (e.g.. data splits, hyperparameters, how they were chosen)? [Yes] We provided the broad overview of our experiments in the main text Sections and Exhaustive details are included in supplementary material Appendices A
and G including data splits, how data was generated, model and optimization parameters, and hardware used.
B
(c) Did you report error bars (e.g.. with respect to the random seed after running experi- ments multiple times)? [Yes] For all experiments in the main text we experiments 5 times with different random seeds and report means, standard deviations, and signifi- cance tests where relevant.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] In Appendices A and we report a rough estimate of time and the GPU used.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] We cite [18 and 40)
(b) Did you mention the license of the assets? [Yes] See Appendix
(c) Did you include any new assets either the supplemental material or as URL? [Yes]
See Appendix B
(d) Did you discuss whether and how consent obtained from people whose data you's 're using/curating? [N/A] We have not obtained any new data.
(c) Did you discuss whether the data you are using/curating contains personally identifiable information offensive content? [N/A] We have obtained any new data, and the ShapeWorld and Birds datasets obviously do not contain personally identifiable information or offensive content.
5. If you used crowdsoureing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A] No human subjects experiments
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "rTc637UZBPn", "reviewer": "Reviewer_13s9", "summary": "The current manuscript proposes to study the emergence of language when agents communicate with each other about generalizations over a set of objects, as part of a referential game. Experiments with object sets yield improved and more interpretable language that evolves between the agents. Additionally, the work also explores finding approximate logical operators in the emergent language.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Strengths**  \n(S1) The manuscript does a good job of clearly stating the hypothesis about analyzing emergent language between agents in referential games about sets of objects. The paper is well-written and easy to understand. I thoroughly enjoyed reading the paper.\n\n(S2) Experiments are well-rounded and back up claims introduced earlier in the paper. Further, analysis of emergent language through sunburst visualizations is insightful to understand the distribution of tokens. Though ambitious, Sec 7.1 to approximately search for operations in the emergence language is an interesting read. Kudos to the authors for attempting it.\n\n**Weaknesses**  \n(W1) The paper seems to be missing a related work section. I urge the authors to include discussions around prior work related to other environmental pressures, placing this work in the context.\n\n**Comments**  \n(C1) Are there any linguistic connections as to why generalizing to a set of objects results in more systematic and interpretable (as defined by metrics in the paper) emergent language?\n\n(C2) How does the behavior of the agents and the emergent language change if the number of attributes and attribute values increases? The first dataset has multiple valued attributes but limited (5 shapes x 6 colors), whereas the second dataset has boolean attributes. Any insights on what happens if there are N attributes where each of them take M values, with increasing N and M?\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The work briefly mentions limitations (in conclusion) and broader impact.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "im9-Mm2Z5l", "reviewer": "Reviewer_Azs8", "summary": "In the referential game setup that is widely used in the emergent communication literature, this paper proposes expanding the target into a target set, and studies the effects of different choices for the distractor and target sets. The authors find that choosing suitable sets leads to more consistent communication protocols and higher topographic similarity between concept and message spaces, but at the same time causes a drop in communicative success.\n\nThe other contribution of the paper is a composition-based evaluation of the protocols that emerge from the set-based training. Small transformers are trained to mimic operators such as ‘AND’ and ‘NOT’. Evaluating the listener performance on the messages produced by those operator models shows a non-trivial communication performance, albeit significantly reduced relative to the original speaker models. The authors conclude that there is limited evidence for compositionality in the communication protocols based on this experiment.\n", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The method presented here is very reminiscent of, and indeed seems to be inspired by, the prototype networks technique from Snell et al. (NeurIPS 2017). In contrast to Snell et al., the authors here use a discrete intermediate representation (the message) and inspect and analyse its properties. It is hard to compare the accuracies because the datasets are different, and although that is not the focus of the authors here, it would help to more clearly place this work in the context of the existing literature.\n\nThe authors’ focus is the systematicity of the communication protocol, and how that is affected by the support and distractor sets chosen. The main finding, loosely paraphrasable as ‘more systematic information in the support and distractor sets leads to more systematic communication’ is nice to see, if not altogether surprising (e.g. Snell et al. also observe that training with more variation in the distractor set leads to better classification performance, which can be attributed to the same effect).\n\nThe search for operators that combine elementary concepts into composite ones is interesting. I salute the authors for including that section even though it does not contain very clearly positive results - putting the idea into writing, suggesting the technique, and publishing a somewhat neutral finding, are helpful contributions.\n\nThe significant drops in communicative success with sets or concept sets are unfortunate. If more systematic communication does not lead to more accurate performance (also on unseen stimuli), it might not be something to strive for in agents, in spite of the human bias in favor. I do wonder to what extent the model architecture and the complexity of the datasets can be at fault here. More inspection of the effects that various choices for the support and distractor sets have on the communicative success would make the paper more relevant for future research. The main reason for wanting systematic communication is the expected benefit for generalization. If that benefit does not materialize, the systematicity of the communication protocol becomes less interesting altogether.\n\nAll in all, while I believe the paper contains a number of interesting contributions, a big question is left open, with significant implications for the relevance of the paper. The paper has lots of interesting analysis, and I would love to see the same rigor applied to the accuracy drop question.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "See main review; there is ample attention for and recognition of the limitations and impact.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "XZYDaQfmj0Q", "reviewer": "Reviewer_cuD1", "summary": "In this paper, the authors point the limitation of existing reference games and propose different games that require communicating generalizations over sets of objects. In the set reference game, a teacher must communicate to a student a group of objects belonging to a concept. The concept game is more challenging as each agent sees different examples of the concept. The authors also build a teacher model and a student model to play these games. They build two tasks to examine the languages developed for their proposed communication games. \n\n\n", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "It is good that the authors propose a new reference game that requires communicating generalizations over sets of objects.\n\nThe paper is well written and easy to follow.\n\nThe authors propose several metrics to analyze the systematicity and interpretability of the learned languages.\n\nThe experiments are good. The authors test the accuracy, conditional entropy of messages given concepts, and adjusted mutual information score of different tasks.\n\nIt seems the tasks are easy and the concepts/words/language that appeared in different games are limited. It is hard to verify whether the conclusion obtained from the experiments applies to more challenging tasks. For example, does the conclusion also suitable for more challenging object categories, i.e. dogs, cats, fruits, etc., or more challenging tasks, i.e. tasks in 3D environments.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The contribution is limited. The authors propose several new reference games and evaluation metrics. However, the proposed games seem very simple, even the Concept game seems more challenging than the Setref and Ref games. \n\nIt is obvious if the task is about high-level concept prediction, the models tend to use fewer tokens and the tokens are more consistent. But it will lose the detailed information. If the task is about low-level concept prediction, the models tend to use more tokens and capture more details. These seem very obvious and I am not sure why designing such tasks is a big contribution. It is more convincing if the authors can perform their experiments and get corresponding conclusions on more challenging tasks or environments.\n\nThe compositional experiment is interesting, but again the concepts/messages/tokens are very limited. What if the concepts need to be described by a long message? What if the difference between the two concepts is very small? Given the powerful natural networks used in recent works, it might be good to have more challenging tasks or environments.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5 hours"}, "extra_text": {}}, {"review_id": "RuU6vNOJNTH", "reviewer": "Reviewer_uEmD", "summary": "This paper explores language emergence properties through two aspects:\n - extending the classic Lewis Discrimination Game to a multi-reference/concept one. \n - probing language compositionality with a new protocol called ACRe evaluation\t\n\nOverall, it is a well-designed paper. It slowly introduces the concept and detail the intuition. I greatly appreciate the didactic aspects of the paper.\n Experiments look sounds, and there is a nice balance between quantitative and qualitative results. \nSome visualizations -- without being ground-breaking -- are novel and help understanding communication protocols.\n\nThe paper does not have substantial weaknesses. Although the ideas are simple (they slightly extend classic ideas), they are correctly explored, and there is no overclaim.\nMy main concern may be that further work could have been done to bridge these ideas to the ML literature.\n Indeed, the Lewis Game extension is a form of negative sampling (which recently (re)boom with CPC) \nACRe is a naive approach for language automata.\nAs those ideas already in the literature, the contribution sometimes feels a but too incremental.\nTherefore, correctly bridging those concepts would be an actual core contribution that is lacking in the community.\n\n In the end, I enjoy the paper, and it may nicely pile up in the emergent communication literature.\n Yet, it misses this je-ne-sais-quoi that could make it excellent.\n I thus lean toward acceptance (but not clear acceptance).", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "As mentioned, there are no apparent flaws in this paper, so I do not have strong remarks.\n\n A) Could you detail further the link between the game and different fields of the ML. For instance:\n  - The two games can be seen as negative sampling approaches [1]\n  - The reference set could be compared to weakly supervised learning (multiple-instance learning) [2-3]\n - How ACRe compare as a language automata [4-5]\n\nPlease, if you disagree with some of these comparisons, can you elaborate further? Note those are not extensive cites.\n\nB) l70: the sentence with label $\\hat{y}_iˆS$ creates a minor ambiguity as I thought It was a classification task for a few moments\n\nC) l120 ”hypergeometric of randomness”… sounds clever, but it is pretty obscure :) Maybe something like a non-uniform distribution with some details about this non-uniformity would be more precise.\n\nD) In evaluation, can you detail a bit further the accuracy metric. How many positive concepts, images do you have in average. Do you compute the mean accuracy or a success ratio (all concepts are found). \n\nE) In tables,  please use floating numbers as the std accuracy is $10^-1$\n\nF) Can you define further “systematic” communication. There is room for ambiguity.\n\nG) When probing compositionality? Is there some specific reasons you did not implement TRE? Besides, why only computing BLEU1 as there are multiple characters in a message. Would you mind adding B2 at a bare minimum? Differently, 10% test hold may look a bit small. Did you try to have hard held-out composition (in the same spirit as [6])\n\n[1] Kalantidis, Yannis, et al. \"Hard negative mixing for contrastive learning.\" arXiv preprint arXiv:2010.01028 (2020).\n[2] Song, Jiaming, and Stefano Ermon. \"Multi-label contrastive predictive coding.\" arXiv preprint arXiv:2007.09852 (2020).\n[3] Carbonneau, Marc-André, et al. \"Multiple instance learning: A survey of problem characteristics and applications.\" Pattern Recognition 77 (2018): 329-353.\n[4] Languages, Automata, and Logic \t\n[5] Chapter 2 Languages and Automata - Inria\n[6] Cogswell, Michael, et al. \"Emergence of compositional language with deep generational transmission.\" arXiv preprint arXiv:1904.09067 (2019).\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "None", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4h"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This work points out limitations of existing reference games, and proposes using communication for generalization over (concept-level) target sets instead of single targets. Different types of targets, distractors and target sets are explored. The reviewers agree the current work presents several good and useful contributions to the field of emergent language. The authors did an excellent job in addressing any remaining concerns in the rebuttal.
Decision: Accept (Poster)