Catalytic Role Of Noise And Necessity Of Inductive Biases In The Emergence Of Compositional Communication
Authors: Łukasz Kuciński
Tomasz Korbak
Paweł Kołodziej
Piotr Miłoś
Venue: NeurIPS 2021 Poster (2021)
Keywords: compositionality
signaling games
noisy channel
deep learning

Abstract: Communication is compositional if complex signals can be represented as a combination of simpler subparts. In this paper, we theoretically show that inductive biases on both the training framework and the data are needed to develop a compositional communication.  Moreover, we prove that compositionality spontaneously arises in the signaling games, where agents communicate over a noisy channel.  We experimentally confirm that a range of noise levels, which depends on the model and the data, indeed promotes compositionality. Finally, we provide a comprehensive study of this dependence and report results in terms of recently studied compositionality metrics: topographical similarity, conflict count, and context independence.

Catalytic Role Of Noise And Necessity Of Inductive Biases In The Emergence Of Compositional
Communication
Lukasz Kuciński*
Tomasz Korbak University of Sussex
Polish Academy of Sciences #kucinski@impan pl
tomasz. www.kkggwbaagaaaoooooooo com
Pawel Kolodziej
Piotr Miloć
Polish Academy of Sciences p. kol ooliidieee @gmail com
Polish Academy of Sciences, deepsense.ai
pmilos@impan, pl
Abstract
Communication is compositional if complex signals can be represented as a combi- nation of simpler subparts. In this paper, theoretically show that inductive biases on both the training framework and the data are needed to develop compositional communication. Moreover, we prove that compositionality spontaneously arises the signaling games, where agents communicate over a noisy channel. We experi- mentally confirm that a range of noise levels, which depends the model and the data, indeed promotes compositionality. Finally, we provide a comprehensive study of this dependence and report results in terms of recently studied compositionality metrics: topographical similarity, conflict count, and context independence.
Introduction
In emergent communication studies, one often considers agents who can share information about set of objects described by the common features. Such situation is common in multi multi-agent systems with partial observation (Foerster et al. (2016). Lazaridou et al. (2016), Jaques et al. (2018), Raczaszek-Leonee
al. (2018)) and it is the major theme in signaling games (Fudenberg al.) (1991), Lewis 1969), Skyrms (2010), Lazaridou et al. (2018)). In a signaling game, one agent (a sender) conveys information about an object to another agent (a receiver), which then has to infer the object's features. Typically, agents are rewarded if some of the features are correctly identified. During this process, the agents develop a communication protocol. recent line of work has studied conditions under which compositionality emerges (Batali (1998); Kottur et al. (2017); Choi et al. (2018) Korbak et al. (2019); Li and Bowling (2019); Stowik al. 2202bb a): Guo et al. (020
Compositionality is a crucial feature of natural languages and it has been investigated extensively in cognitive science (see e.g. Chomsky 1957) Fodor and Pylyshyn (1988)) It is often measured using dedicated metrics such as topographic similarity (Brighton and Kirby (2006); Lazaridou et al. 2006)iLazzzrdoo and Baroni (2018)), context independence Bogin al. (2018) conflict count Kucinski et al. (2020), or positional disentanglement (Chaabouni et al. (2020). In signaling games bears strong resemblance to the concept of disentangled representations, see (Higgins et al, (2017), Kim and Mnih (2018), Locatello et al. (2019)). In machine learning context,
(2018) Kriegeskorte (
"Corresponding author. Now at Google.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

compositionality is perceived as generalization mechanism (Lake et al. (2016)) and has been used e.g. for goal composition QTiang et al. (2019) or knowledge transfer (Li and Bowling (2019))
In this paper; we theoretically show that inductive biases on both the training framework and the data are needed for compositionality emerge. A similar observation has been made by Kottur iet al. (0077); however, our result is more fundamental and points out a common misconception that compositionality can be learned in a purely unsupervised way. Such a result can be perceived as discrete analog of Locatello et al. (211)) applicable in the communication context.
We then prove that adding an inductive bias in the loss function coupled with communication over noisy channel leads to the spontaneous emergence of compositionality. This shows the catalytic role of noise in this process. Intuitively, this can be attributed to the (partial) robustness of compositional language with respect to message corruption caused by a noisy channel.
We experimentally verify that certain range of noise levels, dependent on the model and the data, promotes compositionality. We provide wide range of experiments that illustrate the influence of different priors. For the inductive biases in the training framework, we look into the impact of the network architecture as well as implementation and temporal variation in noise. On the data side, we study the effect of scrambling visual input or its description. We also study the generalization properties of the proposed training framework.
2 Related work
The topic of communication is actively studied in multi-agent RL, see Hernandez-LLal et al. (2020 Table 2) for recent survey. Compositionality is often investigated in the context of signaling games (Fudenberg et al. (199T), Lewis| (1969). [Skyrms (2010), Lazaridou et al. (0018). Recent research has shown that strong inductive biases grounding of communication protocols protocol to be compositional (see c.g. Kottur et al. (2017). Slowik et al. (2020b)). The inductive bias can be imposed into the architecture of the agents the training procedure. For instance, Das et al. (0177) place pressure on agents, to use symbols consistently across varying contexts, by a frequent reset of the agent's memory. A model-based approach was proposed by Choi et al. (2018) and [Bogin cet al. (2018), who build upon the obverter algorithm (Oliphant and Batali (1997) )) Batali (1998) Slowik et al. (2020a) explore games with hierarchical inputs and shows how agents implemented as graph convolutional networks obtain good generalization, Korbak et al. 2019) implemented the idea of template transfer (Barrett and Skyrms 2017) by pre-training the agents simpler subtasks before the target task. Kirby (2001) studied the iterative learning paradigm, where each generation of agents learns the language spoken by the previous generation before starting communicate. In the machine learning literature, this idea was explored by Li and Bow wing (019)) Cogswell et al. (2019) and Ren et al. (2020) with the generation transfer typically implemented as reinitializing the weights of agents' neural networks. Such an approach inevitably introduces noise into the learning process. This naturally leads to a question of whether the noise itself may be a sufficient mechanism of compositionality, which we will try to address in this paper. [uo et al. (2020) have shown that the choice of a game has a large impact on the properties of a communication protocol emerging in that game, foreshadowing what we call grounding. Kim and Ohl (2021) study emergent language in the group setting with varying population sizes and connectivity. Liu et al. (202T) cast constructing
[1969)|
necessary for the
structured neural architectures
communication problem.
The noisy channel model of communication was famously introduced by Shannon 1948), ). The idea of noise as driving force in the emergence of communication was first proposed by Nowak and Krakauer 1999p, who showed that word-level compositionality is the optimal solution to the problem of communication in noisy environment under particular fitness function. [Tucker et al. (2021) consider a noisy channel communication, where the agents learn to communicate via discrete tokens. Noise is also used in deep learning, e.g. as a regularizer (see e.g. dropout (Srivastava et al.) (2014) or a mechanism allowing backpropagation through a discrete latent (see e.g. Salakhutdinov and Hinton (2009), ,. Kaiser and Bengio 2018)). Noise in the latter context was used in Foerster et al. (2016) in order to learn to communicate. The authors observed that it is essential for successful training.
3 Noisy channel method
We discuss the language and compositionality in Section 3.1 The impossibility result and the need for biases in emergent compositionality is the content of Section 5.2 The communication task considered

---

in this paper as well as the catalytic role of noise is described in Section 3.3 Theoretical results from this section hold somewhat idealized situation, but experiments Section are performed in more realistic setup. The difference is described in Section 4.1
3.1 Language and compositionality
Consider a set of objects described by some features, and let a set F contain com- bination of these features' values. For example, the features could represent shape, say squares and circles, and color, say red and green, in which case F = (red square, red circle, green square, green circle). The features could be identified with partitions of F. More formally, each feature can be defined via equivalence relation, with fea- tures values corresponding to equivalence classes of this relation. In our example, the color cor- responds to the partition {red square, red circle} ("red") and (green square, green circle} ('green'), while for the shape corresponds the partition (red square, green square} ("square') and {red circle, green circle} } (" ('circle").
We assume that objects with feature space F can be defined in space X' and are generated by two-stage process: first, the feature values are sampled from f E F, then an element of X is sampled according a distribution conditioned on
In general, a language defined as a mapping from objects to strings over some finite alphabet A (sometimes called messages), e: X'' - AA" In this paper, we will study a subset of languages E, where the range of the language has a fixed length, equal to the number of features. We say that / compositional with respect given feature if change in i-th feature only impacts corresponding j-th index the message. Continuing the previous example, let = {a, b} and consider language l mapping red squares, red circles, green squares, and green circles aa, ba, ab, and bb, respectively. Then l is compositional with respect to color and shape features since the change color only impacts the first index in the message (and analogously for the shape), see Figure Tab.
our running example, suppose that an identity, ex- cept that it swaps red circle with green circle, i.e. <red circle) green circle and T (green circle) = red circle. The resulting language ly would then map red squares, red circles, green squares, and green circles aa, bb, ab, and ba, respectively. This language is not com- positional with respect to color and shape features, since we change shape value in red circle, both symbols in the message will change (from bb aa), see Figure [b). However, & is composi- tional with respect to a different set of features (shape and 'different color-shape'). see Figure I Consequently, compositionality should be defined together with features, with respect which holds. the next section, we show that this observation has significant implications for learning.
Consider a permutation :: +
ba
bb
ba
ab bb
ab ba
ab bb
(a) language f, where
(b) language fx. where
(c) language Cu. where
e aa, e ba.
(=( aa, Lx
bb.
E aa, &x bb, C ab, &x ba.
ab, E bb.
t, ab, &g ba.
f is compositional w.r.t.
t, is not compositional w.r.t.
E is compositional w.r.t.
color
and
color
and
shape
and
shape
shape
'different color-shape"
Figure 1: Language, features, and compositionality.
3.2
Inductive biases and compositionality
This section investigates whether compositionality can spontaneously emerge in unsupervised way. The answer to this question is negative since the underlying features cannot be inferred from the data.
Theorem 1. For a uniform distribution, μ on F and a permutation म F - F, the distribution - is also uniform.
While this result elementary to prove (see Appendix F has deep implications for the emergence of compositionality in the learning process. Suppose that data represent a balanced set of all features
3

---

(i.e. features are sampled from μu) and recall the two-stage generation process described in Section 3.1 If f μ o π has the same distribution as f~ p. then the distribution of the observed data does not depend on и. Consequently, any emergent compositional language l with respect to some features is not compositional with respect to other (permuted) features. We arrive at the following conclusion:
Any learning process which hopes to achieve compositionality must involve some priors related to the data and learning framework.
Theorem [ can be viewed as a discrete version of Locatello et al. (2019) Theorem 1). There are multiple ways of grounding the learning process, including imposing inductive biases on the agents and designing loss functions to disentangle features. We study these, together with a new mechanism: injecting noise into the communication channel. In the next section, we show that this mechanism provably achieves compositionality.
3.3 Compositionality and communication over a noisy channel
Another major conceptual finding is that compositional communication spontaneously emerges when introducing a relatively simple mechanism noisy channel. This is proved in Theorem provided that the loss function penalizes agents' mistakes, but also rewards for (partially) correct guesses.
Recall, that we consider a signaling game, where agents cooperate and develop a communication protocol (a language f) in order maximize their joint reward. Here sender observes a certain object with features f E F and sends message to the receiver to allow him to infer f. The agents are rewarded if some of the features are identified correctly, see equation
The above setup is standard and now we augment it with a noisy channel. The noisy channel is located between the sender and the receiver and may scramble messages. A message is transformed into a corrupted message s', by replacing each symbol, independently and with probability c E (0, 1), with different, uniformly sampled, symbol.
For the sake of this section and Theorem 2 below, we will make the following series of assumptions. Let F Fi .... x Fx, that is the feature space is factorized into K features. Furthermore, assume that each feature has the same number of values. We will assume that = and define language AK, where A is an alphabet and |.A| = [F
used by the sender a mapping é: Fi ... x FK We will further assume that the message s = E(f) is decoded as (s). The corrupted message corresponding to f F denoted by E(f) and the inferred features corresponding to this corrupted message are given by f' = (l(f)) E F. The setup for this section and for our experiments (Section 4 and Section differ, see Section
Recall that the definition of compositionality (with respect to given features) connects the change in feature values with the change in message symbols. It is thus reasonable to look for loss functions that are somehow factorized in terms of individual symbols. Consider the following loss function:
(C. E[H(p(f' .f))
where is a non-negative, strictly increasing function and p is the Hamming distance Intuitively, p measures the number of changed features in the corrupted message and H controls the degree by which we penalize this quantity.
and X =. Suppose additionally that e < (IA) - 1)/||A, Then a language E minimizes J over all languages E which are one-to-one mappings if and only if E* compositional (with respect to features given by Fi).
Theorem 2.
Assume that K 2
FK, -
Informally. Theorem 2 states that optimization of loss function J promotes compositionality (assump- tion on the one-to-one property technical). What makes this possible is the factorized nature of
the losses and the introduction of a noisy channel. Interestingly, there exist other loss functions with similar properties. We postpone the analysis of this point and the proof of Theorem 2 to Appendix [F]
is instructive to discuss some of Theorem assumptions. Assumption F = X' means that takes semantically meaningful symbolic input. This does not cover many interesting cases, where X'
has representation entangled in terms of features (c.g. an image). Furthermore, the assertion of Theorem 2 holds for a rather wide spectrum of values (up to 0.8 for |A| = 5, which is what we use in the main experiment in Section 5) This stands in contrast to our experimental results, where we
The Hamming distance between two vectors U, W E IR is defined as p(e, w) ΣK 1(m: wi).

---

observe an interaction between different noise levels and compositionality, see Section 5 There is no contradiction here since in Section 5 we study more realistic setup that extends beyond relatively strict assumptions of Theorem 2 see Section 4.1
4 Experiments setup
4.1 Differences between experimental and theoretical setups
The setup of our experiments is more realistic (and common in this area of research) and extends beyond the assumptions of Theorem 2 We assume that a dataset X' contains images, consequently making features entangled in a visual representation (i.e. F + X). We also allow the alphabet size to differ from the number of feature values, |A| * [F]. Additionally, the receiver only sees the messages sent by the sender and has to learn to decode the messages. A further gap stems from the implementation details and the use of neural networks, which may converge suboptimal solutions. We do, however, assume that the feature space F is a Cartesian product (see Section 3.3
4.2 Training pipeline
In this section, we sketch the training pipeline and postpone the details to Appendix C The dataset of images observed by the sender is denoted by D. Each element of D has K independent features fi.....fk (here we consider K 2). Both the sender and the receiver are modeled as neural networks (for details see Appendix B The sender network takes an image from D as input and returns a distribution over the space of messages of length (here we assume L K 2). We assume that conditionally on the image, the symbols in the message are independent and take values in finite alphabet A, = [l.....d.) ). This distribution is then distorted by the noisy channel, which is a function that maps probability vectors into d, dimeensiioal logits. Unless otherwise stated, we assume that noise is defined as a dense layer with a specific choice of (not learnable) weights matrix and a log activation function:
noise(z) = ()...
(2)
Here we assume that W E Red is a fixed matrix, which takes a probability vector to a probability vector with positive entries. An example of such W is stochastic matrix. In this paper
1 i == I i ij..
Wi
Alternative implementations of noise architecture are possible, see Appendix The noisy logits are then used to sample message and pass it to the receiver. To make this operation differentiable we use Gumbel-Softmax (Jang et al. (2016)) with Straight-Througl mode (Kaiser and Bengio (2018) Upon receiving the message, the receiver outputs a distribution over possible values of the features, using an alphabet Ar = {l.....dd, dz}. Finally, the neural networks are trained using linear combination of cross-entropy loss for the receiver, cross-entropy loss for the sender, and Lg-regularization The loss term for the sender incentives the language to be a one-to-one mapping.
4.3 Datasets
We use two datasets: shapes3d (used in Burgess and Kim (2018) and included in the TensorFlow datasets package) and a dataset used by Choi et al. (2018 and Korbak et al. (2019), which we will refer to as the obverter dataset, see Figure 2 The datasets are similar but offer different forms of visual variability. Shapes3d dataset includes images of 3D shapes. Each element is a (64, 64, 3) RGB image, and is characterized by multiple features, such shape object huc. We choose images with values for both features ranging in {0, 1, 2, 3}. The obverter dataset contains images of four shapes (box, cylinder, ellipsoid, sphere) in four colors (blue, cyan, gray, green). Each image has dimensions (128, 128, 3), and we use 1000 images for each shape-color pair. For details see Appendix A
*We believe that Gumbel- research, e.g. [Le al. (217). Mordatch and Abbeel (017)) Chaabouni et al, (2020) report that Gumbel- Sofimax converges to similar solutions as REINFORCE but faster and more stable.
approach is now an established choice in emergent communication
'The dataset is available at httpt::// //i thub. com/ wom//benbogin/bbeee obverterr We used the code provided in the repository to generate 1000 images for each color-shaps pair.
5

---

4.4 Training details
The sender and the receiver are implemented as feed- forward neural networks. To ensure the diversity of ran- dom initializations we run each experiment with 100 ran- dom seeds. For each seed, there were 100 evaluation runs, once every 2000 network updates. We report metrics averaged over the last 20 evaluation runs. For details chitecture and hyperparameters' choice, see Appendix B
4.5 Compositionality measures
We measure compositionality in terms of four metries used in emergent communication literature: topographic simi- larity, conflict count, context independence, and positional disentanglement. For all metrics, the higher values the better, except for conflict count, for which the reverse is true. The results presented Section [ indicate that the metrics agree on the assessment of our results. Parallel compositionality metrics, we also report accuracy, which we often refer to acc.
Figure 2: Top: shapes3d. Middle: obverter.
Bottom: Scrambled shapes3d (32, 16, 8).
Topographic similarity Topographic similarity (Brighton and Kirby, 2006 Lazaridou et al. 2018). or topo for short, is popular measure of structural similarity between messages and features. Let : F F + R+ be a distance over features and Lm A; A + R,, be a distance over messages. The topographical similarity is the Spearman P correlation of L and Lm measured over joint uniform distribution over features and symbols. We choose Lm to be the Levenshtein (1966) distance and treat features f F as ordered pairs or features distance. We use topo as the main metric.
choose L to be the Hamming
Figure 3 shows the expected value of topographic similarity for a random bijective language with message length equal to 2. For 5-symbol alphabet, this value does not exceed 0.2 which sets point of reference for compositionality results. For derivation see Appendix
[L....,K) + (L....,Ay K} be permutation. The principal meaning of a symbol s at position j is defined as m(),jid) de-
Conflict count Let
"max"
"min"
0.9
arg max, count(s, σ)) where count(s,j, fined [iang.)ee (e((mgg),
s, feC) = v), runs over all values of all features, and ties arg max are bro- ken arbitrarily. Then, conflict count metric, conf for short, defined as conf ming E.s score(s, Φ), where score(s.j:d) (j = Exfa((st:)) count j. Ø). Intu- itively, score measures how many times the feature as- signed to symbol
0.6
0.3
0.0
position diverts from its princi- pal meaning m(s,) j: Φ). conf sums these errors and takes min over possible orderings Φ. This metric introduced in Kuciński et al. (2020).
8
9
10
symbols
Figure 3: Expected value of topographic similarity for a random bijective language with message length 2, function of the alphabet size. "min". "max", and "avg" stand for different ways of computing ranks.
Context independence Context independence (Bogin ct al. 2018)), abbreviated here as cont, measures the align- ment between symbols forming message and features fi...., By ff)]f), we mean the probability that the sender maps a feature f to a message containing symbol
define Σ. We define the inverse probability p(f(s) similarly. Finally, sf arg max, p( ()] of is the symbol most often sent in presence of feature f. Then, context independence is E[p(v v k) p(k)e*))) the expectation is taken with respect to the joint uniform distribution over features and symbols.
Positional disentanglement Let 8j denote the j-th symbol of message f(d), and of the fea- ture with the highest mutual information with 8j, and e² with the second highest mutual infor- mation: oi arg max, c(s of arg cae I(s;;c) where I(:-)) is mutual information and is feature value. Then, positional disentanglement (Chaabouni et al. 2020 is defined as

---

topo
conf
cont
pos
acc
90
8.7
189
.76
.9
84
8.6
177
844
78
4.5
164
55
.70
72
2.4
152
45
56
00
10
004
noise
noise
noise
noise
noise
00
100
12
23
Figure 4: Results of the main experiment on shapes3d dataset. Top panel: average value of metries for various noise levels. The shaded area corresponds to bootstrapped 95%6-confidenc intervals. Bottom panel: kernel density estimators for metries and noise levels across seeds. Here lopo stands for topographic similarity, conf for conflict count, cont for context independence, pos positional disentanglement, and ace for accuracy.
E [[s;;:) I(s):c() /H(s)), where the maximum message length and H(ss) ) is entropy over the distribution of symbols at j-th place in messages for each feature. We ignore positions with zero entropy We will call this measure pos, for short.
1
5 Experiment results
In this section, we study how different inductive biases for the model and the data influence composi- tionality. The main experiment presented in Section [.I and the experiments in Section (.2 5.4
its variation. In the main experiment we assume that the message length K = 2, |Ag 5, and |A,| (see Section 4.2). Notice that both A, and A, differ from one another and from the number of possible feature values (which equals 4). It turns out that the qualitative results are similar for both the shapes3d and obverter datasets, and in the interest of brevity we only report the results for both in the main experiment. Supplementary material for this section can be found in Appendix D
Our main findings include the fact that the noise indeed catalyzes the emergence of compositionality. There is an interesting dependence on the noise level: too high noise may impede learning, while small noise vanishes within other sources of noise. Importantly, this phenomenon appears consistently across different biases. Using two head output of the network is the strongest bias toward compositionality (amongst studied). Finally, show that compositionality generalize unseen cases when fine-tuning is allowed.
5.1 Main experiment: emergence compositionaliy
The results, presented in Figure illustrate how noise catalyzes the emergence of compositionality. The top panel of Figure shows the important patterns for metrics: they improve until an extremal point is reached, and decline afterward. Topographic similarity achieves extremum for noise level 0.1, reaching value 0.87, which is significant improvement upon 0.79 for the lack of noise. The accuracy drops down with increase of the noise level, expected, however the speed of the decline increases. This shows that there is an interesting compositionality-acoo accuuracy trade-off. The bottom panel of Figure complements the overall picture with visualization of metries' distribution. We see interesting dynamics in topographic similarity distribution with respect to change in the noise levels. Namely, it starts by accumulating noise 0. after which it transitions to a bimodal distribution, finally shifting its mass more towards the mediocre end of the spectrum. An interesting observation is that the results for experiments with achieved high accuracy, are not only better but also the effect of noise is more pronounced (for
the higher spectrum of its values, reaching peak for
"Positional disentanglement is related to residual entropy proposed by Resnick et al 2020 Chaabouni et al. 2020) also proposed bag- bag-of-war words disentanglementt which assumes order- invariance of messages. Due to our architecture choice, this assumption not met, hence decided not report this metrie.
7

---

90
92-
93
90-
90-
88
88
70
77
06-
85
84
82
200
3200
1700
2200
2700
200
3200
3700
2200
2700
04
74
20
04
20
(a) symmols 4x4
(b) symbols 8x8
(c) variable noise 0.1
variable noise 0.15
89-
02-
85
84
79
72
74
73
000
002
800
005
0006
010
012
014
(c) higger CNN
(f) additional dense layer
(g) alternative noise
(h) message len 3
79-
69
64
81
67
56
52
57
8
3
3
8
3
(i scramble 32
① scramble 16
(k) scramble 8
(1) scrambled labels
Figure 5: Topographic similarity for experiments Sections 5.2 and Section 5.3 The shaded areas correspond
to bootstrapped 95%-confidence intervals for average topo.
instance there are 47 seeds with accuracy exceeding 0.9, for which noise 0.16 yields 0.96 topo). For
the sake of brevity, we defer the discussion of this phenomenon to Appendix [D.T] The accuracy undergoes similar transformation as topographic similarity. The detailed numerical analysis, as well as corresponding results for the obverter dataset, can be found Appendix D.]
5.2 Influence of model inductive bias
Different number of symbols Here we study the impact of different number of symbols on compositionality. In our experiments we used the communication channel with |A, 5, giving total of 25- (5 x 5) possible messages. This is slightly redundant since only 16 = (4 x 4) is required, so this is the first case that we study here. turns out, that allowing for only 16 messages makes the training less stable. For topographic similarity, see Figure 5ia), small to medium values of noise exhibit wide confidence intervals and is statistically hard distinguish between the metric values (this might be attributed to a bimodal distribution of topo in this noise range, see Figure 15 in Appendix D..2p. For larger values of noise (greater than 0.16), topo starts to visibly decline. As second experiment,
considered the total of 64 (8 x 8) messages. Interestingly, the topographic similarity values for the small noise regime (up to 0.08) do not improve over the bascline value (0.79), Figure 5(b.. This behavior changes for medium to large values of noise, where visible increase in topo, peaking at 0.88 with a noise level of 0.24. For details see Appendix D.2
observe
Variable noise Understanding what happens under varying noise is an interesting and subtle problem. would most probably arise in more complex situations when a communication channel is part of bigger system. Additionally, it might be similar to phenomenon observed in supervised learning. indicating that training can benefit from learning rate warmup (see e.g. Goyal et al. (2017). Frankle
et al. (2200). In this paragraph, we discuss simple experiment with increasing noise and leave more nuanced study for further work. More precisely, in the initial stage of training the noise is kept at the level cg. and after T E {200, 700, 1200, 1700, 2200, 2700} network updates it is switched to different value, ET, and kept there for the rest of the training. The results for topographic similarity are presented in Figures ((c). where eg we see that topo increases from 0.85 for T 200, to 0.9 for T = 2700. The effect in Figure 5 is weaker and the variance in the results is quite high. The details can be found in Appendix D.3
and éx 0.1 and ex 0. 0.15, respectively. In Figure 5
Sensitivity with respect to small architecture changes This set experiments aims to check the impact of changing the parameters of CNN and the dense layers in the agents' network. In the former experiment, we change the number of filters the sender's CNN architecture from two layers with 8 filters, to two layers with 16 filters. This results in the slight change of topographic similarity
8

---

topo
521
55
2.50
15
75
85
2.5
18
17
.70
50
88
78
.60
120 160
200
40
60
120
160
200
40
60
120
160
200
80
120
350
200
40
80
120
200
finetune up updotes
Tinetune updates
finetune updates
firetune updates
Tinetune upoates
Figure 6: Fine-tuning for baseline agent trained with noise 0.1 and with several features removed from the training set (the diagonal from the chape-by-coloo matrix). On the x-axis the number of finetuning updates.
profile, see Figure [ie), with the highest average value of 0.86 for noise range 0.14 significantly outperforming the zero-noise case (0.79). For the second experiment, we added a dense layer (with 64 neurons) to the receiver's architecture, see Figure 5 This again shows improvement of compositionality due to noise, although the noise in the range (0.08, 0.16) performs roughly the same (see also Appendix D.4).
Sensitivity to noisy channel implementation Here we consider an alternative noisy channel imple- mentation, where the noise permutes the sampled symbols, as opposed to distorting the distribution of the symbols (see Appendix Cj. It turns out that with this change, the scale of noise where the interesting things happen changes well, see Figure 5[g). Namely, when compared with the main experiment, noise values are in the range smaller by the order of magnitude (we report results for noise levels in {0.000, 0.001, ................................ 0.015). The results suggest that the small values of noise can help, while the larger noise levels lead to a decline in compositionality, see also Appendix [.5]
Longer message In this section, we discuss the experiment with an additional feature (with floor color acting as the third feature) and a message of length The result for topographic similarity can be seen in Figure[]hh see also Appendix D.6. The overall topographic similarity level is lower than the main experiment, however, the distinctive peak is visible, here for noise level 0.08.
5.3 Data inductive biases
Visual priors This experiment was intended to check how much the CNN-backed input is relevant in the compositionality context. We could conjecture that CNN may facilitate shape recognition and therefore be the driving force in the emergence of languages compositional with respect to the canonical shape, color split. To check this we impair the prior by scrambling images, as depicted in the bottom panel Figure 2 An image is scrambled by splitting it into (64/z)² disjoint tiles of height and width equal to x, and randomly reshuffling them. This procedure significantly distorts the accuracy profile of the method (see Appendix D.7]. In particular, each transition from coarser to finer tiles, the accuracy decreases significantly: for zero-noise it drops from 0.97 for no tiling, to 0.95 for x = 32, to 0.79 for 16 and 0.53 for x = 8. The overall compositionality metrics decrease as well, but the characteristic peak for some positive noise level still present, see Figure (k) Having said that, the metrics incur a significant boost, when computed for subset of experiments with high accuracy. We conjecture that the explanation is that the CNN prior is indeed relevant but is not the only one (the output considered in the next section is another).
Scrambled labels In this experiment, we aimed to understand how much the overall architecture output is important in the emergence of compositionality. The receiver's 's network has a two-headed output and the training framework uses a factorized loss function. In the standard setting, we compare the heads' outputs with 'color' and 'shape' respectively, therefore we reflect human priors from the data. In this experiment, we distort this setting, permuting the set of (color, shape) and factorizing them into new labels (see Appendix D.8.. We use random permutation, the new labels are abstract and correspond to some joint color-shape concepts. In our experiments, we show that languages, which emerge are compositional with respect to these new concepts, see Figure ((t). Consequently, they are not compositional in the standard color-shape framework. This is in line with the claims of Section 3.2 and further highlights that the output inductive bias is essential.
5.4 Generalization
Network features this paragraph, we present visualizations of the sender network, to gain some insights into what the network is doing, and whether observe any obvious overfit. In the

---

upper panel of Figure 7 t-SNE (Van der Maaten and Hinton (008)) visualization is shown for both the raw shapes3d dataset and the last dense layer of trained agent with noise 0.1. We observe that the network successfully disentangles features of the data, which in the raw dataset appears to be entangled. This could be good sign in the context of generalization") Furthermore, Figure (bottom panel) shows occlusion and saliency maps for sample image from shapes3d dataset. The network seems to pay attention to parts of the image that could be relevant for the task.
Zero-shot and fine-tuning In this experiment, we study the generalization properties of the training protocol. We trained the baseline agent with noise 0.1 with several fea- tures removed from the training set (a diagonal from the chape-by-colbe matrix) and used the removed set to ana- lyze the disti--dittriuution performance. In the zero-shot setup, the messages on these observations were not in line with the compositional structure acquired on the training set, which is reflected by rather unimpressive outcomes for each metric (initial values for each metric presented in Figure 6 This may suggest that for zero-shot general- ization stronger biases might be required (see e.g. Slowik et al. (2020a b)). However, after a few network updates on the whole dataset (without the removed diagonal) we observed quite significant increase in compositionality metrics, see Figure 6 The highest improvement can be seen roughly in the first 50 additional network updates.
roand, eyy dex greeen
Figure 7: I-SNE visualization (with perplex- ity 50 and 10000 steps), Top left: shapes3d, top right: features of the last dense layer. Bot- tom: occlusion and saliency maps.
Limitations of the method
Here we summarize few limitations of the method, that we hope to overcome in future research.
Theory Theorem uses several restrictive assumptions, e.g. fixed message length or alphabet size equal to the range of feature values. In a general setting, it could be the case that noise would promote non-compositional. error-correcting, communication protocols. Stating the general conditions for the emergence of compositionality is an open problem.
Choice of datasets Confirmation non-synthetic datasets is needed fully underpin our method.
Communication protocol We assume a simple communication protocol with messages of length two and two features, each taking four values (except for experiments in Section 5.2).
Compositionality model We assume a rather simple compositionality model based of independent features. Exploring non-trivial compositionality [Steinert-Threikkle 2020 Korbak et al. 2020) might be an important conceptual development.
Architecture Our architecture might implicitly exhibit some unknown biases that increase composi- tionality. On the other hand, the architecture might be too simple to achieve strong results in tasks such as zero- or few- shot generalization.
Multi-agent interactions We used supervised training setup, it would be natural to test the influence of noise in reinforcement learning scenarios.
7 Conclusions
In this paper, theoretically show that inductive biases both the training framework and the data are needed for the compositionality to emerge spontaneously in signaling games. We then formulate inductive biases the loss function and prove that they are sufficient achieve compositionality when coupled with communication over noisy channel. Consequently, we highlight the catalytic role of noise in the emergence of compositionality. We perform a series of experiments in order to understand different aspects of the proposed framework better. We empirically validate that, indeed, a certain range of noise levels, dependent on the model and the data, promotes compositionality. Our work is foundational research and does not lead to any direct negative applications.
IIt is known that t-SNE does not necessarily represent cluster sizes, distances, and respective positions (see compositionality metrie.
Wattenberg al. 2016)). Hence, t-SNE cannot be expected to
10

---

Acknowledgments and Disclosure of Funding
The work of Piotr Miloš was supported by the Polish National Science Center grant UMO- 2017/26/E/ST6/0062. The work of Tomasz Korbak was supported by the Leverhulme Doctoral Scholarship. This research was supported by the PL-Grrd Infrastructure. Our experiments were managed using https: //tp::////tuue/e/eeeee ai We would like to thank the Neptune team for providing us access to the team version and technical support.
References
Barrett, A. and Skymms, B. (2017). Self assembise Games. The British Journal for the Philosophy of Science, 68(2):329-353.
Batali, J. (1998). Computational simulations of the emergence of grammar. Approach to the Evolution of Language, pages 405-426
Bogin, B., Geva, M., and Berant, J. (2018). Emergence of Communication in an Interactive World with Consistent Speakers. arXiv: 1.XXv:180900055550 1809.00549
[es]. arXiv: 1809.00549.
Bouchacourt, D. and Baroni, M. (2018). How agents see things: On visual representations in an emergent language game. 1rXiv:1808.. 10696 [es]. arXiv: 1808.10696.
Brighton, H. and Kirby. S. (2006). Understanding Linguistic Evolution by Visualizing the Emergence of Topographic Mappings. Artificial Life, 12(2):229-242.
Burgess, C. and Kim, H. (2018). 3d shapes dataset. hutp://hhhbbccommeeeenin////ssss
Chaabouni, R., Kharitonov, E., Bouchacourt, D., Dupoux, E., and Baroni, M. (2020). Compositional- ity and Generalization In Emergent Languages. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4427-4442, Online. Association for Computa- tional Linguistics.
Choi, E., Lazaridou, A., and de Freitas, N. (2018). Compositional Obverter Communication Learning From Raw Visual Input. ICLR 2018. arXiv: 1804.02341.
Chomsky, N. (1957). Syntactic structures. OCLC: 934673149.
Cogswell, M., Lu, J.. Lee. S., Parikh, D., and Batra, D. (2019). Emergence of Compositional Language with Deep Generational Transmission. ArXiv, Abs/1904.09067.
Das, A., Kottur, S., Moura, J. M. F., Lee, S., and Batra, D. (2017). Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning. 2017 IEEE International Conference on Computer Vision (ICCV), Venice, 2017. arXiv: 1703.06585.
Fodor, J. A. and Pylyshyn, Z. W. (1988). Connectionism and cognitive architecture: a critical analysis. Cogmition, (11))33777
Foerster, N., Assael, Y. M., de Freitas, N., and Whiteson, S. (2016). Learning to Communicate with Deep Multi-Agent Reinforcement Learning. NIPS' 16 Proceedings of the 30th International Conference on Neural Information Processing Systems. arXiv: 1605.06676.
Frankle, J., Dziugaite, G. K., Roy, D., and Carbin, M. (2020). Linear mode connectivity and the lottery ticket hypothesis. International Conference on Machine Learning pages 3259-3269. PMLR.
Fudenberg, D., Tirole, J., TIROLE, J., and Press, M. (1991). Game Theory. Mit Press. MIT Press.
Goyal, P., Dollár, P., Girshick, R., Noordhuis, P., Wesolowski, I., Kyrola, A., Tulloch, A., Jia, Y., and He, K. (2017). Accurate, large minibatch sgd: Training imagenet in hour. arXiv preprint ArXiv:1706.0244 706.0267.
Guo, S., Ren, Y., Stowik, A., and Mathewson, K. (2020). Inductive bias and language expressivity in emergent communication. arXiv preprint arXiv:2012.02875.

---

Hernandez- Leal,, P., Kartal, B., and Taylor. M. E. (2020). A very condensed survey and critique of multiagent deep reinforcement learning. In Seghrouchni. A. E. F., Sukthankar, G., An, B.. and Yorke-Smith, N., editors, Proceedings of the 19th International Conference on Autonomous Agents and Multiagent Systems, AAMAS '20, Auckland, New Zealand, May 9-13, 2020, pages 2146-2148. International Foundation for Autonomous Agents and Multiagent Systems.
Higgins, I., Matthey, L.,, Pal, A., Burgess, C., Glorot, X., Botvinick, M., Mohamed, S., and Lerchner, A. (2017). beta- VAE: Learning basic visual concepts with constrained variational framework. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26. 2017, Conference Track Proceedings. OpenReview.net.
Jang, E., Gu, S., and Poole, B. (2016). Categorical Reparameterizatioon with Gumbel-Softmax. Axiv:6611.0111 [cs, stat]. arXiv: 1611.01144.
Jaques, N., Lazaridou, A., Hughes, E., Gulcehre, C., Ortega, P. A., Strouse, D. J., Leibo, J. Z., and de Freitas, N. (2018). Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learning. arXiv:18 1111118100067777 [cs, stat]. arXiv: 18 1810.06647.
Jiang. Y., Gu, S., Murphy, K., and Finn, C. (2019). Language as abstraction for hierarchical deep reinforcement learning. arXiv preprint
Kaiser, L. and Bengio, S. (2018). Discrete autoencoders for sequence models. arXiv preprint arXiv: 1oXiv:1801.00 0099977.7
Kim, H. and Mnih, A. (2018). Disentangling by factorising, arXiv preprint arXiv: 1802.09983.
Kim, J. and Oh, A. (2021). Emergent communication under varying sizes and connectivities. In Beygelzimer, A., Dauphin, Y., Liang, P., and Vaughan, W., editors, Advances in Neural Information Processing Systems.
Kirby, S. (2001). Spontaneous evolution of linguistic structure-an iterated learning model of the emer- gence of regularity and irregularity. IEEE Transactions on Evolutionary Computation, 5(2):102- 110.
Korbak, T., Zubek, J., Kuciński, L., Milos, P., and Raczaszek-Lenardii, J. (2019). Developmentally motivated emergence of compositional communication via template transfer. arXiv preprint arXiv:1910.06079.
Korbak, T., Zubek, J., and Raczaszek-Leonarrd. J. (2020). Measuring non-trivial compositionality in emergent communication.
Kottur, S., Moura, J. M. F., Lec, S., and Batra, D. (2017). Natural Language Does Not Emerge 'Naturally' in Multi-Agent Dialog. arXiv:1706 170.008502 [es]. arXiv: 1706.08502.
Kriegeskorte, N. (2008). Representational similarity analysis connecting the branches of systems neuroscience. Frontiers in Systems Neuroscience.
Kuciński, E., Kolodziej, P., and Milos, P. (2020). Emergence of compositional language in communi- cation through noisy channel.
Lake, B. M., Ullman, T. D., Tenenbaum, J. B., and Gershman, S. J. (2016). Building Machines That Learn and Think Like People. 1rXiv:1604.0000 Nee::1104000000 00289 [cs, stat], arXiv: 1604.00289.
Lazaridou, A., Hermann, K. M., Tuyls, K., and Clark, S. (2018). Emergence Linguistic Commu- nication from Referential Games with Symbolic and Pixel Input. 1rXiv:18044 804,03984 [es]. arXiv: 1804.03984.
Lazaridou, A., Peysakhovich, A., and Baroni, M. (2016). Multi-Agent Cooperation and the Emergence of (Natural) Language. arXiv: 16li:01612.71180 182 [es]. arXiv: 1612.07182.
Lee, J., Cho, K., Weston, J., and Kiela, D. (2017). Emergent Translation in Multi-Agent Communication. _eprint: 1710.06922.
Levenshtein, V. I. (1966). Binary codes capable of correcting deletions, insertions, and reversals, In Soviet physics doklady, volume 10, pages 707-710. Soviet Union
12

---

Lewis, D. K. (1969). Convention: a philosophical study. Blackwell, Oxford, nachdr. edition. OCLC: 837747718.
Li, F. and Bowling, M. (2019). Ease-of-teaching and language structure from emergent communica- tion. In Advances in Neural Information Processing Systems, pages 15825-15835.
Liu, D., Lamb, A., Kawaguchi, K., Goyal, A., Sun, C., Mozer, M. C., and Bengio, Y. (2021). Discrete- valued neural communication. In Beygelzimer, A., Dauphin, Y., Liang, P., and Vaughan, J. W., editors, Advances in Neural Information Processing Systems.
Locatello, F., Bauer, S., Lucic, M., Raetsch, G., Gelly, S., Schölkopf, B., and Bachem, O. (2019). Challenging common assumptions in the unsupervised learning of disentangled representations. In International Conference on Machine Learning, pages 4114-4124.
Mordatch, I. and Abbeel, (2017). Emergence of Grounded Compositional Language in Multi-Agent Populations. In AAAI.
Nowak, M. A. and Krakauer, D. C. (1999). The evolution of language. Proceedings of the National Academy of Sciences, 96(14): (6(14)80288880)
Oliphant, M. and Batali, J. (1997). Learning and the Emergence of Coordinated Communication. Center for Research on Language Newsletter, 11.
Raczaszek-Leonard J., Nomikou, I., Rohlfing, K. J., and Deacon, T. W. (2018). Language Develop- ment From an Ecological Perspective: Ecologically Valid Ways Abstract Symbols. Ecological Psychology, 30(1):39-73.
Ren, Y., Guo, S., Labeau, M., Cohen, S. B., and Kirby, S. (2020). Compositional languages emerge in a neural iterated learning model.
Resnick, C., Gupta, A., Foerster, J., Dai, A. M., and Cho, K. (2020). Capacity, Bandwidth, and Com- positionality in Emergent Language Learning. arXiv:1910.11423 [cs, stat]. arXiv: 1910.11424,
Salakhutdinov, R. and Hinton, G. (2009). Semantic hashing. International Journal of Approximate Reasoning, 50(7):969-978.
Shamnon, C. E. (1948). mathematical theory of communication. The Bell system technical journal, 27(3):379-422 423..
Skyrms, B. (2010). Signals: evolution, learning, & information. Oxford University Press, Oxford New York. OCLC: ocn477256653.
Slowik, A., Gupta, A., Hamilton, W. L., Jamnik, M., and Holden, S. B. (2020a). Towards graph representation learning in emergent communication. arXiv preprint arXiv:2001.0900 000063.
Stowik, A., Gupta, A., Hamilton, W. L.,, Jamnik, M., Holden, S. B., and Pal, C. (2020b). Exploring structural inductive biases in emergent communication. arXiv preprint arXiv:2002.01335.
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R. (2014). Dropout:
simple way to prevent neural networks from overfitting. The journal of machine learning research, 1991)1199-
Seeinert-Theekeeld S. (2020). Towards the Emergence of Non-trivial Compositionaliy. Philosophy of Science.
Tucker, M., Li, H., Agrawal, S., Hughes, D., Sycara, K. P., Lewis, M., and Shah, J. (2021). Emergent discrete communication in semantic spaces. In Beygelzimer, A., Dauphin, Y., Liang, P., and Vaughan, J. W., editors, Advances in Neural Information Processing Systems.
Van der Maaten, L. and Hinton, G. (2008). Visualizing data using t-sne. Journal of machine learning research, 9(11).
Wattenberg, M., Viégas, F., and Johnson, I. (2016). How use t-sne effectively. Distill.
13

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] see Section
(c) Did you discuss any potential negative societal impacts of your work? [Yes] see
Section
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] see Section
3.3 and Appendix
(b) Did you include complete proofs of all theoretical results? [Yes] see Appendix F Fland Appendix B
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material included the supplemental material.
URL)? [Yes] The code is
(b) Did you specify all the training details (c.g.. data splits, hyperparameters, how they were chosen)? [Yes] see Appendix and Appendix B
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [Yes] We used 100 seeds. Confidence intervals are reported in figures and tables in the paper.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, cloud provider)? [Yes] See Appendix B.3
4. If you using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] see Section 4.3 and Appendix A
(b) Did you mention the license of the assets? [Yes] We use MIT license, see Appendix
(c) Did you include any new assets either in the supplemental material or as a URL? [Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you'l 're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "z01NbxHZ3ik", "reviewer": "Reviewer_HGBu", "summary": "The paper presents theoretical and empirical results showing that a specific type of utterance noise in a sender-receiver emergent communications architecture leads to compositional utterances, under certain constraints, and that the resulting loss function has all local minimum as global minimum (they don't state explicitly that all local minimum are global minimum, but there results strongly implies to me that they are). The following constraints need to be met in order to satisfy the theoretical proof:\n- message length = number of attributes\n- the input to the sender network should be a bag of attributes, not a non-symbolic, e.g. pixel, input (note that the paper calls attributes 'factors')\n- the vocab size of the utterance should exactly match the number of attribute values\n- the utterances are discretized using Gumbel, so we can backprop through them, and not using a discrete sampling, which would require REINFORCE\n- a specific type of noise is added to the utterances, which randomly flips/corrupts each specific token with an identical probability $\\epsilon$\n- the noise should be more than 0, and less then (vocab size - 1) / vocab size\n\nThe paper presents theoretical results basd on Locatello et al that show that without sufficiently strong inductive biases, that one cannot the utterances to be disentangled.\n\nThe paper presents comprehensive empirical results for scenarios which both somewhat align with the theoretical constraints above, and also that deviate strongly from these constraints. For example, all the experiments use non-symbolic inputs, which is therefore not a bag of attributes input. Most of the experiments use a message length that does match the number of attributes, but some experiments are done using longer utterances, and show similar results to the shorter messages.\n\nThe paper evaluates the results using 4 existing compositional metrics: topographic similarity, conflict count, context independence, and positional disentanglement.\n", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "# Main review\n\nThe paper provides a theoretical justification showing that, under fairly limitating constraints, noise introduced into the utterances of a sender-receiver emergent communications architecture leads to pure compositional utterances, and that all local minimums of the loss function are global minimum (they don't state that all local minimum are global minimum, but their theoretical results strongly imply this, I think). The constraints around this theoretical result are fairly strict. The message length is tiny. The inputs to the sender must be symbolic bags of attributes. A specific loss function must be used on the receiver, which receiver must predict the bag of attributes that was originally provided to the sender. These are I feel fairly strict constraints, that are not matched by many recent emergent communications works. Nevertheless, I really like that there is any theory at all. I feel that emergent communications could benefit from some theory, rather than just running some experiments, eye-balling the utterances, and saying 'look! compositional!'. I do think that the theoretical results of this paper are a useful baby-step towards more rigorous theoretical results for emergent communications as a whole.\n\nThe explanations given in section 3 are I feel fairly hard to understand. I would prefer that section 3 would be written to be much more intuitive and easier to understand. But I will probably let that slide in my own rating of this work, and approve this work for publication anyway.\n\nThe introduction and related works sections are I feel pretty solid and complete, and were easy for me to follow.\n\nThe experimental results section was I feel extremely comprehensive and nice. I really like the set of experiments and results provided. In addition, the experimental work probed what happens when one violates many of the constraints of the theoretical results, and showed that the empirical results aligned well with the theoretical results in any case under such circumstances. I suppose the main deviation from the theoretical results (which wasnt highlighted as such, and perhaps could/should be), is that the theoretical results hold for any noise $\\epsilon$ that is greater than zero, and smaller than (vocab size - 1) / vocab size, which is a pretty broad range of values, extending to most of the range of $(0, 1)$. However, the empirical results using non-symbolic, pixel, inputs showed that the noise must be relatively small. This discrepancy between theory and practice was neither I feel pointed out clearly, nor was any explanation provided for why it is so. Still, I do feel that the provision of theoretical results, however tightly constrained, is an important step forward; and the empirical results did align fairly well with the theoretical results, even when the constraints of the theoretical results were violated somewhat.\n\nOverall, whilst the explanations in section 3 are I feel hard to understand, and the constraints on the theoretical results of the work are quite strict, I feel that this work provides a nice baby step towards more rigorous theoretical understanding of emergent communications; and I feel the empirical work in the experiments section is very nice. At this point in the review lifecycle, my intent is to approve it for publication.\n\n# Specific points I'd like to see addressed\n\n- section 3 target audience\n    - I feel that section 3 was written with a target audience of e.g. the person's advisor. Thus, any description of the setup of the task has been entirely skipped, and only the briefest description of the theorems is provided. I feel that a target audience that would make the work easier to understand for me would be, well me, but more broadly, perhaps a good target audience to keep in mind could be the author themselves, imagining that they only know eg question-answering domain, or machine translation domain, and knows nothing about emergent communications domain\n- cut $J_1$ loss\n    - No justification is provided for the inclusion of the $J_1$ loss, neither intuitive, nor theoretical (appendix F just states that a compositional language would reduce the loss to zero, but does not prove that all local minima are global minima, for example). Nor is any explanation provided anywhere as to how $J_1$ and $J_2$ are combined (i.e. some sort of $\\lambda$ trade-off presumably). The explanations of $J_1$ loss are complicated, hard to understand, and not backed by any intuition insight/jusification. They take up lots of space that could be used to present the task etc instead. I feel that $J_1$ loss should be cut entirely. (Or else much more explanation should be provided to justify why it is needed)\n- constraint on $\\epsilon$ in Theorem 2\n    - the constraint on $\\epsilon$ in theorem 2, as stated in the main paper body, is misleading I feel. They state that a constraint is that \"$\\epsilon$ is small enough\". This is technically true, but 'small enough' means 'smaller than (vocabsize - 1) / vocabsize', which is most of the range $(0, 1)$. It does not need e.g. $\\epsilon \\sim 0$, which I felt was implied here. I feel that the actual constraint on $\\epsilon$ should be concretely given. I feel in hindsight that perhaps it is phrased as it is, because this makes it sound more similar to the empirical results reported later on, which show that $\\epsilon$, when violating $\\mathcal{X} = \\mathcal{F}$, does need to be relatively small. However, I think it is important to highlight this difference in the constraint on $\\epsilon$ for the theoretical results versus the optimal range of $\\epsilon$ in the empirical results. This difference shows potential opportunites for future work perhaps\n\nAny of these specific points could be cited by reviewers as sufficient and complete reasons to reject this work, for now, I feel, depending on what other reviewers feel about this work. I do feel that I'd strongly prefer that all three of these points are addressed prior to publication.\n\n# Notes whilst reading\n\nThe following are notes I made whilst reading the paper. They are not filtered, or edited, in any way, but simply a stream of consciousness whilst I was writing. Make of them what you will :)\n\n## after abstract:\n\n- abstract looks interesting\n- I think I've seen similar results before, possibly, something like \"Learning to Communicate with Deep Multi-Agent Reinforcement Learning\", Foerster et al, 2016 (but no, thats not quite teh same actually)\n\n## 1. Introduction\n\n- pretty comprehensive range of references\n- conflict count Kucinski et al 2020 is new to me.\n\nby the end of the second paragraph of the introduction, I feel the authors are very strong, and know what they are talking about.\n\n[after checking kucinski et al reference] oh I have seen this paper. I just dont remember what is in kucinski et al paper. Oh, it is also about noisy channel, and is probably one of the papers I remember seeing about noisy channel, after reading the abstract.\n\nGoing back to the second paragraph of the paper under review, I like that it cites Locatello et al. This is I feel a key paper for anything involving disentanglement, which trying to emerge compositionality highly resembles, I feel.\n\n\"Such a result can be perceived as a discrete analog of Locatello et al. [2019], applicable in the communication context.\". Hmmm, I worked out the maths for that once, a couple of years ago, but never did anything with it. /me kicks myself :P\n\n\"We then prove that adding an inductive bias in the loss function coupled with communication over a noisy channel leads to the spontaneous emergence of compositionality. This shows the catalytic role of noise in this process.\" Ok, so the authors are probably going further than the tiny baby step I took with discrete Locatello et al proof.\n\n## 2. Related work\n\n\" Such an approach inevitably introduces noise into the learning process. This naturally leads to a question of whether the noise itself may be a sufficient mechanism of compositionality\". Agreed. I have pondered the same thing, but again, done nothing useful with it :P\n\nNowak and Krakauer 1999 is new to me. I need to read that.\n\nGood observation about noise allowing backpropagation through a discrete latent. An interesting way of looking at this, I feel.\n\nThe Foerster et al 2016 reference was cited too, which was the only reference I remembered after reading the abstract earlier. So, good.\n\nBy the end of Related work section, I'm pretty convinced this is going to be a very solid paper.\n\n## 3. Noisy channel method\n\n### 3.1 Language and compositionality\n\nI like the use of the word 'factor' instead of 'attribute'. Although it's not clear to me after reading the first three sentences of section 3.1 what is the definition of a factor. Is it a single attribute, e.g. 'red', or a set of attributes for a single object, e.g. 'red square'? I find myself reading the sentence over and over to figure out the answer to this. You might consider helping me to understand which of these choices is the definition of a factor, in the paper text, perhaps. Ok, so, the fourth sentence implies strongly that a factor is a set of attributes for a single object, eg 'red square'. However, this doesn't align perfectly with the sentence 'the factor could represent shape, ..., and color'. Perhaps the sentence could be reworded slightly to 'the factor could represent a combination of shape, ..., and color'?\n\nI like the definition of $\\mathcal{X}$ and $\\mathcal{F}$, and the description of the two-stage sampling process. Nice.\n\n\nI like the definition of a language as mapping to $\\mathcal{A}_s^*$. That's pretty compact and descriptive, I feel.\n\n\"We will say that l is compositional with respect to a given factor if a change in i-th factor only impacts\n85 a corresponding j-th index of the message\"\n\nI'm a bit dubious about this definition of compositoinality. Unless you're using two letter utterances, I feel there's going to be more than a single index that is going to be impacted by changing a factor. (In practice, I'd expect most of the characters to be impacted to be honest; but assuming perfect compositionality, if we have 6 letter.... hmmm ... here factor is being defined as a single attribute I think, eg 'red'. Otherwise it doesn't make sense to talk about the effect of the $i$-th factor on the $j$-th index of the message. I think you might want to think carefully about what definition of factor you are using. I'm not sure you are being consistent :)\n\nAt this point, in the last two paragraphs, I feel less in the hands of an expert, and am ready to start to revert my earlier impressions on the paper.  I'm wondering if the paper was written by >1 person, and section 3 was written by a different person, perhaps?\n\nBy the way, what does the $_s$ in $\\mathcal{A}_s$ mean? It has not been stated yet. I feel that either this should be stated, or the $_s$ should be dropped for now, for simplicity?\n\nBy the end of the second paragraph of 3.1, I mean, yes, such a language would be compositional, but using only two letters to represent two attributes (I'm using 'attributes' instead of 'factors', since attribute is widely used in the literature, and the definition is widely understood; whereas it's not clear to me which definition of 'factor' the authors intend here?)\n\n3rd paragraph\nnit: 'and denoted' => 'and denote'\n\nSo, I'm unsure why permuting $\\mathcal{F}$ would affect the language. $\\mathcal{F}$ is simply a set. It has no order. It was stated in the first sentence of 3.1 that $\\mathcal{F}$ is a set. But even if it had order (which it does not, being a set), the language would not change simply because we had swapped the order of our ordered set, since the language is a mapping from sets of attributes/factors to utterances, and as long as $\\mathcal{F}$ contains the same set of sets of attributes/factors, which it does, since we merely permuted it, then the language should be unchanged, I feel?\n\npossibly, what is intended to be meant is that if one samples some elements $\\{f_1, f_2, \\dots, f_n}$ from $\\mathcal{F}$, and permute those? I think that paragraph three needs to be carefully rethought/reworded. I don't feel it makes sense currently. At the very least, I feel it is hard to understand (e.g. perhaps I am misunderstanding what it is saying?).\n\n### 3.2 Inductive biases and compositionality\n\nI couldnt really follow 3.2, since $\\mathcal{F}$ is a set, so I'm unclear how we can permute it? The other sentences in 3.2 don't really seem to clearly express their meaning to me either.\n\nI'm a deep believer (if that's the right word for a mathematical proof) in Locatello et al. And I think Locatello et al have stated pretty clearly that you cannot in general disentangle representations, since any axis-aligned distribution can be projected to a uniform sphere, spun arbitrarily, then reprojected back into the original space, which is now entangled. Since the intermediate sphere is uniform, and can be rotated arbitrarily, there is no way of determining how to rotate it to obtain the original axis-aligned distribution. (I dont think I explained this really well). In any case, the point is that if one has a latent vector, in the absence of supervision, under very general conditions, one cannot expect e.g. the first element of the vector to become aligned with e.g. shape, and the second to become aligned with e.g. color. The vector will in general be entangled, so that all attribute are \"smeared\" across the entire vector. We would expect the same thing to hold true also for utterances: each attribute will be \"smeared\" across the entire utterance. How this relates to what is being stated in 3.2 is unclear to me. So, I feel that 3.2 needs to be reworded somewhat.\n\nIn addition, I'm not sure that 3.2 shows understanding of what Locatello et al states. I will wander off to find what they state:\n\n... well they state \"We theoretically prove that (perhaps unsurprisingly) the unsupervised learning of disentangled representations is fundamentally impossible without inductive biases both on the considered learning approaches and the data sets\" (which aligns with what the paper under review states)\n\nBut I think they state something more strongly than that. Let's see...\n\nwell Locatello et al state in section 3 \"Theorem 1 shows that unsupervised disentanglement learning is fundamentally impossible for arbitrary generative models\".\n\nThey do however continue \"this does not necessarily mean it is an impossible endeavour in practice. After all, real world generative models may have a certain structure that could be exploited through suitably chosen inductive biases. However, Theorem 1 clearly shows that inductive biases are required both for the models (so that we find a specific set of solutions) and for the data sets\"\n\nHowever, I'm not sure I agree with that. If they cannot be disentangled unsupervised, they cannot be: they must have supervision. Perhaps I am misunderstanding Locatello et al myself :P\n\nHowever, going back to the introduction of Locatello et al, their empirical results align with my own point of view \"We do not find any evidence that the considered models can be used to reliably learn disentangled representations in an unsupervised manner as random seeds and hyperparameters seem to matter more than the model choice. Furthermore, good trained models seemingly cannot be identified without access to ground-truth labels even if we are allowed to transfer good hyperparameter values across data sets.\"\n\nIn any case, going back to section 3.2 of the paper under review, the presentation doesn't really discuss any of the above points at all, but simply gives what I feel is a very hand waving sequence of statements, that involves permuting a set, which as far as I know is something that does not possess an ordering?\n\nI googled 'set', doubting my maths. It seems it is possible to have ordered sets, and partially ordered sets, but the general concept of a set does not possess order, e.g. wikipedia says \"in mathematics, a set is a collection of distinct elements\", https://en.wikipedia.org/wiki/Set_(mathematics)#cite_note-JainAhmad1995-1\n\nLooking back at the earlier 3.1, it says that $\\mathcal{F}$ contains a 'combination' of values, and a combination is also unordered, c.f. a permutation. I think that section 3.1 and 3.2 should be at least rewritten to be more clearly understandable.\n\n### 3.3 Compositionality and communication over a noisy channel\n\n\"a loss function that ... rewards for (partially) correct guesses\". Since we don't have a task yet, I have no idea what the loss function is comparing. Is it comparing attributes? Utterances? Is it a referential task? If it is a referential task, what does it mean to have partially correct guesses. I feel that the presentation is out of order at this point: we need to know what task is being discussed, before we can be told about constraints on the loss function, I feel.\n\nIn addition, I mean, I have a pretty good idea of the architecture of a sender, receiver, intermediate utterances, however I feel that in general, a diagram and description should be provided at this point.\n\nparagraph 3 of 3.3: starts to talk about factorized loss functions, but again, without presenting what is the task yet. I'm going to guess the task is a symbolic task, where the sender is provided a set of symbolic attributes, sends a message to the receiver, and the receiver attempts to reconstruct the original set of attribute. But this is basically a guess on my part, and should be stated explicitly in the paper.\n\nNote that using symbolic tasks for experiments on compositionality is I feel a relatively weak approach to studying compositionality, since the inputs to the sender are already factorized/compositional, so I'm not incredibly impressed when the sender turns around and sends something somewhat compositional as a result. In fact, in general, I'm more stunned by how incredibly uncompositional the resulting utterance turns out to be. But that's off-topic: in general, I'm not a fan of compositionality experiments that use symbolic, structured inputs.\n\nOk, I scrolled ahead a bit, and there are pictures of 3d images in figure 1.\n\nOk, back to 3.3. I mean, I have no idea what task is being described. This needs to be fleshed out considerably I feel.\n\nSo, in equation 1, $l$ has been previously described to be a language, not an utterance, but a language (second paragraph of 3.1). So, is $l$ here being redefined to be an utterance? or does the loss function take as one of its inputs an entire language? Many languages might not even fit in memory on a standard computer, so it surprises me for the loss function to take as an input an entire language.\n\n$\\textbf{f}$ has not been defined. We had $f$ earlier (without bold), but not $\\mathbf{f}$. I feel it should be defined.\n\noh, $l$ has been defined to mean 'the corrupted message corresponding to $\\mathbf{f}$'. So, $l$ is being redefined. I feel that symbols should ideally be used consistently throughout the paper.\n\nl maybe means the sender network, and $l^{-1}$ is the receiver network? This is not stated.\n\nAnd maybe the loss is going to be a reconstruction loss between $l^{-1}(l(\\mathbf{f})')$ ? I think the notation $'$ being used to mean a function applying noise is very compact, but I'm not sure that using `'` to denote a function is standard? I guess it is readable-ish though, and I'm not a mathematician, so I guess it is probably ok.\n\nthere is no intuition being supplied as to why we are taking the KL wrt a uniform distribution. I feel more intuition might be useful.\n\nOk, so working through this... $\\mathbf{f}_j$ means presuably the $j$ attribute. and $e_j$ is the probability that passing the set of attribute through the sender-noise-receiver network has flipped the $j$ attribute. (note: I'm not sure I like the word 'flipped' here, which implies a binary choice. at least to me. Maybe simply 'changed'? I'm not sure). so anyway, $e_j$ is the probability that passing ... so wait... is $e_j$ the probability marginalized over all possible $j$-th attributes? Or is it the probability given some specific input set of attributes? I'm going to guess the latter, though it's kind of odd, since none of the $\\mathbf{f}$ are indexed by e.g. an example index number or similar. Nor is it stated that $\\mathbf{f}$ represents the attributes of a single example. But I guess it does. So, given some specific example, which is a set of attributes, e.g. a color and a shape, $e_j$ is the probability that after passing through the sender-noise-receiver, that the $j$-th attribute has been corrupted (I prefer 'corrupted' to 'flipped' I feel).\n\nHow is this probability being calculated? No indication is provided. It's just assumed that we can calculate this. Are we using Monte-Carlo sampling? Something else? Very unclear. Are we in fact marginalizing across all/many examples, or e.g. a mini-batch?\n\nThen we take the KL divergence of a probability vector obtained by calculating $e_j$ for each of the attributes, w.r.t a uniform distribution.\n\nOk, so $x$ is not defined. I guess it's a draw from $\\mathcal{X}$, so each $x$ is an object. But that's doesnt really make sense. Why are there $K$ objects? So does $x_k$ mean attribute $k$ of object $x$? But isn't $x$ an instance having certain attributes, e.g. a picture of a red ball?  It seems like $\\mathbf{f}$ should be representing the attributes, and not $x$? I feel that $x$ needs to be defined. And $x_k$ needs to be defined.\n\nSo, let's assume that $x_k$ means attribute $k$. Now, KL is defined only between two probability distributions. Which means that in the expression $F(\\epsilon, \\dots, \\epsilon)$, $\\sum_{k=1}^K \\epsilon = 1$, therefore $\\epsilon = 1/K$. However, earlier $\\epsilon$ appeared to be an arbitrary probability, configured as a hyper-parameter, something like dropout. It is not clear to me therefore whether $\\epsilon$ is not $1/K$, and therefore $F(\\epsilon, \\dots, \\epsilon)$ is not being calculated over a probability distribution, or else $\\epsilon := 1/K$. I feel this needs to be explained more carefully please.\n\nNo intuition has been provided into what is intended by the mathematical equations.\n\nFor equation (2), I feel it is weird not to use $H$ to mean entropy, especially in this context. I feel that a symbol such as $g$ might be more standard perhaps?\n\nEquation (2) is at least somewhat intuitively underestandable, but note that it does not define what the expectation is over. Is it over the language? A mini-batch? somethign else? I feel it should be defined what the expectation is being taken over.\n\nTheorem 2: why is $\\mathcal{F} = \\mathcal{X}$? Previously, elements of $\\mathcal{X}$ were being sampled on distributions conditioned on draws from $\\mathcal{F}$ (start of section 3.1). It's unclear to me why these spaces are even comparable, let alone equal. I feel that more explanation is required here, please.\n\nOk, so.... stepping back, and thinking about this... the loss function assumes that we have a sender-noise-receiver network, that takes in a set of attributes, and outputs a prediction of the original attributes. the loss function then compares these predictions with the originals by ... calculating the probability of corrupting them. How this probability is being calculated/determined is as far as I can tell unstated/unclear. Let's assume that the authors are marginalizing over a mini-batch or so, i.e. Monte Carlo approximation. So, run a batch of examples through the network, measure empirically how many times each attribute gets flipped, at each position. calculate a loss based on this. and there is a second loss over how far away the prediction is from the ground truth original attributes. Over time, the predicted output will likely resemble the input attributes. But what does this say about the intermediate utterance $l(\\mathbf{f})$? A theorem is asserted, but not proved, that states that a compositional language minimizes the losses. That doesnt sound implausible. And it states that any $l$ that minimizes $J_2$ is compositional. I don't have access to theorem 2, so am unable to judge this.\n\nOk, finally discovered the 'supplementary material' button in the reviewing page :P Downloading and reading...\n\n## Appendix B.1\n\nOk, so the uttearnces are length 2. That seems ... unambitious... to me, but ok.\n\nscanning forward for the theorems, wow there are a ton of experiments in the appendix :)\n\n## Appendix F:\n\nok, so $\\mathcal{F}$ is the space of all possible sets of attributes. $\\mathcal{F}_i$ is the space of values for a single attribute. $K$ is the number of attributes. And the length of the utterances is set to equal the number of attributes, i.e. to $K$. The vocabulary size $|\\mathcal{A}_s|$ is set to the number of attributes values $|\\mathcal{F}_i|$. We assume at least 2 different letters.\n\nThe language maps from the space of all possible sets of attributes to the space of all possible K-length utterances, using vocabulary size $|\\mathcal{A}_s|$. I note that the theorem is relatively constrained in only applying to scenarios where the utterance length is exactly identical to the number of attributes, and the vocabulary size is exactly equal to the number of attribute values.\n\nI note that the definition of compositionality being used basically corresponds to that used in topographic similarity (e.g. Lazaridou et al 2018), albeit a stricter version. So, the authors are asserting that the L0 distance between two attribute sets should equal the L0 distance between teh resulting K-length utterances. Which makes sense to me.\n\n$\\mathbf{f} \\in \\mathcal{F}$ is presumably a specific set of attributes, eg 'red ball', drawn from the space of all possible sets of attributes.\n\neyeballing equation 7, I guess it might make sense. I think it would be good to explain the other two terms, on the left of the right hand side of the equation, just as you have explained the right most term. But they seem plausible-ish to me, eye-balling.\n\n### Theorem 3:\n\nI'm not sure why we are assuming $\\mathcal{F} = \\mathcal{X}$, but ok.\n\nSo, here the expectation is over $\\mathbf{f} \\sim \\nu$, and the explanation makes sense to me.\n\nOk, far too much maths for me. I'm going to assume the maths is probably correct.\n\nSo, the maths assumes:\n- $\\mathcal{F} = \\mathcal{X}$\n- vocab size = number of attribute values\n- utterance length = number of attributes\n- $\\epsilon < (vocabsize - 1) / vocabsize$\n\nAnd the proof says that a loss function which is for example proportional to the L0 distance between the predicted attribute set and the input attribute set will drive the utterances to be perfectly compositional.\n\nI wonder what assumptions it has on the noise? i.e. does it assume noise > 0. checking... well, I guess equation 7 would have a right hand side of 0 if $\\epsilon = 0$. So, ok, makes sense.\n\nLet's move back to the main body of the paper...\n\n### section 3.4\n\nok, so the pipeline is:\n\npixel input => sender => utterance => noise => noised utterance => receiver => predicted attributes\n\nAs stated, I think this should be stated earlier, or at least provided as an example implementation, of a more general architecture.\n\nI guess it makes intuitive sense that the utterances in this case might become compositional. Interesting that one needs noise for that to be the case. Interesting approach to noise. (I recognize that the noise design is not from this work, but I do understand the explanation of the noise better in this work tbh. Good :) )\n\nThe explanation of the noise matrix more or less makes sense to me. I do think that an example might be useful potentially.\n\n### 4.3 Compositionality measures\n\nNit: the caption for figure 1 sort of blends in with the text, and it's hard to figure out where one ends and the next begins. Maybe separate these slightly more?\n\nFigure 2 is interesting.\n\nNice comprehensive list of compositionality metrics :)\n\n## 5. Experiment results\n\nFigure 3 top row: interesting.\n\nFigure 3 bottom row: very interesting. Interesting how the distributions are basically bimodal, but the mass shifts from the rightmost peak to the lowermost peak, ad add noise.\n\neffect of variable noise is interesting to me\n\neffect of changing the type of noise is interesting to me.\n\neffect of visual priors is interesting\n\neffect of scrambled labels is I feel a pretty nice experiment\n\nnetwork features generalization is interesting I feel\n\nthat compositionality is not sufficient to give good generalization is interesting I feel. Note that I'm not sure what is the goal of figure 5. I feel figure 5 could be cut, to make space for more explanations in section 3.\n\n## 6 Limitations of the method\n\nI like the presence of this section. This section seems fairly complete, and I like that.\n\n# Thoughts after reading through the paper\n\nAt this point, I feel that:\n- I like the theoretical contributions of the paper. Yes, they are for very constrained situations, but I feel this is a great start for more complete theoretical contributions by the authors and others in the future.\n- I like the extremely comprehensive set of experiments, given these constraints, and that they use non-symbolic input\n- section 3 is I feel extremely challenging to understand, but probably not enough to tip the balance towards rejection I feel. I do feel that if section 3 could be written more clearly, this would certainly have made it much easier for me to understand the paper\n\nAt this point, I will look (and not before) at the information provided about previous submissions:\n\n# Thoughts after reading submission history\n\n\"The reviewers had concerns about the layout of the paper, the presentation of the results, and the clarity of the description of technical details. To address this, we thoroughly rewritten the paper, reworked the presentation of the experiment, and fixed a number of errors.\"\n- so, the results presentation looks good to me\n- the layout of the paper seems reasonable to me\n- the clarity of the description of the technical details is, as stated above, still a major concern I feel\n\n\"We also run all experiments from scratch on 100 seeds (a 5x more than in the previous submission)\"\n- ok. I'm not sure that that was a concern of mine. ok.\n\n\n\"At the reviewers’ request, we added additional compositionality metrics: positional disentanglement.\"\n- Ok. Good. I do like having multiple compositional metrics, yes\n\n\"Additionally, following the reviewers’ advice, we run experiments with a longer message length.\"\n- technically this is true: there are experiments on a longer message length. And I did like this. The theoretical results of course are still only for the shorter messages, but: baby steps, have to start somewhere\n\n\"The reviewers wondered about the validity of Theorem’s 2 assertions when its assumptions are not met. We computationally verified that for small cases, it is indeed true.\"\n- I have the same concern, but I do feel that the empirical results you've added help alleviate this concern, and I'm happy to see some small baby steps toward some actual theory in emergent communications\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The paper has a 'limitations of the method' section, which is fairly comprehensive, and addresses most/all of the limitations of their theoretical work. The paper presents experiments which show empirically that violating many of these limitations does not cause the empirical results to deviate significantly from the conclusions of the theoretical results.\n\nOverall, I think the paper does a good job of stating clearly the limitations of the work, (albeit, this is left until the section right at the end of the paper).\n\nThere are no obvious negative societal implications of the work, since this line of research is very theoretical, and does not involve classes of people, or similar. Any natural language utterances associated with the experiments are limited to very simple concepts such as `red square'.", "review_text": "", "extra_scores": {"ethical_concerns": "None noticed", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5-6 hours"}, "extra_text": {}}, {"review_id": "IkdVWp9JqJ3", "reviewer": "Reviewer_PkWg", "summary": "The community has been looking at compositional generalisation more recently, and while Fodor and Pyshylynn is an oft cited work, there is a lack of a good definition of compositionality that researchers can agree on. \n\nThis paper suggests that a noisy channel with an encoder decoder setup is crucial for the emergence of compositionality. The authors propose a simple definition of compositionality in a non-hierarchical case, and discuss how 1) if no assumptions or prior information about the data is incorporated into the training process, the resulting methods will be uncompositional, 2) if this inductive bias is injected through the use of a noisy channel, there are losses (namely J_1, J_2) that can be used to encourage the rise of a compositional encoding. \n\nThe basic setup involves communication through a noisy channel. They then discuss how, if the encoding is compositional, there are particular losses (J_1, and J_2) that will be optimal. More importantly, when J_2 is optimal, the encoding is compositional. In their experiments, they use J_1 primarily, and demonstrate compositional emergence (according to some metrics) under some circumstances. \n", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: I am not aware of other work attempting to give a definition for compositionality, even in this limited form.\n\nQuality:\n I find this definition of compositionality potentially useful, perhaps in future it can be extended to hierarchical cases, and therefore to domains like language. For now, there seems to be some similarity to the notion of equivariance, which can potentially be looked at for extending the current work.\n\n1. The losses J_1 and J_2 are claimed to be optimal when the language is compositional. How J_2 works is much clearer to me. J_1 (as I understand it) requires the probability of P(f’!=f’) , which would require integration over the latent state and the output probabilities at the decoder. I’m not sure if this is being accounted for in the appendix, but this is perhaps due to a lack of understanding on my part.\n\n2. Theorem 2: Am I right to say that minimising J_1 does not guarantee compositionality? The implication in the other direction only seems to apply to J_2. \n\n3. Experiments: Given the nature of the theory here, it would be nearly impossible to test it on a “real” task. Still, I need to ask, is there a way to train a simple model on a simple task with J_1?\n\n4. Experiments: are there other benchmarks on compositionality that can be compared with? I understand that the authors are demonstrating how varying particularly the noise parameters affect compositionality, but it would be useful to see how it stacks up against other methods .\n\n\nClarity: The paper is clearly written, but certain aspects of the encoder decoder and its details are hard to find. In fairness, some of these details are in the appendix, and it is hard to fit all details into the page budget. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "\nAs mentioned in their conclusion, their work is largely theoretical, and does not directly lead to negative applications. However, improving our understanding of compositionality may lead to much better natural language understanding systems. As with all improvements in technology, these are fraught with positive and negative uses. \n", "review_text": "", "extra_scores": {"ethical_concerns": "Not applicable", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "9ujVCTZQ68i", "reviewer": "Reviewer_ecJL", "summary": "The authors analyze the effect of a _noisy channel_ on an emergent communication protocol.  In particular, they show that languages that minimize losses involving this noise are compositional [in their sense of the term] and then conduct a very thorough set of experiments while varying noise in a standard emergent communication.  They find that compositionality tends to go up for some low levels of noise, but then goes down after that (and accuracy tends to go down consistently with more noise).  There is also a very thorough set of experiments manipulating myriad factors in the setup and finding that the overall patterns just discussed are robust.  The paper is a very interesting theoretical and experimental contribution to the emergent communication literature.  My only minor complaint would be that there are so many experimental results that it's easy for the reader to lose the forest for the trees; more synthesis of the overall trends would be welcome.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "In addition to the summary above:\n\n# Minor questions\n\n* Why was a feed-forward instead of recurrent network?\n\n* Fig 4(b): what about accuracy for this setting?  Does it still decline in higher noise values, or does it rise with higher topo even with more noise?\n\n\n# Missing references\n\n* It would be useful to discuss Steinert-Threlkeld 2019: https://www.journals.uchicago.edu/doi/10.1086/710628.  The notion of compositionality in the present paper is what he refers to as \"trivial\" compositionality.  This is still a hard and valuable target, but does not go all of the way to capturing the rich forms of composition that exist in natural language.  It would be good to call attention to this aspect of the definition of compositionality in the present paper.\n\n* Pagin and Westerstahl 2010a/b show the following result: on their [relatively standard] definition of compositionality, all one-to-one mappings are compositional.  Is this at all connected to the one-to-one assumption made in the proof of your theorem?  It might be nice to discuss.\n\n\n# Typographic comments\n\n* page 3, line 98: \"Theorem 1....\" is a sentence fragment, with no verb.\n\n* page 3, line 128: missing parenthesis after $e_j$\n\n* page 5, line 214: \"s^k\" should be \"s^f\"\n\n* page 6, line 233: maybe better to use \"main experiment\" instead of \"master experiment\"", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes, the authors provided a very nice limitations section.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "1.5"}, "extra_text": {}}, {"review_id": "6m1iBeK-lJX", "reviewer": "Reviewer_H5rU", "summary": "This paper adds results to the literature on learning compositional emergent communication protocols. Two results are theoretical: that inductive biases are required in order for compositional communication to emerge, and that adding a noisy channel with a specific loss function leads to compositional language emergence. The third result is a small-scale empirical corroboration of the latter theoretical result in a speaker-listener game.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I thought this paper was fairly interesting. It adds to the many other papers that investigate factors that affect the emergence of compositional communication. This paper is unique in that the main contributions are theoretical, rather than empirical -- the paper proves that, \n\nAs far as I can tell, the results are technically sound. But it's a bit difficult for me to assess the significance of the paper: \n\n- The first theorem, that inductive biases are needed for emergent communication, is interesting but also not very surprising (having agents policies' represented by neural networks is already a form of inductive bias?). \n\n- The more consequential result in my view is the idea that adding noise to the communication channel promotes compositionality (Theorem 2 + empirical results). I find it surprising that nobody has published this before -- the closest that I can think of is the work by Foerster et al. (2016) who show that adding noise to a continuous communication channel leads to the formation of discrete symbols (which is different than how the work is described in this paper). If this is indeed true, I find the result pretty interesting (it's also simpler than many previously proposed strategies for attaining compositional emergent language). \n\n- The empirical results are done at quite small scale, though this is fairly standard in the emergent communication literature. \n\n- I'm uncertain as to the value of these kinds of theoretical results in the field of emergent communication, but this could just be a function of differing research interests / tastes.\n\nAs an aside, I greatly appreciate the section on the limitations of the paper, and see this as a big plus. \n\nOverall, I lean slightly in favor of acceptance, but I'm uncertain in my assessment.\n\nMinor points:\n- The citation style seems non-standard for NeurIPS", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "There is a specific section on limitations which is greatly appreciated. Societal impact is briefly mentioned", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper studies the effect of a noisy channel in an emergent communication setting. They formally prove that, under fairly restrictive assumptions (type of noise, loss functions used, capacity of the channel, etc.), noise in the communication channel promotes emergence of compositional languages. The paper presents extensive empirical results studying the emergence of compositional languages in conditions that also diverge from the theoretical ones. 

--

This paper offers to the literature of emergent communication a first, restrictive, but interesting, formal result on the emergence of compositional languages. All reviewers agree that the paper is interesting and the experiments are extensive. The section on the limitation of the current approach and the restrictive assumptions of the theorems have been found useful and important to inspire future work. I suggest the authors to incorporate the precious reviewers' feedback, and especially: (a) the sheer amount of experiments is hard to process, one possible direction for improvement would be to add a synthesis of the overall trends and most important observations; (b) improve writing and clarity of Section 3, expanding on the relevance and interpretation of the theorem and the task setup.

Overall, I think this paper brings a worthy contribution to the field and can inspire future theoretical work to relax the strict assumptions considered here. Therefore, I recommend this paper for acceptance.

Decision: Accept (Poster)